| Property | Value |
|---|---|
| Author | AI-Growth-Lab |
| Vector Dimension | 768 |
| Paper | arXiv:2103.11933 |
| Architecture | MPNet-based SBERT with CLS Pooling |
What is PatentSBERTa?
PatentSBERTa is a specialized deep learning model developed by Aalborg University Business School's AI Growth-Lab for patent analysis and classification. It's built on the SBERT architecture and maps patent text to dense 768-dimensional vector spaces, enabling sophisticated semantic search and similarity analysis in patent documents.
Implementation Details
The model utilizes a transformer-based architecture with MPNet as its backbone, implementing CLS token pooling for sentence embeddings. It's trained using cosine similarity loss with AdamW optimizer and WarmupLinear scheduler, fine-tuned with a batch size of 16 and learning rate of 2e-05.
- Maximum sequence length: 512 tokens
- Optimized for patent-specific language and technical content
- Supports both sentence-transformers and HuggingFace Transformers implementations
Core Capabilities
- Dense vector representation of patent documents
- Semantic similarity computation between patents
- Patent classification and clustering
- Technical language understanding and processing
Frequently Asked Questions
Q: What makes this model unique?
PatentSBERTa is specifically designed for patent analysis, combining SBERT architecture with patent-specific optimizations. Its 768-dimensional vector space is particularly effective for capturing technical and legal language nuances in patent documents.
Q: What are the recommended use cases?
The model excels in patent similarity search, classification tasks, technical document clustering, and patent landscape analysis. It's particularly useful for patent offices, R&D departments, and intellectual property researchers.