| Property | Value |
|---|---|
| Parameter Count | 125M |
| License | Apache 2.0 |
| Paper | Sentence-BERT Paper |
| Output Dimensions | 768 |
What is roberta-base-nli-stsb-mean-tokens?
This is a deprecated sentence transformer model based on RoBERTa architecture that maps sentences and paragraphs to 768-dimensional dense vector representations. While historically significant, it's now considered outdated due to producing lower quality embeddings compared to newer alternatives.
Implementation Details
The model utilizes a RoBERTa base architecture combined with mean pooling strategy. It processes input text through the transformer and applies mean pooling on token embeddings while accounting for attention masks. The implementation supports both sentence-transformers and HuggingFace Transformers frameworks.
- Maximum sequence length: 128 tokens
- Pooling strategy: Mean tokens (no CLS token or max pooling)
- Word embedding dimension: 768
Core Capabilities
- Sentence and paragraph embedding generation
- Semantic similarity computation
- Text clustering applications
- Semantic search functionality
Frequently Asked Questions
Q: What makes this model unique?
The model combines RoBERTa with specific pooling strategies and was trained on NLI and STS benchmark datasets. However, it's now deprecated in favor of more modern alternatives.
Q: What are the recommended use cases?
Given its deprecated status, it's recommended to use newer models from SBERT.net's pretrained collection instead. However, if using this model, it's suitable for basic sentence similarity tasks and semantic search applications.