sbert-roberta-large-anli-mnli-snli
| Property | Value |
|---|---|
| Model Architecture | RoBERTa-large |
| Embedding Dimension | 768 |
| Training Datasets | ANLI, MNLI, SNLI |
| Paper | Machine-Assisted Script Curation |
What is sbert-roberta-large-anli-mnli-snli?
This is a sophisticated sentence transformer model developed by USC-ISI that converts sentences and paragraphs into 768-dimensional dense vector representations. Built on RoBERTa-large architecture, it's specifically trained on three major natural language inference datasets: ANLI, MNLI, and SNLI, making it particularly effective for semantic similarity tasks and text matching applications.
Implementation Details
The model was trained using careful parameter optimization, including a learning rate of 2e-5 and a batch size of 8. It employs mean pooling for sentence embeddings and was trained for approximately 20 hours on an NVIDIA GeForce RTX 2080 Ti. The implementation supports both sentence-transformers and Hugging Face Transformers frameworks, offering flexibility in deployment.
- Utilizes RoBERTa-large as the base architecture
- Implements mean pooling strategy for embedding generation
- Maximum sequence length of 128 tokens
- Supports both simple and advanced implementation approaches
Core Capabilities
- Sentence and paragraph embedding generation
- Semantic similarity computation
- Text clustering support
- Natural language inference tasks
- Cross-sentence relationship understanding
Frequently Asked Questions
Q: What makes this model unique?
This model stands out due to its comprehensive training on three major NLI datasets and its use of the robust RoBERTa-large architecture. The combination provides superior semantic understanding capabilities while maintaining practical usability through the sentence-transformers interface.
Q: What are the recommended use cases?
The model excels in applications requiring semantic similarity matching, such as document clustering, semantic search, information retrieval, and text classification. It's particularly well-suited for tasks where understanding the relationship between text passages is crucial.
