| Property | Value |
|---|---|
| Model Size | 149M parameters |
| Output Dimension | 768 |
| Max Sequence Length | 8192 tokens |
| MTEB Score | 64.38 |
| Developer | Alibaba-NLP (Tongyi Lab) |
| Primary Language | English |
What is gte-modernbert-base?
gte-modernbert-base is an advanced text embedding model developed by Alibaba's Tongyi Lab, built upon the modernBERT pre-trained encoder-only foundation models. It represents a significant advancement in text embedding technology, offering competitive performance across various evaluation benchmarks while maintaining a relatively compact model size of 149M parameters.
Implementation Details
The model leverages modern architecture optimizations and can be easily integrated using popular frameworks like transformers and sentence-transformers. It supports Flash Attention 2 for improved efficiency and can process sequences up to 8192 tokens in length, outputting 768-dimensional embeddings.
- Achieves 64.38 on MTEB benchmark evaluation
- Demonstrates strong performance in BEIR (55.33) and LoCo (87.57) evaluations
- Supports both text embedding and reranking tasks
- Compatible with transformers.js for browser-based applications
Core Capabilities
- Long-context understanding with 8192 token support
- High-quality text embeddings for semantic search
- Effective document retrieval and comparison
- Strong performance in code retrieval tasks (79.31 COIR score)
Frequently Asked Questions
Q: What makes this model unique?
The model combines the benefits of modernBERT architecture with optimized training for text embeddings, offering an excellent balance between model size and performance. Its support for long sequences (8192 tokens) and competitive benchmark scores make it particularly valuable for practical applications.
Q: What are the recommended use cases?
The model excels in semantic search, document retrieval, text similarity analysis, and code search applications. It's particularly well-suited for applications requiring long-context understanding and efficient text representation.