Granite-Embedding-107m-Multilingual
| Property | Value |
|---|---|
| Parameters | 107M |
| Embedding Dimension | 384 |
| License | Apache 2.0 |
| Release Date | December 18th, 2024 |
| Developer | IBM Granite Team |
What is granite-embedding-107m-multilingual?
Granite-Embedding-107m-multilingual is a sophisticated dense biencoder embedding model designed for generating high-quality text embeddings across multiple languages. This model is part of IBM's Granite Embeddings suite and produces 384-dimensional embedding vectors, making it particularly efficient for text similarity and retrieval applications. The model supports 12 languages including English, German, Spanish, French, Japanese, and others, with the flexibility for users to fine-tune it for additional languages.
Implementation Details
The model is built on an encoder-only XLM-RoBERTa-like transformer architecture, featuring 6 layers, 12 attention heads, and an intermediate size of 1536. It employs GeLU activation functions and can handle sequences up to 512 tokens. The model's training incorporates contrastive fine-tuning, knowledge distillation, and model merging techniques for enhanced performance.
- Trained on diverse datasets including MC4, Webhose, Wikipedia, and Stack Exchange
- Supports integration with both SentenceTransformers and Hugging Face Transformers
- Optimized for performance with twice the speed of comparable models
- Uses CLS pooling for embedding generation
Core Capabilities
- Multilingual text embedding generation
- Text similarity computation
- Information retrieval across languages
- Search applications
- Cross-lingual document matching
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient architecture that delivers strong performance across multiple languages while maintaining a relatively small parameter count. It's trained without using MS-MARCO dataset, making it suitable for commercial applications, and achieves impressive metrics across various benchmarks like MTEB and Miracl.
Q: What are the recommended use cases?
The model excels in text similarity tasks, document retrieval, semantic search, and cross-lingual information retrieval. It's particularly valuable for enterprise applications requiring multilingual capabilities and fast processing times.
