Granite-Embedding-125m-English
| Property | Value |
|---|---|
| Developer | IBM Granite Embedding Team |
| Parameter Count | 125M |
| Embedding Dimension | 768 |
| License | Apache 2.0 |
| Release Date | December 18th, 2024 |
| Max Sequence Length | 512 tokens |
What is granite-embedding-125m-english?
Granite-Embedding-125m-English is a sophisticated embedding model developed by IBM that generates high-quality text embeddings for English language content. Unlike many other open-source models, it's uniquely positioned for enterprise use, having been trained exclusively on datasets with permissive licensing. The model achieves competitive scores on academic benchmarks like BEIR while maintaining strong performance in enterprise applications.
Implementation Details
The model is built on a RoBERTa-like transformer architecture with 12 layers, 12 attention heads, and an intermediate size of 3072. It processes text sequences up to 512 tokens and outputs 768-dimensional embedding vectors. The model can be easily implemented using either the SentenceTransformers library or Hugging Face Transformers.
- Trained on diverse datasets including academic citations, Stack Exchange content, and Wikipedia
- Implements CLS pooling for embedding generation
- Utilizes GeLU activation functions
- Vocabulary size of 50,265 tokens
Core Capabilities
- Text similarity computation
- Information retrieval applications
- Semantic search implementation
- Document comparison and matching
- Query-passage relevance scoring
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its exclusive training on enterprise-friendly licensed datasets, deliberately avoiding restrictive licenses like MS-MARCO. This makes it particularly suitable for commercial applications while maintaining competitive performance.
Q: What are the recommended use cases?
The model excels in text similarity tasks, retrieval applications, and search functionalities. It's particularly well-suited for enterprise environments requiring reliable text embeddings with clear licensing terms.
