jina-embeddings-v2-small-en
| Property | Value |
|---|---|
| Parameter Count | 32.7M parameters |
| License | Apache 2.0 |
| Paper | Technical Report |
| Architecture | BERT with ALiBi |
What is jina-embeddings-v2-small-en?
jina-embeddings-v2-small-en is a compact yet powerful English embedding model developed by Jina AI. It's built on a modified BERT architecture incorporating symmetric bidirectional ALiBi, enabling support for sequences up to 8,192 tokens while maintaining just 33M parameters. The model was pretrained on C4 dataset and fine-tuned on over 400 million carefully curated sentence pairs.
Implementation Details
The model utilizes a unique architecture combining BERT with ALiBi (Attention with Linear Biases) to handle long sequences effectively. While trained on 512-length sequences, it can extrapolate to 8K tokens thanks to ALiBi's properties. The model requires mean pooling for optimal performance and supports both transformers and sentence-transformers implementations.
- Supports sequence lengths up to 8,192 tokens
- Optimized for English language understanding
- Trained on 400M+ high-quality sentence pairs
- Efficient 33M parameter footprint
Core Capabilities
- Long document retrieval
- Semantic textual similarity
- Text reranking
- RAG applications
- Recommendation systems
Frequently Asked Questions
Q: What makes this model unique?
This model combines compact size with extensive sequence length support, making it ideal for production deployments where both efficiency and long-text processing are required. The ALiBi attention mechanism allows it to handle sequences up to 8K tokens despite being trained on shorter sequences.
Q: What are the recommended use cases?
The model excels in applications requiring semantic search, document retrieval, and RAG implementations. It's particularly effective for scenarios involving long documents while maintaining efficient resource usage due to its small parameter count.
