NV-Embed-v1
| Property | Value |
|---|---|
| Base Model | Mistral-7B-v0.1 |
| Embedding Dimension | 4096 |
| License | Non-commercial use only |
| Paper | arXiv:2405.17428 |
What is NV-Embed-v1?
NV-Embed-v1 is NVIDIA's cutting-edge embedding model that currently holds the #1 position on the Massive Text Embedding Benchmark (MTEB). Built on the Mistral-7B foundation, it introduces innovative techniques for generating high-quality text embeddings, particularly excelling in retrieval tasks with a score of 59.36 across 15 different retrieval scenarios.
Implementation Details
The model implements several groundbreaking technical features, including a novel latent-attention pooling mechanism and a two-stage instruction tuning method. It generates 4096-dimensional embeddings and is specifically designed to handle both retrieval and non-retrieval tasks with superior accuracy.
- Latent-Attention Pooling Architecture
- Two-stage instruction tuning pipeline
- 4096-dimensional embedding output
- Built on Mistral-7B-v0.1 decoder-only architecture
Core Capabilities
- State-of-the-art performance on MTEB benchmark
- Superior retrieval capabilities across diverse tasks
- Excellent performance in reranking, classification, and clustering
- Support for both query and passage encoding
- Flexible instruction-based task adaptation
Frequently Asked Questions
Q: What makes this model unique?
NV-Embed-v1 stands out through its innovative latent-attention mechanism and two-stage instruction tuning approach, enabling superior performance across various embedding tasks. It's particularly notable for achieving the highest scores on the MTEB benchmark, demonstrating its versatility and effectiveness.
Q: What are the recommended use cases?
The model excels in text retrieval, semantic similarity matching, document classification, and clustering tasks. It's particularly well-suited for applications requiring high-quality text embeddings, though it's important to note that it's licensed for non-commercial use only.
