NV-Embed-v2
| Property | Value |
|---|---|
| Base Model | Mistral-7B-v0.1 |
| Parameter Count | 7.85B |
| Embedding Dimension | 4096 |
| License | CC-BY-NC-4.0 (Non-commercial use only) |
| Paper | NV-Embed Paper |
What is NV-Embed-v2?
NV-Embed-v2 is a cutting-edge embedding model that currently ranks #1 on the Massive Text Embedding Benchmark (MTEB) with an impressive score of 72.31 across 56 different tasks. Built on the Mistral-7B architecture, it introduces innovative approaches to text embedding, including latent-attention pooling and a novel two-staged instruction tuning method.
Implementation Details
The model implements several groundbreaking technical features: a latent-attention mechanism for improved pooled embedding output, an advanced two-staged instruction tuning process, and sophisticated hard-negative mining techniques that consider positive relevance scores for better false negative removal. It generates 4096-dimensional embeddings and is particularly strong in retrieval tasks, where it leads the MTEB sub-category with a score of 62.65.
- Latent-Attention Pooling Architecture
- Two-Stage Instruction Tuning
- Advanced Hard-Negative Mining
- 4096D Embedding Output
- Built on Mistral-7B Foundation
Core Capabilities
- State-of-the-art performance on MTEB benchmark
- Excellence in retrieval tasks (ideal for RAG applications)
- Support for long sequences (up to 32768 tokens)
- Flexible instruction-based querying
- Efficient batch processing capabilities
Frequently Asked Questions
Q: What makes this model unique?
NV-Embed-v2's unique strength lies in its innovative latent-attention mechanism and two-staged instruction tuning approach, resulting in state-of-the-art performance across diverse embedding tasks, particularly in retrieval applications.
Q: What are the recommended use cases?
The model excels in retrieval-augmented generation (RAG), semantic search, document similarity, and various text embedding tasks. However, due to its non-commercial license, it's primarily suitable for research and non-commercial applications.
