nv-embed-v2-ontada-twab-peft
| Property | Value |
|---|---|
| Parameter Count | 7.85B |
| Output Dimensions | 4096 |
| Base Model | nvidia/NV-Embed-v2 |
| Maximum Sequence Length | 1024 tokens |
| Training Dataset Size | 16,186 samples |
| Paper | SBERT Paper |
What is nv-embed-v2-ontada-twab-peft?
This is a specialized sentence transformer model fine-tuned from NVIDIA's NV-Embed-v2 base model, designed specifically for medical text similarity tasks. It transforms medical text into high-dimensional vector representations, enabling sophisticated semantic search and similarity comparisons in healthcare contexts.
Implementation Details
The model implements a sophisticated architecture combining a transformer encoder with special pooling and normalization layers. It was trained using MultipleNegativesRankingLoss with a batch size of 4 and learning rate of 2e-05, achieving impressive evaluation metrics including a cosine NDCG@10 score of 0.8649.
- BF16 mixed precision training for improved efficiency
- Custom prompt-based input formatting
- Specialized pooling mechanism for optimal sentence embeddings
- Linear learning rate scheduler with 10% warmup
Core Capabilities
- Semantic textual similarity in medical contexts
- Patient question-answering support
- Medical document retrieval and comparison
- Support for sequences up to 1024 tokens
- High-dimensional (4096) semantic embeddings
Frequently Asked Questions
Q: What makes this model unique?
This model combines the power of NVIDIA's NV-Embed-v2 with specialized medical domain fine-tuning, offering state-of-the-art performance for medical text similarity tasks with impressive accuracy metrics, particularly its 0.8649 NDCG@10 score.
Q: What are the recommended use cases?
The model excels at medical document retrieval, patient question answering, and semantic similarity tasks in healthcare contexts. It's particularly well-suited for applications requiring precise understanding of medical terminology and context.
