Neural Chat 7B v3.1
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Base Model | Mistral-7B-v0.1 |
| License | Apache 2.0 |
| Context Length | 8192 tokens |
| Training Dataset | Open-Orca/SlimOrca |
What is neural-chat-7b-v3-1?
Neural Chat 7B v3.1 is an advanced language model developed by Intel, built upon the Mistral-7B architecture and fine-tuned using the Open-Orca/SlimOrca dataset. This model represents a significant improvement over its predecessors, featuring enhanced performance across multiple benchmarks and optimized for deployment on Intel Gaudi 2 processors.
Implementation Details
The model employs Direct Preference Optimization (DPO) for alignment and supports multiple inference modes including FP32, BF16, and INT4 quantization. It was trained using 8 Gaudi2 cards with DeepSpeed Zero2 optimization, achieving impressive benchmark scores across various tasks.
- Supports flexible deployment options with both Transformers and Intel Extensions
- Features comprehensive quantization capabilities for efficient inference
- Implements advanced training techniques including DPO alignment
Core Capabilities
- Strong performance on ARC (66.21%) and HellaSwag (83.64%) benchmarks
- Enhanced truthfulness with TruthfulQA score of 59.65%
- Improved mathematical reasoning with GSM8K score of 19.56%
- Versatile text generation and conversation capabilities
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its balanced performance across multiple benchmarks and its optimization for Intel hardware. It shows significant improvements over the base Mistral-7B model while maintaining efficient deployment options through various quantization schemes.
Q: What are the recommended use cases?
The model is well-suited for general language tasks, conversational AI, and educational applications. It performs particularly well in scenarios requiring truthful responses and can handle both analytical and creative tasks effectively.
