Dolphin-2.6-Phi-2
| Property | Value |
|---|---|
| Parameter Count | 2.78B |
| License | MIT |
| Tensor Type | FP16 |
| Training Time | 2 days on 4x A100s |
| Training Method | qLoRA and Axolotl |
What is dolphin-2_6-phi-2?
Dolphin-2.6-Phi-2 is an advanced language model based on Microsoft's Phi-2 architecture, specifically designed to be uncensored and highly compliant while maintaining strong performance across various tasks. This iteration includes significant improvements in training configuration and carefully curated datasets for enhanced quality and capabilities.
Implementation Details
The model utilizes the ChatML prompt format and was trained for 3 epochs using qLoRA and Axolotl framework. It incorporates seven diverse datasets including Dolphin, Airoboros, Magicoder, and Capybara, creating a well-rounded knowledge base for various applications.
- Benchmark Performance: 61.7% average across major metrics
- Enhanced training configuration for improved quality
- Integrated empathy data from Samantha-based datasets
- Replaced certain datasets with Capybara for better performance
Core Capabilities
- Strong performance in reasoning tasks (ARC: 59.81%)
- Excellent common sense understanding (HellaSwag: 74.65%)
- Robust mathematical reasoning (GSM8K: 58.07%)
- Enhanced empathy and conversational abilities
- Compliance with complex instructions
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its uncensored nature combined with high compliance, making it suitable for research and controlled environments. It's specifically designed to be adaptable to custom alignment layers while maintaining strong performance across various benchmarks.
Q: What are the recommended use cases?
The model excels in conversational AI, code generation, and general instruction-following tasks. However, due to its uncensored nature, it requires implementation of appropriate alignment layers before deployment in production environments.
