YiXin-Distill-Qwen-72B
| Property | Value |
|---|---|
| Base Model | Qwen2.5-72B |
| Model Size | 72B parameters |
| Author | YiXin-AILab |
| Model Hub | Hugging Face |
What is YiXin-Distill-Qwen-72B?
YiXin-Distill-Qwen-72B is a high-performance language model specifically optimized for mathematical and general reasoning tasks. Derived from Qwen2.5-72B through advanced distillation techniques, it maintains strong computational efficiency while achieving state-of-the-art performance across various benchmarks. The model demonstrates significant improvements of 5-11 percentage points compared to similar distilled models.
Implementation Details
The model employs a progressive two-stage distillation approach with continuous refinement through intelligent data selection. The training process includes comprehensive quality control using DeepSeek-R1 as an LLM judge, ensuring optimal performance across varying complexity levels.
- Utilizes structured multi-stage data processing pipeline
- Implements rigorous quality control framework
- Features balanced representation across difficulty tiers
- Employs systematic mathematical content validation
Core Capabilities
- Advanced mathematical reasoning with 97% accuracy on MATH-500 benchmark
- Strong performance on GPQA-Diamond (69.2%)
- Impressive AIME results (76.7% on AIME-24, 73.3% on AIME-25)
- High MMLU-Pro score of 92.6%
- Balanced performance across both mathematical and general knowledge tasks
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its specialized optimization for mathematical reasoning while maintaining strong general capabilities. Its progressive distillation approach and comprehensive data validation process set it apart from other distilled models.
Q: What are the recommended use cases?
The model is particularly well-suited for applications requiring strong mathematical reasoning, educational tools, and general knowledge tasks. It excels in scenarios where both computational efficiency and high accuracy are crucial.
