DeepSeek-R1-Distill-Qwen-32B-AWQ
| Property | Value |
|---|---|
| Base Model | Qwen2.5-32B |
| Quantization | 4-bit AWQ |
| License | MIT License |
| Context Length | 32,768 tokens |
What is DeepSeek-R1-Distill-Qwen-32B-AWQ?
DeepSeek-R1-Distill-Qwen-32B-AWQ is a 4-bit quantized version of the powerful DeepSeek-R1-Distill-Qwen-32B model, which was distilled from the larger DeepSeek-R1 model. This model represents a significant achievement in AI compression, maintaining exceptional performance while reducing the computational requirements through quantization.
Implementation Details
The model utilizes AutoAWQ version 3.2.7.post3 for quantization, with specific configurations including zero_point enabled, q_group_size of 128, and 4-bit weight quantization. The quantization process preserves the model's capabilities while making it more efficient for deployment.
- Outperforms OpenAI-o1-mini across various benchmarks
- Achieves impressive scores on AIME 2024 (72.6% pass@1) and MATH-500 (94.3% pass@1)
- Demonstrates strong performance in code tasks with a Codeforces rating of 1691
Core Capabilities
- Advanced mathematical reasoning and problem-solving
- Strong coding and software engineering capabilities
- Robust performance on general knowledge and reasoning tasks
- Efficient processing with 4-bit quantization
Frequently Asked Questions
Q: What makes this model unique?
This model combines the powerful reasoning capabilities of DeepSeek-R1 with efficient 4-bit quantization, making it both powerful and practical for deployment. It particularly excels in mathematical reasoning and coding tasks, outperforming many larger models.
Q: What are the recommended use cases?
The model is particularly well-suited for mathematical problem-solving, coding tasks, and general reasoning applications. It's recommended to use a temperature setting between 0.5 and 0.7 to avoid issues with repetition or incoherent output.
