OpenThinker-32B
| Property | Value |
|---|---|
| Base Model | Qwen2.5-32B-Instruct |
| Training Dataset | OpenThoughts-114k |
| License | Apache 2.0 |
| Training Infrastructure | AWS SageMaker (8xH100 P5 nodes) |
What is OpenThinker-32B?
OpenThinker-32B is an advanced language model that represents a significant achievement in open-source AI development. It's a fine-tuned version of Qwen2.5-32B-Instruct, trained on the OpenThoughts-114k dataset, which was created through distillation of DeepSeek-R1. The model demonstrates exceptional performance across various mathematical and reasoning benchmarks, achieving notably high scores in MATH500 (90.6%) and LCBv2 (68.9%).
Implementation Details
The model was trained for 3 epochs with a 16k context length using LlamaFactory. The training process utilized AWS SageMaker with 8xH100 P5 nodes, taking approximately 90 hours on 4 nodes. The implementation employs key hyperparameters including a learning rate of 1e-05, AdamW optimizer, and cosine learning rate scheduling with 0.1 warmup ratio.
- Fully open-source architecture with publicly available weights, datasets, and code
- Trained on OpenThoughts-114k dataset using advanced distillation techniques
- Implements a 16k context length for comprehensive text processing
- Utilizes state-of-the-art training infrastructure and optimization techniques
Core Capabilities
- Strong performance in mathematical reasoning (MATH500: 90.6%)
- Advanced problem-solving abilities (AIME24 I/II: 66.0%)
- Robust logical reasoning (LCBv2: 68.9%)
- High accuracy in complex question answering (GPQA Diamond: 61.6%)
Frequently Asked Questions
Q: What makes this model unique?
OpenThinker-32B stands out for its fully open-source nature, combining high performance with complete transparency. Unlike many competitors, it provides open access to its weights, training data, and implementation code, making it valuable for both research and practical applications.
Q: What are the recommended use cases?
The model excels in mathematical reasoning, problem-solving, and complex question answering tasks. It's particularly well-suited for educational applications, scientific computing, and scenarios requiring advanced logical reasoning capabilities.
