OlympicCoder-32B
| Property | Value |
|---|---|
| Parameters | 32 Billion |
| Base Model | Qwen2.5-Coder-32B-Instruct |
| License | Apache-2.0 |
| Primary Language | English |
| Repository | GitHub |
What is OlympicCoder-32B?
OlympicCoder-32B is a specialized large language model designed for competitive programming tasks, particularly excelling in environments like the International Olympiad in Informatics (IOI) and LiveCodeBench. The model represents a significant advancement in AI-powered coding assistance, being fine-tuned on a carefully decontaminated version of the Codeforces dataset.
Implementation Details
The model was trained using 16 H100 nodes with specific hyperparameters including a learning rate of 4.0e-5, FSDP distributed training, and a cosine learning rate scheduler with warmup. Training was conducted over 10 epochs with careful attention to batch size and optimization parameters.
- Utilizes FSDP (Fully Sharded Data Parallel) training across 128 devices
- Implements Adam optimizer with specialized beta parameters
- Features a custom chat template with think token for enhanced reasoning
- Supports both C++ and Python code generation
Core Capabilities
- Exceptional performance on IOI'2024 benchmark problems
- Strong results on LiveCodeBench's Python programming challenges
- Advanced chain-of-thought reasoning through specialized prompting
- Efficient handling of competitive programming tasks
Frequently Asked Questions
Q: What makes this model unique?
OlympicCoder-32B stands out for its specialized training on competitive programming tasks and its integration of a think token mechanism for enhanced reasoning. The model was specifically post-trained on C++ solutions from DeepSeek-R1, making it particularly strong in competitive programming scenarios.
Q: What are the recommended use cases?
The model is best suited for competitive programming tasks, algorithmic problem-solving, and technical coding challenges. It excels in both C++ and Python programming, though its primary strength lies in C++ solutions due to its training focus.
