| Property | Value |
|---|---|
| Base Model | google/gemma-3-12b-it |
| Training Framework | TRL 0.17.0 |
| Dataset | open-r1/codeforces-cots |
| Model Hub | HuggingFace |
What is GemmaCoder3-12B?
GemmaCoder3-12B is a specialized code generation model fine-tuned from Google's Gemma 12B architecture. Created by burtenshaw, this model has been specifically optimized for coding tasks using the codeforces-cots dataset and trained through Supervised Fine-Tuning (SFT).
Implementation Details
The model leverages the Transformer Reinforcement Learning (TRL) framework and demonstrates significant improvements in code generation capabilities. It runs on PyTorch 2.6.0 and uses the Transformers 4.51.0 library for implementation.
- Improved LiveCodeBench performance from 21.9% to 32.9%
- Maintains comparable performance on general tasks like Winogrande (63.9%)
- Built with modern ML frameworks including Datasets 3.4.1 and Tokenizers 0.21.1
Core Capabilities
- Specialized code generation and completion
- Enhanced performance on practical coding tasks
- Balanced performance across multiple benchmarks
- Easy integration with HuggingFace's pipeline API
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its significant improvement in code generation capabilities, showing a 50% increase in LiveCodeBench performance while maintaining reasonable performance on general language tasks.
Q: What are the recommended use cases?
GemmaCoder3-12B is best suited for code generation tasks, particularly in competitive programming scenarios. It can be effectively used for code completion, generation, and assistance in software development workflows.