Qwen2.5-Coder-0.5B-Instruct-bnb-4bit
| Property | Value |
|---|---|
| Parameter Count | 0.49B (0.36B non-embedding) |
| Context Length | 32,768 tokens |
| Architecture | Transformers with RoPE, SwiGLU, RMSNorm |
| Model Type | Causal Language Model |
| Authors | Unsloth Team |
| Paper | Qwen2.5-Coder Technical Report |
What is Qwen2.5-Coder-0.5B-Instruct-bnb-4bit?
This is a lightweight 4-bit quantized version of the Qwen2.5-Coder model, specifically designed for code generation and understanding tasks. It represents the smallest variant in the Qwen2.5-Coder series, which spans from 0.5B to 32B parameters. The model has been optimized using binary neural network (BNB) 4-bit quantization to reduce memory usage while maintaining performance.
Implementation Details
The model features a sophisticated architecture with 24 layers and uses grouped-query attention (GQA) with 14 heads for queries and 2 for key/values. It implements modern transformer enhancements including RoPE for positional encoding, SwiGLU activation, and RMSNorm for normalization.
- Full 32k context window support
- Efficient 4-bit quantization
- 24 transformer layers
- GQA attention mechanism
- Tied word embeddings for efficiency
Core Capabilities
- Code generation and completion
- Code reasoning and analysis
- Bug fixing and code optimization
- Mathematical problem-solving
- General programming tasks
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient 4-bit quantization while maintaining the core capabilities of the Qwen2.5-Coder architecture. It offers an excellent balance between performance and resource usage, making it suitable for deployment in environments with limited computational resources.
Q: What are the recommended use cases?
The model is primarily designed for code-related tasks but requires post-training (like SFT or RLHF) for conversational use. It's particularly well-suited for code generation, analysis, and fixing in resource-constrained environments. However, it's recommended to apply additional training for specific use cases or conversation-based applications.
