StarCoder2-15B-Instruct-GPTQ
| Property | Value |
|---|---|
| Parameter Count | 2.66B (Quantized) |
| License | BigCode OpenRAIL-M |
| Model Type | Code Generation / Instruction Following |
| Precision | 4-bit GPTQ Quantization |
What is StarCoder2-15B-Instruct-GPTQ?
StarCoder2-15B-Instruct-GPTQ is a quantized version of the StarCoder2 model specifically fine-tuned for code generation tasks. The model has been trained on an additional 0.7 billion high-quality code-related tokens for 3 epochs, utilizing DeepSpeed ZeRO 3 and Flash Attention 2 for optimized training performance.
Implementation Details
The model implements the Alpaca instruction format (without system prompts) and has been quantized using GPTQ to reduce its size while maintaining performance. It achieves an impressive 77.4% pass@1 rate on HumanEval-Python benchmarks.
- 4-bit precision quantization for efficient deployment
- Supports both direct model usage and text-generation pipeline implementation
- Optimized for code-related tasks and instruction following
- Compatible with Hugging Face Transformers library
Core Capabilities
- Code generation and completion
- Understanding and following coding instructions
- Memory-efficient deployment through quantization
- Support for long-context generation (up to 2048 tokens)
Frequently Asked Questions
Q: What makes this model unique?
This model combines the power of StarCoder2 with efficient quantization, making it more accessible for deployment while maintaining high performance on code-related tasks. Its instruction-following capabilities and specific fine-tuning for code make it particularly suitable for programming applications.
Q: What are the recommended use cases?
The model is best suited for code generation tasks, understanding programming instructions, and generating code-related content. It's particularly effective when used within its intended domain of software development and coding assistance.
