Nxcode-CQ-7B-orpo
| Property | Value |
|---|---|
| Parameter Count | 7.25B |
| Model Type | Code Generation |
| Architecture | Qwen2-based Transformer |
| License | Tongyi Qianwen Research |
| Paper | ORPO Paper |
What is Nxcode-CQ-7B-orpo?
Nxcode-CQ-7B-orpo is a specialized code generation model developed by NTQAI, built upon the CodeQwen1.5-7B foundation and fine-tuned using the Monolithic Preference Optimization without Reference Model (ORPO) technique. The model was trained on 100,000 samples of high-quality ranking data, achieving remarkable performance on code generation benchmarks.
Implementation Details
The model utilizes BF16 tensor type for efficient computation and leverages the Qwen2 architecture. It implements state-of-the-art preference optimization techniques without requiring a reference model, making it more efficient in training while maintaining high performance.
- Achieves 86.6% pass@1 on HumanEval benchmark
- 83.5% pass@1 on HumanEval+ benchmark
- 82.3% pass@1 on MBPP benchmark
- Compatible with text-generation-inference endpoints
Core Capabilities
- Advanced code completion and generation
- High performance on complex programming tasks
- Efficient processing with BF16 precision
- Competitive performance against GPT-4 and other leading models
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its impressive performance metrics, achieving scores that compete with much larger models like GPT-4, while maintaining a relatively compact 7B parameter size. Its ORPO fine-tuning approach enables high-quality code generation without the need for a reference model.
Q: What are the recommended use cases?
The model excels in code generation tasks, particularly in Python programming. It's ideal for automated code completion, function implementation, and solving programming challenges. The high performance on HumanEval and MBPP benchmarks makes it suitable for professional development environments.
