li-14b-v0.4
| Property | Value |
|---|---|
| Model Size | 14B parameters |
| Base Model | Qwen/Qwen2.5-14B-Instruct |
| Hugging Face | Link |
| Average Score | 43.66 on Open LLM Leaderboard |
What is li-14b-v0.4?
li-14b-v0.4 is a sophisticated merged language model that combines seven different Qwen-based models using the Model Stock merge method. Currently ranked #1 among models up to 15B parameters on the Open LLM Leaderboard, it represents a significant achievement in model optimization through careful combination of specialized capabilities.
Implementation Details
The model uses mergekit to combine multiple specialized models including Qwen2.5-Coder-14B, DeepSeek-R1-Distill-Qwen-14B, and others. It employs bfloat16 dtype and includes features like int8_mask and normalization for optimal performance.
- Utilizes Model Stock merge method with Qwen2.5-14B-Instruct as base
- Implements normalization and int8 masking for efficiency
- Combines seven specialized models for comprehensive capabilities
Core Capabilities
- Strong performance in coding tasks (via Qwen2.5-Coder-14B)
- Enhanced logical reasoning (via DeepSeek-R1-Distill)
- Improved mathematics handling (via Impish_QWEN_14B-1M)
- Advanced text generation capabilities
- Impressive benchmark scores: 81.33 on IFEval, 55.74 on MATH Lvl 5
Frequently Asked Questions
Q: What makes this model unique?
This model's uniqueness lies in its careful merger of seven specialized models, each bringing specific strengths in areas like coding, logic, and mathematics. The Model Stock merge method ensures optimal integration of these capabilities while maintaining performance.
Q: What are the recommended use cases?
The model excels in diverse applications including coding tasks, mathematical problem-solving, logical reasoning, and general text generation. It's particularly well-suited for applications requiring a balance of technical and creative capabilities.
