Lamarckvergence-14B
| Property | Value |
|---|---|
| Model Size | 14B parameters |
| Author | suayptalha |
| Merge Method | SLERP |
| Base Models | Lamarck-14B-v0.7, Qwenvergence-14B-v12-Prose-DS |
| Model Hub | Hugging Face |
What is Lamarckvergence-14B?
Lamarckvergence-14B is a cutting-edge merged language model that combines the strengths of Lamarck-14B-v0.7 and Qwenvergence-14B-v12-Prose-DS using the SLERP (Spherical Linear Interpolation) merge method. As of February 2025, it holds the prestigious position of #1 among models up to 15B parameters and ranks #56 overall on the Open LLM Leaderboard.
Implementation Details
The model utilizes a sophisticated merging configuration with varying interpolation weights across different components. The self-attention and MLP layers are specifically tuned with different mixing ratios to optimize performance. The model operates in bfloat16 precision and spans 48 layers from each parent model.
- SLERP merge with customized interpolation weights
- Specialized attention and MLP layer mixing
- bfloat16 precision for efficient computation
- 48-layer architecture from both parent models
Core Capabilities
- IFEval (0-Shot): 76.56% accuracy
- BBH (3-Shot): 50.33% performance
- MATH Level 5 (4-Shot): 54.00% success rate
- GPQA (0-shot): 15.10% accuracy
- MuSR (0-shot): 16.34% performance
- MMLU-PRO (5-shot): 47.59% accuracy
- Overall average: 43.32%
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its carefully calibrated merge of two powerful base models using SLERP, with specialized attention and MLP layer mixing ratios. This results in state-of-the-art performance among sub-15B parameter models.
Q: What are the recommended use cases?
Based on its evaluation metrics, the model excels in tasks requiring inference and mathematical reasoning, making it particularly suitable for academic and analytical applications. It shows strong performance in zero-shot and few-shot learning scenarios.
