Luminis-phi-4
| Property | Value |
|---|---|
| Author | suayptalha |
| Model Type | Merged Language Model |
| Base Models | Phi-4-RR-Shoup, Phi-4-Model-Stock-v4 |
| Merge Method | SLERP |
| Model URL | Hugging Face |
What is Luminis-phi-4?
Luminis-phi-4 is a sophisticated merged language model created using mergekit, combining two powerful Phi-4 variants. Currently ranked #3 on the Open LLM Leaderboard among models up to 15B parameters and #4 among models up to 32B parameters, it demonstrates exceptional performance across various benchmarks.
Implementation Details
The model employs the SLERP (Spherical Linear Interpolation) merge method with carefully tuned parameters for different model components. The merge configuration uses varying interpolation values for self-attention layers and MLP components, optimizing the blend between the base models.
- Custom interpolation values for self-attention (ranging from 0.0 to 1.0)
- Inverse interpolation for MLP layers
- Layer-specific merging across 40 layers
- Implementation in bfloat16 format for efficiency
Core Capabilities
- Strong performance on IFEval (69.00% accuracy on 0-shot tasks)
- Impressive BBH results (55.80% on 3-shot tasks)
- Solid MATH Level 5 performance (43.66% on 4-shot problems)
- Competitive MMLLU-PRO scores (49.15% on 5-shot tasks)
- Overall average benchmark score of 41.30
Frequently Asked Questions
Q: What makes this model unique?
The model's uniqueness lies in its sophisticated merge strategy, combining the strengths of two Phi-4 variants using SLERP with carefully calibrated interpolation values for different neural network components. This results in superior performance across various benchmarks while maintaining efficiency.
Q: What are the recommended use cases?
Given its strong performance across multiple benchmarks, Luminis-phi-4 is particularly well-suited for tasks requiring reasoning (BBH), mathematical problem-solving (MATH Level 5), and general knowledge applications. It shows exceptional capability in few-shot learning scenarios.
