WinterEngine-24B-Instruct
| Property | Value |
|---|---|
| Base Model | mistralai/Mistral-Small-24B-Base-2501 |
| License | Apache-2.0 |
| Context Length | 32,768 tokens |
| Language | English |
What is WinterEngine-24B-Instruct?
WinterEngine-24B-Instruct is an advanced language model created through a sophisticated merge of PersonalityEngine and Redemption Wind models, built on the Mistral-24B architecture. It's specifically designed to excel in creative tasks, roleplay scenarios, and instruction following while maintaining an uncensored approach to content generation.
Implementation Details
The model employs a unique merging strategy using LazyMergekit, combining Dans-PersonalityEngine-V1.2.0-24b and Redemption_Wind_24B with carefully calibrated layer mixing. The implementation uses SLERP merging with sophisticated attention and MLP parameter tuning across 40 layers.
- Custom attention parameter distribution: [0, 0.5, 0.3, 0.7, 1]
- MLP layer optimization: [1, 0.5, 0.7, 0.3, 0]
- BFloat16 precision for optimal performance
- Extensive 32K context window
Core Capabilities
- Enhanced instruction following
- Advanced roleplay capabilities
- Creative content generation
- Balanced personality traits
- Uncensored response generation
Frequently Asked Questions
Q: What makes this model unique?
WinterEngine-24B-Instruct stands out through its balanced combination of instruction-following capabilities and creative freedom, achieved through strategic model merging of PersonalityEngine and Redemption Wind models.
Q: What are the recommended use cases?
The model excels in creative writing, roleplay scenarios, conversational tasks, and situations requiring both structured responses and creative freedom. The recommended temperature setting of 1.2 and min_p of 0.05 makes it particularly suitable for generating diverse and engaging content.
