| Property | Value |
|---|---|
| Base Architecture | Mistral 24B |
| Parameter Count | 24B |
| Model Type | Language Model (Merged) |
| Source Repository | HuggingFace |
What is MS3-RP-Broth-24B?
MS3-RP-Broth-24B is a sophisticated merged language model created as part of the Tantum project. It represents a complex fusion of multiple Mistral-based models using advanced merging techniques including SCE (Sparse Crossed Entropy) and della_linear methods. The model is specifically optimized for roleplaying applications with custom sampling presets.
Implementation Details
The model implements a multi-stage merging process utilizing various base models including unsloth/Mistral-Small-24B variants, ArliAI/Mistral-Small-24B-ArliAI-RPMax, and others. The merging process employs sophisticated techniques with carefully tuned parameters for density, epsilon, and lambda values.
- Uses bfloat16 dtype for efficient computing
- Implements specialized weight distributions across different projection layers
- Employs both static and Imatrix quantization approaches
- Features custom merging ratios and density parameters
Core Capabilities
- Optimized for roleplaying scenarios
- Supports Weird preset and Forgotten-Safeword preset samplers
- Compatible with Mistral-V7-Tekken prompt format
- Uses specialized lorebook instead of system prompts
Frequently Asked Questions
Q: What makes this model unique?
The model's uniqueness lies in its complex merging architecture that combines multiple specialized models with carefully calibrated weights and parameters, specifically optimized for roleplaying applications.
Q: What are the recommended use cases?
This model is primarily designed for roleplaying scenarios and works best with specific sampling presets (Weird preset and Forgotten-Safeword preset) and the Mistral-V7-Tekken prompt format.