Mistral-Large-Instruct-2411-exl2
| Property | Value |
|---|---|
| Base Model | mistralai/Mistral-Large-Instruct-2411 |
| License | Mistral Research License (MRL) |
| Supported Languages | 10 (English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, Korean) |
| Quantization Framework | ExLlama v2 |
What is Mistral-Large-Instruct-2411-exl2?
This is a quantized version of the Mistral-Large-Instruct model, optimized using ExLlama v2 framework. The model offers multiple quantization options ranging from 2.2 to 6.5 bits per weight, allowing users to balance between model size and performance based on their specific needs. The quantization was performed using turboderp's ExLlamaV2 v0.2.4 with default calibration datasets.
Implementation Details
The model implements different quantization levels through separate branches, each offering a different compression ratio:
- 6.5 bits per weight (with 8-bit lm_head layer)
- 5.0 bits per weight
- 4.25 bits per weight
- 3.75 bits per weight
- 3.5 bits per weight
- 3.0 bits per weight
- 2.2 bits per weight
Core Capabilities
- Multi-language support across 10 major languages
- Flexible quantization options for different deployment scenarios
- Maintained text generation quality with reduced model size
- Optimized for research purposes
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its variety of quantization options while maintaining the core capabilities of the original Mistral-Large-Instruct model. It's specifically optimized for research purposes and offers a range of compression levels to suit different hardware constraints.
Q: What are the recommended use cases?
The model is strictly limited to non-commercial research purposes as per the Mistral Research License. It's ideal for academic research, personal scientific exploration, and non-profit applications. Commercial use requires explicit permission from Mistral AI.
