Mistral-Large-Instruct-2411-exl2
| Property | Value |
|---|---|
| License | Mistral AI Research License (MRL) |
| Supported Languages | 10 (English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, Korean) |
| Base Model | mistralai/Mistral-Large-Instruct-2411 |
| Quantization Framework | ExLlamaV2 v0.2.4 |
What is Mistral-Large-Instruct-2411-exl2?
This is a specialized quantized version of the Mistral-Large-Instruct model, optimized using turboderp's ExLlamaV2 framework. The model offers multiple quantization options ranging from 2.2 to 6.5 bits per weight, allowing users to balance between model size and performance based on their specific needs.
Implementation Details
The model implements advanced quantization techniques with varying bits-per-weight configurations. For quantization levels above 6.0, the lm_head layer is specifically quantized at 8 bits per weight, while lower quantization levels use the default 6 bits for this layer. The quantization process utilized the default calibration dataset to ensure optimal performance.
- Multiple quantization options: 6.5, 5.0, 4.25, 3.75, 3.5, 3.0, and 2.2 bits per weight
- Optimized using ExLlamaV2 v0.2.4 framework
- Customized lm_head layer quantization for higher bit versions
Core Capabilities
- Multilingual support across 10 major languages
- Flexible deployment options with different quantization levels
- Maintained model quality through careful calibration
- Compatible with vllm library for inference
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its variety of quantization options and multilingual capabilities while maintaining the core strengths of the original Mistral-Large-Instruct model. The different quantization levels allow users to choose the optimal trade-off between model size and performance for their specific use case.
Q: What are the recommended use cases?
According to the license, this model is strictly for research purposes only. It can be used for personal, scientific, or academic research, but cannot be used for commercial purposes or in business operations without a commercial license from Mistral AI.
