Mistral-Large-Instruct-2411-AWQ
| Property | Value |
|---|---|
| Parameter Count | 17.1B (AWQ Quantized) |
| Model Type | Large Language Model (Instruct) |
| License | Mistral Research License (MRL) |
| Supported Languages | 10 languages including English, French, German, Spanish, etc. |
| Framework | vLLM |
What is Mistral-Large-Instruct-2411-AWQ?
Mistral-Large-Instruct-2411-AWQ is an AWQ-quantized version of the powerful Mistral-Large-Instruct-2411 model, designed to maintain high performance while reducing the model's memory footprint. This version preserves the advanced capabilities of the original 123B parameter model while making it more accessible for deployment.
Implementation Details
The model utilizes AWQ (Activation-aware Weight Quantization) technology to compress the original model while maintaining its performance. It's optimized for use with the vLLM framework and supports tensor parallelism for efficient deployment.
- 4-bit precision quantization for efficient memory usage
- Compatible with vLLM serving infrastructure
- Supports 128k context window
- Implements improved function calling capabilities
Core Capabilities
- Multi-lingual support across 10 major languages
- Advanced coding capabilities in 80+ programming languages
- Strong mathematical and reasoning capabilities
- Native function calling and JSON output support
- Robust context adherence for RAG applications
- System prompt handling with improved reliability
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient AWQ quantization while maintaining the advanced capabilities of the original Mistral-Large-Instruct model. It offers a perfect balance between performance and resource efficiency, making it suitable for production deployments.
Q: What are the recommended use cases?
The model excels in multi-lingual applications, coding tasks, mathematical reasoning, and agent-based implementations. It's particularly well-suited for RAG applications and scenarios requiring long context understanding.
