AI21-Jamba-Large-1.6
| Property | Value |
|---|---|
| Parameter Count | 94B active / 398B total |
| Model Type | Joint Attention and Mamba (Jamba) |
| Context Length | 256K tokens |
| License | Jamba Open Model License |
| Languages | English, Spanish, French, Portuguese, Italian, Dutch, German, Arabic, Hebrew |
| Knowledge Cutoff | March 5, 2024 |
What is AI21-Jamba-Large-1.6?
AI21-Jamba-Large-1.6 represents a breakthrough in AI language models, combining SSM-Transformer architecture to deliver exceptional performance with long-context handling. This hybrid model features 94B active parameters (398B total), making it one of the most powerful and efficient long-context models available, delivering up to 2.5X faster inference than competing models.
Implementation Details
The model leverages innovative architecture combining traditional transformer attention mechanisms with Mamba SSM components. It requires specialized deployment considerations, including support for 8x80GB GPUs and ExpertsInt8 quantization for optimal performance.
- Supports context lengths up to 256K tokens
- Implements efficient ExpertsInt8 quantization
- Requires mamba-ssm and causal-conv1d dependencies
- Compatible with both vLLM and transformers frameworks
Core Capabilities
- Superior benchmark performance (76.5 on Arena Hard, 78.2 on CRAG)
- Advanced tool use capabilities with JSON-formatted outputs
- Structured output generation and function calling
- Multi-language support across 9 languages
- Efficient long-context processing
Frequently Asked Questions
Q: What makes this model unique?
The model's hybrid SSM-Transformer architecture enables superior long-context handling while maintaining high performance. It's the first non-Transformer model to achieve comparable quality to market-leading models while offering significantly faster inference.
Q: What are the recommended use cases?
The model excels in business applications requiring long-context understanding, structured output generation, and function calling. It's particularly well-suited for tasks involving document processing, complex reasoning, and multi-language applications.
