| Property | Value |
|---|---|
| Parameter Count | 7 Billion |
| Model Type | Causal Language Model |
| Base Architecture | Mistral-7B-v1.0 |
| Context Length | 200,000 tokens |
| Author | Yam Peleg |
| HuggingFace URL | Link |
What is Hebrew-Mistral-7B-200K?
Hebrew-Mistral-7B-200K is an innovative bilingual Large Language Model that extends the capabilities of Mistral-7B to excel in both Hebrew and English language processing. Built upon the Mistral-7B-v1.0 architecture, this model features an enhanced tokenizer with 64,000 tokens specifically optimized for Hebrew language representation.
Implementation Details
The model implements a sophisticated architecture that builds upon the Mistral foundation while incorporating specialized Hebrew language capabilities. It can be deployed using various configurations including standard CPU/GPU implementations and memory-efficient 4-bit quantization options.
- Extended tokenizer with 64,000 tokens optimized for Hebrew
- 200K context length for handling extensive text sequences
- Supports multiple deployment options (CPU, GPU, 4-bit quantization)
- Built on the robust Mistral-7B architecture
Core Capabilities
- Bilingual understanding and generation in Hebrew and English
- Long-context processing up to 200K tokens
- General-purpose language processing tasks
- Memory-efficient deployment options
- Flexible integration through HuggingFace Transformers library
Frequently Asked Questions
Q: What makes this model unique?
The model's primary distinction lies in its specialized Hebrew language capabilities while maintaining English proficiency, combined with an extended context length of 200K tokens and a Hebrew-optimized tokenizer with 64,000 tokens.
Q: What are the recommended use cases?
The model is suitable for a wide range of natural language processing tasks, particularly those involving Hebrew and English content. This includes text generation, translation assistance, content analysis, and general language understanding tasks requiring bilingual capabilities.