Yi-9B-200K
| Property | Value |
|---|---|
| Parameter Count | 8.83B |
| Context Length | 200K tokens |
| License | Apache 2.0 |
| Paper | Yi Tech Report |
What is Yi-9B-200K?
Yi-9B-200K is part of the next generation of open-source large language models developed by 01.AI. It features an extended context window of 200K tokens while maintaining the powerful capabilities of the base Yi-9B model. The model is built on the Llama architecture and trained on a comprehensive 3T multilingual corpus, making it especially effective for both English and Chinese language tasks.
Implementation Details
The model utilizes BF16 tensor types and is optimized for efficient deployment. It's built upon the proven Transformer architecture with Llama-style optimizations, offering a balance between performance and computational requirements. The 200K context window is approximately equivalent to 400,000 Chinese characters, making it suitable for processing very long documents.
- Architecture: Llama-based Transformer
- Training Data: 3T multilingual corpus
- Tensor Type: BF16
- Context Window: 200K tokens
Core Capabilities
- Exceptional performance in code generation and mathematical reasoning
- Strong common-sense reasoning and reading comprehension abilities
- Bilingual proficiency in English and Chinese
- Extended context handling for long-form content
- Competitive performance against larger models like SOLAR-10.7B and Mistral-7B
Frequently Asked Questions
Q: What makes this model unique?
Yi-9B-200K stands out for its exceptional performance despite its relatively compact size. It particularly excels in coding and mathematical tasks, often outperforming larger models, while offering an extensive 200K token context window.
Q: What are the recommended use cases?
The model is well-suited for code generation, mathematical problem-solving, long-form content analysis, and bilingual applications requiring both English and Chinese language processing. It's particularly effective for tasks requiring extended context understanding.
