XVERSE-13B
| Property | Value |
|---|---|
| License | Apache-2.0 |
| Context Length | 8K tokens |
| Architecture | Decoder-only Transformer |
| Training Data | 3.2T tokens |
| Vocabulary Size | 100,534 tokens |
What is XVERSE-13B?
XVERSE-13B is a sophisticated multilingual large language model developed by Shenzhen Yuanxiang Technology. It represents a significant advancement in multilingual AI capabilities, supporting over 40 languages including Chinese, English, Russian, and Spanish. The model has been extensively trained on 3.2 trillion tokens, making it particularly powerful for both Chinese and English language tasks.
Implementation Details
The model employs a standard Decoder-only Transformer architecture with several innovative features. It utilizes a BPE-based tokenizer trained on hundreds of gigabytes of data, enabling efficient multilingual processing without vocabulary expansion.
- 8K context length - highest among similar-sized models
- Advanced training framework with 58.5% peak computational power utilization
- Sophisticated parallel scheduling and memory optimization
- Custom tokenizer supporting 40+ languages efficiently
Core Capabilities
- Strong performance in multilingual tasks (C-Eval: 63.5%, MMLU: 61.2%)
- Advanced reasoning and mathematical problem-solving (GSM8K: 54.9%)
- Coding capabilities (HumanEval: 39.6%)
- Robust common sense reasoning (CommonSenseQA: 74.0%)
- Excellent performance in both Chinese and English educational assessments
Frequently Asked Questions
Q: What makes this model unique?
XVERSE-13B stands out for its exceptional context length of 8K tokens and comprehensive multilingual support. The model's training on 3.2T tokens and sophisticated tokenization system allows it to handle multiple languages without compromising performance.
Q: What are the recommended use cases?
The model excels in multilingual applications, including long-form content generation, educational assessments, technical documentation, and cross-lingual tasks. It's particularly well-suited for applications requiring both Chinese and English language capabilities.
