| Property | Value |
|---|---|
| Author | BlinkDL |
| Training Data | 3.1T tokens (World-v3) |
| Language Distribution | 80% English, 10% multilingual, 10% code |
| Model URL | huggingface.co/BlinkDL/rwkv-7-world |
What is rwkv-7-world?
RWKV-7 World is a sophisticated language model trained on a diverse corpus of over 100 languages. It represents a significant advancement in multilingual AI capabilities, with different versions trained on varying token counts (v3: 3.1T, v2.9: 2T, v2.8: 1T tokens). The model demonstrates impressive performance improvements, particularly in MMLU evaluations, where the 2.9B parameter version achieves 54.56% accuracy, a substantial improvement over its predecessor.
Implementation Details
The model offers multiple architecture sizes to accommodate different computational requirements:
- 0.1B parameters: 12 layers, 768 dimensions
- 0.4B parameters: 24 layers, 1024 dimensions
- 1.5B parameters: 24 layers, 2048 dimensions
- ~3B parameters: 32 layers, 2560 dimensions
- ~7B parameters: 32 layers, 4096 dimensions
Core Capabilities
- Multilingual processing across 100+ languages
- Significantly improved MMLU performance
- Flexible architecture sizes for different deployment scenarios
- Specialized prompt formats for chat and QA tasks
- Support for code generation and processing
Frequently Asked Questions
Q: What makes this model unique?
RWKV-7 World stands out for its balanced approach to multilingual processing, with a thoughtful distribution of training data (80% English, 10% multilingual, 10% code) and significant improvements in benchmark performance. The model's architecture allows for various sizes while maintaining quality across scales.
Q: What are the recommended use cases?
The model is well-suited for chat applications, question-answering tasks, and code-related operations. It includes specific prompt formats for optimal performance in these scenarios, making it particularly valuable for multilingual applications and development environments.