| Property | Value |
|---|---|
| Base Model | Qwen 2.5 |
| Training Data | 140B tokens total |
| Model URL | Hugging Face |
| Author | t-tech |
What is T-lite-it-1.0?
T-lite-it-1.0 is an advanced language model built upon the Qwen 2.5 architecture, specifically enhanced for Russian language capabilities while maintaining strong performance in English. The model underwent a comprehensive training process involving 100B tokens of diverse Russian data in the initial pre-training stage, followed by 40B tokens of mixed instruction and pre-training data in stage 2.
Implementation Details
The model's training pipeline consists of multiple stages, including continual pre-training and alignment techniques. The training data encompasses diverse sources such as Common Crawl, books, code, and proprietary datasets. The model underwent both Supervised Fine-Tuning (SFT) with 1B tokens and preference tuning for enhanced helpfulness.
- Two-stage pre-training process totaling 140B tokens
- Specialized instruction-following capabilities
- Comprehensive Russian language optimization
- Integration with both Transformers and VLLM frameworks
Core Capabilities
- Superior performance on Russian language benchmarks (MERA: 0.552, MaMuRaMu: 0.775)
- Strong mathematical reasoning (ruGSM8K: 0.856, ruMATH: 0.679)
- Advanced code evaluation capabilities (ruCodeEval)
- High-quality instruction following (MT Bench Ru: 7.87)
Frequently Asked Questions
Q: What makes this model unique?
T-lite-it-1.0 stands out for its specialized Russian language capabilities while maintaining strong performance across various tasks. It outperforms several comparable models in benchmarks and offers a balanced approach to both general language understanding and specific task completion.
Q: What are the recommended use cases?
The model is designed for further fine-tuning rather than direct deployment as a conversational assistant. It's particularly well-suited for tasks requiring Russian language processing, mathematical reasoning, and instruction following. However, users should implement appropriate safety measures and ethical considerations before deployment.