Arcee-VyLinh
| Property | Value |
|---|---|
| Parameter Count | 3.4B |
| Model Type | Text Generation |
| Architecture | Qwen2.5-3B based |
| Context Length | 32K tokens |
| Language Support | Vietnamese, English |
What is Arcee-VyLinh?
Arcee-VyLinh is a specialized Vietnamese language model that combines advanced training techniques with efficient architecture. Built on the Qwen2.5-3B foundation, this model represents a significant advancement in Vietnamese language processing, offering competitive performance against larger models while maintaining a relatively compact 3B parameter size.
Implementation Details
The model employs a sophisticated multi-stage training process, incorporating evolved hard questions and iterative Direct Preference Optimization (DPO). Built using transformers architecture, it supports both F32 tensor types and offers a comprehensive 32K token context window.
- Custom evolved dataset integration with ORPO-Mix-40K
- Proprietary merging technique with Qwen2.5-3B-Instruct
- 6 epochs of iterative DPO training
- Innovative EvolKit implementation for question generation
Core Capabilities
- Advanced Vietnamese language understanding and generation
- Strong instruction-following abilities
- Efficient text completion and content creation
- Competitive performance with 4B-8B parameter models
- Extensive context handling with 32K token window
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its optimized performance for Vietnamese language tasks achieved through an innovative training pipeline combining evolved hard questions, iterative DPO, and proprietary merging techniques. Despite its compact 3B parameter size, it demonstrates competitive performance with larger models.
Q: What are the recommended use cases?
The model excels in Vietnamese language tasks including chat applications, instruction following, text generation, question answering, and content creation. It's particularly well-suited for general language understanding tasks while maintaining efficiency in deployment.
