EEVE-Korean-10.8B-v1.0
| Property | Value |
|---|---|
| Parameter Count | 10.8B |
| Base Model | SOLAR-10.7B-v1.0 |
| License | Apache 2.0 |
| Paper | Research Paper |
| Tensor Type | BF16 |
What is EEVE-Korean-10.8B-v1.0?
EEVE-Korean-10.8B-v1.0 is an advanced language model specifically enhanced for Korean language processing. Built upon the SOLAR-10.7B foundation, it incorporates 8,960 carefully selected Korean tokens through a sophisticated vocabulary expansion process. The model represents a significant advancement in multilingual AI, particularly focusing on Korean language capabilities while maintaining the strong performance of its base model.
Implementation Details
The model employs a seven-stage training process with strategic parameter freezing, focusing on efficient vocabulary expansion. The implementation includes partial fine-tuning of lm_head embeddings for existing tokens while preserving the base model's original parameters. The training process involved comprehensive token selection based on frequency analysis in a 100GB Korean corpus.
- Sophisticated vocabulary expansion with 8,960 new Korean tokens
- Seven-stage training methodology with parameter freezing
- Specialized embedding training for new tokens
- Frequency-based token selection (minimum 6,000 occurrences)
Core Capabilities
- Enhanced Korean language understanding and generation
- Efficient cross-linguistic knowledge transfer
- Preserved base model capabilities
- Optimized for Korean web-based content processing
Frequently Asked Questions
Q: What makes this model unique?
The model's unique approach to vocabulary expansion and its specialized training methodology for Korean language integration, while maintaining the base model's capabilities, sets it apart from traditional multilingual models.
Q: What are the recommended use cases?
While the model excels in Korean language tasks, it hasn't undergone instruction-based fine-tuning. It's best suited for general Korean language processing tasks but may require additional training for specific applications.
