Latxa-Llama-3.1-8B-Instruct
| Property | Value |
|---|---|
| Base Model | Meta-Llama-3.1-8B-Instruct |
| Language | Basque (eu) |
| License | Llama 3.1 |
| Training Data | 4.3M documents, 4.2B tokens |
| Developer | HiTZ Research Center & IXA Research Group |
What is Latxa-Llama-3.1-8B-Instruct?
Latxa-Llama-3.1-8B-Instruct is a specialized Large Language Model designed to bridge the gap between high and low-resource languages, specifically focusing on Basque language processing. Built upon Meta's Llama-3.1 architecture, this model has been extensively trained on a high-quality Basque corpus to achieve superior performance in Basque language tasks.
Implementation Details
The model maintains Llama-3.1's architecture while incorporating specialized training on the Latxa Corpus v1.1. Training was conducted on CINECA HPC's infrastructure using 4 A100 64GB nodes, resulting in significant improvements across various Basque language benchmarks.
- Advanced language adaptation techniques applied to Llama-3.1
- Comprehensive training on 4.3M Basque documents
- Achieved third place in public arena evaluation, competing with GPT-4 and Claude
- Environmental impact: 173.45kg CO2 eq from training
Core Capabilities
- Superior performance in Basque language tasks compared to baseline models
- Enhanced reading comprehension (62.78% accuracy on EusReading)
- Improved performance on multiple-choice tasks (80% accuracy on Belebele)
- Specialized handling of Basque trivia and professional examination content
- Chat assistant capabilities with Basque language understanding
Frequently Asked Questions
Q: What makes this model unique?
This model specifically addresses the limitations of existing LLMs in processing Basque language content, showing significant improvements over the original Llama-3.1-Instruct model across all benchmark tests. It's particularly notable for achieving performance levels close to leading proprietary models like GPT-4 and Claude.
Q: What are the recommended use cases?
The model is optimized for Basque language tasks including reading comprehension, chat assistance, and professional content processing. It's particularly suitable for educational applications, professional examination preparation, and general Basque language understanding tasks. However, it's not recommended for use with other languages or for any malicious activities.
