Llama-3.2-1B-Instruct-korQuAD-v1
| Property | Value |
|---|---|
| Parameter Count | 1.24B |
| License | llama3.2 |
| Base Model | Llama-3.2-1B-Instruct |
| Training Dataset | KorQuAD v1.0 |
| Latest Performance | EM: 36.07%, F1: 59.03% |
What is Llama-3.2-1B-Instruct-korQuAD-v1?
This is a specialized Korean language question-answering model that builds upon the Llama-3.2-1B-Instruct architecture. Fine-tuned using the KorQuAD dataset, it represents a significant advancement in Korean language understanding and response generation.
Implementation Details
The model utilizes Low-Rank Adaptation (LoRA) for efficient fine-tuning, with specific configurations including an r value of 16, lora_alpha of 16, and targeted modules including q_proj, v_proj, k_proj, and others. Training was conducted over 5 epochs with a learning rate of 2e-4 using the AdamW optimizer.
- Training Method: LoRA with 7 target modules
- Optimization: AdamW (32-bit)
- Batch Size: 1
- Learning Rate: 2e-4
Core Capabilities
- Korean Question-Answering
- Context-based response generation
- Improved performance over base model (36.07% Exact Match)
- Support for both academic and general knowledge queries
Frequently Asked Questions
Q: What makes this model unique?
This model specializes in Korean language question-answering, utilizing the powerful Llama-3.2-1B-Instruct architecture with specific optimizations for Korean language understanding. The LoRA fine-tuning approach ensures efficient adaptation while maintaining core model capabilities.
Q: What are the recommended use cases?
The model is particularly suited for Korean language applications requiring question-answering capabilities, such as educational tools, customer service automation, and information extraction from Korean text. It performs well with both factual and contextual questions.
