Cabrita-LoRA v0.1
| Property | Value |
|---|---|
| Language | Portuguese |
| Base Model | LLaMA 7B |
| Training Method | LoRA Fine-tuning |
| License | OpenRAIL |
What is cabrita-lora-v0-1?
Cabrita-LoRA is a Portuguese language instruction-tuned variant of the LLaMA model, developed using the LoRA (Low-Rank Adaptation) fine-tuning approach. The model was trained on a Portuguese translation of the Stanford Alpaca dataset, making it particularly effective for Portuguese language instruction-following tasks.
Implementation Details
The model leverages the PEFT (Parameter-Efficient Fine-Tuning) framework from Hugging Face and was trained on a single A100 GPU for approximately 4 hours. The training data was created by translating the original Alpaca dataset using ChatGPT, providing a cost-effective solution for Portuguese language model development.
- Built on LLaMA-7B base model
- Uses LoRA for efficient fine-tuning
- Trained on translated Alpaca dataset
- Implements PEFT methodology
Core Capabilities
- Portuguese language instruction following
- Creative text generation
- Structured response formatting
- Natural language understanding in Portuguese
Frequently Asked Questions
Q: What makes this model unique?
This model represents one of the first Portuguese-language instruction-tuned variants of LLaMA, offering efficient fine-tuning through LoRA while maintaining high-quality outputs in Portuguese. The cost-effective approach to dataset translation (US$8.00) demonstrates an innovative way to create language-specific models.
Q: What are the recommended use cases?
The model is particularly suitable for Portuguese language applications requiring instruction following, creative text generation, and structured responses. It can be deployed for tasks ranging from content generation to providing detailed explanations in Portuguese.
