| Property | Value |
|---|---|
| Parameter Count | 6.74B |
| Max Sequence Length | 2048 tokens |
| License | Research Only (Same as LLaMA-1) |
| Paper | Sabiá: Portuguese Large Language Models |
| Language | Portuguese |
What is sabia-7b-GGUF?
Sabia-7b-GGUF is a quantized version of the Maritaca AI's Sabiá-7B model, optimized for efficient deployment while maintaining strong performance on Portuguese language tasks. The model was pretrained on 7 billion tokens from the Portuguese subset of ClueWeb22, building upon LLaMA-1-7B architecture.
Implementation Details
This model utilizes the LLaMA architecture and tokenizer, with specialized training for Portuguese language understanding. The model was further trained on an additional 10 billion tokens (approximately 1.4 epochs) after initial pretraining.
- Based on LLaMA-1-7B architecture
- GGUF quantization for improved efficiency
- Supports context window of 2048 tokens
- Trained on data up to mid-2022
Core Capabilities
- Strong performance on Portuguese benchmarks (47.09% average across tasks)
- Excels in ENEM Challenge (55.07% accuracy)
- Effective hate speech detection (64.13% f1-macro on PT Hate Speech Binary)
- Maintains 49.0 NPM score on English tasks
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its specialized Portuguese language capabilities while maintaining reasonable performance on English tasks. It achieves a higher NPM score (48.5) compared to both LLaMA-1-7B (33.0) and LLaMA-2-7B (43.7) on Portuguese tasks.
Q: What are the recommended use cases?
The model is best suited for few-shot learning tasks rather than zero-shot applications. It excels in Portuguese text generation, classification, and analysis tasks, particularly in educational and sentiment analysis contexts.