LLaMa-30b-instruct-2048
| Property | Value |
|---|---|
| Developer | Upstage |
| Base Model | LLaMA |
| License | Non-commercial Bespoke License |
| Context Length | 2048 tokens (expandable to 10k+) |
| Language | English |
What is llama-30b-instruct-2048?
LLaMa-30b-instruct-2048 is an instruction-tuned language model developed by Upstage, based on Meta's LLaMA architecture. This model stands out for its extended context length handling and impressive performance on scientific and general instruction tasks, achieving a 67.0 score on the Open LLM Leaderboard.
Implementation Details
The model leverages dynamic rope scaling for handling extended input sequences up to 10,000+ tokens. It's optimized for deployment on A100 GPUs and supports 8-bit quantization for efficient inference.
- Trained on multiple high-quality datasets including OpenOrca, LIMA, ScienceQA, and OpenBookQA
- Implements a structured prompt template with System, User, and Assistant roles
- Supports both float16 and 8-bit inference options
Core Capabilities
- Strong performance on scientific question-answering tasks
- Extended context handling beyond standard 2048 tokens
- Competitive benchmark scores: ARC (64.9), HellaSwag (84.9), MMLU (61.9), TruthfulQA (56.3)
- Efficient streaming text generation with built-in support
Frequently Asked Questions
Q: What makes this model unique?
The model's ability to handle extended context lengths up to 10k+ tokens through rope scaling, combined with its strong performance on scientific tasks and general instruction following, makes it particularly suitable for complex, long-context applications.
Q: What are the recommended use cases?
The model excels in scientific question-answering, instruction following, and tasks requiring longer context windows. It's particularly well-suited for academic and research applications, though it requires proper licensing for use.
