Llama-3.1-Tulu-3-8B-DPO
| Property | Value |
|---|---|
| Base Model | Llama-3.1-8B |
| License | Llama 3.1 Community License |
| Language | English |
| Paper | arXiv:2411.15124 |
What is Llama-3.1-Tulu-3-8B-DPO?
Llama-3.1-Tulu-3-8B-DPO is a sophisticated language model developed by Allen AI, representing an advanced stage in the Tulu 3 model family. It's built upon the Llama 3.1 architecture and optimized using Direct Preference Optimization (DPO) techniques. The model demonstrates exceptional performance across various tasks, particularly excelling in mathematical reasoning, problem-solving, and instruction following.
Implementation Details
The model implements specific hyperparameters including a learning rate of 5×10⁻⁷, linear learning rate schedule, effective batch size of 32, and maximum sequence length of 2,048. It utilizes a specialized chat template format with distinct user and assistant markers.
- Trained using DPO on carefully curated datasets
- Implements Llama 3.1's architecture with comprehensive optimizations
- Supports both regular text generation and chat-based interactions
Core Capabilities
- Strong performance in mathematical reasoning (87.6% on GSM8K)
- High accuracy in code generation (83.9% on HumanEval)
- Excellent instruction following capabilities (82.4% on IFEval)
- Robust safety features (85.5% average on safety tasks)
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its balanced performance across diverse tasks, particularly excelling in mathematical reasoning and instruction following. It's part of a fully open-source ecosystem with transparent training procedures and comprehensive documentation.
Q: What are the recommended use cases?
The model is particularly well-suited for mathematical problem-solving, code generation, and general instruction following tasks. It's designed for research and educational purposes, with strong capabilities in both conversational and analytical applications.
