Llama3-70B-SteerLM-RM
| Property | Value |
|---|---|
| Model Size | 70B parameters |
| Context Length | 8,192 tokens |
| License | Llama 3 Community License |
| Paper | HelpSteer2 Paper |
What is Llama3-70B-SteerLM-RM?
Llama3-70B-SteerLM-RM is an advanced reward model built on the Llama 3 70B Base architecture. Unlike conventional reward models that output a single score, this model provides multi-aspect evaluation of responses across five key dimensions. It's specifically designed to rate conversational AI outputs with sophisticated metrics, making it particularly valuable for training and evaluating language models.
Implementation Details
The model is implemented using NVIDIA's NeMo-Aligner framework and trained on the HelpSteer2 dataset. It processes conversations between users and assistants, providing numerical ratings (0-4) for each response attribute. The model has achieved impressive benchmark scores, particularly in safety evaluation (92.8%) and overall performance (88.8%) on the RewardBench Primary Dataset.
- Built on Llama 3 70B Base architecture
- Trained with NVIDIA NeMo-Aligner toolkit
- Supports up to 8,192 token context length
- Compatible with NeMo ecosystem for deployment and customization
Core Capabilities
- Evaluates response helpfulness and overall utility
- Assesses factual correctness and accuracy
- Measures coherence and clarity of expression
- Rates complexity of intellectual content
- Analyzes verbosity relative to prompt requirements
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its multi-dimensional evaluation approach, providing separate scores for five different aspects of response quality, unlike traditional reward models that only output a single score. It's particularly notable for its strong performance in safety evaluation and challenging conversational scenarios.
Q: What are the recommended use cases?
The model is ideal for training other language models through reinforcement learning, evaluating chatbot responses, and filtering or ranking model outputs. It can be used both as a multi-aspect evaluation tool or as a conventional reward model using the recommended weight configuration.
