Llama-3.1-Nemotron-70B-Reward-HF
| Property | Value |
|---|---|
| Parameter Count | 70.6B |
| License | Llama 3.1 |
| Architecture | Transformer (Llama 3.1) |
| Paper | HelpSteer2-Preference |
| Training Data | HelpSteer2 Dataset |
What is Llama-3.1-Nemotron-70B-Reward-HF?
Llama-3.1-Nemotron-70B-Reward-HF is a sophisticated reward model developed by NVIDIA, built on the Llama-3.1-70B-Instruct foundation. This model represents a significant advancement in AI evaluation capabilities, designed to predict and score the quality of language model responses. It employs a novel approach combining Bradley Terry and SteerLM Regression Reward Modelling techniques.
Implementation Details
The model processes conversations of up to 4,096 tokens and provides a reward score indicating response quality. It requires at least 2 80GB GPUs (NVIDIA Ampere or newer) and 150GB of free disk space for deployment.
- Achieves state-of-the-art performance on RewardBench with 94.1% overall accuracy
- Excels in Chat (97.5%), Safety (95.1%), and Reasoning (98.1%) categories
- Implemented using HuggingFace Transformers library
Core Capabilities
- Comparative response evaluation across multiple conversation turns
- Robust performance across different evaluation categories
- Support for both direct inference and integration into RLHF pipelines
- Compatible with major NVIDIA GPU architectures (Ampere, Hopper, Turing)
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its ability to evaluate LLM responses without relying on GPT-4 generated training data, achieving top performance using only permissive licensed data. It's currently #1 on multiple automatic alignment benchmarks, surpassing models like GPT-4 and Claude 3.5 Sonnet.
Q: What are the recommended use cases?
The model is particularly suited for: evaluating chatbot responses, assessing AI safety compliance, analyzing reasoning capabilities in AI outputs, and integration into RLHF pipelines for model fine-tuning.
