| Property | Value |
|---|---|
| Parameter Count | 7.5B |
| License | cc-by-nc-4.0 |
| Base Model | Meta-Llama-3-8B-Instruct |
| Paper | RLHF Workflow Paper |
| Tensor Type | BF16 |
What is FsfairX-LLaMA3-RM-v0.1?
FsfairX-LLaMA3-RM-v0.1 is a cutting-edge reward modeling system built on the LLaMA3 architecture, specifically designed for Reinforcement Learning from Human Feedback (RLHF). As of April 2024, it represents the state-of-the-art in open-source reward models, achieving remarkable performance across various benchmarks.
Implementation Details
The model is implemented using the Meta-Llama-3-8B-Instruct base architecture and utilizes advanced training techniques from the RLHF Workflow framework. It supports multiple RLHF approaches, including PPO, iterative SFT, and iterative DPO, making it highly versatile for various alignment tasks.
- Built with transformers architecture and safetensors implementation
- Optimized for text-generation-inference
- Implements BF16 tensor type for efficient computation
- Includes comprehensive chat templating functionality
Core Capabilities
- Chat Performance: 99.44% accuracy on standard benchmarks
- Hard Chat Scenarios: 65.13% accuracy on challenging cases
- Safety Features: 88.76% effectiveness in safety evaluations
- Reasoning Capabilities: 88.3% accuracy in logical reasoning tasks
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its exceptional performance in reward modeling, particularly its state-of-the-art results on Reward-Bench. It's specifically optimized for RLHF applications and offers a balanced approach to safety, reasoning, and chat capabilities.
Q: What are the recommended use cases?
The model is ideal for implementing RLHF pipelines, particularly in scenarios requiring robust reward modeling. It excels in chat applications, safety-critical implementations, and tasks requiring strong reasoning capabilities.