llama-3.1-70b-abliterated-lora
| Property | Value |
|---|---|
| Base Model | meta-llama/Llama-3.1-70B-Instruct |
| License | Apache 2.0 |
| Developer | @reissbaker |
| Organization | Synthetic Lab |
What is llama-3.1-70b-abliterated-lora?
This is a LoRA adapter model extracted from mlabonne/Meta-Llama-3.1-70B-Instruct-lorablated, designed to enhance the capabilities of the base Llama-3.1-70B-Instruct model. The model underwent specialized training using carefully curated datasets and advanced training techniques.
Implementation Details
The model was trained using sophisticated technical specifications, including BF16 mixed-precision training, a 4e-4 learning rate with a linear schedule, and the Fused AdamW optimizer. Training was conducted on 8xA100 GPUs using FSDP (Fully Sharded Data Parallel) for optimal performance.
- Training utilized mlabonne/harmful_behaviors dataset
- Incorporated user prompts from Guilherme34/uncensor
- Underwent 2 epochs of training
- Dataset was cleaned to remove unwanted refusals
Core Capabilities
- Enhanced instruction following abilities
- Optimized response generation
- Efficient adaptation through LoRA architecture
- Seamless integration with base model
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines the powerful Llama 3.1 70B architecture with specialized LoRA adaptation, trained using state-of-the-art techniques and carefully curated datasets.
Q: What are the recommended use cases?
The model is optimized for general instruction following and can be accessed through glhf.chat for immediate deployment. It's particularly suited for applications requiring robust language understanding and generation capabilities.
