Dolphin 2.2.1 Mistral 7B GGUF
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| License | Apache 2.0 |
| Base Model | Mistral 7B |
| Training Time | 48 hours on 4x A100s |
| Author | Eric Hartford (TheBloke quantized) |
What is dolphin-2.2.1-mistral-7B-GGUF?
Dolphin 2.2.1 is an advanced language model built on Mistral-7B, implementing Microsoft's Orca approach with significant enhancements for conversation and empathy. This GGUF version, quantized by TheBloke, offers various compression levels for efficient deployment while maintaining model quality.
Implementation Details
The model utilizes the ChatML prompt format and features multiple quantization options ranging from 2-bit to 8-bit precision. It was trained for 4 epochs using the Axolotl framework, incorporating datasets from Dolphin, Airoboros, WizardLM, and Samantha.
- Multiple quantization options (Q2_K through Q8_0)
- Optimized for both CPU and GPU deployment
- Supports context lengths up to 2048 tokens
- Uses advanced GGUF format for improved compatibility
Core Capabilities
- Enhanced multi-turn conversation abilities
- Improved empathy and personal advice capabilities
- Uncensored responses for flexible deployment
- Balanced quality-size tradeoffs across different quantizations
Frequently Asked Questions
Q: What makes this model unique?
This model combines Mistral's powerful base architecture with special training in conversation and empathy, while offering various quantization options for different deployment scenarios. It's particularly notable for its uncensored nature and compliance with requests.
Q: What are the recommended use cases?
The model is suitable for conversational AI applications, personal assistance, and general text generation tasks. Users should implement their own alignment layer for production deployments, especially given its uncensored nature.
