Dolphin 2.7 Mixtral 8x7b
| Property | Value |
|---|---|
| License | Apache 2.0 |
| Context Window | 16k tokens |
| Training Duration | 3 days on 4x A100s |
| Base Model | Mixtral-8x7b |
What is dolphin-2.7-mixtral-8x7b?
Dolphin 2.7 is an advanced language model built on the Mixtral-8x7b architecture, specifically designed to excel in coding tasks while maintaining strong general-purpose capabilities. This iteration represents a significant improvement over its predecessors, featuring fixes in the transformers library and unfrozen gate layers to enhance overall performance.
Implementation Details
The model was trained using qLoRA and Axolotl frameworks for 1.5 epochs, incorporating data from seven diverse datasets including Magicoder, OpenHermes, and Dolphin-coder. It implements the ChatML prompt format and requires trust_remote_code for operation.
- Retrained with Mixtral-specific optimizations
- Enhanced code generation capabilities
- Uncensored response generation
- 16k context window implementation
Core Capabilities
- Advanced coding and programming assistance
- Compliant and detailed response generation
- Structured output handling
- Enhanced role-playing capabilities
- General chat and conversation support
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its exceptional coding capabilities, combined with uncensored and unbiased responses. The integration of Mixtral-specific optimizations and unfrozen gate layers makes it particularly effective for complex programming tasks while maintaining natural conversation abilities.
Q: What are the recommended use cases?
The model excels in coding tasks, technical documentation, general conversation, and structured output generation. It's particularly suitable for developers, technical writers, and users requiring detailed, unfiltered responses for various applications.
