Magnum v4 72B GGUF
| Property | Value |
|---|---|
| Parameter Count | 72.7B |
| License | Apache 2.0 |
| Base Model | Qwen2.5-72B-Instruct |
| Training Hardware | 8x mi300x GPUs |
| Format | GGUF Quantized |
What is magnum-v4-72b-gguf?
Magnum v4 72B GGUF is a sophisticated language model designed to replicate the prose quality of Claude 3 models (Sonnet and Opus). Built on top of Qwen2.5-72B-Instruct, this model has been fine-tuned using multiple high-quality datasets and optimized for deployment through GGUF quantization.
Implementation Details
The model utilizes the ChatML format for interactions and was trained using the Axolotl framework with advanced optimizations including Liger plugins for enhanced performance. The training process involved full-parameter fine-tuning across six carefully curated datasets, focusing on maintaining high-quality prose and instruction-following capabilities.
- Implements ChatML conversation format with system, user, and assistant messages
- Utilizes advanced training optimizations including Liger RoPE, RMS norm, and SwiGLU
- Trained with a sequence length of 32,768 tokens
- Incorporates gradient checkpointing and flash attention for efficient training
Core Capabilities
- High-quality prose generation similar to Claude 3 models
- Comprehensive instruction-following abilities
- Extended context window handling
- Optimized for both chat and general text generation tasks
- Support for SillyTavern integration with custom templates
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its careful optimization to match Claude 3's prose quality while maintaining the benefits of open deployment through GGUF quantization. It combines the strengths of Qwen2.5-72B-Instruct with extensive fine-tuning on quality datasets.
Q: What are the recommended use cases?
The model excels in conversational AI applications, creative writing, and general text generation tasks. It's particularly well-suited for applications requiring high-quality prose output and detailed contextual understanding.
