| Property | Value |
|---|---|
| Base Model | Mistral-Nemo-Instruct-2407 |
| Parameters | 12 Billion |
| Context Length | 32,768 tokens |
| Training Hardware | 8x H100 GPUs |
| Model Type | AutoModelForCausalLM |
| Repository | Hugging Face |
What is magnum-v4-12b-gguf?
Magnum v4 12B is an advanced language model developed by Anthracite Organization, specifically designed to emulate the high-quality prose generation capabilities of Claude 3 models (Sonnet and Opus). Built upon the Mistral-Nemo-Instruct-2407 architecture, this model has been fine-tuned using a diverse set of high-quality datasets and optimized for GGUF quantization.
Implementation Details
The model underwent a comprehensive training process spanning 2 epochs using 8 H100 GPUs, implementing advanced features like flash attention, gradient checkpointing, and specialized plugins including Liger optimizations. The training utilized a sophisticated setup with AdamW 8-bit optimizer and cosine learning rate scheduling.
- Employs Liger optimizations: RoPE, RMS normalization, and SwiGLU activation
- Supports sample packing with 32,768 sequence length
- Implements flash attention for improved performance
- Uses specialized instruction format with support for system messages
Core Capabilities
- High-quality prose generation similar to Claude 3
- Extended context window handling (32k tokens)
- Efficient inference through GGUF quantization
- Comprehensive instruction following with system message support
- Optimized for both casual and professional writing tasks
Frequently Asked Questions
Q: What makes this model unique?
The model combines the powerful Mistral-Nemo-Instruct architecture with specialized training on carefully curated datasets, specifically designed to match Claude 3's writing capabilities while maintaining efficient inference through GGUF quantization.
Q: What are the recommended use cases?
The model excels in tasks requiring high-quality prose generation, including content creation, writing assistance, and general instruction following. Its 32k context window makes it particularly suitable for handling longer documents and conversations.