EXAONE-Deep-32B-GGUF
| Property | Value |
|---|---|
| Parameters | 30.95B |
| Context Length | 32,768 tokens |
| Layers | 64 |
| Attention Heads | 40 Q-heads, 8 KV-heads (GQA) |
| Vocab Size | 102,400 |
| License | EXAONE AI Model License Agreement 1.1 - NC |
What is EXAONE-Deep-32B-GGUF?
EXAONE-Deep-32B-GGUF is an advanced language model developed by LG AI Research, specifically engineered for superior reasoning capabilities across various tasks including mathematics and coding. This GGUF-formatted model represents the culmination of LG AI's research in creating efficient, powerful language models that can compete with leading open-weight models in the field.
Implementation Details
The model employs a sophisticated architecture featuring Grouped-Query Attention (GQA) with 40 Q-heads and 8 KV-heads, optimized for efficient processing. It supports multiple quantization options including Q8_0, Q6_K, Q5_K_M, Q4_K_M, and IQ4_XS in GGUF format, with BF16 weights available for high-precision applications.
- Extensive context window of 32,768 tokens
- Large vocabulary size of 102,400 tokens
- Optimized for reasoning tasks with specialized thought process handling
- Compatible with multiple inference frameworks including TensorRT-LLM, vLLM, and llama.cpp
Core Capabilities
- Advanced reasoning in mathematics and coding tasks
- Structured thought process generation with
tags - High-performance multi-turn conversations
- Competitive performance against leading open-weight models
- Flexible deployment options across various frameworks
Frequently Asked Questions
Q: What makes this model unique?
EXAONE-Deep-32B-GGUF stands out for its specialized reasoning capabilities and structured thought process approach, utilizing a unique architecture with GQA and extensive context length. The model's ability to handle complex reasoning tasks while maintaining efficiency through various quantization options makes it particularly valuable for technical applications.
Q: What are the recommended use cases?
The model excels in scenarios requiring deep reasoning, particularly in mathematics and coding. It's best utilized with specific prompting patterns that leverage its thought process capabilities, making it ideal for educational applications, technical problem-solving, and complex analytical tasks. For optimal results, users should follow the recommended temperature (0.6) and top-p (0.95) settings.
