EXAONE-Deep-7.8B-GGUF
| Property | Value |
|---|---|
| Parameters | 6.98B |
| Context Length | 32,768 tokens |
| License | EXAONE AI Model License Agreement 1.1 - NC |
| Author | LG AI Research |
| Architecture | 32 layers, GQA with 32 Q-heads and 8 KV-heads |
What is EXAONE-Deep-7.8B-GGUF?
EXAONE-Deep-7.8B-GGUF is an advanced language model developed by LG AI Research, specifically designed for superior reasoning capabilities in mathematics and coding tasks. The model represents a significant advancement in AI reasoning, outperforming both open-weight models of comparable scale and proprietary models like OpenAI's o1-mini.
Implementation Details
The model features a sophisticated architecture with 32 layers and employs Grouped-Query Attention (GQA) with 32 query heads and 8 key-value heads. It supports multiple quantization options including Q8_0, Q6_K, Q5_K_M, Q4_K_M, and IQ4_XS in GGUF format, with BF16 weights available.
- Vocabulary size of 102,400 tokens
- Extensive context window of 32,768 tokens
- Optimized for reasoning tasks with specialized thought process handling
- Compatible with various inference frameworks including TensorRT-LLM, vLLM, and llama.cpp
Core Capabilities
- Advanced mathematical reasoning and problem-solving
- Superior coding capabilities
- Structured thought process with
tags - High-performance quantization options for different deployment scenarios
Frequently Asked Questions
Q: What makes this model unique?
EXAONE-Deep-7.8B-GGUF stands out for its exceptional reasoning capabilities and optimized configuration for math and coding tasks. It incorporates a unique thought process structure and outperforms models of similar size, including some proprietary solutions.
Q: What are the recommended use cases?
The model excels in mathematical reasoning, coding tasks, and general problem-solving scenarios. It's particularly effective when used with step-by-step reasoning prompts and specialized instructions for math problems using \boxed{} notation.
