OpenHermes-2.5-Mistral-7B-GGUF
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| License | Apache 2.0 |
| Model Type | Mistral Architecture |
| Format | GGUF (Various Quantizations) |
What is OpenHermes-2.5-Mistral-7B-GGUF?
OpenHermes-2.5-Mistral-7B-GGUF is a quantized version of Teknium's OpenHermes 2.5, optimized for efficient local deployment. The model represents a significant advancement in open-source language models, trained on 1,000,000 entries of primarily GPT-4 generated data and featuring enhanced capabilities in both code and general tasks.
Implementation Details
This GGUF version comes in multiple quantization options ranging from 2-bit to 8-bit, allowing users to balance between model size (3.08GB - 7.70GB) and performance. It uses the ChatML format for interactions and supports system prompts for consistent multi-turn conversations.
- Multiple quantization options (Q2_K through Q8_0)
- ChatML prompt format support
- Improved HumanEval score of 50.7% @ Pass 1
- Enhanced benchmark performance across GPT4All, AGIEval, and TruthfulQA
Core Capabilities
- Strong code generation and comprehension
- Advanced reasoning and problem-solving
- Multi-turn conversation support
- System prompt integration
- Balanced performance across various quantization levels
Frequently Asked Questions
Q: What makes this model unique?
The model combines Mistral's efficient architecture with extensive training on high-quality data, including code instruction (7-14% of dataset), resulting in improved performance across multiple benchmarks while maintaining reasonable model sizes through various quantization options.
Q: What are the recommended use cases?
The model excels in code-related tasks, general knowledge queries, and multi-turn conversations. It's particularly suitable for local deployment scenarios where balanced performance and resource usage are important.
