Arithmo-Mistral-7B
| Property | Value |
|---|---|
| Base Model | Mistral-7B-v0.1 |
| License | Apache 2.0 |
| Training Approach | QLoRA Fine-tuning |
| Primary Task | Mathematical Reasoning |
| Language | English |
What is Arithmo-Mistral-7B?
Arithmo-Mistral-7B is a specialized mathematical reasoning model that builds upon the Mistral-7B foundation model. Developed by Ashvini Kumar Jindal and Ankur Parikh, it represents a significant advancement in mathematical problem-solving capabilities, outperforming many larger models including 13B parameter versions. The model excels in both Chain-of-Thought (CoT) reasoning and Program-of-Thought (PoT) approaches, achieving impressive scores of 74.7% on GSM8K and 25.3% on MATH benchmarks.
Implementation Details
The model was fine-tuned using QLoRA on a single RTX 4090 GPU, making it an efficient and accessible implementation. It supports two primary interaction modes: Zero-Shot Chain-of-Thought for generating reasoning steps and answers, and Zero-Shot Program-of-Thought for generating executable Python code that solves mathematical problems.
- Trained using QLoRA fine-tuning technique
- Implements both CoT and PoT reasoning approaches
- Optimized for single GPU training and inference
- Compatible with text-generation-inference systems
Core Capabilities
- Solve complex mathematical word problems with detailed reasoning
- Generate executable Python code for mathematical solutions
- Process both direct questions and step-by-step reasoning tasks
- Achieve state-of-the-art performance among 7B parameter models
- Handle various mathematical reasoning tasks including arithmetic, algebra, and word problems
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its ability to match or exceed the performance of much larger models while maintaining efficiency with a 7B parameter architecture. It's uniquely capable of both reasoning through problems and generating executable code solutions.
Q: What are the recommended use cases?
The model is ideal for educational applications, automated math tutoring, problem-solving assistance, and mathematical reasoning tasks. It can be used both for explaining solutions step-by-step and for generating programmatic solutions.
