Granite-3.2-8B-Instruct
| Property | Value |
|---|---|
| Parameter Count | 8 Billion |
| License | Apache 2.0 |
| Release Date | February 26th, 2025 |
| Developer | IBM Granite Team |
| Model URL | https://huggingface.co/unsloth/granite-3.2-8b-instruct-bnb-4bit |
What is granite-3.2-8b-instruct-bnb-4bit?
Granite-3.2-8B-Instruct is an advanced language model that builds upon its predecessor, featuring enhanced reasoning capabilities and multi-lingual support. This quantized version (4-bit) offers improved efficiency while maintaining performance across various tasks. The model demonstrates significant improvements in benchmark scores, particularly in ArenaHard (55.25) and Alpaca-Eval-2 (61.19), showing substantial gains over its previous version.
Implementation Details
The model is trained on IBM's Blue Vela supercomputing cluster using NVIDIA H100 GPUs, combining permissively licensed open-source datasets with synthetic data specifically designed for reasoning tasks. It implements a unique controllable thinking capability that can be toggled based on task requirements.
- 4-bit quantization for efficient deployment
- Supports 12 languages including English, German, Spanish, and more
- Controllable thinking capability via API parameter
- Long-context support for document processing
Core Capabilities
- Advanced reasoning and thinking tasks
- Text summarization and classification
- Question-answering and information extraction
- Retrieval Augmented Generation (RAG)
- Code-related tasks and function calling
- Multi-lingual dialogue processing
- Long document analysis and summarization
Frequently Asked Questions
Q: What makes this model unique?
The model's distinguishing feature is its controllable thinking capability and significant performance improvements in reasoning tasks, demonstrated by its superior benchmark scores compared to previous versions. The 4-bit quantization makes it more efficient while maintaining high performance.
Q: What are the recommended use cases?
The model excels in business applications requiring complex reasoning, document processing, and multi-lingual support. It's particularly suitable for AI assistants, document analysis, code-related tasks, and scenarios requiring detailed analytical thinking.
