Meditron-7B-AWQ
| Property | Value |
|---|---|
| Base Model | Llama-2-7B |
| Parameters | 7 Billion |
| Context Length | 4096 tokens |
| Quantization | 4-bit AWQ |
| License | LLAMA 2 Community License |
| Paper | MediTron-70B: Scaling Medical Pretraining for Large Language Models |
What is Meditron-7B-AWQ?
Meditron-7B-AWQ is a quantized version of the Meditron medical language model, specifically optimized for healthcare applications. It's been trained on a comprehensive medical corpus including PubMed articles, medical guidelines, and general domain knowledge. The model employs AWQ (Activation-aware Weight Quantization) to reduce its size while maintaining performance, making it more accessible for deployment on resource-constrained systems.
Implementation Details
The model uses a 4-bit AWQ quantization scheme with a group size of 128, optimized on medical datasets. It maintains the original Llama 2 architecture with 32 attention heads and 32 layers, while reducing the model size to just 3.89 GB. The implementation supports various inference frameworks including vLLM, Hugging Face's TGI, and text-generation-webui.
- Optimized 4-bit quantization for efficient inference
- Support for multiple deployment options
- Compatible with major inference frameworks
- Maintains medical domain expertise despite compression
Core Capabilities
- Medical exam question answering
- Differential diagnosis support
- Disease information querying
- General health information processing
- Medical reasoning and analysis
Frequently Asked Questions
Q: What makes this model unique?
The model combines specialized medical knowledge from high-quality sources with efficient 4-bit quantization, making it both capable and deployable in resource-constrained environments. It significantly outperforms base Llama-2-7B on medical tasks while maintaining a small footprint.
Q: What are the recommended use cases?
The model is best suited for research and development in medical AI applications, including medical exam preparation, clinical decision support systems, and health information retrieval. However, it should not be used directly in clinical settings without proper validation and testing.
