| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Base Model | Mistral-7B-v0.1 |
| Context Length | 4096 tokens |
| License | Apache 2.0 |
| Training Data | 2.3B tokens (Medical + General Domain) |
| Paper | Link to Research Paper |
What is base-7b-v0.2?
Internist.ai 7b represents a breakthrough in medical AI, being the first 7B parameter model to achieve above 60% pass threshold on MedQA (USMLE). Developed by medical professionals at UCLouvain and Cliniques Universitaires Saint-Luc, it demonstrates the effectiveness of a physician-in-the-loop approach to medical language model development.
Implementation Details
The model was trained using Axolotl on 4 NVIDIA A100 80GB GPUs for 450 GPU hours. It incorporates advanced techniques including FlashAttention and NEFTune, with a training corpus of 2.3B tokens carefully curated from medical guidelines, textbooks, and general domain knowledge.
- Trained using BF16 precision with cosine learning rate scheduling
- Utilizes sample packing and 4096 token sequence length
- Implements the Alpaca chat template format
- Incorporates both medical and general domain capabilities
Core Capabilities
- Achieved 60.5% accuracy on MedQA (USMLE)
- Scores 79.4% on PubMedQA benchmarks
- Demonstrates strong performance across MMLU medical categories
- Maintains general domain competency while excelling in medical tasks
Frequently Asked Questions
Q: What makes this model unique?
This is the first 7B parameter model to achieve a passing score on the USMLE benchmark, demonstrating that carefully curated training data and physician involvement can create more effective medical AI models than larger, less focused approaches.
Q: What are the recommended use cases?
The model is designed for medical professionals as an assistant in clinical decision support and documentation. It requires additional task-specific training and safety evaluation before deployment in real-world settings, and is not recommended for use by non-medical professionals.