| Property | Value |
|---|---|
| License | Microsoft Research License |
| Paper | View Paper |
| Framework | PyTorch |
| Type | Delta weights for medical vision-language model |
What is llava-med-7b-delta?
LLaVA-Med is a specialized biomedical vision-language model created by Microsoft Research. It's built upon the LLaVA architecture and specifically trained for biomedical image understanding and question answering. This repository contains the delta weights that need to be applied to the original LLaMA model to create the full LLaVA-Med model.
Implementation Details
The model uses a curriculum learning approach to adapt LLaVA to the biomedical domain. It's trained on the PMC-15M dataset, which contains 15 million figure-caption pairs from biomedical research articles.
- Trained using curriculum learning methodology
- Built on top of LLaVA architecture
- Requires original LLaMA weights for implementation
- Optimized for biomedical visual understanding
Core Capabilities
- Medical visual question answering
- Biomedical image interpretation
- Figure-caption understanding
- Support for various medical image types (microscopy, radiography, histology)
Frequently Asked Questions
Q: What makes this model unique?
LLaVA-Med is specifically designed for biomedical applications, with demonstrated improved performance on medical visual question answering tasks like PathVQA and VQA-RAD. It's trained on a comprehensive medical image-text dataset and uses a specialized curriculum learning approach.
Q: What are the recommended use cases?
The model is strictly for research purposes only and should not be used in clinical settings. It's ideal for AI researchers studying biomedical vision-language processing and visual question answering in academic contexts.