LLaVA-OneVision-Qwen2-0.5B
| Property | Value |
|---|---|
| Parameter Count | 894M |
| Model Type | Multimodal LLM |
| Architecture | SO400M + Qwen2 |
| License | Apache 2.0 |
| Paper | arXiv:2408.03326 |
What is llava-onevision-qwen2-0.5b-ov-hf?
LLaVA-OneVision is an innovative multimodal language model that represents a significant advancement in visual understanding capabilities. Built on the Qwen2 architecture, this 894M parameter model is specifically designed to handle single-image, multi-image, and video scenarios with remarkable efficiency.
Implementation Details
The model underwent a comprehensive training process including multiple stages: LCS-558K pretraining, 4.7M high-quality synthetic data training, 3.6M single-image data training, and finally 1.6M mixed modality training. It supports FP16 precision and can be optimized using 4-bit quantization and Flash-Attention 2.
- Supports both pipeline and pure transformers implementation
- Compatible with transformers.js for JavaScript deployment
- Includes multi-image and multi-prompt generation capabilities
- Offers optimization options for improved performance
Core Capabilities
- Single-image understanding and analysis
- Multi-image comparison and reasoning
- Video comprehension through transfer learning
- Cross-modal conversation handling
- Bilingual support (English and Chinese)
Frequently Asked Questions
Q: What makes this model unique?
This model is distinctive for its ability to handle multiple visual scenarios (single-image, multi-image, and video) within a single architecture, demonstrating strong transfer learning capabilities across different modalities.
Q: What are the recommended use cases?
The model is ideal for applications requiring visual understanding and natural language interaction, such as image analysis, visual question answering, multi-image comparison, and basic video understanding tasks.
