Qwen-VL-Chat-Int4
| Property | Value |
|---|---|
| Parameter Count | 4.05B params |
| Model Type | Visual Language Model |
| Research Paper | arXiv:2308.12966 |
| Architecture | Transformer-based with 4-bit quantization |
What is Qwen-VL-Chat-Int4?
Qwen-VL-Chat-Int4 is a quantized version of the Qwen visual language model that maintains high performance while significantly reducing memory usage. This model can process both images and text, enabling sophisticated multimodal interactions while being more efficient than its full-precision counterpart.
Implementation Details
The model leverages 4-bit precision quantization to achieve improved inference speed and reduced memory footprint. Benchmarks show that the Int4 version achieves comparable performance to the original model while using significantly less GPU memory - peak usage is reduced from 22.60GB to 11.82GB for encoding 2048 tokens.
- Supports high-resolution image processing (448x448)
- Maintains competitive performance on benchmarks like TouchStone
- Achieves 37.79 tokens/sec for 2048 token generation
Core Capabilities
- Zero-shot image captioning with SOTA performance
- General visual question-answering
- Text-based visual QA for documents and charts
- Multi-language support (Chinese and English)
- Referring expression comprehension
Frequently Asked Questions
Q: What makes this model unique?
The model combines high performance in visual-language tasks with efficient 4-bit quantization, making it more practical for deployment while maintaining competitive accuracy across various benchmarks.
Q: What are the recommended use cases?
The model excels in image captioning, visual QA, document analysis, and multilingual visual-language tasks. It's particularly suitable for applications requiring efficient deployment with limited computational resources.
