ViT-L-16-HTxt-Recap-CLIP
| Property | Value |
|---|---|
| License | CC-BY-4.0 |
| Research Paper | arXiv:2406.08478 |
| Downloads | 51,187 |
| Primary Task | Zero-Shot Image Classification |
What is ViT-L-16-HTxt-Recap-CLIP?
ViT-L-16-HTxt-Recap-CLIP is a cutting-edge Vision Transformer model that leverages the power of CLIP architecture and LLaMA-3 generated captions. Developed by UCSC-VLAA, this model represents a significant advancement in zero-shot image classification capabilities, trained on the extensive Recap-DataComp-1B dataset.
Implementation Details
The model implements a Vision Transformer (ViT) architecture with a 16x16 patch size and utilizes contrastive learning between image and text pairs. It can be easily integrated using the OpenCLIP library and supports efficient inference with torch.cuda.amp.autocast for mixed precision acceleration.
- Built on Vision Transformer architecture
- Supports context-aware image-text matching
- Implements efficient mixed-precision inference
- Utilizes LLaMA-3 generated captions for training
Core Capabilities
- Zero-shot image classification
- Contrastive image-text learning
- Feature extraction for both images and text
- Normalized embedding space for efficient similarity matching
Frequently Asked Questions
Q: What makes this model unique?
This model stands out due to its training on the Recap-DataComp-1B dataset with LLaMA-3 generated captions, offering improved zero-shot classification capabilities while maintaining efficient processing through its ViT architecture.
Q: What are the recommended use cases?
The model is particularly well-suited for zero-shot image classification tasks, content retrieval, and image-text matching applications where pre-training or fine-tuning on specific categories isn't feasible.
