SketchVideo
| Property | Value |
|---|---|
| Model Type | Video Diffusion Model |
| Base Model | CogVideo-2b (720x480) |
| License | Apache-2.0 |
| Paper | Research Paper |
What is SketchVideo?
SketchVideo is an innovative AI model designed for sketch-based video generation and editing. It enables users to create and modify video content using a combination of text prompts and keyframe sketches. The model specializes in generating short video clips of approximately 6 seconds at 8 frames per second, with a resolution of 720x480 pixels.
Implementation Details
Built upon the CogVideo-2b architecture, SketchVideo generates 49 video frames while maintaining high-quality spatial and motion control. The model accepts one or two keyframe sketches at arbitrary time points, allowing for precise control over the video's content and movement.
- Generates 720x480 resolution videos
- Produces 49 frames at 8 FPS
- Supports both single and dual keyframe sketch inputs
- Enables fine-grained editing of real or synthetic videos
Core Capabilities
- Text-prompted video generation
- Sketch-based spatial control
- Motion control through keyframe sketches
- Video editing functionality
- Real-time processing of arbitrary timepoint inputs
Frequently Asked Questions
Q: What makes this model unique?
SketchVideo stands out for its ability to combine text prompts with sketch-based controls, offering unprecedented precision in video generation and editing. The model's dual-keyframe capability allows for complex motion control, while maintaining consistency across generated content.
Q: What are the recommended use cases?
The model is ideal for creative professionals and researchers who need to generate or edit short video clips with specific visual requirements. It's particularly useful for prototyping animations, creating concept videos, and making quick edits to existing footage. The model is available for research purposes under the Apache-2.0 license.
