VideoLLaMA3-7B-Image

Overview

This is the image checkpoint of VideoLLaMA 3, published by DAMO-NLP-SG, Alibaba DAMO Academy's Language Technology Lab. Send one picture with an instruction and it returns a caption, a direct answer about the scene, or a reading of a chart or document. Qwen2.5-7B handles the language side, and the vision side is an encoder derived from SigLIP.

In the first of four training stages, that encoder was adapted to accept images of varying sizes, so a large picture turns into more vision tokens than a small one, which the paper says helps it capture fine-grained detail. The alignment stage after that mixes scene photos, documents and charts with text-only data. On Hugging Face the checkpoint is tagged for visual question answering.

Pick this page for stills. VideoLLaMA3-7B, whose card lists this checkpoint among its base models, is the one that takes video.

What you can build

DAMO-NLP-SG's notebooks for VideoLLaMA 3 show general image understanding, chart analysis, table understanding, document recognition and visual code analysis. Product ideas along those lines:

• Alt text and captions for product, editorial or user-uploaded photos
• Photo questions in support apps, like the card's own example asking what a woman in a picture is wearing
• Text summaries of charts, tables and scanned pages
• Checks that ask what looks odd in an image, the kind of reply shown in the washing machine sample

Inputs

Each request takes one image, uploaded or given by URL, and a prompt (the preset one is "Please describe this image in detail."). An advanced field caps the reply length anywhere from 10 to 4096 new tokens, starting at 128.

Outputs

Text only. One sample reads a fantasy portrait as a blonde woman in a wide-brimmed witch's hat and a blue cloak with gold trim, holding a glowing orb against a deep blue background. Another spots a person's legs sticking out of an open washing machine door in a laundry room.

Recommended settings

Both sample replies run past 100 words, while the default cap of 128 tokens suits short answers. Raise it well above the default for full descriptions and leave it alone for one-line replies.

Limitations

• The card names English as its one language, so write prompts in English.
• The VideoLLaMA 3 notebooks also show multi-image comparison, which this page does not offer.
• Open prompts invite commentary. In the washing machine reply, two of three paragraphs are interpretation and mood, so ask a narrower question when you only want facts.

Safety & compliance

Apache 2.0 is the license on this checkpoint's card. Before commercial use, read the license note in the VideoLLaMA 3 repository: it calls the service a non-commercial research preview and ties it to the Qwen model licenses, OpenAI and Gemini terms for generated data, and ShareGPT's privacy practices. Photos of identifiable people need their consent.