VideoLLaMA3 2B Image

Overview

VideoLLaMA3-2B-Image is the image checkpoint of VideoLLaMA 3, a vision-language series from DAMO-NLP-SG. Its language model is Qwen2.5-1.5B-Instruct and its vision encoder was initialized from SigLIP. You give it one picture and an instruction, and it answers in English.

In the first training stage the authors fine-tuned the encoder to take images at their own resolution and aspect ratio, so the number of visual tokens follows the size of the picture. Large screenshots and scans therefore keep more of their detail, which matters for small print on receipts and slides. The card's list of training data includes two document sets, PixMo-Docs and Docmatix, next to LLaVA-OneVision.

DAMO-NLP-SG reports 91.9 on DocVQA (test), 79.8 on ChartQA, 69.4 on InfoVQA (test) and 59.2 on MathVista (testmini) for this model. On those four it has the top score among the 2B models in the card's table, which also lists SmolVLM, InternVL2.5 and Qwen2-VL. InternVL2.5 2B is ahead on OCRBench (804 against 779) and on MME.

For video, use VideoLLaMA3 2B. Its card names this checkpoint as its base model, and the authors train it further on video. This page takes one still image per run, so it is the one for photos, charts and scans.

What you can build

• Alt text and captions for photo archives and user uploads
• Answers about scanned invoices and app screenshots, for example the total on a receipt
• Reading a single value or a trend off a chart or table image
• Flagging odd content in a picture, as one of the example prompts on this page does when it asks what is unusual in the image
• Questions about diagrams and math figures in textbooks or homework

Inputs

• One image per run, uploaded or given as a URL.
• A prompt. It starts as "Please describe this image in detail." Replace it with a direct question when you want a short answer.
• Max new tokens, in the advanced settings: 10 to 4096, default 128. Generation stops when the reply reaches this length.

Outputs

Plain text. On this page, a fantasy illustration of a woman in a blue hat and cloak holding a glowing orb gets two sentences, the second noting that the image contains no text. A photo of a laundromat with a person's legs sticking out of a washer door gets two paragraphs.

Recommended settings

128 tokens is enough for one-line answers and short captions. For a detailed description like the two-paragraph laundromat answer, raise the limit to 256 or more. If you want a single value back, say so in the prompt, for example "Answer with the number only."

Limitations

• It can misname large objects and misread poses. In the laundromat sample the model gets the laundry room and the person right, then calls the washer doors ventilation windows and describes the figure as leaning out of a window, when only the legs stick out of the drum.
• The model card lists English as the only language, so write prompts in English.
• This page accepts one image per run, so comparing several images side by side is not possible here.
• A reply that stops mid-sentence has hit the token limit. Raise it and run again.

Safety & compliance

The model card gives the license as Apache 2.0. The project's GitHub page adds that VideoLLaMA 3 is a research preview intended for non-commercial use only, subject to the Qwen model licenses, the terms of use of data generated by OpenAI and Gemini, and ShareGPT's privacy practices. Check those terms before using it in a commercial product. If a photo shows a real person's face, get that person's consent before you upload it.