NVIDIA · Vision–language grounding

LocateAnything 3B

A sentence in, coordinates out. LocateAnything 3B finds objects, UI elements and text inside an image and answers with normalized boxes or points — so natural language becomes something your code can actually act on.

Run the model API documentation

~50 s

Typical run

Boxes or points

Returns

0 – 1000

Coordinate grid

NVIDIA research

Licence

Samples

Grounding, one prompt at a time

Every result below came from a single image plus a short phrase — no category training, no bounding-box annotation pass.

LocateAnything 3B grounding sample — ground_single

What you can build

Six things that need a coordinate, not a caption

LocateAnything pairs a vision encoder with a language model and decodes whole coordinate blocks at once, instead of emitting numbers one token at a time. That makes it a fit wherever a pipeline has to turn words into pixels.

detect

Open-vocabulary detection

Pass your own category list and get every instance found, labelled — no retraining when the taxonomy changes.

ground_single

Referring expressions

“The person in the red jacket” resolves to one box, even in a crowded frame.

gui

GUI grounding

Point an agent at a screenshot and ask for “the search button”. Get a box, or a single click point.

detect_text

Text region localization

Find text regions before OCR so document workflows crop tightly instead of reading the whole page.

point

Pointing for agents & robots

When a policy needs one target coordinate rather than a rectangle, ask for a point.

ground_multi

Annotation assist

Generate first-pass boxes for a dataset, then keep a human in the loop to review and correct.

Task modes

Pick the mode, not the prompt trick

One task field switches what the model is asked to return. Choosing correctly matters more than phrasing the prompt cleverly.

task	What you send	What comes back
`detect`	A comma-separated category list	Every instance of every category, each with a label and a box
`ground_single`	One describing phrase	The single best-matching box
`ground_multi`	One describing phrase	Every box that matches the phrase
`ground_text`	The text you are looking for	A box around that specific text
`detect_text`	Nothing — the image is enough	Boxes for every text region in the frame
`gui`	A UI element description	A box, or a click point when `outputType` is `point`
`point`	An object description	One point marking the object

Integration

Four fields and a task id

Send the run request with your project API key, keep the returned task id, then poll task detail until the outputs are ready.

Served on Wiro · nvidia/locateanything-3b

# Start the run
curl -X POST https://api.wiro.ai/v1/Run/nvidia/locateanything-3b \
  -H "x-api-key: $WIRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputImage": "https://example.com/harbour.jpg",
    "task": "ground_single",
    "prompt": "the largest boat",
    "outputType": "box"
  }'

# Then poll until it finishes
curl -X POST https://api.wiro.ai/v1/Task/Detail \
  -H "x-api-key: $WIRO_API_KEY" \
  -d '{"tasktoken": "…", "taskid": "…"}'

# Coordinates arrive on a 0–1000 grid.
# Scale them by the image size to get pixels:

x_px = x / 1000 * image_width
y_px = y / 1000 * image_height

# A box is 4 integers, a point is 2.
# When nothing matches, the model returns
# an explicit "none" marker instead of a
# guessed region — check for it before you
# draw, crop or click.

Before you ship

What it will not do for you

Worth reading once, so the failure modes turn up in review rather than in production.

Coordinates are quantizedVery small objects can need a higher-resolution input to keep boxes tight on the 0–1000 grid.
Ambiguity is realPrompts like “the biggest one” fail when the size cue is unclear. Describe the target, don't rank it.
Dense scenes drop instancesHeavy overlap causes missed matches and occasionally mixed labels.
Validate the tokensFast structured decoding can emit a malformed location token. Check tags and coordinate counts before parsing.
Input quality carries throughBlur, heavy compression, glare and scan artefacts all degrade box quality.
Research licenceReleased under an NVIDIA research licence — intended for research and development, not unrestricted commercial deployment.

Try it

Run it on your own image

Open the playground, drop in a screenshot or a photo, and watch a phrase turn into coordinates. No setup, no GPU to provision.

Open the playground Browse every model