Qwen2.5 7B Instruct
Qwen2.5's 7B chat model (7.61B parameters) with JSON mode, function calling and up to 32,768 input tokens on Wiro. Qwen reports 75.5 on MATH and 84.8 on HumanEval.
Overview
Qwen released this 7B chat checkpoint with Qwen2.5 in September 2024. It has 7.61B parameters (6.53B outside the embeddings), 28 layers and grouped-query attention with 4 key-value heads. The card's 131,072-token context depends on YaRN scaling, which Wiro does not enable, so a request holds up to 32,768 input tokens.
Against Qwen2-7B-Instruct, Qwen's table shows the largest gains on Arena-Hard (25.0 to 52.0), MATH (52.9 to 75.5) and IFEval strict-prompt (54.7 to 71.2). The same table puts it ahead of Gemma2-9b-IT and Llama3.1-8B-Instruct everywhere except IFEval, where Llama scores 75.9. Qwen also lists better table and JSON handling among the Qwen2.5 changes.
Choosing a variant: start with the 7B for general assistant work. Qwen2.5 32B Instruct scores higher on reasoning and code but takes only about 17,000 input tokens here. Qwen2 7B Instruct is the same size one generation back, and Qwen2.5 3B Instruct, Qwen2.5 1.5B Instruct and Qwen2.5 0.5B Instruct suit narrower jobs.
What you can build
- A support or sales assistant with a persona set in the system prompt.
- Answers grounded in retrieved passages that fit in one request.
- Extraction of rows and fields from tables into a JSON object.
- Small agents that call your functions and answer from the results.
- Multilingual replies: Qwen reports 74.87 on its multilingual IFEval set across eight languages.
Inputs
Use the prompt field for a single message or a messages array for a full conversation, not both. The pre-filled system prompt is a generic safety-minded instruction; Qwen's own examples use the shorter "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."
A user ID and session ID make Wiro remember the conversation. The 32,768-token budget covers system prompt, history, tool definitions and the new message. Over it, the oldest turns are dropped, then the end of the latest message is trimmed without an error. Tool definitions are never trimmed, so if they and the system prompt leave no room for the message, the request fails instead.
For tool use, send up to 128 function definitions, set tool choice (auto, none, required, or one function by name), and allow or block parallel calls (allowed by default). JSON mode, sampling and length controls, stop sequences and a seed complete the form.
Outputs
Plain text by default, a JSON object in JSON mode, tool calls when functions are defined. The sample for the default Eiffel Tower prompt is one paragraph on the tower's planned 20-year life and Eiffel's push to keep it standing.
Recommended settings
- Raise top-k from 0 to 20, lower top-p to 0.8 and use a 1.05 repetition penalty, matching Qwen's generation config.
- For extraction, request a JSON object, name every key you expect and turn sampling off.
Limitations
- Qwen2.5 can write 8K tokens, but with max tokens left at 0 a reply to a prompt of 2,048 tokens or less is limited to 2,048 (a longer prompt gets twice its length). For longer output, set max tokens yourself; it is honoured within the room the input leaves in the window.
- Hard science is a weak area: 36.4 on GPQA.
Safety & compliance
Apache 2.0 covers this checkpoint, so you can ship it in a commercial product and modify it.
API quick start
Run Qwen2.5-7B-Instruct with a single API call.
{
"prompt": "Can you say something interesting about t…",
"user_id": "...",
"session_id": "...",
"system_prompt": "You are a helpful, respectful and honest …"
}curl -X POST "https://api.wiro.ai/v1/Run/qwen/qwen2-5-7b-instruct" \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_WIRO_API_KEY" \
--data-binary @- <<'JSON'
{
"prompt": "Can you say something interesting about t…",
"user_id": "...",
"session_id": "...",
"system_prompt": "You are a helpful, respectful and honest …"
}
JSONOther models you might like
Explore similar llm & chat models.
