DeepSeek R1 Distill Qwen 7B

Overview

DeepSeek made this checkpoint from Qwen2.5-Math-7B, using the 800k samples it curated with DeepSeek-R1 as training data. Training was supervised fine-tuning only, so its long chains of thought imitate R1's answers.

For its size it is strong on math. DeepSeek's table gives it 92.8 on MATH-500, above the 90.0 listed for OpenAI o1-mini, and 55.5 pass@1 on AIME 2024. On GPQA Diamond it gets 49.1, next to GPT-4o's 49.9. Code is the weak side: 37.6 on LiveCodeBench, 1189 on Codeforces.

Choosing a variant: DeepSeek R1 Distill Llama 8B has about the same parameter count but starts from general-purpose Llama 3.1. DeepSeek's results put this 7B ahead on AIME and MATH-500, with the Llama model slightly ahead on LiveCodeBench and Codeforces. DeepSeek R1 Distill Qwen 1.5B applies the same training to the 1.5B math base and scores lower everywhere. When code or graduate-level science matters, DeepSeek R1 Distill Qwen 14B is the next step up, and DeepSeek R1 Distill Qwen 32B is the strongest Qwen distill.

What you can build

• Math tutoring features where a learner reads every line of a worked solution, up to competition problems such as AIME.
• Grading aids that solve a submitted problem and flag answers that disagree.
• Self-consistency setups that sample one question several times and keep the most common answer.

Inputs

The prompt is mandatory. Everything goes in it, answer format included, since DeepSeek's guidance for R1 models is to keep instructions in the user turn. The optional system prompt arrives filled with a stock assistant line; clear it.

Multi-turn chat works when each request carries the same session ID together with a user ID. Wiro stores the history per session.

Temperature defaults to 0.7 (0 to 5 allowed) and top-p to 0.95. Other controls: top-k, the repetition and length penalties, token counts, stop sequences, a seed, a sampling switch and quantization, which starts enabled.

Outputs

Text comes back in two parts. The reasoning, which the model writes between think tags, is returned as its own thinking segment, followed by the answer.

Recommended settings

• Lower temperature from the 0.7 default to 0.6. DeepSeek advises 0.5 to 0.7 for this series so the output does not loop or turn incoherent.
• On math, have the model work step by step and box the final answer with \boxed{}, following DeepSeek's advice.
• When accuracy matters, run the question several times and compare answers. In DeepSeek's AIME 2024 results, majority voting over 64 samples took this model from 55.5 to 83.3.
• Set max tokens generously so the reasoning can finish.

Limitations

• On LiveCodeBench it sits about 15 points below the 14B distill (37.6 vs 53.1), so code-heavy work belongs on a larger model.
• R1-series models sometimes return an empty think block and answer without reasoning; DeepSeek reports this hurts accuracy.
• Per DeepSeek, R1 handles Chinese and English best, so prompts in other languages may come back in mixed languages.
• Few-shot prompting degrades R1, DeepSeek says; use zero-shot prompts with a clear output format.

Safety & compliance

DeepSeek licenses the weights under MIT, which allows commercial use and derivative works. Its base, Qwen2.5-Math-7B, belongs to a series originally licensed under Apache 2.0.