Llama 3.2 3B Instruct

Overview

Llama 3.2 3B Instruct is the larger of the two text models in Meta's September 2024 Llama 3.2 release, next to a 1B. Meta built it by pruning and distillation, feeding logits from Llama 3.1 8B and 70B in as token-level targets during pretraining on up to 9 trillion tokens. Alignment then ran in rounds of supervised fine-tuning, rejection sampling and Direct Preference Optimization. The model has 3.21B parameters, a December 2023 knowledge cutoff and eight supported languages.

Meta aims it at chat, knowledge retrieval, summarization and query or prompt rewriting, and reports 63.4 on 5-shot MMLU, 77.4 on IFEval and 67.0 on the BFCL V2 tool-use test.

On Wiro a request takes in about 51,200 tokens, counting tool definitions, system message, history and prompt; the checkpoint's 131,072-token window is not available. Longer input is trimmed, oldest history first and then the tail of the latest message. Tool definitions are never cut, so if they and the system message leave no room for your message, the run fails with a context-length error. Leave the output limit at 0 and replies stop at 2,048 tokens, as long as the prompt is no longer than that.

Choosing a variant: pick this one for tool calling and JSON, or when a small model is enough. Llama 3.1 8B Instruct is the bigger multilingual chat model, without tool or JSON inputs. The Llama 2 pair is older and English only: Llama 2 7B Chat for dialogue, Llama 2 13B for text continuation.

What you can build

• Agents that pick one of your functions and return its arguments
• Field extraction from emails or tickets into a JSON object
• Summaries of notes or threads, and query rewriting in front of a search index

Inputs

Send either one prompt or a messages array with the whole conversation, never both. Stored history follows the session ID: send the same one on each turn to continue a thread.

Tools: up to 128 definitions in the OpenAI style, function type only. Tool choice takes auto, none, required or one named function, and parallel calls are on by default. Tool results go back in a follow-up run, each matched to its call ID.

The response format is plain text or a JSON object; schema-constrained output is not offered. Sampling controls, a seed (123456 unless you change it) and quantization, on by default, complete the list.

Outputs

Text, tool calls or a JSON object. The sample answer numbers six events with bold titles, then adds four bulleted story concepts.

Recommended settings

• Use a low temperature, or turn sampling off, when the model fills tool arguments or JSON fields.
• Keep tool descriptions short; they come out of the same input allowance.

Limitations

• It makes factual slips. The sample says Mary, Queen of Scots was held in the Tower of London, which she never was.
• Meta says the model should run inside a system with extra safeguards, especially for agents, and suggests the lighter Llama Guard 3-1B for constrained setups.

Safety & compliance

License: the Llama 3.2 Community License, which Meta calls a custom commercial license agreement. Companies past 700 million monthly active users at release must ask Meta for a separate license. The Acceptable Use Policy rules out uses such as fraud and malware, and languages beyond the eight supported ones are out of scope.