Meta Llama 3 8B Instruct

Overview

Meta Llama 3 8B Instruct is the smaller instruction-tuned model of the Llama 3 family, released by Meta on April 18, 2024 alongside a 70B version. It is a transformer with grouped-query attention, pretrained on more than 15 trillion tokens of publicly available data, then aligned for dialogue with supervised fine-tuning and RLHF on public instruction datasets and over 10 million human-annotated examples.

The model reads text and writes text and code. Its context window is 8k tokens. Meta reports 68.4 on MMLU (5-shot), 62.2 on HumanEval (0-shot) and 79.6 on GSM-8K (8-shot, CoT) for this model.

What you can build

• English chat assistants and support bots, the use Meta tuned it for.
• Answers drawn from your own documents: paste retrieved passages into the prompt and tell the model to stay within them.
• Coding helpers that explain snippets or write small functions.
• Drafting tools for emails, summaries and story ideas. The sample on this page answers a question about the Tower of London with four events and legends from its history, each paired with a story idea.

Inputs

Only the prompt is required. The system prompt is prepended to it to set the model's role and rules; the default asks for helpful, respectful and honest answers and can be replaced.

A user_id stores and recalls chat history for that user, and a session_id splits one user's history into separate conversations. Both can be left blank.

Advanced settings cover temperature (default 0.7, range 0 to 5), top_p (0.95), top_k (0 to 100), repetition and length penalties, minimum and maximum tokens, semicolon-separated stop sequences and a seed (default 123456). Sampling (do_sample) is on by default. So is quantization, which loads the weights at lower precision to save memory and may slightly affect quality; switch it off for full precision.

Outputs

A plain text reply, which can include numbered lists or code.

Recommended settings

• Turn do_sample off for code and other precise tasks, so the same prompt returns the same reply.
• Keep sampling on for creative writing. Meta's own generation examples use temperature 0.6 and top_p 0.9.
• Set max_tokens and stop sequences to keep replies short.

Limitations

• Meta intends the model for English and treats other languages as out of scope unless you fine-tune for them.
• The system prompt, the prompt and the reply share the 8k-token window, and recalled chat history takes space too, so split long documents.
• Knowledge stops at March 2023. Put recent facts in the prompt.
• Meta warns that outputs can be inaccurate or biased. The sample reply here writes "Richard of Shewsbury" for Richard of Shrewsbury, so check names and facts before publishing.

Safety & compliance

The model is released under the Meta Llama 3 Community License, which permits commercial and research use under Meta's Acceptable Use Policy. The license forbids using the model or its outputs to improve any other large language model, except Llama 3 and its derivatives. Anyone distributing the model, or a product or service that uses it, must prominently display "Built with Meta Llama 3", and companies whose products or services had more than 700 million monthly active users when Llama 3 was released must request a separate license from Meta, which Meta can grant or refuse. Meta recommends safety testing for your own application and adding safeguards such as Llama Guard 2 and Code Shield.