Llama 2 7B Chat
Meta's 7B Llama 2 chat model, aligned for English dialogue with supervised fine-tuning and RLHF. Text in, text out, with a 4k-token context window.
Overview
Llama 2 7B Chat is the dialogue-tuned 7B checkpoint from Meta's Llama 2 release, which shipped 7B, 13B and 70B sizes, each in a pretrained and a chat version. Meta trained the family between January and July 2023. Pretraining used 2 trillion tokens of public data, and the chat models were then aligned with supervised fine-tuning and reinforcement learning from human feedback (RLHF). It takes text and returns text, with a 4k-token context, and Meta scopes it to English.
For this checkpoint Meta reports 57.04 on TruthfulQA (share of answers both truthful and informative) and 0.00 on ToxiGen (share of toxic generations, lower is better).
Choosing a variant: this is the only chat-tuned Llama 2 here, for a small English assistant with Meta's safety tuning. Llama 2 13B is bigger but has no chat tuning, so it suits text continuation. Llama 3.1 8B Instruct is a newer generation that answers in eight languages. Llama 3.2 3B Instruct is the one to pick when you need function tools or JSON output.
What you can build
- English FAQ or help-desk bots whose persona and refusal rules live in the system prompt
- Brainstorming helpers: the page sample turns one question about the Tower of London into ten story hooks
Inputs
You send a prompt. The system prompt in front of it comes prefilled with about 80 words of safety instructions, which tell the model to avoid harmful content and to admit when it does not know an answer.
To continue a conversation, send the same session ID on every turn (a user ID is optional), and Wiro stores and recalls that history. With the session ID left empty, each run starts a new conversation, which suits one-off questions.
You can also tune temperature (0.7 by default, up to 5), top-p, top-k, repetition and length penalties, token limits, semicolon-separated stop sequences and a seed. Two switches, sampling and quantization, are both on by default.
Outputs
Plain text. The sample reply is a numbered list of ten items, two sentences each, followed by a short wrap-up paragraph.
Recommended settings
- Shorten or replace the default system prompt when space is tight. It shares the 4k window with the stored history, your prompt and the reply.
- Turn sampling off for extraction or classification, where you want the most likely wording. Leave it on near 0.7 for brainstorming lists.
- Quantization lowers memory use and might slightly affect output quality; switch it off to run at full precision.
Limitations
- The 4k context rules out long documents and long chat histories.
- Pretraining data stops at September 2022; some tuning data runs to July 2023.
- It can state wrong facts with confidence. The sample dates the Ceremony of the Keys to 1871, though the nightly ceremony is centuries older.
- Meta tested it in English only and says its outputs cannot be predicted in advance.
Safety & compliance
Released under the Llama 2 Community License, a custom license from Meta that allows commercial and research use. Out of scope: uses that break laws or trade compliance rules, uses barred by Meta's Acceptable Use Policy, and languages other than English. Before deploying, Meta asks developers to run safety testing and tuning for their own application.
API quick start
Run Llama-2-7b-chat-hf with a single API call.
{
"prompt": "What are some interesting historical even…",
"user_id": "...",
"session_id": "...",
"system_prompt": "You are a helpful, respectful and honest …"
}curl -X POST "https://api.wiro.ai/v1/Run/meta-llama/llama-2-7b-chat-hf" \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_WIRO_API_KEY" \
--data-binary @- <<'JSON'
{
"prompt": "What are some interesting historical even…",
"user_id": "...",
"session_id": "...",
"system_prompt": "You are a helpful, respectful and honest …"
}
JSONOther models you might like
Explore similar llm & chat models.
