Llama 2 13B

Overview

Llama 2 13B is the mid-size pretrained base model of Meta's Llama 2 family, in its Hugging Face Transformers conversion. It was pretrained on 2 trillion tokens from publicly available sources, with a data cutoff of September 2022, and received no supervised fine-tuning or RLHF. The model treats your input as the start of a document and writes what is likely to come next, within a 4k-token context.

Meta offers its pretrained Llama 2 models as a base to adapt for English text generation. For the 13B checkpoint it lists 54.8 on MMLU, 66.9 on a commonsense reasoning average and 65.8 on a reading comprehension average.

Choosing a variant: the 13B is the base model of the group and fits jobs where text should simply be continued. For an English assistant that answers questions, Llama 2 7B Chat carries Meta's dialogue tuning. Llama 3.2 3B Instruct is the smallest and the only one here with function tools and JSON mode, while Llama 3.1 8B Instruct follows instructions in German, Hindi, Thai and five other languages.

What you can build

• Fiction drafting from an opening paragraph; the page sample carries a Westminster Bridge scene forward with the same character.
• Pattern completion: write three or four examples of a format, such as product blurbs or headlines, and leave the last one open.
• Autocomplete in writing tools, where a person keeps or discards each suggestion.

Inputs

The prompt is the text to continue, so write it as the opening of the document you want. The page default is a two-sentence story opening.

The form also has a system prompt field, prefilled with "Please, complete my short story.", but this page runs the model as plain completion: only your prompt reaches it, so put any framing at the top of the prompt itself.

Decoding fields cover temperature (default 0.7), top-p and top-k filtering, penalties for repetition and for length, minimum and maximum token counts, stop sequences, a seed and quantization. User and session IDs are accepted, but earlier turns are not fed back to this base model, so each run continues only the text you send.

Outputs

You get raw continuation text. The sample stays in the past tense and third person of its opening and ends mid-sentence.

Recommended settings

• Set a maximum token count. A base model has no sense of when a piece is finished, and the sample kept going until its token budget ran out.
• Raise the repetition penalty a little above the default 1.0 if you see loops. The sample repeats one sentence more than 30 times in a row.
• For short completions, use the label that starts each of your examples (for instance "Headline:") as a stop sequence.

Limitations

• It does not follow instructions or hold a conversation. A question may be continued as if it were part of a story.
• Long continuations can echo training data: the sample borrows character names and plot points from a published spy novel series. Review text before you publish it.
• On ToxiGen, Meta measured toxic text in 26.10 percent of this model's generations, so filter output before end users see it.

Safety & compliance

The Llama 2 Community License, written by Meta for this release, permits commercial work as well as research. Illegal uses, anything the Acceptable Use Policy forbids and any language besides English fall outside the intended use.