Skip to content
Home: How language models work

03Book 02 · Chapter 03 of 07· 5 min · Choose among the high percents

means picking one next word from the list of percents. A 62% is picked often. An 18% is picked sometimes. Two runs can differ.

means: imagine the possible next words for “The capital of Australia is....” laid out as a pie chart. Glue that pie to a wheel and spin it. Canberra’s 62% slice is large, so it comes up more often. Sydney’s 18% slice still wins some of the time. The model writes whatever the wheel lands on. A large slice is usual because that word looked usual in training text — not because the model looked up a school fact.

That is how a chat can answer the same question two ways. Both ways were on the list. The wheel landed differently.

does not include or invent a that has 0% on this list. A word with no share has no slice on the wheel, so it cannot be the next word this step.

is a spin of that pie. A 62% slice comes up often. An 18% slice still wins sometimes. Spin it.

The capital of Australia is....

  • Canberra62%
  • Sydney18%
  • Melbourne9%
  • a6%
  • The5%

Quiz

6 questions

Before moving forward, test your knowledge of what you’ve just learned!

  1. 1.Sampling picks the next token by:

  2. 2.Two sampled runs of the same prompt can differ because:

  3. 3.A token at 0% in sampling:

  4. 4.Sampling is the reason a chat can feel:

  5. 5.18% for Sydney means:

  6. 6.The list sampling uses comes from:

0 of 6 answered

3 / 7