01Book 02 · Chapter 01 of 07· 5 min · A percent for every possible
The list of next words
Before the model writes the next word, it gives every possible next piece of text a percent. Those percents add up to 100%.
Book 1 showed how a reply is built. The model looks at the words already on the page, chooses the next small piece of text, adds it, and does that again. This book zooms in on that choice — the moment before the next piece is written.
Suppose the words so far are “The capital of Australia is....” The model does not hold only one guess. It gives a percent to every next word it knows — tens of thousands of them. Those percents add up to 100%. Canberra might get 62%, so it is the usual next word. Sydney might get 18%, so it is a real runner-up. The leftover 20% is split across every other possible next word, most of them with a tiny share.
Those percents are called a distribution: a list of chances that add up to 100%. The next chapters are about how one word is selected from that list, and used by the model as the next word.
The capital of Australia is....
█ next word — a list, not one guess
- Canberra62%
- Sydney18%
- everything else20%
Adds up to 100%
Canberra is usual. Sydney is a real runner-up. The leftover 20% is not empty — it is thousands of other words sharing what is left.
Quiz
6 questions
Before moving forward, test your knowledge of what you’ve just learned!
1.Right before the model writes the next word, it holds:
2.If Canberra is 62% and Sydney is 18%, the remaining tokens share:
3.A distribution in this guide is:
4.The writing loop from Book 1 still applies because:
5.Tens of thousands of tokens on the list means:
6.The next chapters are about:
0 of 6 answered
1 / 7