Book 02

How a reply is chosen

At each step the model holds a list of likely next words. How it picks from that list is why two replies to the same question can differ.

After this: Say why two runs differ. · 7 chapters · about 40 minutes

1. 01The list of next wordsA percent for every possible token5 min
2. 02The usual pickAlways take the highest percent5 min
3. 03SamplingChoose among the high percents5 min
4. 04How wild the pick isTemperature5 min
5. 05Only the plausible wordsTop-k and top-p5 min
6. 06Why two runs differSame prompt, different landings5 min
7. 07The first word can refuseTone sits on the same list as facts5 min

Book 02 examNext: How meaning is stored