Book 01
How language models work
A reply is built one at a time. lets every word use every other word in the same step. Later scores change which get picked.
After this: See the next-word loop and attention. · 9 chapters · about 45 minutes
- 01IntroductionHow we got here6 min
- 02Why older models were slowSequential processing5 min
- 03What attention meansWeighted connections between tokens6 min
- 04Query, key, and valueHow one attention step is computed6 min
- 05Multi-head attentionSeveral attention operations in parallel5 min
- 06Encoder and decoderMasking, and chat models today6 min
- 07Why word order mattersPositional encoding5 min
- 08How the model generates textNext-token prediction6 min
- 09How annotators train the modelHow labels become training7 min