Book 03

How meaning is stored

The model stores a word as a point among other points. Nearby points are related. Attention mixes those points when it reads a sentence.

After this: Treat a word as a pin on a map. · 7 chapters · about 35 minutes

1. 01A word is a pin on a mapNot the spelling — a list of numbers5 min
2. 02Nearby is relatedClusters — the right neighborhood, the wrong house5 min
3. 03Sliding across the mapSome relations are directions5 min
4. 04One spelling, two neighborhoods“Bank” needs the rest of the sentence5 min
5. 05Attention is mixing pinsQuestion, tag, copy5 min
6. 06Longer lists, louder matchesWhy the scores get shrunk5 min
7. 07The wrong neighborFluent, and still pointing at the dog5 min

Book 03 examNext: How the model is trained