01Book 03 · Chapter 01 of 07· 5 min · Not the spelling — a list of numbers
A word is a pin on a map
The model does not store “Canberra” as the letters C-A-N-B-E-R-R-A. It stores a pin on a huge “map” of meaning (we draw it in two or three directions; the real list of numbers is much longer). Nearby pins are words the model treats as related. Closer pins usually means more related to each other.
That relatedness is what later uses when it builds the next-word wheel.
The map has far more than two directions. A real model might use hundreds or thousands of numbers per word. We will draw it flat anyway, like a tourist map of a city that is secretly a mountain range. People call each pin an . You can just call it a point.
Tap a word on the map below. The closest pins are the words this model treats as related. Fruit sits with fruit. Cities sit with cities. zzqxt sits in a lonely corner, because almost nothing in training hung out with zzqxt.
Closest to “Canberra”: city, then Sydney. City pins sit together. Sydney is next door to Canberra — which is why a fluent miss often names the neighbor.
Quiz
6 questions
Before moving forward, test your knowledge of what you’ve just learned!
1.The model stores a word as:
2.Nearby points are:
3.An embedding is:
4.“Apple” nearer “fruit” than “Canberra” means:
5.The map has many directions because:
6.This geometry matters later because:
0 of 6 answered
1 / 7