Skip to content
Home: How language models work

06Chapter 06 of 09· 6 min · , and chat models today

and

The first had two parts: an and a . is the rule: each new word is mixed only from the input and the words already written.

The first model was built for translation. The read the source sentence. The wrote the target sentence one word at a time. It used the result and the words it had already written.

is a rule on

At each step, on words that have not been written yet is set to 0%. Those words add nothing this step. The current word is mixed from the result and the words already written.

During training, the full target sentence is already there. makes training match writing: each prediction is made only from what came before, which is how the model writes when it produces a reply.

A small translation. Cells marked used can add to the mix. Cells marked have 0% and add nothing this step.

· input

thecat

Available at every step.

· output

le· · ·

Writing “le”.

Mask grid. Rows are decoder steps. Columns are tokens attention can use.
WritingEncoderDecoder
thecatlechat
leusedusedmaskedmasked
chatusedusedusedmasked

Writing “le”. can use “the” and “cat”. “le” and “chat” have 0% and add none of the mix.

Chat models you use today are mostly -only. The prompt and the reply are one . The same rule applies: when the model writes the 12th word, the 13th word is still unwritten, so its share is 0%.

Quiz

9 questions

Before moving forward, test your knowledge of what you’ve just learned!

  1. 1.Why mask future tokens in the decoder?

  2. 2.A typical chat model is mostly:

  3. 3.The encoder:

  4. 4.The decoder:

  5. 5.A weight of 0% in masking means:

  6. 6.During training the full target sequence is present. Masking:

  7. 7.When writing “chat” in the translation “the cat” → “le chat”, attention can use:

  8. 8.When the model writes the 12th word of a reply:

  9. 9.A demonstration teaches the decoder:

0 of 9 answered

6 / 9