05Chapter 05 of 09· 5 min · Several operations in parallel
One step mixes all the information from all the text into a single result. runs that same step several times at the same time. Each run uses different settings for what to pay to. Then it combines those results.
A single step makes only one mix. Grammar, who a word like “she” points to, and the tone of words all go into that one mix, so they can get in each other’s way. runs several copies of the step at the same time. Each copy has its own settings for what to look at. Then it joins the results.
Many designs use eight copies. That number is simply a design choice and can be altered. What matters is that the different copies can focus on different patterns. A reply can follow grammar rules and still be false. Those are different patterns.
This puts a large share of on “she” as the subject after a verb of saying.
Quiz
6 questions
Before moving forward, test your knowledge of what you’ve just learned!
1.Why use several attention heads instead of one?
2.If everything went through one attention mix:
3.A common design uses eight heads. That number is:
4.Grammar, who a pronoun refers to, and tone:
5.After several heads run, the model:
6.A reply can be grammatical and still false. That is:
0 of 6 answered
5 / 9