02Book 04 · Chapter 02 of 07· 5 min · Fluent — and still sure Sydney is the capital
What the pot leaves behind
After , the model already writes fluent sentences. It also repeats whatever was common, including popular mix-ups.
A model already sounds like language. It can finish a proverb, write a function, mimic a news lede, and tell you about Hugh Jackman with alarming confidence.
It also finishes patterns that were common and wrong. Travel blogs name Sydney constantly. So “What is the capital of Australia?” can put Sydney on top of the wheel. The sentence sounds great. Sydney is a nearby city, so it sits close to Canberra on the stored map — close enough to win the next-word list even though it is the wrong capital.
A product chat still needs later stages so the usual pick is a useful reply to a request — not only a plausible continuation based on what “the Internet would say.”
What is the capital of Australia?
- Sydney54%
- Canberra28%
- Melbourne12%
- a kangaroo6%
Fluent and sure: Sydney. The travel blogs won this spin. Fluency was never the same thing as the true pin.
Quiz
6 questions
Before moving forward, test your knowledge of what you’ve just learned!
1.After pretraining the model can already:
2.Common false claims in the first text:
3.A useful chat reply is:
4.Stereotypes in pretraining text:
5.This is why later training exists:
6.Fluency after pretraining means:
0 of 6 answered
2 / 7