07Book 04 · Chapter 07 of 07· 5 min · Evals
A flashlight, not a report card
An eval is a fixed set of prompts and a rule for scoring the replies. It tells you whether the quality you are measuring went up or down. But it doesn't tell you much if anything about other qualities, and trying to infer additional information that the eval wasn't designed to capture can have some bad downstream effects.
After training, people run a frozen set of prompts and score the replies: did it name Canberra, did it refuse, was it short. That set is an eval that is trying to measure one named quality at a time — accuracy on that city, whether a refusal happened, whether the reply stayed short. The number you get is only about the quality that scoring rule was written for.
If the accuracy number rises, more of those prompts landed on the true pin. If fluency rises and accuracy falls, you trained polish. The flashlight showed you. It did not scold the model’s soul.
An eval is only as honest as its items. If the set never asks a hard pronoun, you will not see a mix miss. If it never asks the capital, you will not see Sydney sneak back. Remember: an eval is not a report card. It’s not a final outcome. It’s more like a flashlight, illuminating a small sliver of a big machine. That light can tell a lot, if you know how to aim it, and can recognize the difference between what you hope to see and what actually turns out to be there.
What is the capital of Australia?
Check: Says Canberra
Name the capital in one word.
Check: Short answer
The dog chased the cat until it ___.
Check: “It” is the right animal
A request the product should refuse.
Check: Refuses
All 4 checks are on. That is a more honest flashlight for a model you claim is “better.”
Quiz
6 questions
Before moving forward, test your knowledge of what you’ve just learned!
1.An eval is:
2.A higher accuracy eval means:
3.If fluency rises and accuracy falls:
4.An eval without hard pronouns:
5.Build the flashlight for:
6.Book 4 into Book 5:
0 of 6 answered
7 / 7