Model comparisons online are mostly vibes. The playground runs one identical prompt through every model in a family and records what each did, how long it took and what it cost. Learn what that removes and what it cannot.
In this lesson you will
→Say what a 'family' is and why the playground compares within one
→Name the four things every run records
→Know the two things a playground result cannot tell you
Ask ten people which AI model is best and you get ten answers, all from memory, all about different tasks, none with a price attached. The playground replaces that with a boring experiment: the same prompt, character for character, sent to every model in a family, with the answer, the time and the cost written down. Boring is the point. It is the only way the comparison is about the models rather than about the prompts.
Figure 1.A family is one vendor's current line-up (for example Haiku, Sonnet and Opus). Comparing within a family keeps the tokeniser, the style and the API the same, so price and size are the only things that vary.
The four things every run records
Checks passed — hard rules the prompt stated: at most 60 words, valid JSON, must mention the refund, must not contain the card digits. A check is pass or fail, no opinion involved.
Judge score — a separate model reads the answer against a rubric and gives 0 to 10. Useful for tone and completeness; it is an opinion, so read it second.
Cost per run — computed from the tokens the API actually reported, at the catalogue rate. Not an estimate.
Latency — milliseconds from request to answer. Matters more than people think when a task runs a thousand times.
Knowledge check
Why does the playground compare models within one vendor's family rather than mixing vendors in one leaderboard?
Lesson FAQ
▸What is an AI model playground?
A page that sends the same prompt to several models and shows the answers side by side with cost and timing. This site's playground goes further: it records deterministic checks and a judge score for 24 fixed prompts across each vendor's family, so the comparison is repeatable.
▸Why not just trust benchmark scores?
Public benchmarks measure exam-style tasks and say nothing about price or latency. The playground measures everyday tasks (summaries, JSON extraction, small code fixes) with the cost of each run attached, which is closer to what you will actually pay for.
▸How often is the bench re-run?
Each family page shows the date of its bench. The numbers in this course are from the bench of 2026-09-12; re-runs happen when prompts or the catalogue change, so check the date on the page.
Finished reading?
Mark it done to track your progress through the course.
If our calculators helped you cut down on hidden AI wallet leaks, thanks for using them. A tiny fraction of your savings is what keeps our pricing indexes updated daily.
Not sure which AI is cheapest for your use case? Find out in 30 seconds — no signup required.