Rewrite · 4 checks · max 400 tokens · benched 2026-09-16

Claude Opus 5 vs Claude Sonnet 5 vs Claude Sonnet 4.6 vs Claude Haiku 4.5 on product update to a 280-character post

Anthropic models side by side on "Product update to a 280-character post": Claude Haiku 4.5 scores 9/10; Claude Haiku 4.5 is the cheapest answer scoring 8+ at $0.30 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

User

Turn this update into a single social media post of at most 280 characters. It must mention "v2.0" and end with the hashtag #OpenSource. Output only the post.

Update: Version 2.0 of our command-line tool is out. It is three times faster on large repositories, adds a built-in diff viewer, and is now licensed under MIT so anyone can contribute.

Rubric for the judge: Punchy post under 280 characters that mentions v2.0, the speed gain, the diff viewer and MIT, ending with #OpenSource.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Claude Opus 5

anthropic/claude-opus-5

9/10

Latency: 1.6s
Cost: $0.00205
Per 1,000: $2.06

121 in · 58 out · 24 words · checks 4/4

max_chars
contains
regex
contains

v2.0 of our CLI is here: 3x faster on large repos, a built-in diff viewer, and now MIT licensed so anyone can contribute. #OpenSource

Judge: Post is 127 characters, mentions v2.0, speed gain (3x faster), diff viewer, MIT licensing, and ends with #OpenSource hashtag as required; only minor issue is 'CLI' instead of 'command-line tool' but this is a reasonable abbreviation for social media.

Claude Sonnet 5

anthropic/claude-sonnet-5

9/10

Latency: 1.1s
Cost: $0.00094
Per 1,000: $0.94

121 in · 70 out · 27 words · checks 4/4

max_chars
contains
regex
contains

🚀 v2.0 of our CLI tool is here! 3x faster on large repos, built-in diff viewer, and now MIT licensed for anyone to contribute. Upgrade today! #OpenSource

Judge: Post is 139 characters (well under 280), mentions v2.0, speed gain, diff viewer, and MIT licensing, ends with #OpenSource, and is punchy and engaging; only minor issue is 'Upgrade today!' adds slight filler rather than pure information.

Claude Sonnet 4.6

anthropic/claude-sonnet-4.6

9/10

Latency: 745ms
Cost: $0.00100
Per 1,000: $1.00

94 in · 48 out · 26 words · checks 4/4

max_chars
contains
regex
contains

🚀 v2.0 of our CLI tool is here! 3x faster on large repos, built-in diff viewer, and now MIT licensed — everyone's welcome to contribute! #OpenSource

Judge: Post is 127 characters (well under 280), mentions v2.0, speed gain, diff viewer, and MIT licensing, ends with #OpenSource, and is punchy and engaging; only minor issue is it doesn't explicitly state 'anyone can contribute' though 'everyone's welcome' conveys similar meaning.

Claude Haiku 4.5

anthropic/claude-haiku-4.5

9/10

top judge scorecheapest scoring 8+

Latency: 696ms
Cost: $0.00030
Per 1,000: $0.30

94 in · 42 out · 20 words · checks 4/4

max_chars
contains
regex
contains

🚀 v2.0 is here! 3x faster on large repos, built-in diff viewer, and now MIT licensed. Everyone can contribute! #OpenSource

Judge: Post is 109 characters (well under 280), mentions v2.0, speed gain, diff viewer, MIT licensing, and ends with #OpenSource as required; only minor issue is it doesn't explicitly state 'anyone can contribute' though 'Everyone can contribute' conveys the same meaning.

Frequently asked

What does this prompt test?

Rewrite: Punchy post under 280 characters that mentions v2.0, the speed gain, the diff viewer and MIT, ending with #OpenSource. The deterministic checks are max_chars, contains, regex, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Claude Haiku 4.5 at $0.30 per 1,000 runs. If you need the top score, Claude Haiku 4.5 at $0.30 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Anthropic with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

Claude Opus 5

Claude Sonnet 5

Claude Sonnet 4.6

Claude Haiku 4.5

Frequently asked

Same prompt, other families

Stop overpaying for AI tools