Rewrite · 4 checks · max 400 tokens · benched 2026-09-16

Grok 4.6 vs Grok 4.5 vs Grok 4.20 vs Grok 4.3 on product update to a 280-character post

xAI models side by side on "Product update to a 280-character post": Grok 4.20 scores 9/10; Grok 4.20 is the cheapest answer scoring 8+ at $1.71 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

User

Turn this update into a single social media post of at most 280 characters. It must mention "v2.0" and end with the hashtag #OpenSource. Output only the post.

Update: Version 2.0 of our command-line tool is out. It is three times faster on large repositories, adds a built-in diff viewer, and is now licensed under MIT so anyone can contribute.

Rubric for the judge: Punchy post under 280 characters that mentions v2.0, the speed gain, the diff viewer and MIT, ending with #OpenSource.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Grok 4.6

x-ai/grok-4.6

9/10

Latency: 528ms
Cost: $0.00315
Per 1,000: $3.15

290 in · 460 out (424 reasoning) · 21 words · checks 4/4

max_chars
contains
regex
contains

Our CLI tool v2.0 is out! 3x faster on large repos, built-in diff viewer, now MIT-licensed so anyone can contribute. #OpenSource

Judge: Post is 119 characters, mentions v2.0, speed gain, diff viewer, MIT licensing, and ends with #OpenSource as required; only minor issue is it doesn't explicitly state 'anyone can contribute' though it's implied by 'MIT-licensed so anyone can contribute.'

Grok 4.5

x-ai/grok-4.5

9/10

Latency: 448ms
Cost: $0.00194
Per 1,000: $1.94

290 in · 263 out (225 reasoning) · 23 words · checks 4/4

max_chars
contains
regex
contains

v2.0 of our CLI tool is out! 3x faster on large repos, adds built-in diff viewer, now MIT-licensed so anyone can contribute. #OpenSource

Judge: Post is 119 characters, mentions v2.0, speed gain, diff viewer, MIT licensing, and ends with #OpenSource as required; only minor issue is 'CLI tool' is slightly less descriptive than 'command-line tool' but still clear and punchy.

Grok 4.20

x-ai/grok-4.20

9/10

top judge scorecheapest scoring 8+

Latency: 401ms
Cost: $0.00171
Per 1,000: $1.71

267 in · 604 out (567 reasoning) · 24 words · checks 4/4

max_chars
contains
regex
contains

v2.0 of our command-line tool is out: 3x faster on large repos, built-in diff viewer, and now MIT licensed so anyone can contribute! #OpenSource

Judge: Post is 127 characters, mentions v2.0, speed gain, diff viewer, MIT licensing, and ends with #OpenSource as required; only minor issue is not explicitly stating 'anyone can contribute' is due to MIT, though it's implied.

Grok 4.3

x-ai/grok-4.3

9/10

Latency: 520ms
Cost: $0.00198
Per 1,000: $1.98

273 in · 711 out (677 reasoning) · 22 words · checks 4/4

max_chars
contains
regex
contains

CLI tool v2.0 is here: 3x faster on large repos, adds built-in diff viewer, now MIT licensed so anyone can contribute! #OpenSource

Frequently asked

What does this prompt test?

Rewrite: Punchy post under 280 characters that mentions v2.0, the speed gain, the diff viewer and MIT, ending with #OpenSource. The deterministic checks are max_chars, contains, regex, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Grok 4.20 at $1.71 per 1,000 runs. If you need the top score, Grok 4.20 at $1.71 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on xAI with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

Grok 4.6

Grok 4.5

Grok 4.20

Grok 4.3

Frequently asked

Same prompt, other families

Stop overpaying for AI tools