Summarise · 3 checks · max 400 tokens · benched 2026-09-16

DeepSeek V4 Pro vs DeepSeek V4 Flash 0423 vs DeepSeek V4.1 Flash on one-sentence changelog summary

DeepSeek models side by side on "One-sentence changelog summary": DeepSeek V4 Flash 0423 scores 9/10; DeepSeek V4 Flash 0423 is the cheapest answer scoring 8+ at $0.05 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

User

Summarise this release note in one sentence of at most 30 words. Output only the sentence.

Release 4.7.0: The public API now enforces a rate limit of 600 requests per minute per key (previously unlimited). Requests over the limit return HTTP 429 with a Retry-After header. Webhook deliveries are now signed with HMAC-SHA256 and the old unsigned mode will be removed in 4.9. We also fixed a bug where exporting more than 10,000 rows produced an empty file.

Rubric for the judge: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

DeepSeek V4 Pro

deepseek/deepseek-v4-pro

9/10

Latency: 1.3s
Cost: $0.00053
Per 1,000: $0.53

194 in · 237 out (193 reasoning) · 23 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600 requests/minute/key API rate limit with 429 responses, signs webhooks via HMAC-SHA256, and fixes empty exports over 10,000 rows.

Judge: Accurate single sentence under 30 words (24 words), prioritizes rate limit and signed webhooks as required, no preamble or bullet list, minor: could emphasize deprecation of unsigned mode more explicitly.

DeepSeek V4 Flash 0423

deepseek/deepseek-v4-flash

9/10

top judge scorecheapest scoring 8+

Latency: 534ms
Cost: $4.9e-5
Per 1,000: $0.05

115 in · 312 out (270 reasoning) · 16 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600 request/min rate limit, HMAC-SHA256 webhook signing, and fixes an export bug.

Judge: Accurate single sentence under 30 words (16 words), prioritizes rate limit and webhook signing as required, no preamble or bullets, though could have mentioned the deprecation timeline for completeness.

DeepSeek V4.1 Flash

deepseek/deepseek-v4.1-flash

9/10

Latency: 8.5s
Cost: $0.00023
Per 1,000: $0.23

141 in · 347 out (297 reasoning) · 26 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600/min API rate limit with HTTP 429 and Retry-After, signs webhooks with HMAC-SHA256 (unsigned removed in 4.9), and fixes large export bug.

Judge: Accurate single sentence under 30 words (27), prioritizes rate limit and signed webhooks as required, no preamble or bullet list, minor: could emphasize the deprecation timeline slightly more clearly.

Frequently asked

What does this prompt test?

Summarise: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble. The deterministic checks are max_words, contains, regex.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, DeepSeek V4 Flash 0423 at $0.05 per 1,000 runs. If you need the top score, DeepSeek V4 Flash 0423 at $0.05 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on DeepSeek with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

DeepSeek V4 Pro

DeepSeek V4 Flash 0423

DeepSeek V4.1 Flash

Frequently asked

Same prompt, other families

Stop overpaying for AI tools