Summarise · 3 checks · max 400 tokens · benched 2026-09-16

GLM 5.3 vs GLM 5.2 vs GLM 5.3 Flash on one-sentence changelog summary

Z Ai models side by side on "One-sentence changelog summary": GLM 5.3 Flash scores 9/10; GLM 5.3 Flash is the cheapest answer scoring 8+ at $0.02 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

User

Summarise this release note in one sentence of at most 30 words. Output only the sentence.

Release 4.7.0: The public API now enforces a rate limit of 600 requests per minute per key (previously unlimited). Requests over the limit return HTTP 429 with a Retry-After header. Webhook deliveries are now signed with HMAC-SHA256 and the old unsigned mode will be removed in 4.9. We also fixed a bug where exporting more than 10,000 rows produced an empty file.

Rubric for the judge: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

GLM 5.3

z-ai/glm-5.3

9/10

Latency: 1.0s
Cost: $0.00037
Per 1,000: $0.37

123 in · 44 out (1 reasoning) · 23 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600-requests-per-minute API rate limit, HMAC-SHA256 webhook signing, and fixes a bug where exporting over 10,000 rows produced empty files.

Judge: Accurate single sentence under 30 words (24 words), prioritizes rate limit and signed webhooks as required, no preamble or bullet list, though could have emphasized the deprecation timeline slightly more.

GLM 5.2

z-ai/glm-5.2

9/10

Latency: 1.9s
Cost: $0.00162
Per 1,000: $1.62

123 in · 774 out (605 reasoning) · 21 words · checks 2/3

max_words
contains
regex

Release 4.7.0 enforces a 600 requests per minute API rate limit, adds HMAC-SHA256 webhook signing, and fixes a large export bug.

Judge: Accurate single sentence under 30 words (17 words) that prioritises rate limit and signed webhooks as required, with no preamble or bullet list.

GLM 5.3 Flash

z-ai/glm-5.3-flash

9/10

top judge scorecheapest scoring 8+

Latency: 228ms
Cost: $2.1e-5
Per 1,000: $0.02

123 in · 48 out · 25 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600 requests/minute API rate limit (HTTP 429 on excess), signs webhooks with HMAC-SHA256, and fixes empty exports for over 10,000 rows.

Judge: Accurate single sentence under 30 words (26 words), prioritizes rate limit and signed webhooks as required, no preamble or bullet list, minor: could emphasize deprecation of unsigned mode more explicitly.

Frequently asked

What does this prompt test?

Summarise: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble. The deterministic checks are max_words, contains, regex.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, GLM 5.3 Flash at $0.02 per 1,000 runs. If you need the top score, GLM 5.3 Flash at $0.02 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Z Ai with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

GLM 5.3

GLM 5.2

GLM 5.3 Flash

Frequently asked

Same prompt, other families

Stop overpaying for AI tools