Summarise · 3 checks · max 400 tokens · benched 2026-09-16

Kimi K3 vs Kimi K2.7 Code vs Kimi K2.6 vs Kimi K2.5 on one-sentence changelog summary

Moonshotai models side by side on "One-sentence changelog summary": Kimi K3 scores 9/10; Kimi K3 is the cheapest answer scoring 8+ at $1.40 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

User

Summarise this release note in one sentence of at most 30 words. Output only the sentence.

Release 4.7.0: The public API now enforces a rate limit of 600 requests per minute per key (previously unlimited). Requests over the limit return HTTP 429 with a Retry-After header. Webhook deliveries are now signed with HMAC-SHA256 and the old unsigned mode will be removed in 4.9. We also fixed a bug where exporting more than 10,000 rows produced an empty file.

Rubric for the judge: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Kimi K3

moonshotai/kimi-k3

9/10

top judge scorecheapest scoring 8+

Latency: 934ms
Cost: $0.00140
Per 1,000: $1.40

200 in · 81 out (29 reasoning) · 19 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds API rate limiting (600/min, HTTP 429), HMAC-SHA256 webhook signatures, and fixes empty exports exceeding 10,000 rows.

Judge: Accurate single sentence under 30 words (21 words), prioritizes rate limit and signed webhooks as required, no preamble or bullets, though could have been slightly more explicit about the deprecation timeline.

Kimi K2.7 Code

moonshotai/kimi-k2.7-code

9/10

Latency: 1.3s
Cost: $0.00144
Per 1,000: $1.44

119 in · 388 out (303 reasoning) · 25 words · checks 2/3

max_words
contains
regex

Release 4.7.0 adds a 600-request-per-minute API rate limit with 429/Retry-After, signs webhooks with HMAC-SHA256, deprecates unsigned webhooks in 4.9, and fixes exports over 10,000 rows.

Judge: Accurate single sentence under 30 words (28), prioritizes rate limit and signed webhooks as required, no preamble or bullets, though slightly dense phrasing.

Kimi K2.6

moonshotai/kimi-k2.6

9/10

Latency: 1.9s
Cost: $0.00396
Per 1,000: $3.96

119 in · 976 out (940 reasoning) · 16 words · checks 1/3

max_words
contains
regex

Release 4.7.0 enforces a 600-request-per-minute API limit, adds HMAC-SHA256 webhook signing, and fixes a large-export bug.

Judge: Accurate single sentence under 30 words (17 words), prioritizes rate limit and signed webhooks as required, no preamble or bullet list, minor: could emphasize the deprecation timeline more explicitly.

Kimi K2.5

moonshotai/kimi-k2.5

9/10

Latency: 1.1s
Cost: $0.00439
Per 1,000: $4.38

119 in · 1925 out (1890 reasoning) · 17 words · checks 2/3

max_words
contains
regex

Release 4.7.0 enforces 600-per-minute API rate limits and HMAC-SHA256 webhook signatures while fixing exports exceeding 10,000 rows.

Judge: Accurate single sentence under 30 words (17 words), prioritizes rate limits and webhooks as required, but slightly downgraded for omitting the HTTP 429 detail and Retry-After header context.

Frequently asked

What does this prompt test?

Summarise: A single accurate sentence that prioritises the rate limit and signed webhooks; no bullet list, no preamble. The deterministic checks are max_words, contains, regex.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Kimi K3 at $1.40 per 1,000 runs. If you need the top score, Kimi K3 at $1.40 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Moonshotai with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

Kimi K3

Kimi K2.7 Code

Kimi K2.6

Kimi K2.5

Frequently asked

Same prompt, other families

Stop overpaying for AI tools