SQL · 5 checks · max 500 tokens · benched 2026-09-16

Kimi K3 vs Kimi K2.7 Code vs Kimi K2.6 vs Kimi K2.5 on monthly active users

Moonshotai models side by side on "Monthly active users": Kimi K2.7 Code scores 10/10; Kimi K2.7 Code is the cheapest answer scoring 8+ at $0.72 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a database engineer. Reply with a single SQL statement and nothing else.

User

Table: events(user_id, event_type, occurred_at). Write one PostgreSQL query returning, for each calendar month of 2025, the month (as the first day of that month) and the number of distinct users who had at least one event in that month, ordered by month ascending.

Rubric for the judge: Uses date_trunc('month', occurred_at), COUNT(DISTINCT user_id), a 2025 filter, grouped and ordered by month.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Kimi K3

moonshotai/kimi-k3

7/10

Latency: 1.3s
Cost: $0.00142
Per 1,000: $1.42

173 in · 119 out (12 reasoning) · 36 words · checks 4/5

regex
regex
regex
regex
contains

SELECT m.month_start, COUNT(DISTINCT e.user_id) AS distinct_users
FROM generate_series('2025-01-01'::date, '2025-12-01'::date, interval '1 month') AS m(month_start)
LEFT JOIN events e
  ON e.occurred_at >= m.month_start
 AND e.occurred_at <  m.month_start + interval '1 month'
GROUP BY m.month_start
ORDER BY m.month_start;

Judge: Query correctly returns months of 2025 with distinct user counts ordered by month, but uses generate_series with manual date range logic instead of the rubric-specified date_trunc('month', occurred_at) approach for grouping.

Kimi K2.7 Code

moonshotai/kimi-k2.7-code

10/10

top judge scorecheapest scoring 8+

Latency: 1.1s
Cost: $0.00072
Per 1,000: $0.72

87 in · 187 out (103 reasoning) · 29 words · checks 5/5

regex
regex
regex
regex
contains

```sql
SELECT 
    DATE_TRUNC('month', occurred_at) AS month,
    COUNT(DISTINCT user_id) AS distinct_users
FROM 
    events
WHERE 
    occurred_at >= '2025-01-01' 
    AND occurred_at < '2026-01-01'
GROUP BY 
    DATE_TRUNC('month', occurred_at)
ORDER BY 
    month ASC;
```

Judge: Query correctly uses date_trunc('month', occurred_at), COUNT(DISTINCT user_id), filters for 2025, groups and orders by month as required by the rubric.

Kimi K2.6

moonshotai/kimi-k2.6

1/10

failed: empty output (finish_reason length)

Latency: 1.3s
Cost: $0.00684
Per 1,000: $6.84

88 in · 1700 out (1700 reasoning) · 0 words · checks 0/5

regex
regex
regex
regex
contains

(empty output)

Judge: No query was provided; the output is completely empty when a specific PostgreSQL query was explicitly required.

Kimi K2.5

moonshotai/kimi-k2.5

unjudged

failed: The operation was aborted due to timeout

Latency: 0ms
Cost: $0
Per 1,000: $0

0 in · 0 out · 0 words · checks 0/5

regex
regex
regex
regex
contains

(empty output)

Judge: —

Frequently asked

What does this prompt test?

SQL: Uses date_trunc('month', occurred_at), COUNT(DISTINCT user_id), a 2025 filter, grouped and ordered by month. The deterministic checks are regex, regex, regex, regex, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Kimi K2.7 Code at $0.72 per 1,000 runs. If you need the top score, Kimi K2.7 Code at $0.72 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Moonshotai with your own OpenRouter key, or tweak the wording and see what changes.