SQL · 6 checks · max 500 tokens · benched 2026-09-16

GPT-5.5 vs GPT-5.4 vs GPT-5.4 mini vs GPT-5.4 Nano vs GPT-5.3-Codex on top five customers by 2025 spend

OpenAI models side by side on "Top five customers by 2025 spend": GPT-5.4 scores 10/10; GPT-5.4 Nano is the cheapest answer scoring 8+ at $0.13 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a database engineer. Reply with a single SQL statement and nothing else.

User

Tables: customers(id, name) and orders(id, customer_id, amount, ordered_at). Write one standard SQL query that returns each customer's name and their total order amount for calendar year 2025, highest total first, limited to the top 5. Use a JOIN, GROUP BY and ORDER BY.

Rubric for the judge: Correct join and aggregate, filters ordered_at to 2025, sorts descending, limits to 5, portable SQL.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

GPT-5.5

openai/gpt-5.5

10/10

Latency: 442ms
Cost: $0.00548
Per 1,000: $5.48

88 in · 168 out (76 reasoning) · 40 words · checks 5/6

regex
regex
regex
regex
regex
contains

SELECT c.name, SUM(o.amount) AS total_order_amount
FROM customers AS c
JOIN orders AS o
  ON o.customer_id = c.id
WHERE o.ordered_at >= DATE '2025-01-01'
  AND o.ordered_at < DATE '2026-01-01'
GROUP BY c.id, c.name
ORDER BY total_order_amount DESC
FETCH FIRST 5 ROWS ONLY;

Judge: Query correctly joins customers and orders, filters 2025 using portable DATE syntax, groups by customer with proper aggregation, orders descending by total, limits to 5 rows, and uses standard SQL throughout.

GPT-5.4

openai/gpt-5.4

10/10

top judge score

Latency: 403ms
Cost: $0.00170
Per 1,000: $1.70

88 in · 99 out (11 reasoning) · 38 words · checks 5/6

regex
regex
regex
regex
regex
contains

SELECT c.name, SUM(o.amount) AS total_order_amount
FROM customers c
JOIN orders o ON o.customer_id = c.id
WHERE o.ordered_at >= DATE '2025-01-01'
  AND o.ordered_at < DATE '2026-01-01'
GROUP BY c.id, c.name
ORDER BY total_order_amount DESC
FETCH FIRST 5 ROWS ONLY;

GPT-5.4 mini

openai/gpt-5.4-mini

9/10

Latency: 387ms
Cost: $0.00102
Per 1,000: $1.02

88 in · 212 out (123 reasoning) · 39 words · checks 5/6

regex
regex
regex
regex
regex
contains

SELECT c.name, SUM(o.amount) AS total_order_amount
FROM customers AS c
JOIN orders AS o
  ON o.customer_id = c.id
WHERE o.ordered_at >= DATE '2025-01-01'
  AND o.ordered_at < DATE '2026-01-01'
GROUP BY c.name
ORDER BY total_order_amount DESC
FETCH FIRST 5 ROWS ONLY;

Judge: Query correctly joins tables, filters 2025 orders, groups by customer, orders descending by total, and limits to 5 rows using standard SQL syntax; minor deduction only because GROUP BY uses c.name instead of c.id (though functionally correct, c.id is more robust for grouping).

GPT-5.4 Nano

openai/gpt-5.4-nano

9/10

cheapest scoring 8+

Latency: 290ms
Cost: $0.00013
Per 1,000: $0.13

88 in · 88 out · 34 words · checks 6/6

regex
regex
regex
regex
regex
contains

SELECT
  c.name,
  SUM(o.amount) AS total_amount_2025
FROM customers c
JOIN orders o
  ON o.customer_id = c.id
WHERE o.ordered_at >= DATE '2025-01-01'
  AND o.ordered_at < DATE '2026-01-01'
GROUP BY c.name
ORDER BY total_amount_2025 DESC
LIMIT 5;

Judge: Query correctly implements JOIN, GROUP BY, ORDER BY, and LIMIT with proper 2025 filtering; minor deduction only because GROUP BY should include c.id to be fully standard-compliant and avoid potential issues in some SQL dialects.

GPT-5.3-Codex

openai/gpt-5.3-codex

10/10

Latency: 406ms
Cost: $0.00274
Per 1,000: $2.74

88 in · 185 out (97 reasoning) · 38 words · checks 5/6

regex
regex
regex
regex
regex
contains

SELECT c.name, SUM(o.amount) AS total_order_amount
FROM customers c
JOIN orders o ON o.customer_id = c.id
WHERE o.ordered_at >= DATE '2025-01-01'
  AND o.ordered_at < DATE '2026-01-01'
GROUP BY c.id, c.name
ORDER BY total_order_amount DESC
FETCH FIRST 5 ROWS ONLY;

Frequently asked

What does this prompt test?

SQL: Correct join and aggregate, filters ordered_at to 2025, sorts descending, limits to 5, portable SQL. The deterministic checks are regex, regex, regex, regex, regex, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, GPT-5.4 Nano at $0.13 per 1,000 runs. If you need the top score, GPT-5.4 at $1.70 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on OpenAI with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

GPT-5.5

GPT-5.4

GPT-5.4 mini

GPT-5.4 Nano

GPT-5.3-Codex

Frequently asked

Same prompt, other families

Stop overpaying for AI tools