SQL · 4 checks · max 500 tokens · benched 2026-09-16

Muse Spark 1.3 vs Muse Glimmer 30B vs Llama 4 Maverick vs Llama 4 Scout on products over 100 units with having

Meta models side by side on "Products over 100 units with HAVING": Llama 4 Maverick scores 10/10; Llama 4 Maverick is the cheapest answer scoring 8+ at $0.04 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a database engineer. Reply with a single SQL statement and nothing else.

User

Tables: products(id, name) and order_items(product_id, quantity). Write one standard SQL query that lists product names whose total quantity sold across all order_items is greater than 100. Filter the aggregate with a HAVING clause.

Rubric for the judge: Join, SUM(quantity), GROUP BY product, HAVING > 100; returns names only; no WHERE on the aggregate.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Muse Spark 1.3

meta/muse-spark-1.3

unjudged

failed: OpenRouter HTTP 403: This model requires you to complete the following before use: 18+ age confirmation. Confirm at https://openrouter.ai/settings/preferences.

Latency: 0ms
Cost: $0
Per 1,000: $0

0 in · 0 out · 0 words · checks 0/4

regex
regex
regex
regex

(empty output)

Judge: —

Muse Glimmer 30B

meta/muse-glimmer-30b

10/10

Latency: 98ms
Cost: $0.00020
Per 1,000: $0.20

90 in · 168 out (136 reasoning) · 22 words · checks 4/4

regex
regex
regex
regex

```sql
SELECT p.name
FROM products p
JOIN order_items oi ON p.id = oi.product_id
GROUP BY p.id, p.name
HAVING SUM(oi.quantity) > 100;
```

Judge: Query correctly joins products and order_items, groups by product, uses HAVING with SUM(quantity) > 100, returns only names, and contains no WHERE clause on the aggregate.

Llama 4 Maverick

meta-llama/llama-4-maverick

10/10

top judge scorecheapest scoring 8+

Latency: 181ms
Cost: $4.1e-5
Per 1,000: $0.04

78 in · 37 out · 21 words · checks 4/4

regex
regex
regex
regex

```sql
SELECT p.name FROM products p JOIN order_items oi ON p.id = oi.product_id GROUP BY p.name HAVING SUM(oi.quantity) > 100;
```

Judge: Query correctly joins products and order_items, groups by product name, uses HAVING with SUM(quantity) > 100, returns only names, and contains no WHERE clause filtering the aggregate.

Llama 4 Scout

meta-llama/llama-4-scout

7/10

Latency: 364ms
Cost: $4.2e-5
Per 1,000: $0.04

78 in · 113 out · 62 words · checks 4/4

regex
regex
regex
regex

```sql
SELECT p.name
FROM products p
JOIN (
  SELECT product_id, SUM(quantity) as total_quantity
  FROM order_items
  GROUP BY product_id
) oi ON p.id = oi.product_id
WHERE oi.total_quantity > 100;
```

Alternatively, a more idiomatic SQL query using a HAVING clause would be:

```sql
SELECT p.name
FROM products p
JOIN order_items oi ON p.id = oi.product_id
GROUP BY p.name
HAVING SUM(oi.quantity) > 100;
```

Judge: The second query correctly uses JOIN, SUM, GROUP BY, and HAVING as required; however, the output includes unnecessary commentary and the first query violates the rubric by using WHERE instead of HAVING on the aggregate.

Frequently asked

What does this prompt test?

SQL: Join, SUM(quantity), GROUP BY product, HAVING > 100; returns names only; no WHERE on the aggregate. The deterministic checks are regex, regex, regex, regex.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Llama 4 Maverick at $0.04 per 1,000 runs. If you need the top score, Llama 4 Maverick at $0.04 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Meta with your own OpenRouter key, or tweak the wording and see what changes.