Extract JSON · 6 checks · max 400 tokens · benched 2026-09-16

Gemini 3.1 Pro Preview vs Gemini 3.8 Flash vs Gemini 3.5 Flash Lite on invoice text to json

Google models side by side on "Invoice text to JSON": Gemini 3.5 Flash Lite scores 10/10; Gemini 3.5 Flash Lite is the cheapest answer scoring 8+ at $0.23 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a data extraction engine. Reply with JSON only, no prose, no code fences.

User

Extract the fields below from the invoice text and return one JSON object with exactly these keys: vendor (string), invoice_number (string), total (number), currency (ISO 4217 string), due_date (YYYY-MM-DD string).

Invoice text:
Northwind Print Co.
Invoice INV-2041
Issued: 15 September 2026
Payment due: 15 October 2026
Items: 500 x A5 flyers @ 1.20 = 600.00; 4 x roll-up banners @ 171.13 = 684.50
Total due: USD 1,284.50
Thank you for your business.

Rubric for the judge: Valid JSON with the five keys, numeric total 1284.5, ISO due date, no extra keys or commentary.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

Gemini 3.1 Pro Preview

google/gemini-3.1-pro-preview

10/10

Latency: 2.2s
Cost: $0.00495
Per 1,000: $4.95

185 in · 382 out (312 reasoning) · 14 words · checks 6/6

json_valid
json_keys
contains
contains
contains
contains

{
  "vendor": "Northwind Print Co.",
  "invoice_number": "INV-2041",
  "total": 1284.50,
  "currency": "USD",
  "due_date": "2026-10-15"
}

Judge: All five required keys present with correct values: vendor, invoice_number, total (1284.50), currency (USD), and due_date (2026-10-15 in YYYY-MM-DD format); valid JSON with no extra keys or commentary.

Gemini 3.8 Flash

google/gemini-3.8-flash

10/10

Latency: 1.1s
Cost: $0.00035
Per 1,000: $0.35

185 in · 57 out · 12 words · checks 6/6

json_valid
json_keys
contains
contains
contains
contains

{"vendor": "Northwind Print Co.", "invoice_number": "INV-2041", "total": 1284.50, "currency": "USD", "due_date": "2026-10-15"}

Judge: Output is valid JSON with exactly the five required keys, correct numeric total (1284.50), proper ISO 4217 currency code (USD), correctly formatted due date (2026-10-15), no extra keys or commentary.

Gemini 3.5 Flash Lite

google/gemini-3.5-flash-lite

10/10

top judge scorecheapest scoring 8+

Latency: 443ms
Cost: $0.00023
Per 1,000: $0.23

185 in · 70 out · 14 words · checks 6/6

json_valid
json_keys
contains
contains
contains
contains

{
  "vendor": "Northwind Print Co.",
  "invoice_number": "INV-2041",
  "total": 1284.50,
  "currency": "USD",
  "due_date": "2026-10-15"
}

Frequently asked

What does this prompt test?

Extract JSON: Valid JSON with the five keys, numeric total 1284.5, ISO due date, no extra keys or commentary. The deterministic checks are json_valid, json_keys, contains, contains, contains, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, Gemini 3.5 Flash Lite at $0.23 per 1,000 runs. If you need the top score, Gemini 3.5 Flash Lite at $0.23 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Google with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

Gemini 3.1 Pro Preview

Gemini 3.8 Flash

Gemini 3.5 Flash Lite

Frequently asked

Same prompt, other families

Stop overpaying for AI tools