Extract JSON · 5 checks · max 400 tokens · benched 2026-09-16

DeepSeek V4 Pro vs DeepSeek V4 Flash 0423 vs DeepSeek V4.1 Flash on event announcement to structured json

DeepSeek models side by side on "Event announcement to structured JSON": DeepSeek V4 Flash 0423 scores 10/10; DeepSeek V4 Flash 0423 is the cheapest answer scoring 8+ at $0.03 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a data extraction engine. Reply with JSON only, no prose, no code fences.

User

Return one JSON object with keys title (string), date (YYYY-MM-DD), start_time (24-hour HH:MM), location (string), attendees (integer). Use only facts stated in the announcement.

Announcement: Join us for the Autumn Data Meetup on Tuesday 3 November 2026. Doors open at 2 pm and talks start at 2:30 pm at The Loft, 14 Harbour Street. We have confirmed 48 attendees so far and can take up to 60.

Rubric for the judge: Correctly picks the talk start (14:30, not doors), ISO date, integer attendees 48, and a sensible title and location.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

DeepSeek V4 Pro

deepseek/deepseek-v4-pro

10/10

Latency: 3.1s
Cost: $0.00044
Per 1,000: $0.44

127 in · 216 out (158 reasoning) · 18 words · checks 5/5

json_valid
json_keys
contains
contains
regex

{
  "title": "Autumn Data Meetup",
  "date": "2026-11-03",
  "start_time": "14:30",
  "location": "The Loft, 14 Harbour Street",
  "attendees": 48
}

Judge: All fields correctly extracted: title matches announcement, date is proper ISO format, start_time correctly uses talk start (14:30) not door opening, location is complete and accurate, and attendees count is 48 as stated.

DeepSeek V4 Flash 0423

deepseek/deepseek-v4-flash

10/10

top judge scorecheapest scoring 8+

Latency: 776ms
Cost: $3.2e-5
Per 1,000: $0.03

127 in · 179 out (120 reasoning) · 18 words · checks 5/5

json_valid
json_keys
contains
contains
regex

{
  "title": "Autumn Data Meetup",
  "date": "2026-11-03",
  "start_time": "14:30",
  "location": "The Loft, 14 Harbour Street",
  "attendees": 48
}

DeepSeek V4.1 Flash

deepseek/deepseek-v4.1-flash

10/10

Latency: 1.5s
Cost: $0.00043
Per 1,000: $0.43

153 in · 673 out (630 reasoning) · 7 words · checks 5/5

json_valid
json_keys
contains
contains
regex

{"title":"Autumn Data Meetup","date":"2026-11-03","start_time":"14:30","location":"The Loft, 14 Harbour Street","attendees":48}

Judge: All fields correctly extracted: title matches announcement, date in ISO format (2026-11-03), start_time correctly uses talk start time (14:30) not door opening, location fully specified, and attendees set to confirmed count (48).

Frequently asked

What does this prompt test?

Extract JSON: Correctly picks the talk start (14:30, not doors), ISO date, integer attendees 48, and a sensible title and location. The deterministic checks are json_valid, json_keys, contains, contains, regex.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, DeepSeek V4 Flash 0423 at $0.03 per 1,000 runs. If you need the top score, DeepSeek V4 Flash 0423 at $0.03 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on DeepSeek with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

DeepSeek V4 Pro

DeepSeek V4 Flash 0423

DeepSeek V4.1 Flash

Frequently asked

Same prompt, other families

Stop overpaying for AI tools