Tool call · 7 checks · max 400 tokens · benched 2026-09-16

DeepSeek V4 Pro vs DeepSeek V4 Flash 0423 vs DeepSeek V4.1 Flash on emit a calendar tool call

DeepSeek models side by side on "Emit a calendar tool call": DeepSeek V4 Flash 0423 scores 10/10; DeepSeek V4 Flash 0423 is the cheapest answer scoring 8+ at $0.03 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a function-calling engine. Reply with JSON only, no prose, no code fences.

User

You have one tool:
{"name": "create_event", "parameters": {"title": "string", "start": "ISO 8601 date-time without timezone, e.g. 2026-01-31T09:00", "duration_minutes": "integer", "attendees": "array of email strings"}}

Produce the tool call for this user message as a JSON object of the shape {"name": string, "arguments": object}.

User: Book a 45-minute design review with [email protected] and [email protected] on 2 October 2026 at 3pm.

Rubric for the judge: Correct tool name, ISO start 2026-10-02T15:00, integer duration 45, both attendees in an array, a sensible title.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

DeepSeek V4 Pro

deepseek/deepseek-v4-pro

10/10

Latency: 564ms
Cost: $0.00038
Per 1,000: $0.38

229 in · 129 out (81 reasoning) · 2 words · checks 7/7

json_valid
json_keys
regex
contains
regex
contains
contains

{"name":"create_event","arguments":{"title":"Design review","start":"2026-10-02T15:00","duration_minutes":45,"attendees":["[email protected]","[email protected]"]}}

Judge: All parameters are correct: tool name is 'create_event', ISO start datetime is '2026-10-02T15:00', duration is 45 minutes as an integer, both attendees are in an array, and title 'Design review' is sensible and matches the user request.

DeepSeek V4 Flash 0423

deepseek/deepseek-v4-flash

10/10

top judge scorecheapest scoring 8+

Latency: 729ms
Cost: $3.5e-5
Per 1,000: $0.03

150 in · 186 out (117 reasoning) · 17 words · checks 7/7

json_valid
json_keys
regex
contains
regex
contains
contains

{
  "name": "create_event",
  "arguments": {
    "title": "Design Review",
    "start": "2026-10-02T15:00",
    "duration_minutes": 45,
    "attendees": ["[email protected]", "[email protected]"]
  }
}

Judge: All rubric requirements met: correct tool name, ISO start datetime 2026-10-02T15:00, integer duration 45, both attendees in array, and sensible title 'Design Review'.

DeepSeek V4.1 Flash

deepseek/deepseek-v4.1-flash

10/10

Latency: 598ms
Cost: $0.00011
Per 1,000: $0.11

176 in · 138 out (90 reasoning) · 2 words · checks 7/7

json_valid
json_keys
regex
contains
regex
contains
contains

{"name":"create_event","arguments":{"title":"Design review","start":"2026-10-02T15:00","duration_minutes":45,"attendees":["[email protected]","[email protected]"]}}

Frequently asked

What does this prompt test?

Tool call: Correct tool name, ISO start 2026-10-02T15:00, integer duration 45, both attendees in an array, a sensible title. The deterministic checks are json_valid, json_keys, regex, contains, regex, contains, contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, DeepSeek V4 Flash 0423 at $0.03 per 1,000 runs. If you need the top score, DeepSeek V4 Flash 0423 at $0.03 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on DeepSeek with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

DeepSeek V4 Pro

DeepSeek V4 Flash 0423

DeepSeek V4.1 Flash

Frequently asked

Same prompt, other families

Stop overpaying for AI tools