Summarise · 3 checks · max 400 tokens · benched 2026-09-16

MiniMax M3 vs MiniMax M2.7 vs MiniMax M2.5 on summarise a support thread in 60 words

Minimax models side by side on "Summarise a support thread in 60 words": MiniMax M2.5 scores 9/10; MiniMax M2.5 is the cheapest answer scoring 8+ at $0.19 per 1,000 runs. Outputs, checks, judge reasons, latency and cost.

The prompt every model received

System

You are a concise assistant. Follow length limits exactly and output plain text only.

User

Summarise the following customer support thread in no more than 60 words. Mention what the customer wanted and how it was resolved.

---
Customer (Mon 09:12): Hi, I was charged twice for my Team plan on 1 September. Order references are 88213 and 88214. I only have one workspace. Can you fix this?
Agent (Mon 10:40): Sorry about that. I can see the duplicate charge. Could you confirm the last four digits of the card so I can raise the refund?
Customer (Mon 11:02): 4471.
Agent (Mon 11:30): Thanks. I have refunded order 88214 in full (USD 96.00). It should appear on your statement within 5 business days. I have also added a note so it cannot recur next cycle.
Customer (Mon 11:35): Great, thanks for the quick turnaround.
---

Rubric for the judge: Accurate, complete summary of the double charge and its refund, within 60 words, no leaked card digits.

Side by side

Every cell is one OpenRouter call at temperature 0 with the prompt's token cap and reasoning effort "low" where the model supports it. Cost is usage × the catalogue rate in models.json. Quality is one judge call to anthropic/claude-haiku-4.5 against the prompt's rubric, cached per prompt version.

MiniMax M3

minimax/minimax-m3

9/10

Latency: 1.9s
Cost: $0.00028
Per 1,000: $0.28

386 in · 133 out (76 reasoning) · 40 words · checks 3/3

max_words
contains
not_contains

The customer reported being double-charged for their Team plan on 1 September (orders 88213 and 88214), despite having only one workspace. The agent verified the card, refunded the duplicate charge of USD 96.00, and added a note to prevent recurrence.

Judge: Accurate, complete summary within 60 words (47 words), mentions what customer wanted and resolution, no card digits leaked; minor: could have explicitly stated 5-day timeline.

MiniMax M2.7

minimax/minimax-m2.7

9/10

Latency: 1.6s
Cost: $0.00044
Per 1,000: $0.44

219 in · 310 out (277 reasoning) · 40 words · checks 3/3

max_words
contains
not_contains

Customer was charged twice for their Team plan (orders 88213 and 88214). Agent verified the duplicate charge and refunded USD 96.00 for order 88214, which will appear within 5 business days. A note was added to prevent future duplicate charges.

Judge: Accurate, complete summary covering what the customer wanted (duplicate charge fixed) and resolution (refund processed with preventative measures), well under 60 words (47 words), no card digits leaked, minor: could have mentioned the date for full context.

MiniMax M2.5

minimax/minimax-m2.5

9/10

top judge scorecheapest scoring 8+

Latency: 673ms
Cost: $0.00019
Per 1,000: $0.19

219 in · 118 out (51 reasoning) · 49 words · checks 3/3

max_words
contains
not_contains

Customer was charged twice for their Team plan (order references 88213 and 88214) and requested a refund for the duplicate charge. Agent verified the card details and refunded $96.00 for order 88214, which will appear within 5 business days, and added a note to prevent recurrence. Customer confirmed satisfaction.

Judge: Accurate, complete summary covering the double charge and refund resolution within 60 words (57 words), no card digits leaked, minor: could have specified 'Team plan on 1 September' for full context but not required.

Frequently asked

What does this prompt test?

Summarise: Accurate, complete summary of the double charge and its refund, within 60 words, no leaked card digits. The deterministic checks are max_words, contains, not_contains.

Which model should I pick for this task?

If the judge's bar of 8/10 is good enough for you, MiniMax M2.5 at $0.19 per 1,000 runs. If you need the top score, MiniMax M2.5 at $0.19 per 1,000 runs.

Same prompt, other families

Run it yourself

Re-run this prompt on Minimax with your own OpenRouter key, or tweak the wording and see what changes.

Side by side

MiniMax M3

MiniMax M2.7

MiniMax M2.5

Frequently asked

Same prompt, other families

Stop overpaying for AI tools