Every AI launch now comes with a chart: our model scored 66%, theirs scored 58%, ours wins. Business owners are expected to take those charts as a buying guide. This year, some of the people who make and use those tests started saying, in public, that the numbers are shakier than they look.
What happened
In February, OpenAI published a post explaining why it no longer evaluates on SWE-bench Verified, one of the best-known tests of whether AI can fix real software bugs. OpenAI said the benchmark had become contaminated: the test problems had leaked into the material models were trained on, so higher scores increasingly reflected memory rather than real coding skill. It stopped reporting the score, urged others to do the same, and pointed to a newer test while it builds uncontaminated ones.
Then came a problem with the answer keys themselves. Epoch AI runs FrontierMath, a set of very hard, unpublished math problems used to measure advanced reasoning. On June 12 it released a corrected version, and its FrontierMath v2 page says the update addressed errors in 42% of the problems. Epoch corrected well over a hundred problems and removed a dozen. In other words, for a period the industry was ranking models partly on questions that had flawed answers.
Even model makers now hedge their own charts. When Anthropic launched Claude Opus 5.5 in September, it published the usual scores but also noted that benchmark margins have become less reliable guides to real-world differences, and that the gap between two of its models was narrower in its own use than the numbers suggested.
Why it matters for your business
A benchmark is a standardized exam. Like any exam, it can be studied for, it can contain mistakes, and it can stop telling top students apart once everyone scores near the top. Three things follow:
- Small gaps mean little. A few points' difference between two leading models on a public test is often noise. It rarely predicts which one will do better on your invoices, contracts or customer emails.
- Published scores are curated. Companies choose which tests to report. No single number is false, but the selection tells a story.
- Your work is not on the test. Public benchmarks measure math, coding and trivia. Most business tasks involve your formats, your terminology and your customers. No public exam covers that.
The case for benchmarks is real too. They are still a fast way to rule out weak options, and the fact that OpenAI and Epoch publicly fixed their own problems shows the field correcting itself. The mistake is treating a leaderboard as a purchase decision.
What we'd do
- Build your own small exam. Collect 20 to 50 real examples of a task you want AI to handle, with the answer you would accept. Keep them private so no model has seen them.
- Score what you care about: accuracy, tone, format, time saved and cost per task. Have the person who normally does the work grade the results.
- Use public benchmarks only to shortlist. Narrow to two or three candidates, then let your own exam decide.
- Rerun your exam on every major release. It takes an afternoon and protects you from both hype and inertia.
- Ask vendors for evidence on tasks like yours, not leaderboard ranks. Case studies in your industry beat a percentage on a math test.
Our take
We think public AI benchmarks are useful for researchers and close to useless for buying decisions. The tests leak, the answer keys have errors, and the margins between leading models are now smaller than the noise. That is not a scandal. It is what happens when a measuring stick gets used as a marketing tool. The good news for business owners is that the best benchmark is cheap to build: a folder of your own real work, graded by someone who knows what good looks like.
Want a system like this in your business?
A discovery conversation is where we map it: what to automate first, where the human checkpoints go, and what it would take.
Book a discovery conversation →