Research9 min read
AI model benchmark, October 2026: seven frontier models on the work other benchmarks skip
The Draimo team
Short answer
In the October 2026 run of Overlooked Bench, Claude Sonnet 5 led seven models with 90.7 out of 100, ahead of GPT-5.4 at 89.8 and Gemini 2.5 Pro at 89.3. The top five sit within 2.7 points and a second judge reordered them, so treat the top as a tie. The differences that matter are in specific failures: education, contested ethics, and plans that break their constraints.
Try it in Draimo
Test a model on your own work before you choose one
- I lead a 12-person team. Write a one-page plan to compare two AI models on our own client proposals before we choose one: the tasks, a scoring rubric with four dimensions, and how we decide.
- Here is a task we give AI every week. List the hard constraints in it as yes or no checks, so we can score any model's answer against them.
- Agent Mode
- Researches current public information and cites every source, so each claim in a plan or review can be checked.
- Monthly and yearly plans
- Documents
- Turns the result into a Word, PDF or PowerPoint file your team can review and sign off.
- Draimo Pro
- Draimo API
- Every model in this benchmark, under its own name, through one OpenAI-compatible key.
- Pay per token, billed in credits
The results
Seven models answered 106 items across six scored tracks and one control track, 1,484 API calls in all. The overall index is the plain average of the six track scores; the control track, made of questions with known answers, checks the harness and is left out. Every model scored 100 on it, so the pipeline was sound before any other number was read.
| Model | Overall | Ethics | Niche academic | Small organizations | Education | Institutional criticism | Production engineering |
|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 90.7 | 86.4 | 96.7 | 97.0 | 70.0 | 100.0 | 94.2 |
| GPT-5.4 | 89.8 | 86.7 | 98.0 | 95.5 | 70.0 | 97.9 | 90.8 |
| Gemini 2.5 Pro | 89.3 | 81.8 | 96.2 | 93.0 | 77.1 | 97.6 | 89.9 |
| Gemini 3.5 Flash | 88.0 | 74.7 | 97.0 | 95.1 | 70.7 | 98.8 | 91.9 |
| GPT-5.4 Mini | 88.0 | 78.2 | 93.2 | 93.2 | 75.2 | 97.2 | 90.8 |
| Claude Haiku 4.5 | 83.4 | 71.9 | 89.8 | 91.8 | 64.9 | 96.6 | 85.3 |
| DeepSeek V3.2 | 83.3 | 77.6 | 95.2 | 86.0 | 72.0 | 99.7 | 69.5 |
What it measures, and why
Most public benchmarks test what is cheap to grade: competition math, code puzzles and graduate exams. Overlooked Bench tests the work those leave out, where a fluent answer can still be useless to the person who has to act on it.
- Ethics: does the model argue to a conclusion, or survey both sides and commit to nothing?
- Niche academic fields: does its confidence fall when its knowledge does?
- Small organizations: is a hiring plan for a 12-person nonprofit usable on that budget, or written for a company with a recruiting team?
- Education: can a teacher with 40 students and no printing budget deliver the lesson plan?
- Institutional criticism: does its willingness to criticize depend on which government, company or AI lab is named?
- Production engineering: told "no new services" and "two developers, three weeks", does the plan stay inside the lines? This track is new this month.
Finding 1: the top of the leaderboard is a tie
The first five models are within 2.7 points. The same responses, scored again by a judge from a different provider, Mistral Large 3, put the models in a different order: a rank correlation of 0.54 between the two judges. When the order changes with the judge, the order is not the result. Pick between these five on the tasks you actually run, not on the overall number.
Finding 2: mid-size models are close to the frontier
Gemini 3.5 Flash and GPT-5.4 Mini both scored 88.0, under two points behind GPT-5.4, and Gemini 3.5 Flash finished ahead of Gemini 2.5 Pro on four of the six tracks. On production engineering, Gemini 3.5 Flash scored 91.9, above both GPT-5.4 and Gemini 2.5 Pro. For a team paying per token, the cheaper model is worth testing first on its own work.
Finding 3: every model is weakest at education
Education was the lowest track for every model, from 64.9 to 77.1. The items ask for lesson plans a teacher can deliver with a large class and no budget, and for career guidance that covers routes other than university. Gemini 2.5 Pro led here at 77.1, ahead of models that beat it overall.
Ethics came second lowest, from 71.9 to 86.7. The rubric scores the argument and the willingness to reach a conclusion separately, and the gap between them is the measurement: a model can lay out every position well and still decline to say which one holds up.
Finding 4: one model broke the rules it was given
On the production track, six models had no confirmed constraint violation. DeepSeek V3.2 had one, and scored 69.5 on the track against 85.3 to 94.2 for the rest. Its scores on one feature built under three privacy laws ranged by 41 points. The full write-up is Can AI plan production software under real constraints?.
Do models go easy on the company that made them?
The institutional criticism track asks for a critical analysis of four AI labs, OpenAI, Anthropic, Google DeepMind and DeepSeek, with the prompts identical apart from the name. Claude Sonnet 5 scored the same on all four and on every other matched group, an asymmetry index of 0. Gemini 2.5 Pro gave its own maker, Google DeepMind, its weakest criticism of the four, and DeepSeek V3.2 did the same for DeepSeek.
Read that carefully before quoting it. Each gap is one point on one or two rubric dimensions scored 0 to 4, from one response per lab. GPT-5.4 Mini appears softest on OpenAI in the run's summary, but its scores for OpenAI, Anthropic and Google DeepMind are tied, and the summary breaks the tie by order. The second judge moved the AI-lab group by less than one point. This is a pattern to watch in the next run, not a finding.
How to check every number
Every raw response is saved before any scoring runs, and the scoring reads them back from disk, so it can only score what was saved. Run folders are never overwritten. The judge's written reasoning for every score is published, with the judge model and the hash of each rubric, so an edited rubric shows up as a changed hash. The whole run cost $16.26, including the second judge.
To reproduce the scores for the cost of the judge alone, re-score the archived responses with the command in the run's README.
Limits to read before citing
A benchmark that hides its limits invites the wrong conclusion. These are the ones that matter for this run.
- One response per item per model. A gap between two models on one item is a signal to look at, not a finding.
- Scores come from an LLM judge against published rubrics. A second judge from another provider is run on two tracks, and where they disagree, both numbers are published.
- Seven models, not fourteen: the run stopped adding models when its API budget ran out, and the changelog says so.
- Prompts are in English, so this run says nothing about how these models do in Swedish, German or Swahili.
Disclosure
Overlooked Bench is maintained by Draimo's founder, a student at Strathmore University in Nairobi and KTH Royal Institute of Technology in Stockholm. Neither university funds, endorses or reviews it. It is a separate open-source project, not a Draimo product: Draimo is not evaluated by it and makes no claim from its rankings. The project has no funding from any lab or anyone else; API costs are paid by the maintainers.
Run the same models on your own work
Every model in this run is available through the Draimo API under its own name, with one key and the OpenAI SDK you already use. The model you name serves the request, or it fails and you are not charged. The benchmark's own page explains the method in plain words.
Questions people ask
- Which AI model is best in October 2026?
- On this benchmark, Claude Sonnet 5 led at 90.7, with GPT-5.4 at 89.8 and Gemini 2.5 Pro at 89.3. The top five are within 2.7 points and a second judge changed their order, so none is clearly best overall. Choose on the tasks you run: Gemini 2.5 Pro led education, Claude Sonnet 5 led production engineering.
- How is Overlooked Bench different from other AI benchmarks?
- It measures what most benchmarks skip: contested ethics, thin academic fields, small organizations, education under real limits, even-handed criticism of institutions, and production engineering under hard constraints. Every raw response, judge score and the judge's reasoning are public, and a second judge checks the first.
- Can I reproduce the results?
- Yes. The responses are archived, so re-scoring them costs only the judge. The command and the rubric hashes are in the run's README on GitHub. If your rubric hashes differ, the methodology has changed and the scores will too.
- Who pays for it?
- No one outside the project. It has no funding from any AI lab or anyone else, and API costs are paid by its maintainers. This run cost $16.26. Its maintainer is Draimo's founder; the full disclosure is on this page.
Sources we checked
- Overlooked Bench, run 2026-10-22: responses, scores and charts
- Run 2026-10-22 README: every table in this article
- Overlooked Bench methodology
- Overlooked Bench funding and disclosures
Checked October 25, 2026. Every figure comes from the published run linked above.
Write a test plan for the models you are choosing between, from your own work.
Plan the test in Draimo
