The highest score on the benchmark
At xhigh effort Gemini 4 Neon passes 370 of 657 tasks, 56.3%. That is 2.4 points above Gemini 4 Carbon's 53.9% and 3.9 above GPT-6.2 Astra's 52.4%, at $1.69 a task, 13% less than Carbon in this comparison.
Release report · November 2026Gemini 4 Neon
Gemini 4 Neon completes 56.3% of AutomationBench's strict, end-to-end business tasks at xhigh effort, the highest score on the benchmark so far. That is 2.4 points above Gemini 4 Carbon, and at high effort Neon edges past Carbon's best for 55% less per task.
At xhigh effort Gemini 4 Neon passes 370 of 657 tasks, 56.3%. That is 2.4 points above Gemini 4 Carbon's 53.9% and 3.9 above GPT-6.2 Astra's 52.4%, at $1.69 a task, 13% less than Carbon in this comparison.
High effort reaches 54.3% for $0.88 a task, above Carbon's best for 55% less. Medium reaches 53.3% for $0.58, above GPT-6.2 Astra at max for 30% less.
Going from low to xhigh adds 13% more passes, 326 to 370, while cost per task rises 320%, from $0.40 to $1.69. Every Neon setting sits on the cost and accuracy frontier.
At its best effort per domain, Neon leads Marketing (68.0% vs. Carbon's 64.0%), Finance (65.0% vs. 60.0%) and Sales (59.0% vs. Gemini 4 Argon's 58.1%), and ties Carbon in Operations at 68.0%, which it reaches at low effort. HR is the gap: 37.5% at best, 5.8 points behind GPT-6.2 Astra.
A shallow curve: 49.6% at low, 56.3% at xhigh.
Scoring is strict. A task passes only if every assertion about the final state of the simulated company holds. All 657 tasks stay in the denominator at every effort. The dashed line marks Gemini 4 Carbon at high effort, the strongest other result in this comparison; Neon clears it from high effort up.
Official pass rate · task_completed_correctly · 657 tasks per effort, all in the denominator
2.4 points above Gemini 4 Carbon, the previous high.
Each bar is a model's best effort setting on AutomationBench 1.0.6. Gemini 4 Neon is shown at xhigh effort; the other results are the ones published on zapier.com/benchmarks. Claude Fable 5.2 and Gemini 4 Hydrogen appear only here, since they aren't in this run's cost comparison.
Best observed pass rate per model · AutomationBench 1.0.6
Four of the five frontier points are Neon's.
Each line is one model's effort settings, pass rate against cost per task. GPT-6.2 Sol at high is the cheapest result, at $0.21 a task; from $0.40 up, the cheapest way to any score in this comparison is a Neon setting. Medium, at $0.58, beats GPT-6.2 Astra at max on both cost and accuracy.
| モデル | Low | Medium | High | XHigh |
|---|---|---|---|---|
| Gemini 4 Neon | 49.6%$0.40 | 53.3%$0.58 | 54.3%$0.88 | 56.3%$1.69 |
| Gemini 4 Carbon | — | — | 53.9%$1.94 | — |
| GPT-6.2 Astra | — | — | — | — |
| Gemini 4 Argon | — | — | 51.3%$1.63 | — |
| GPT- 6 . 2ソル | — | — | 41.2%$0.21 | — |
API cost per task, list-price estimate, with caching · pass rate at each effort, low to max · Cost comparisons use the models in this run's metrics export. Claude Fable 5.2 and Gemini 4 Hydrogen are pass-rate results from the zapier.com/benchmarks leaderboard; Claude Fable 5.2's uses Anthropic's default fallbacks.
Marketing gains the most, and Operations doesn't move.
Each line runs across the four effort settings. Marketing climbs 13.0 points from low to xhigh, 55.0% to 68.0%. Operations scores 68.0% at low, medium and xhigh, with a dip to 65.0% at high. HR is the hardest domain at every effort and ends at 35.0%.
| ドメイン | Low | Medium | High | XHigh |
|---|---|---|---|---|
| 販売 | 47.9%56/117 | 52.1%61/117 | 53.8%63/117 | 59.0%69/117 |
| マーケティング | 55.0%55/100 | 59.0%59/100 | 64.0%64/100 | 68.0%68/100 |
| オペレーションズ | 68.0%68/100 | 68.0%68/100 | 65.0%65/100 | 68.0%68/100 |
| 支える | 37.0%37/100 | 41.0%41/100 | 43.0%43/100 | 45.0%45/100 |
| 金融 | 57.5%69/120 | 65.0%78/120 | 64.2%77/120 | 65.0%78/120 |
| HR | 34.2%41/120 | 35.8%43/120 | 37.5%45/120 | 35.0%42/120 |
Pass rate · passed / tasks per cell · darker = higher
$0.58 a task at medium, $1.69 at xhigh.
Low has the lowest spend per passed task, $0.81, against $3.01 at xhigh. At xhigh a failed task costs $2.21 and a passed one $1.29, so misses drive the bill. Spend per pass counts the cost of every failed task too. These are estimates at Google's stated rates with an assumed cache-read rate, not billed costs.
| Effort | Cost / task | Passed task | Failed task | Spend per pass |
|---|---|---|---|---|
| Low | $0.40 | $0.44 | $0.37 | $0.81 |
| Medium | $0.58 | $0.60 | $0.55 | $1.08 |
| High | $0.88 | $0.87 | $0.90 | $1.62 |
| XHigh | $1.69 | $1.29 | $2.21 | $3.01 |
USD, list-price estimate, with caching · spend per pass = all task costs ÷ passes, failures included
Reasoning grows 8.8×, steps grow 64%.
Tool calls rise from 57.6 per task at low to 106.3 at xhigh, and steps from 10.4 to 17.1. Reasoning tokens grow from 6,657 to 58,544 per task and make up 82% of output at xhigh. One task, at high effort, hit the 50-step cap.
| Effort | Tool calls | ステップ | Calls / step | Input tokens | Cached share | At step cap |
|---|---|---|---|---|---|---|
| Low | 57.6 | 10.4 | 5.5 | 272.3k | 81。3% | 0 |
| Medium | 66.1 | 11.4 | 5.8 | 411.3k | 83。3% | 0 |
| High | 82.9 | 14.3 | 5.8 | 751.6k | 87。2% | 1 |
| XHigh | 106.3 | 17.1 | 6.2 | 1310.8k | 88。5% | 0 |
Mean per task, all tasks included · step cap 50
AutomationBench tests AI agents on end-to-end workflow execution using 47 simulated apps across six business functions: Sales, Marketing, Operations, Support, Finance and HR. Each task boots a small simulated company, hands the agent one request a real operations person would get, then grades the world it leaves behind with deterministic assertions.
CRM records, inbox threads, spreadsheets and support cases, including traps like stale rows and near-duplicate names.
The agent gets one trigger message and tools to search, read, update and send. It can't ask clarifying questions, and there's no human in the loop.
Deterministic assertions check what changed. The task passes only if every one holds.
CostEstimates at Google's stated rates for Gemini 4 Neon: $3.00 input and $15.00 output per 1M tokens, with reasoning billed as output, and a $0.15 cache-read rate assumed until Google publishes one. Gemini implicit caching records no cache writes. These are not billed costs.
ComparisonsCost comparisons use the models in this run's metrics export. Claude Fable 5.2 and Gemini 4 Hydrogen are pass-rate results from the zapier.com/benchmarks leaderboard; Claude Fable 5.2's uses Anthropic's default fallbacks.