Measuring the ROI of AI coding tools when you pay by the token

The bill stopped being a fixed number

For most of the last three years an AI coding assistant was priced like any other developer tool: a fee per seat per month. Finance could forecast it to the penny. That is no longer how the main tools charge.

Two details in GitHub's change matter more than the price. Admins can now set budgets at enterprise, cost-centre and user level. And the old fallback, where someone who ran out of allowance dropped to a cheaper model and carried on, has gone: work now stops at the budget. A spending limit has become a delivery decision.

The FinOps Foundation, which writes the cloud cost-management practices most finance teams use, put token cost at the top of its practitioners' list in its June 2026 *State of Tokenomics* paper. It also noted that AI developer tools "frequently rival direct API spend within the first year of adoption". And it pointed out that for seat-plus-usage tools, cost visibility is limited to what the vendor exposes.

So the first half of the ROI calculation, the cost, has gone from known to variable. The second half was never well measured in the first place.

What the evidence says about the return

The studies that exist don't agree, and the reasons they disagree are the useful part.

Read together, three things come out.

Perception isn't measurement. METR's developers were slower and felt faster. Surveys of felt productivity, which is most of what organisations collect, overstate the gain in at least some settings, so a self-reported "hours saved" figure can't carry an ROI case on its own.

Individual output isn't organisational output. Faros saw more code merged and no company-level improvement, because review, testing and release absorbed it. Atlassian's numbers say the same thing from the other side: the time AI saves is matched by time lost to everything around the code.

The measurement itself is getting harder. When developers won't work without the tool and run several agents in parallel, the clean comparison a trial needs stops existing. That's METR's own conclusion, and it's why internal measurement has to lean on comparing cohorts and tracking trends rather than one-off experiments.

What moves the number

Developer experience

The field experiments found the gains concentrated in less experienced staff. Short-tenure developers increased output by 27% to 39%, against 8% to 13% for long-tenure developers. Juniors gained 21% to 40%, against 7% to 16% for seniors. The authors are clear that these splits are noisy and not statistically significant on their own, but every measure points the same way. METR's slowdown came from exactly the other end: experienced people in code they already knew deeply.

Task type and codebase

Stanford researcher Yegor Denisov-Blanch, presenting data from more than 100,000 developers at a conference, reported gains of roughly 30% to 35% on simple tasks in new codebases, falling to 5% to 10% on complex work in large existing ones, and about 15% to 20% net once rework is counted. These figures come from the talk as reported by others, and I couldn't find a paper with them, so treat them as directional. They match what the trials suggest: the less the tool has to understand about an existing system, the more it helps.

The delivery system around the code

DORA's central finding is that AI is an amplifier. Teams with strong foundations (clear policies, good internal context, solid testing and safety nets, a working internal platform) get more out of it. Teams without them get more of their existing problems, faster. That is the same mechanism as the Faros result: if review and release are the bottleneck, more code just makes a longer queue.

The tool and how it's used

Under the new billing, how the tool is used decides the cost as much as the benefit. On Copilot, completions stay inside the seat price while agentic work burns credits at token rates. So a team that mostly uses inline suggestions and a team that runs agents on long tasks can pay very different amounts for the same seat.

How the frameworks are being used

None of the established frameworks were built to price AI, but they're being adapted rather than replaced.

• DORA's four delivery metrics (deployment frequency, lead time, change failure rate, time to restore) are the outcome check. If AI adoption rises and change failure rate rises with it, DORA's 2025 instability finding is happening on your team. DORA's 2025 report adds a seven-capability AI model describing what makes the gains stick.
• SPACE (Forsgren, Storey and colleagues, *ACM Queue*, 2021) covers satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. It exists to stop any single activity metric being read as productivity, which is exactly the trap with "lines of AI code accepted".
• DX's AI Measurement Framework sorts AI metrics into utilisation (who is using it, and on how much of the work), impact (time saved, throughput, quality) and cost (spend per developer, and whether it pays). It is used alongside DX Core 4, which DX describes as combining DORA, SPACE and DevEx. DX also treats agents as extensions of the developers who supervise them, so agent-written pull requests count towards that team's output. DX sells measurement software, and the results it publishes for customers, such as Booking.com, are its own claims.
• The FinOps Foundation brings the finance side, with unit metrics such as cost per user per month and cost per successful outcome. For coding, the natural outcome is a merged, shipped change.

The pattern across all four is the same. Measure use, impact and cost together, and judge the impact on delivery outcomes, not on activity.

A worked model

This is illustrative. Every number below is an assumption chosen to show the mechanics, not a finding, so replace each with your own.

Cost per month

• 10 developers on Copilot Business at $19: $190, including $190 of credits
• Agent usage beyond the included credits: assume $600
• Total: $790 a month

Return per month

• Validated time saved: assume 2 hours per developer per week, net. That means measured against a baseline, not self-reported. 10 × 2 × 4.33 weeks = 86.6 hours
• At an assumed loaded cost of $60 an hour: $5,196
• Less extra review: assume 10 team hours a month at $60, $600, the cost Faros measured as review time rising
• Net value: $4,596

ROI = (4,596 − 790) ÷ 790 = about 4.8 times the spend.

The useful number isn't the 4.8. It's the break-even. Under these assumptions the spend pays for itself at about 23 team hours a month, roughly half an hour per developer per week. Below that it's a loss. And if your situation looks like METR's (experienced people, a mature codebase, complex changes), the honest starting assumption for that line is close to zero until you've measured it.

So the whole case rests on one input, validated time saved, which is the input most organisations don't measure. That's where the effort should go.

A 90-day plan

1. Baseline first. Before changing anything, record four weeks of cycle time, review time, change failure rate and merged changes per developer from your repository and deployment tools.
2. Tag the work. Mark which pull requests were AI-assisted or agent-written, by label or by the tool's own telemetry where it offers it. Without this nothing downstream can be compared.
3. Put the spend where the work is. Use the budget controls the tools now have (cost centre or team level on Copilot, spend limits on Claude Enterprise and Cursor Enterprise) so the cost lands against the same team as the output.
4. Track cost per merged change. AI spend divided by AI-assisted changes that reach production. It's the FinOps "cost per successful outcome" applied to code, and it moves when people burn tokens on work that never ships.
5. Compare by task type and seniority. Split the numbers into new features against changes to existing systems, and junior against senior. The evidence says the averages hide the story.
6. Watch the downstream signals. If review time, PR size or change failure rate climb, the gain is being spent in the queue. Fix review capacity and testing before buying more usage.
7. Pair telemetry with a short survey. Perception and measurement disagree, so you need both, and the gap between them is a finding.
8. Re-price at 90 days. Put your measured numbers into the model above. Keep, cut or reallocate the budget per team on that basis, not on the vendor's case study.

What isn't known yet

There's no independent, peer-reviewed figure for organisation-level ROI of agentic coding under token pricing. The controlled studies predate most of the current agent tools, METR has said its newest data isn't reliable, and the large datasets come from companies that sell measurement. Anyone quoting a single ROI multiple for "AI coding" is quoting a vendor or a guess. Your own baseline and your own cost per merged change are the only numbers that will hold up in a budget review.