Boris Cherny is right about AI ROI. He’s just measuring it from inside a lab. Here’s the number the enterprise actually pays for.
Boris Cherny — the person behind Claude Code — keeps getting the same question from engineers at other companies: how are you getting so much value out of the same tool we have?
Same license. Same model. Wildly different returns.
His answer is the interesting part, and it’s worth stealing before I take it apart:
- Stop benchmarking AI against your old tooling. Benchmark it against the engineer-hours it replaces. That’s the yardstick.
- The proof point: a developer rewrote an entire codebase from one language to another in six days — work that used to be a year of engineering.
- The playbook: underfund the team, hand them “as many tokens as possible,” and let them “Claude-ify.”
Clean logic. Airtight — inside a frontier lab where adoption is frictionless, the buyer and the user are the same person, and there is no security review standing between a good idea and production.
Now walk that logic into a bank. Or an OEM. Or a government-adjacent enterprise running on a BPO contract.
It breaks. Not because Boris is wrong — because the enterprise is buying a different thing than he’s selling.
The token is not the product. The workflow completion is not the win.
Both are inputs. The CFO isn’t buying inputs.
Boris’ instinct — measure against engineer-hours saved — is correct, and the denominator is still wrong the moment you cross into a governed org. Because the enterprise doesn’t capture value when the agent ships a PR. It captures value after the org tax: procurement cycles, security review, data residency, change management, audit.
“Successful workflows executed” and “tokens consumed” are the vendor’s revenue metrics. They are not your value metrics. If your ROI story is “we burned X billion tokens and completed Y workflows,” you’ve measured how much you paid — not what you kept.
So here’s what we actually move in the field. Three numbers. Not one of them is a token count.
1. Self-service capture — the internal developer portal number
The IDP taught us the right question a decade ago: how much work does the org complete without opening a ticket to a human?
That’s the number. Not “the agent succeeded.” The agent succeeding is table stakes. The ROI shows up as the ticket that was never filed — the request that resolved itself, the platform team that stopped being a bottleneck, the internal customer who self-served at 2 am.
Value captured = the queue that shrank. You can put that on a slide the CFO signs.
2. SLA adherence — the number your BPO contract already tracks
In a BPO-run process, the clock is contractual. There’s a service level, there’s a penalty, and someone owns both.
Drop an agent into that flow, and it inherits the SLA. It doesn’t get graded on elegance or token efficiency — it gets graded against a number your legal and finance teams already agreed to in writing.
Value captured = the SLA held, at lower unit cost, at higher volume, without renegotiating the contract. That’s not a productivity anecdote. That’s margin.
3. Time-to-audit — the housekeeping number nobody markets, and everybody pays for
How long does a security and compliance audit take against your agentic system?
Unsexy. Decisive. Because the cost of owning an enterprise system isn’t the build — it’s the maintenance, the housekeeping, the standing burden of proving to a regulator that the thing is behaving.
A system that is easy to track, audit, and maintain is a system that is cheap to own over time. Shrink the audit from six weeks to six days, and you’ve moved a recurring line item on the balance sheet — every quarter, forever.
Boris optimizes token cost late. Fine, in his world. In ours, the compounding cost was never the tokens. It was the audit you run four times a year.
Why this is a Financial Sovereignty problem, not a productivity one
Residency is geography. Sovereignty is control.
The financial cut of that is simple: renting a capability is not the same as owning the economics of the value it creates. Boris’ model optimizes adoption early and cost late — the correct move when you are the vendor. The sovereign enterprise optimizes for a different variable entirely: who captures the surplus.
Get seduced by token efficiency and successful-workflow dashboards, and you will happily fund the vendor’s growth while reporting it as your own ROI. The two look identical on a usage graph. They are opposites on a P&L.
Measure what lands with the org, not what leaves the meter:
- Tickets that never got filed.
- SLAs that held at lower cost.
- Audits that got shorter.
That’s the pillar. Miss the denominator, and you miss the whole number.
