Boris Cherny is right about AI ROI. He’s just measuring it from inside a lab. Here’s the number the enterprise actually pays for.
Boris Cherny — the person behind Claude Code — keeps getting the same question from engineers at other companies: how are you getting so much value out of the same tool we have?
Same license. Same model. Wildly different returns.
His answer is the interesting part, and it’s worth stealing before I take it apart:
- Stop benchmarking AI against your old tooling. Benchmark it against the engineer-hours it replaces. That’s the yardstick.
- The proof point: a developer rewrote an entire codebase from one language to another in six days — work that used to be a year of engineering.
- The playbook: underfund the team, hand them “as many tokens as possible,” and let them “Claude-ify.”
Clean logic. Airtight — inside a frontier lab where adoption is frictionless, the buyer and the user are the same person, and there is no security review standing between a good idea and production.
Now walk that logic into a bank. Or an OEM. Or a government-adjacent enterprise running on a BPO contract.
It breaks. Not because Boris is wrong — because the enterprise is buying a different thing than he’s selling.
The token is not the product. The workflow completion is not the win.
Both are inputs. The CFO isn’t buying inputs.
Boris’ instinct — measure against engineer-hours saved — is correct, and the denominator is still wrong the moment you cross into a governed org. Because the enterprise doesn’t capture value when the agent ships a PR. It captures value after the org tax: procurement cycles, security review, data residency, change management, audit.
“Successful workflows executed” and “tokens consumed” are the vendor’s revenue metrics. They are not your value metrics. If your ROI story is “we burned X billion tokens and completed Y workflows,” you’ve measured how much you paid — not what you kept.
So here’s what we actually move in the field. Three numbers. Not one of them is a token count.
1. Self-service capture — the internal developer portal number
The IDP taught us the right question a decade ago: how much work does the org complete without opening a ticket to a human?
That’s the number. Not “the agent succeeded.” The agent succeeding is table stakes. The ROI shows up as the ticket that was never filed — the request that resolved itself, the platform team that stopped being a bottleneck, the internal customer who self-served at 2 am.
Value captured = the queue that shrank. You can put that on a slide the CFO signs.
2. SLA adherence — the number your BPO contract already tracks
In a BPO-run process, the clock is contractual. There’s a service level, there’s a penalty, and someone owns both.
Drop an agent into that flow, and it inherits the SLA. It doesn’t get graded on elegance or token efficiency — it gets graded against a number your legal and finance teams already agreed to in writing.
Value captured = the SLA held, at lower unit cost, at higher volume, without renegotiating the contract. That’s not a productivity anecdote. That’s margin.
3. Time-to-audit — the housekeeping number nobody markets, and everybody pays for
How long does a security and compliance audit take against your agentic system?
Unsexy. Decisive. Because the cost of owning an enterprise system isn’t the build — it’s the maintenance, the housekeeping, the standing burden of proving to a regulator that the thing is behaving.
A system that is easy to track, audit, and maintain is a system that is cheap to own over time. Shrink the audit from six weeks to six days, and you’ve moved a recurring line item on the balance sheet — every quarter, forever.
Boris optimizes token cost late. Fine, in his world. In ours, the compounding cost was never the tokens. It was the audit you run four times a year.
Why this is a Financial Sovereignty problem, not a productivity one
Residency is geography. Sovereignty is control.
The financial cut of that is simple: renting a capability is not the same as owning the economics of the value it creates. Boris’ model optimizes adoption early and cost late — the correct move when you are the vendor. The sovereign enterprise optimizes for a different variable entirely: who captures the surplus.
Get seduced by token efficiency and successful-workflow dashboards and you will happily fund the vendor’s growth while reporting it as your own ROI. The two look identical on a usage graph. They are opposites on a P&L.
Measure what lands with the org, not what leaves the meter:
- Tickets that never got filed.
- SLAs that held at lower cost.
- Audits that got shorter.
But measuring the right number is only half the job. You can’t control a number whose inputs sit on someone else’s meter.
The panacea: own the stack the number comes from
If your intelligence, your compute, and your inference all run on rented infrastructure, your unit economics aren’t yours. They’re a line in a vendor’s pricing deck, repriced whenever they choose. A sharper dashboard won’t fix that. Owning the stack will.
- Bring the intelligence in-house. Open weights and open source, on infrastructure you control. You stop renting the model and start owning the cost curve — and the model stops being a black box you take on faith when the auditor shows up. Open beats closed on exactly the three numbers above: self-service, SLA adherence, and time-to-audit all get easier when you can actually see inside the thing.
- Own the compute and the inference. Where it runs is where your margin gets decided. Own that layer and unit cost becomes a lever you pull, data residency becomes a default instead of a negotiation, and time-to-audit becomes a config you control instead of a vendor ticket you wait on.
- Don’t do it alone — get forward-deployed engineering. Open weights on your own infra is a strategy, not a weekend project. The distance between “we could self-host” and “we captured value self-hosting” is implementation — and that gap is the adoption gap Boris named. It closes with forward-deployed engineers who sit inside your stack and wire the intelligence to your workflows, your SLAs, and your audit posture. On your terms. On your infra.
Own the compute. Own the intelligence. Own the inference. Do that and your unit economics stop being something that happens to you — and start being something you set.
Full disclosure, full conviction: this is the work we do at Agentcy Labs. Forward-deployed, on your infrastructure, building the stack you actually own.
That’s the pillar. Measure what lands — then own the stack it lands on.
Rent your intelligence, you rent your economics. Own it, and the number is finally yours.
— Amit
