Sovereign AI · Pillar 5, and then all five
Last week Dave Vellante and I argued that tokenmaxxing is a trap and that the real question is who controls your AI economics.
Half of you wrote back with the same question.
“Fine. Show me the model.”
So here it is. Six strategies. Three years. Every line item. Cost per token, cost per task, cost per outcome that actually ships — and then the whole thing back through all five pillars, because a TCO model tells you what something costs and not what you own.
Every price below is from a live vendor page. Every assumption is labelled. The answer is not the one I expected when I opened the spreadsheet.
Start with the two stories everyone already knows
Uber burned through its entire 2026 AI coding tools budget in four months — after incentivising the burn itself with an internal leaderboard ranking teams by total AI tool usage. Then president and COO Andrew Macdonald said the quiet part out loud:
“Maybe implicitly there’s more that is getting shipped, but it’s very hard to draw a line between one of those stats and ‘Okay now we’re actually producing like 25% more useful consumer features.'”
Canva launched Canva AI 2.0 in April and slowed the rollout because it was unaffordable — “We decided to slow the rollout while we rebuilt the architecture, reduced unit costs and strengthened the business model.” It cut 2026 revenue growth guidance by a third, from 30% to 20%. Blackbird and Airtree marked their stake down 17%, from $42bn to $34.9bn.
Two of the most sophisticated product companies on earth. Both blindsided by the same line item.
Now watch what they each did next.
Uber’s CTO Praveen Neppalli Naga, three months later: “I think it’s another signal that we’re coming to the end of the so-called tokenmaxxing era.” Prompt caching, a changed default model setting, evaluating new models for efficiency, and letting engineers see their own usage and cost per hour. Cost per token fell while the number of people using frontier AI tools more than quadrupled.
Canva: a 140-strong research team building in-house models, with Leonardo.AI — acquired in 2024 — playing a key role in the alternative platform. Co-founder Cliff Obrecht on the frontier models they moved off: “They were amazing at what they did, but they were also very expensive.” On costs: “We’ve managed to do that by about 90 per cent, which means we can really scale it up now.”
Neither of them negotiated a better per-token rate.
That should tell you where the money isn’t.
The equation your CFO should be using
Everyone is optimising this:
cost = tokens × price per token
That equation has one lever, and it is the one lever your vendor controls.
Here is the real one:
cost per delivered outcome = (tokens per attempt × price per token) + (1 − pass rate) × human cleanup
Three terms. Not one.
- Price per token — the only term on your invoice
- Tokens per attempt — how efficiently the model works. Bad models flail. Flailing is billable.
- Pass rate — how often the work ships without a human touching it
Renting a cheaper model moves one term and can move the other two against you.
Post-training moves all three in the same direction. That is the entire argument, and the rest of this piece is the arithmetic.
First, the meter. Cost per token.
Reference workload — my assumption, stated so you can change it: an agentic enterprise task of 25,000 input tokens (system prompt, retrieved context, tool results across the trajectory) and 3,000 output tokens. 28,000 tokens per task.
| Strategy | In $/M | Out $/M | Tokens/task | $/task | Blended $/M |
|---|---|---|---|---|---|
| Rent frontier — unoptimised | $2.00 | $10.00 | 28,000 | $0.0800 | $2.86 |
| Rent frontier — cached + routed | blended | blended | 28,000 | $0.0180 | $0.64 |
| Rent open weights — serverless | $0.22 | $0.66 | 28,000 | $0.0043 | $0.15 |
| Post-trained specialist — managed dedicated | blended | blended | 11,760 | $0.0054 | $0.46 |
| Post-trained specialist — self-hosted | blended | blended | 11,760 | $0.0061 | $0.52 |
Frontier is gpt-5.6-sol at $2.00 / $10.00. Open weights is DeepSeek V4-Flash at its official off-peak rate — the cheapest number DeepSeek publishes, so the comparison runs against my own argument. Caching assumes a 60% hit rate, which is what Coinbase actually achieved, up from 5%.
A pricing note worth having, because almost everyone gets it wrong. You will see the OpenAI flagship quoted at $5/$30. That is the price of chat-latest, a different model on the same page. The flagship is $2/$10. If you are benchmarking your spend against a number you read in a cost-comparison blog, check which row it came from.
Four things worth stopping on.
One. The post-trained model uses 11,760 tokens, not 28,000. Harvey measured a 58% cut in total tokens in completed trajectories after reward-shaping for efficiency. Their earlier run cut tool calls from 104 to 42 and payload tokens from 461k to 250k. You don’t just buy cheaper tokens. You buy fewer of them.
Two. That is why the ratios diverge. Frontier versus post-trained self-hosted is 5.5x cheaper per token — and 13.2x cheaper per task. The gap between those two numbers is pure token efficiency, and it is trained in. Against Claude Opus 5 at $5/$25 the per-task gap is 32.9x.
Three. Self-hosting is a utilisation bet, not a price bet — and at realistic utilisation it loses to managed dedicated capacity:
| Sustained utilisation | Own cluster, $3.99/hr | Managed dedicated, $5.49/hr |
|---|---|---|
| 5% | $5.68/M | $7.82/M |
| 20% | $1.42/M | $1.96/M |
| 55% | $0.52/M | $0.71/M |
| 100% | $0.28/M | $0.39/M |
Both rates are Together’s live listed prices — GPU Clusters and Dedicated Inference respectively. At 5% utilisation you are paying more than frontier prices for a model you also have to operate. The published rule of thumb: “self-hosting only pencils out when you can keep roughly 50 to 100 concurrent requests in flight most of the time and the API you are replacing is a premium model.” Note the second condition — it does half the work.
Four, and this one surprised me: in the full model, managed dedicated beats self-hosting — $25.3M against $26.3M over three years. A provider running at 85% utilisation on a $5.49/hr GPU beats you running at 55% on a $3.99/hr one. Utilisation beats the hourly rate. Sovereignty has a price here and it is worth naming rather than hiding.
Now the problem with everything above.
That table is the entire conversation the industry is having. And in the full three-year model, it never accounts for more than about a sixth of the bill — and usually for under 2%.
Now, the actual bill. Three years, all in.
Same workload. Two profiles, because “cost per task” means nothing until you say what a failure costs.
Pass rates are anchored on the only published head-to-head I trust that reports genuine accuracy on private data: Bridgewater’s work with Thinking Machines — best frontier 78.2%, post-trained 84.7%. The off-the-shelf open-weight number is my assumption, not a measurement. I test it hard below.
Profile A — high volume, low unit value
Support triage, classification, routing. 2M tasks/month. A failed task costs $2 of human cleanup.
| Strategy | Pass rate | $/delivered | Inference/yr | Human cleanup/yr | People/yr | Build | 3-yr TCO |
|---|---|---|---|---|---|---|---|
| Rent frontier — unoptimised | 0.782 | $0.516 | $1.9M | $10.5M | $32k | — | $37.6M |
| Rent frontier — cached + routed | 0.766 | $0.486 | $431k | $11.2M | $160k | — | $35.9M |
| Rent open weights — serverless | 0.720 † | $0.564 | $103k | $13.4M | $240k | — | $41.6M |
| Post-train + managed dedicated | 0.847 | $0.311 | $130k | $7.3M | $480k | $710k | $25.3M |
| Post-train + self-host | 0.847 | $0.312 | $146k | $7.3M | $800k | $710k | $26.3M |
| Continued pretraining (the sovereign extreme) | 0.847 | $0.312 | $146k | $7.3M | $1.3M | $10.7M | $37.7M |
Profile B — low volume, high unit value
Diligence, claims adjudication, contract review. 100k tasks/month. A failed task costs $60 of expert cleanup.
| Strategy | Pass rate | $/delivered | Inference/yr | Human cleanup/yr | People/yr | Build | 3-yr TCO |
|---|---|---|---|---|---|---|---|
| Rent frontier — unoptimised | 0.782 | $13.16 | $96k | $15.7M | $32k | — | $47.9M |
| Rent frontier — cached + routed | 0.766 | $14.06 | $22k | $16.8M | $160k | — | $51.5M |
| Rent open weights — serverless | 0.720 † | $16.80 | $5k | $20.2M | $240k | — | $61.4M |
| Post-train + managed dedicated | 0.847 | $9.19 | $6k | $11.0M | $480k | $710k | $35.9M |
| Post-train + self-host | 0.847 | $9.19 | $7k | $11.0M | $800k | $710k | $36.9M |
| Continued pretraining (the sovereign extreme) | 0.847 | $9.19 | $7k | $11.0M | $1.3M | $10.7M | $48.3M |
† assumption, not a measurement — see below
Post-training: 33% cheaper in Profile A. 25% cheaper in Profile B.
And it is robust. I ran the off-the-shelf open-weight pass rate from 0.60 all the way to 0.782 — i.e. all the way up to frontier parity. Post-training is the cheapest strategy at every single value. That is the one result in this piece I am confident survives contact with your numbers.
The cheap-model trap has a precise crossover
Here is what I got wrong on my first pass, and it is worth showing the working.
Renting off-the-shelf open weights is $0.15 per million tokens — nineteen times cheaper than frontier on the meter, the cheapest line item in the model by a mile. My first instinct was that this makes it the obvious cost play.
It isn’t, and the reason is that it buys nothing but price. The quality is whatever the base model happens to have on your task, and the tokens per attempt are whatever the base model happens to burn.
The crossover is sharp:
- In Profile A, renting open weights beats renting frontier only once its pass rate exceeds 0.748 — within 3.4 points of frontier.
- In Profile B, where a failure costs $60, it needs to exceed 0.783 — it has to match frontier outright before the cheaper tokens pay for themselves.
At 2M tasks a month, every point of pass rate is worth roughly $480,000 a year in avoided cleanup. The entire annual token bill for the open-weight option is $103,000.
You cannot save your way past a three-point quality gap. There isn’t enough money in the token line to do it.
This is the part Bret Taylor is right about — and he is worth quoting because he chairs OpenAI’s board and runs Sierra, so he has every reason to say it:
“Token efficiency, compute efficiency, is something that I think the frontier labs are exceptional [at]… At the end of the day, it’s going to be how efficient your inference is. And I think the frontier labs are still far ahead here, and that’s really where the value is going to accrue.”
He is right that efficiency decides it. He assumes efficiency is something only frontier labs can produce. Harvey’s counter-number is 190.8 intelligence-per-token against 129.3 for the best frontier configuration — earned by explicitly reward-shaping for token efficiency during training.
Efficiency is not a property of the lab. It is a property of the training objective. You can buy it.
AT&T is the real-world midpoint here: routing cut costs by as much as 56% for a 2% quality decline, with open models now 40% of employee queries and a target of 60–70%. VP Mark Austin’s read: open-source models are “just as good or better” than older models from Anthropic and OpenAI, running six to ten months behind the frontier. Note what AT&T is not doing — it is not using DeepSeek or Moonshot, and says it is still evaluating the risks.
Where the money actually goes
Share of three-year TCO:
| Inference | Human cleanup | People | Build | |
|---|---|---|---|---|
| Profile A — rent frontier, unoptimised | 15.3% | 83.4% | 0.2% | — |
| Profile A — post-train + managed | 1.5% | 87.2% | 5.7% | 2.8% |
| Profile B — rent frontier, unoptimised | 0.6% | 98.4% | 0.2% | — |
| Profile B — post-train + managed | 0.1% | 92.0% | 4.0% | 2.0% |
Your token bill tops out around a sixth of your AI bill, and in most configurations it is under 2%.
You have a procurement function, a FinOps dashboard, and probably a steering committee pointed at the 15%. Nobody owns the 83%.
Cheap is a price. Sovereign is a position. And quality is the actual invoice.
What post-training costs — the honest build sheet
The number that surprises people: the GPUs are the cheap part.
| Line item | Cost | Basis |
|---|---|---|
| SFT run — 500M tokens on a ~30B model | $1,500 | Fireworks LoRA SFT, 16.1–80B tier, $3.00/M |
| RL run — 8× H100 × 14 days | $18,816 | 2,688 GPU-hours at Fireworks on-demand H100, $7.00/hr today, $8.00 from 1 Sept |
| Task environments — 300 @ ~$800 | $240,000 | Epoch AI: “$200 to $2000 mostly” per task |
| Expert data — 2,000 hours @ $150 | $300,000 | my assumption — replace with your rate |
| Eval + grader harness | $150,000 | my assumption |
| Total build | ~$710,000 | |
| Retune, 2× per year | ~$113,000/yr | reruns plus a 15% environment refresh |
| Eval passes, 12 per year | $96,000/yr |
Training compute is 2.9% of the build. Data, environments and evals are 97%.
That ratio is the sovereignty argument, and it is why the naive version of this story is wrong. The scarce asset was never GPUs — you rent those from anyone, and they got more expensive in 2026, not cheaper: H100 one-year contract pricing rose almost 40% from a $1.70/hr low in October 2025 to $2.35/hr by March 2026, and SemiAnalysis concluded “GPU rental pricing is more likely to continue rising than falling.”
The scarce asset is adjudicated expert judgment and verifiable environments.
And unlike GPUs, you may already own yours. Every bank with thirty years of adjudicated claims. Every firm with a decade of reviewed contracts. Every carrier with a million resolved tickets. That is an RL environment you haven’t built yet, sitting in a database you already pay to maintain.
Which is exactly why the money moved there. Mercor passed $2B gross annualised revenue in June 2026 and pays over $2 million a day to more than 30,000 weekly active contractors — and by July was reportedly in talks at a $20B valuation. Sacra estimates Handshake hit $1.1B annualised, up 349% YoY, with data labelling now 86% of revenue, up from 29% eight months earlier. Epoch reports Anthropic leaders discussing over $1B on RL environments in a year, per-task pricing of $200–$2,000, and roughly $2,400 of compute burned per task during RL training.
The frontier labs are buying, at enormous cost, the thing sitting on your servers.
The head-to-heads, adjusted for what actually ships
Published comparisons report raw cost advantage. None adjust for pass rate. When you do, the advantage gets bigger:
| Case | Pass rate | Raw cost advantage | Acceptance-adjusted |
|---|---|---|---|
| Bridgewater — financial document triage | 0.782 → 0.847 | 13.8x inference | 14.9x |
| Harvey × Applied Compute — legal agent | 0.853 → 0.913 rubric | 5x evaluation cost | 5.4x |
One correction I want to make explicitly, because I nearly printed it wrong. Harvey’s “5x cheaper than OpenAI and 15x cheaper than Opus” is Applied Compute’s figure for grading cost during training — achieved by switching to GPT-5 Mini and batching criteria, which produced 40x to 100x savings while “staying above 97% alignment” with frontier graders. It is not the cost of serving the trained agent. You will see it quoted as a serving number. It isn’t one.
Bridgewater is the one to sit with, because of why it worked. Frontier models scored “a mere ~50% accuracy” with a plain prompt, and expert prompting “boosted their accuracy from a coin flip to the mid-70s.” The tuned Qwen3-235B hit 84.7% — “29.8% fewer mistakes than the best frontier model we evaluated,” at “a 13.8x reduction in inference costs per task.”
(Three frontier models tie at 78.2%, so “the best frontier model” is genuinely plural. Coverage naming a single one is over-reading the chart.)
The reason is the whole thesis in one sentence: the right answers were never public.
Bridgewater’s judgment was not in the frontier model. It could not be. No amount of prompting retrieves something that was never in the weights.
Alex Karp has been making the qualitative version all year. On the Q2 call, on frontier labs: “They are superior to you. They deserve to colonize your enterprise.“ And five weeks earlier on CNBC, the sharper one:
Bridgewater is that argument with a number attached. Post-training is the only mechanism that puts your judgment into weights you hold.
Now run it against all five pillars
A TCO model tells you what something costs. It does not tell you what you own.
So before anyone signs anything, the strategy goes through the litmus test — all five pillars, not just the financial one. And it grades the way it always grades here: not pass/fail. Control, with a named risk sidecar.
Start with the standing rule, because it disqualifies most of the field before we begin.
If a hosted frontier model sits anywhere in the request path, you auto-fail Territorial and Financial by default. Capping spend on someone else’s meter is cost control. It is not sovereignty. The only clean escape is running open weights you hold.
That rule is why this piece is about post-training rather than about procurement discipline. Caching and routing are good engineering. They do not move a single pillar.
The scorecard
| Pillar | The question it asks | Rent frontier | Rent open weights | Post-train + managed | Post-train + self-host |
|---|---|---|---|---|---|
| Territorial | Where do data and compute physically sit — at rest and in motion? | ❌ | ❌ | ⚠️ | ✅ |
| Operational | Who runs and secures it — keys, paging, audit logs? | ❌ | ❌ | ⚠️ | ✅ |
| Technological | Who owns the stack and the IP — can you audit, fork, self-host? | ❌ | ⚠️ | ✅ | ✅ |
| Legal | Which jurisdiction governs access to it? | ❌ | ⚠️ | ⚠️ | ⚠️ |
| Financial | Predictable cost, no forced migration, no forced dependence? | ❌ | ⚠️ | ✅ | ✅ |
Nothing scores five out of five. If your vendor’s deck says otherwise, that’s the sovereign-washing tell.
Post-training buys three clean passes and two partials. That’s a real position — and the two partials are where the work is.
Territorial — conditional, and the condition is inference
Post-training does not make you territorially sovereign. Where you run the result does.
You can post-train a model and then serve it from a US region on someone else’s silicon, and you have moved nothing. The pillar is satisfied at the point of inference, not the point of training.
The reference implementation is OneAdvanced’s NHS model — NVIDIA Nemotron-Nano-9B, trained on pseudonymised NHS triage data:
“Model weights, fine-tuning, hosting and inference all sit within the UK perimeter.”
Four nouns. All four inside the boundary. That is the whole test.
The sidecar: the training run is territorial too. Harvey’s Tenet ran on approximately 150 NVIDIA B300 GPUs over two months, somewhere, on data that had to be somewhere. If your regulator cares where inference happens, ask whether they care where training happened. Most haven’t been asked yet.
Operational — post-training makes this pillar worse before it makes it better
This is the pillar nobody wants to write down, so here it is.
You gain control by taking on the burden. Renting fails Operational because someone else holds the keys and answers the page. Self-hosting passes it because you hold the keys and answer the page.
That is 2.5 fully loaded FTE in the model — $800,000 a year before a single GPU — plus a 3am rotation, a capacity plan, a serving stack, and an upgrade cycle you own forever.
Sovereignty here is not abstract. It is headcount.
The sidecar: if you cannot staff it, managed dedicated is the honest middle — keep the weights, rent the operations, grade yourself ⚠️ rather than pretending. The model says it also costs about $1M less over three years, because the provider’s utilisation beats yours.
Technological — the strongest pass, with the sharpest caveat
You hold weights. You can fork them, freeze them, air-gap them, and run them in ten years when the vendor is gone. Nothing else on the list gives you that.
And you cannot audit them.
A December 2025 preprint on persistent backdoors is the one to read before anyone signs off: backdoors optimised to survive fine-tuning achieved “over 99% persistence while preserving clean-task accuracy,” where representative prior methods “suffer 50% to 70% effectiveness drops after several rounds of model fine-tuning.” Tested on Qwen2.5 and LLaMA3 — the exact families sovereign programmes build on — and surviving full-parameter updates, data replay and parameter freezing. The authors’ framing: “end users often download pretrained LLMs from public repositories without the ability to audit their integrity.”
Owning a weight file lets you freeze it and fork it. It does not tell you what is in it.
Signing doesn’t close the gap. OpenSSF’s Model Signing spec establishes who published a file and that it hasn’t been altered since — and says so itself: “A perfectly signed model may still produce biased results, make inaccurate predictions, or behave unethically.” Provenance is not integrity is not trustworthiness.
For genuine auditability the bar is Ai2’s Olmo 3 and Switzerland’s Apertus: full training corpus, full pipeline, and intermediate checkpoints from every stage, so the run is reproducible. Qwen, Llama, DeepSeek and Kimi release weights — Qwen under Apache 2.0 and DeepSeek under MIT, both genuinely permissive — but not their pretraining corpora or intermediate checkpoints. The licence and the auditability are separate axes, and most commentary collapses them.
Legal — and here is where post-training creates new exposure
The one pillar where the honest grade is ⚠️ even when you self-host.
The good news first. The European Commission’s guidance gives downstream modifiers an indicative threshold:
“An indicative criterion for when a downstream modifier is considered to be the provider of a general-purpose AI model is that the training compute used for the modification is greater than a third of the training compute of the original model.”
And Recital 109 limits what you inherit even if you cross it: “the obligations for providers of general-purpose AI models should be limited to that modification or fine-tuning.” Post-train below a third and you take the capability without taking full provider liability. The regulator has effectively blessed the post-training path.
One correction to the timeline everyone is getting wrong, including a law-firm headline I nearly repeated: GPAI obligations did not become enforceable on 2 August 2026. Under Article 113, Chapter V — the GPAI chapter — has applied since 2 August 2025. What arrived on 2 August 2026 is Article 101, the Commission’s power to fine GPAI providers up to 3% of worldwide turnover or €15M. Models on the market before August 2025 have until 2 August 2027. If your compliance plan is built on the 2026 date, you are a year late, not early.
Now the exposures you did not have as a renter:
- The base model’s licence is not “open source.” The Kimi K3 licence — Harvey’s base — requires a separate commercial agreement with Moonshot once a licensee operating a “Model as a Service” business exceeds “20 million US dollars… in total over any consecutive 12 months.” A separate clause requires prominent “Kimi K3” attribution above 100 million MAU or $20M monthly revenue. There are carve-outs for purely internal use. And note: K3 is a bespoke licence, not the Modified MIT that K2 shipped under. Open weight ≠ open source ≠ yours.
- You may not distil your way there. OpenAI’s terms prohibit using Output “to develop models that compete with OpenAI.” Anthropic’s usage policy is broader still, barring “utilization of inputs and outputs to train an AI model (e.g., ‘model scraping’ or ‘model distillation’) without prior authorization” — note that covers training any model, not just a competing one. Enforcement isn’t theoretical: Anthropic revoked OpenAI’s Claude API access in August 2025 over exactly this.
- You have adopted a foreign maintainer’s jurisdiction. OSTP Director Michael Kratsios accused Moonshot of running “a sophisticated internal platform to conduct large scale distillation against U.S. models,” calling “large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology” unacceptable; Treasury Secretary Scott Bessent said “sanctions and Entity List designations will be on the table” (Lawfare, 24 July 2026). Be precise about what that would and would not do: the Entity List is an outbound control — it requires a licence to export US-origin items to a listed entity. It does not make it unlawful for you to use weights you already hold. That distinction matters and it is widely misreported, including in my own first draft.
- Your customers’ data probably cannot go in. Harvey pre-empted the question in its announcement — “We did not use any customer data in any of our post-training efforts” — and said it because in regulated industries the answer is usually no.
The sidecar: you have not removed jurisdictional risk. You have swapped a US vendor’s jurisdiction for a base-model maintainer’s jurisdiction plus a licence. That may be the better trade. It is not a clean pass, and anyone grading it as one hasn’t read the licence.
Financial — the clean pass, and the reason for the whole exercise
| Rent frontier | Rent open weights | Post-train + host | |
|---|---|---|---|
| Can you forecast cost 36 months out? | ❌ list price is unilateral | ⚠️ cheaper, still unilateral | ✅ your capacity, your curve |
| Do you hold the weights? | ❌ | ❌ | ✅ |
| Can you fork, freeze, air-gap? | ❌ | ⚠️ if you self-host | ✅ |
| Does your quality bar belong to you? | ❌ vendor sets it | ❌ | ✅ you own the graders |
| Cost of exit | ~$300k, recurring | ~$120k | ~$50k, once |
| Who captures the compounding value of your usage? | Them | Nobody | You |
That last row is the one that matters, and no TCO model has a column for it.
Every task through a rented frontier model makes their next model better and your position weaker. Every task through a model you post-trained makes your weights better and your position stronger.
Same workload. Opposite direction of compounding.
The one that will bite you: the burst path
My standing recommendation is to baseline on open weights and burst to frontier when you genuinely need the intelligence. It’s the right architecture and I’ll keep recommending it.
It is also a partial fail, and I should grade my own advice.
The moment a hosted frontier model is in the request path — even for 5% of traffic, even only for hard cases — Territorial and Legal go back to ⚠️ for that slice. Which slice? The hard one. The interesting one. The one most likely to contain your crown jewels.
Defensible trade. Not a free one. Route by sensitivity, not just difficulty, and put a classifier in front of the burst path so the decision to leave your perimeter is made by policy rather than by a latency threshold.
The Risk Acceptance Record
Sovereignty is not binary and it is not a certificate. It is a signed piece of paper with a name on it.
| Field | Entry |
|---|---|
| Strategy | Post-train an open-weight base; managed dedicated inference in-region; burst to frontier under policy |
| Exposure | Base-model supply, licence, and audit opacity |
| Pillars affected | Technological (partial), Legal (partial), Territorial (burst path only) |
| Commercial justification | 33% / 25% three-year TCO reduction; +6.5pp pass rate; exit cost falls from ~$300k recurring to ~$50k once |
| Compensating controls | Weights mirrored and hashed in-region · signed model manifest · safety and refusal regression suite every release · sensitivity classifier gating the burst path · second base model qualified and kept warm |
| Executive owner | Named individual. Not “the AI Council.” |
| Expiry | 12 months |
| Reassessment triggers | Base model export-controlled or sanctioned · licence revenue threshold crossed · maintainer ceases releases · a base release lands that beats your tuned model on your own evals |
That last trigger is the one people forget and the one most likely to fire. For most of the last three years, post-training was a bad bet precisely because the next base model absorbed whatever you had tuned in. The claim that this has stopped being true is a claim about the current rate of pretraining progress — and nobody has shown it stopped permanently. Put it in the register with a date on it.
What changed, and why now
Six months ago the honest answer to “can we do this?” was no. Three things changed it:
LoRA stopped being a compromise. Thinking Machines’ result: “LoRA learns with the same sample efficiency as FullFT and achieves the same ultimate performance.” And for RL specifically — “policy gradient algorithms learn roughly 1 bit of information per episode, given that there’s a single reward value at the end… It follows that the number of bits of useful information gleaned per episode is O(1), independent of model size.” Their docs put it flatly: “LoRA performs equivalently to FullFT for reinforcement learning even with small ranks.” A rank-32 LoRA on a 200B MoE is not a budget option. It is the correct algorithm.
GSPO made MoE reinforcement learning stable. Sequence-level clipping instead of token-level: it “notably stabilizes Mixture-of-Experts (MoE) RL training.” It’s what Harvey ran on Tenet. And GRPO is cheap because it “obviates the need for additional value function approximation as in PPO,” using the group mean as baseline — one fewer huge model in memory.
RL became a purchase order. Fireworks, Applied Compute, Tinker, Baseten. You rent the training stack and keep the weights.
How far this has already gone is easy to miss. Fireworks reports serving “more than 40 trillion tokens every day” and that “more than 95% of those tokens come from models specialized on customers’ proprietary data.” That is a vendor’s unaudited number in a funding announcement, and I’d treat the 95% as positioning rather than measurement — but even discounted heavily, the direction is the story.
Four things that would change my mind
I don’t publish a model without publishing what breaks it.
1. The pass-rate assumptions are the load-bearing wall. The +6.5pp delta comes from one study, on one domain, published by the platform vendor and its customer. The off-the-shelf open-weight number is my assumption outright. If your domain’s delta is 3 points instead of 6.5, the case narrows sharply. Measure yours before you budget.
2. Fine-tuning breaks safety, and your evals won’t catch it. Ten examples and under $0.20 jailbroke GPT-3.5 Turbo’s guardrails via OpenAI’s own API — and the same authors found that “even without malicious intent, simply fine-tuning with benign and commonly used datasets can also inadvertently degrade the safety alignment of LLMs, though to a lesser extent.” Under $200 and one GPU undid Llama 2-Chat’s safety training to “refusal rates of about 1%” on the 70B, and — the part that matters — “our method retains capabilities across two general performance benchmarks.” Standard benchmarks will not show you this. And narrow finetuning on insecure code produced broad misalignment on unrelated prompts. Every benchmark Harvey published for Tenet is a capability benchmark. Ask for a safety regression. Run one.
3. Most people who think they need RL need SFT. TensorZero’s independent numbers: “RFT costs 100-700x more than SFT on equivalent datasets” — and on one of three real tasks RFT made performance worse than baseline. Prompt, then eval, then SFT. Reach for RL only with a verifier in hand and SFT plateaued.
4. The window is open on someone else’s sufferance — but read the direction carefully. Both governments are circling. The US has floated sanctions and Entity List designations for Moonshot. Beijing has signalled tiered governance of its own open weights — a framework proposed by Nankai University’s Chen Bing and amplified by state media, under which “core technology bearing on national security would in principle not be open-sourced at all.”
And yet the tap has been widening, not narrowing. In May, Meta’s AI chief said Muse Spark was “not suitable for open sourcing” after safety checks. On 10 August Meta open-sourced Muse Spark 1.2 anyway, alongside a new laptop-friendly open family, with Zuckerberg saying “our goal should be for American open source models to be the best globally.”
So: not a closing window, but a politically contingent one. Which is the more useful warning, because it means the risk is a policy event you cannot forecast rather than a trend you can. Mirror the weights you’re betting on, and qualify a second base while you’re at it.
What I’d tell a board
Post-training is not a cost-reduction project. Pitch it that way and you’ll be measured on the 2% of the bill that is tokens, and you’ll lose.
It is a quality and control project that happens to be cheaper, and the mechanism isn’t mysterious: it moves all three terms of the equation instead of one.
The threshold is arithmetic, not ideology. Against renting frontier, post-training pays for itself once a failed task costs more than about $8.90 of human cleanup at 100,000 tasks a month — and above roughly a million tasks a month it wins on cost alone, before you count a single escalation.
Below that, rent. Rent without embarrassment, name the dependency, and put it in the risk register with an owner and an expiry date.
Above it, the question stops being what a token costs.
You are not buying tokens. You are buying the right to be good at your own job — and to still own that after the invoice changes.
Own the crown jewels. Stay nimble at the edges.
Action Item for Procurement Officers
Stop treating AI spend as a pure token-pricing negotiation. Immediately require every AI vendor RFP, renewal, or new contract to include a full three-year economic model that accounts for tokens per task, measured pass rates on your private data, and the resulting human cleanup/escalation costs—not just the headline $/M token rate. Benchmark at least three strategies side-by-side (unoptimized frontier, cached/routed frontier, and post-trained open-weight models on managed dedicated capacity) using the same workload volumes and failure costs that apply to your use cases. Where post-training shows a clear TCO advantage (typically when failed tasks cost more than ~$9 of expert cleanup or volumes exceed ~1M tasks/month), shift procurement focus from locking in cheaper frontier tokens to securing the rights, data pipelines, and evaluation harnesses needed to own specialist weights. Put a named owner, 12-month expiry, and explicit reassessment triggers on any residual frontier dependency so the organization retains the option to exit or rebalance as base models and economics evolve.
— Amit
Assumptions, so you can argue with the model
Everything is reproducible. Change the numbers and the conclusions move — that’s the point.
Workload [assumption]: 25,000 input + 3,000 output tokens per agentic task. Three-year horizon.
Prices [sourced, live Aug 2026]: frontier gpt-5.6-sol $2.00/$10.00 per M, cached input $0.20 (OpenAI); routing tier gpt-5.6-luna $0.10/$0.60; upper band Claude Opus 5 $5.00/$25.00 (Anthropic); open-weight serverless DeepSeek V4-Flash off-peak $0.22 cache-miss / $0.007 cache-hit / $0.66 output — peak rates are double (DeepSeek); GPU capacity from Together (Clusters H100 $3.99/hr, Dedicated Inference H100 $5.49/hr) and Fireworks (on-demand H100 $7.00/hr today, $8.00 from 1 September).
Throughput [sourced, with caveats]: 3,900 tok/s aggregate for Qwen3-30B-A3B on one H100 under vLLM at batch 256, per Spheron — a GPU marketplace, and the authors call their figures “engineering approximations, not vendor-guaranteed peak specs.” Treat as indicative.
Token reduction from post-training [sourced]: 58%, measured by Harvey on Tenet.
Pass rates: frontier 0.782 and post-trained 0.847 [sourced — Bridgewater]. Routed 0.766 [derived from AT&T’s measured 2% quality decline]. Off-the-shelf open weights 0.720 — my assumption; tested across 0.60–0.782 with no change to the ranking.
People [assumption]: $320,000 fully loaded. 0.1 FTE to rent frontier, 0.75 for open weights, 1.5 for post-trained managed, 2.5 for self-hosted.
Derived, not sourced — replace with yours: expert data rate, environment count, eval pass cost, human escalation cost, exit cost, FTE loading, utilisation.
What I cut during fact-checking, and why: a widely-circulated quote attributed to Harvey’s Gabe Pereyra about post-training being absorbed by the next base model — traceable only to an AI-generated aggregator page with no episode, date or link. A claim that an Entity List designation would bar US persons from using listed models — the Entity List is an outbound export control and does not do that. A claim that Alibaba was considered for the Entity List — it was added to the Pentagon’s Section 1260H list, a different instrument. And a set of Fermisense figures I had misread as raw benchmark scores when they were percentages of an achievable ceiling.
Full working, all references, and the counter-case in more depth: the research dossier.
