Understanding the drivers of enterprise AI costs

The number everyone negotiates is the one that matters least

Rate per million tokens is the number that appears in every vendor conversation, every business case and every board slide, because it is the only one that looks like a price. It is also the number an enterprise has the least control over and the most competition working on its behalf. Frontier model pricing has fallen repeatedly without anyone in your organisation doing anything at all.

Meanwhile the bill went up. That is not a contradiction, it is arithmetic. Cost is price multiplied by volume, and volume is the term nobody owns. Token consumption has been reported at roughly 13x its early-2025 level. No procurement outcome survives a 13x multiplier on the other side of the equation.

So the discipline that gets applied, negotiate harder, is aimed at the smaller of the two terms. The larger one is set by engineering decisions made months earlier, usually without a cost conversation attached.

Every figure is sourced at the end of this article. The ranges are wide because the methodologies differ. The direction is not in dispute.

Inference is the bill, and it never stops arriving

The budgeting instinct formed during the model-training era, when the expensive event was building something once. In production the expensive event is answering, repeatedly, forever. Every additional user, every additional workflow and every retry lands on the same meter, and none of them appear in the business case that approved the programme.

This is why AI cost control resembles cloud FinOps far more than software procurement. Software licensing rewards negotiation. Metered infrastructure rewards attribution: put the unit cost in front of the team that creates it, and review it on a cadence. That worked on cloud for the same structural reason it works here, and it is the part enterprises consistently defer because it is organisational rather than technical.

Agents did not make inference expensive. They made it honest.

An agent that plans, calls tools, checks its own work and retries on failure performs far more inference per unit of business value than a chatbot answering a question. The reported spread is 5 to 30x. The reflex is to read that as waste. It is not. It is the mechanism by which agents do work a chatbot cannot.

What it does expose is that the earlier number was flattering. A chatbot was cheap because it was doing very little. The moment a system takes on a real task, with judgement and recovery and multiple steps, the cost of that task becomes visible for the first time. Most enterprises met that number after the architecture was fixed.

Which is the actual failure. Where an agentic system may retry, how many tools it may call, which model tier handles which step, and what happens when a task exceeds its allowance are all decisions with a price attached, and all of them get made at design time whether or not anyone prices them. By the optimisation pass they are expensive to change.

There is a useful accident buried in this. The boundary you draw around an agent for safety reasons turns out to be the same boundary that caps its cost. Greenlight sets that line per use case, from confidence thresholds, explicit rules and human guardrails, so an agent runs unattended on the high-volume path and stops at a defined point on everything else. An agent that stops at a defined point has a token ceiling. An agent permitted to keep trying until it succeeds has neither a ceiling nor an owner, and it is a risk problem and a cost problem in the same breath. Enterprises tend to fund the first framing and inherit the second.

Four levers, and the one that decides whether the other three hold

Four levers do most of the work. Their savings overlap, so a real programme is a stack rather than a sum.

Each one is covered properly, with the reported returns and the trade each carries, in AI inference cost optimisation. The short version is that the first three are engineering work and they are the ones that get funded. The fourth is the one that gets skipped, and it is the only one that makes the other three survive contact with a roadmap. A compression win nobody owns decays the moment the next feature ships, and it decays silently, because the thing that regressed was a ratio nobody was watching.

The one number worth governing

Here is the threshold I would hold a programme to. If you cannot state the cost of one completed task, in dollars, for your three highest-volume AI use cases, you are not managing AI cost. Our cost per completed task calculator works it out from your own figures, and the field most people cannot fill in is the one that matters. Price per million tokens will not tell you this. It is an input to the number, and on its own it is close to meaningless, because it says nothing about how many of those tokens one useful outcome takes.

This is also the argument for proving a use case on one real process before it becomes a programme. Our own governed agent work reports what a run cost against how the same work ran before we touched it, which is cost per completed task with a baseline attached rather than a rate in isolation. Not because measurement is virtuous, but because a number without a baseline cannot tell you whether to extend the pattern or stop.

Cost per completed task is the number that makes every other decision tractable. It tells you whether a use case earns frontier access or gets routed down a tier. It tells you what a retry is worth and when a task should fail loudly rather than quietly consuming its way through an allowance. It survives a model price change, because when the rate drops the number improves and you can see by how much.

For Australian enterprises setting FY27 budgets, this matters more than it does elsewhere, and for an unglamorous reason: regional inference capacity, data residency requirements and cross-region latency all push work toward architectures with more calls in them, not fewer. A budget built on a global rate card and an offshore assumption will read low. Build the number from your own tasks, in your own region, before the year is committed.

None of that is a procurement exercise. It is architecture and operating model, which is why cost control tends to fail as a negotiation and hold as an engineering practice. That is the reason our AI FinOps work sits inside managed services rather than beside a contract, and if the immediate bill is cloud rather than tokens, cloud optimisation is the faster place to start on the same discipline one layer down.

Sources

Figures in this article are compiled from the following published sources. Each should be read in full before a number is used in a business case.

1. Correlation One. How to Manage AI Token Costs in the Enterprise: The 2026 Playbook. correlation-one.com
2. Spheron Network. AI Inference Cost Economics in 2026: GPU FinOps Playbook. spheron.network
3. Optimum Partners. AI Token Costs: Why Enterprise AI Bills Keep Rising in 2026. optimumpartners.com
4. TrueFoundry. AI Cost Optimization Strategies for 2026. truefoundry.com