We have not accounted for the true costs of AI coding

As AI models continue to become more capable, it is easy to focus on the immediate productivity gains they offer after watching one-shot demos all over YouTube and social media. And do not get me wrong, there are real gains to be had, no question about it. However, we often forget to talk about the real costs involved in making AI-generated code trustworthy for things that really matter.

We used to rely more on human steering and correcting the output as it was being created. But we quickly realised that this was inefficient, so we shifted more of the responsibility onto the AI itself, hoping it could handle the reasoning and checking that humans used to do. Unfortunately, this required more reasoning tokens, tool calls, and context—shifting the previous time cost from human steering to the token costs of the AI.

And what many people are starting to notice is that these incredible gains come at a huge cost. We see this with cutbacks on spending, more conservative subscription offerings, lower usage limits, and more. It seems to me the reality is starting to catch up with the excitement; with venture capital money running out, we are beginning to see the true costs of relying on AI for coding.

That is what I mean when I say we have not accounted for the true costs of AI coding. We are making claims about the economics of finished software using a partial account of what it takes to produce it to a reliable standard.

The cost of AI coding includes everything required to make the generated change worth keeping.

Reliability Has a Recurring Cost#

Imagine asking an agent to add a report to an existing application. It produces a page, a query and a few tests. The page loads. The numbers look plausible. The first result arrives quickly enough to make the old way of working feel painfully slow.

Then the questions begin. Does the query use the right date boundary? Does it respect the existing access rules? What happens when a customer has no records? Did the tests check the requirement, or simply confirm the assumptions in the generated implementation?

This is a hypothetical example, but it captures the accounting problem. The first response is only part of the work. Every investigation, correction and validation step belongs in the cost of delivering that report.

An agent can do some of that checking itself. It can inspect more files, run tests, reason through a failure and revise the change. Those are useful capabilities, and they consume resources. A second agent reviewing the first agent adds another opportunity to catch mistakes, along with another bill. Agreement between them still needs evidence.

More computation can help on difficult tasks. It can also prolong an approach that should have been abandoned. The relevant question is how much additional confidence the next round buys, and whether that confidence is justified.

This is the operational risk I am concerned about: a workflow can appear affordable at the point of generation, then become expensive at the level of reliability we actually need. The extra work recurs with each change. It is not settled by buying access to a capable model.

The Subscription Is an Incomplete Price Signal#

A fixed monthly fee is convenient for a developer. It makes experimentation easier because every prompt does not require a spending decision. But it can also make very different workloads feel as though they cost the same.

A short completion and a long agent session both fit inside the same subscription experience. Underneath that experience, one may require much more model work, tool execution and retained context than the other.

GitHub made this distinction explicit in its announcement of usage-based Copilot billing. It described absorbing rising inference costs and moving towards billing based on token consumption. That is one provider making more of the underlying cost visible.

The broader point holds regardless of which tool we use. An included allowance tells us what the vendor currently charges us. It does not establish what our workflow costs to serve, or what the same volume of work will cost under a different commercial arrangement.

That matters when a prototype becomes a routine way of working. A process that is attractive while usage is bundled needs another look when each retry has a visible price. The work has not changed. Our view of its cost has.

Human Attention Still Belongs in the Calculation#

There is also a tempting way to overstate the savings: count the time the agent spent writing, then treat the engineer as free for everything that followed.

Writing the brief takes time. So does selecting useful context, answering questions, inspecting the diff and deciding whether the tests provide enough evidence. When an agent takes the wrong direction, someone has to recognise that and explain the correction.

I want to apply this standard to my own work. If I spend an afternoon supervising an agent through a change, that afternoon belongs in the calculation even if I typed very little code. My role changed; my time still has a cost.

This does not mean AI assistance has failed to save time. A good run can remove substantial implementation work and leave me with a manageable review. The comparison needs to include the whole task in both cases.

Human-written code also requires review, tests and correction. Charging all verification to AI while pretending the alternative arrives correct would be just as misleading. What matters is the difference in total effort at a comparable quality level.

The difficult cases are those where fast generation hides slow understanding. An agent can hand over a large change before the reviewer has formed a clear picture of it. Reading that change carefully may become the longest part of the job. Asking the same agent to explain it helps with navigation, but the explanation is still something to verify.

Cheap Generation Can Increase What We Have to Maintain#

The cost continues after the pull request is merged.

Suppose an agent implements a feature by adding a new dependency, another service and a separate configuration path. The feature works, and generating those pieces took little time. The team now owns the upgrades, failure modes and interactions between them.

That obligation exists whether a human or an agent wrote the code. AI changes how quickly we can accumulate it.

When implementation becomes easier, more features become tempting. So do larger rewrites and custom versions of things we could have done without. A team can save time on each individual change while increasing its total maintenance burden by accepting more changes.

This is why I am wary of treating output volume as a productivity measure. A smaller implementation that reuses an existing component can be more valuable than a larger one produced at impressive speed.

There is a similar distinction in agent workflow design. Some tasks deserve extensive investigation because they are difficult. Others consume repeated reasoning because the ownership, requirements or architecture are unclear. In the second case, we are paying the model to rediscover decisions that could have been made explicit.

Clear contracts and smaller changes have their own preparation cost. They can reduce repeated investigation across future tasks. That is a trade-off worth measuring, rather than assuming either more prompting or more structure is always cheaper.

We Need to Count the Whole Change#

I would start with a few comparable tasks and follow them far enough to see what happened after generation.

The account should include model usage, tool execution and test infrastructure, along with human time spent specifying, supervising, reviewing and correcting the work. It should also record whether the change was accepted, whether it needed rework and what new maintenance obligations it introduced.

Failed attempts belong in that account. If several abandoned runs preceded a successful one, the final run did not deliver the result on its own.

Some costs are straightforward to measure. Others, especially future maintenance, need estimates that can be revised as evidence arrives. That is still more useful than silently assigning them a value of zero.

For each task, I want to know:

• What did it cost to reach an accepted change, including unsuccessful attempts?
• How much human attention did it require compared with similar work?
• Did the change remain correct after integration and use?
• What did we add that somebody will now have to maintain?

Those questions also change how I would respond to a rising bill. A more capable model may be worth its price if it reduces correction and review. A cheaper model may be the better choice for a narrow, well-specified task. Another round of reasoning may help, or the task may need a clearer requirement before any model continues.

The useful comparison is the cost of completing the work to the required standard. Token prices and generation speed are inputs to that comparison.

The Savings Need to Survive the Full Account#

I still see a strong case for AI coding. Faster implementation, easier experimentation and help with unfamiliar code can all be valuable. I want those benefits assessed against the work we actually depend on, including the less impressive parts that happen after the demonstration.

My concern is that we have been too quick to convert a good generation experience into a claim about cheap software delivery. Reliability, human judgment and maintenance remain part of the process, and some of the ways we secure them are expensive.

We should be able to say where the savings came from and what we spent to obtain them. Until I can include the retries, supervision, checking and later corrections in that account, I cannot confidently say how much a fast first result saved me.