AL+

RO EN

Email us

The cost of adding AI to an app is mostly engineering

GPT-4o halved GPT-4-class prices in May 2024. Two and a half years on, the model bill rarely decides an AI feature's cost; integration and upkeep do.

. By . AI and agentic engineering

Citiți articolul în română

On 13 May 2024 OpenAI released GPT-4o and made it available in the API the same day. Most of the coverage was about the voice demo. The line that mattered to anyone paying for an AI feature was further down: GPT-4o matched GPT-4 Turbo on English text and code and was "2x faster, half the price, and has 5x higher rate limits". In the developer announcement, that meant $5 per million input tokens and $15 per million output tokens. Simon Willison pointed out the same day that this was exactly ten times the price of GPT-3.5.

Two and a half years on, the price drop is the part of that week worth remembering, because of what it did to the arithmetic. For a small or medium business adding AI to an existing app, the model bill is now rarely what decides whether the feature pays off. Integration, evaluation, guardrails and maintenance decide it, and that is where the budget should go.

The prices kept falling

GPT-4o was not a one-off discount. Two months later OpenAI released GPT-4o mini, which Simon Willison worked out was 33 times cheaper than GPT-4o for input and 25 times cheaper for output. In August a new GPT-4o snapshot, gpt-4o-2024-08-06, cut the price again to $2.50 per million input tokens and $10 for output, half the input price of the May version.

Those are still the list prices for GPT-4o and GPT-4o mini on OpenAI's pricing page today, and the cheapest model in the current flagship table at the top of that page, gpt-6-luna, is listed at $0.10 per million input tokens and $0.50 for output.

A cost model you can apply

Take an ordinary feature: the app reads an incoming customer request of about 2,000 tokens and drafts a 300-token reply for a person to check. Assume 10,000 requests a month. The arithmetic below uses list prices only and ignores caching and batch discounts.

Model and price Input per month Output per month Total per month
GPT-4o, May 2024 ($5 / $15) $100 $45 $145
GPT-4o, August 2024 ($2.50 / $10) $50 $30 $80
GPT-4o mini ($0.15 / $0.60) $3 $1.80 $4.80

These numbers are illustrative, not measured on a real system. The point is the order of magnitude. At these prices the model costs somewhere between a few dollars and about $150 a month. Put that next to what a week of an engineer's time costs you, and the question of where the money goes answers itself.

The useful split for a business owner is between what scales with use and what does not.

Costs that scale with use: tokens, hosting for the parts that call the model, and the time your people spend checking drafts. Only the first of these fell sharply in 2024.

Fixed engineering costs: connecting the model to your data and tools, deciding what it may and may not do, building the screen where a person approves the result, testing it on past examples, logging what it did, and handling the cases where it is wrong, slow or unavailable.

Recurring engineering costs: keeping all of that working as the provider changes models, prices and APIs.

The fixed and recurring lines did not get cheaper when tokens did. A prompt that drafts replies is a weekend's work. Making sure the draft quotes the current price list, never promises a delivery date the warehouse cannot meet, and reaches a named person before it reaches a customer is the project.

The recurring line is larger than it looks

The best evidence for the maintenance cost is OpenAI's own deprecations page. The original gpt-4o-2024-05-13 snapshot, the model launched that week, was announced for deprecation on 22 April 2026 and shut down on 23 October 2026. On 14 October 2026 OpenAI notified developers that the later GPT-4o snapshots, GPT-4o mini and GPT-4.1 will be removed from the API on 14 January 2027.

So a feature built on GPT-4o in May 2024 has already had its model changed or retired at least once. If the feature remains in use and still depends on a snapshot named in the notice, it needs a replacement model by 14 January 2027. Each move is engineering work: run the new model on the same past examples, compare the answers, adjust the prompts and limits, and release. A team that never built a set of test examples in the first place has to do that comparison by hand, or skip it and find out from customers.

This is why I think the budget should sit with the evaluation and the integration. Betting that tokens would get cheaper was right. Betting that the integration would get cheaper, or would not need touching, was not.

The strongest counterargument

The per-token price of a given level of capability fell. The total token bill did not always follow.

Reasoning models generate internal tokens before they answer, and OpenAI's documentation says those reasoning tokens are billed as output tokens, from a few hundred to tens of thousands per request depending on the problem. An agent that calls tools in a loop sends the growing conversation back to the model at every step. The frontier also did not get cheaper: gpt-6-astra, at the top of today's pricing table, lists at $10 per million input tokens and $50 for output, well above GPT-4o's launch price, and the "pro" models cost more still.

That is a fair objection, and it changes the advice in one way. Usage cost is no longer something to estimate once and forget. It is a number to measure on real traffic and cap in code, for example with a limit on output tokens per request and a monthly budget alert. But it reinforces the thesis rather than overturning it. Choosing the cheapest model that passes your test examples, limiting the steps an agent may take, and watching the bill are all engineering work. High bills can also reflect valuable complex work or legitimate volume; the bill alone does not diagnose engineering quality.

Where this does not hold

The conclusion is for businesses adding AI to a defined task with moderate volume. It does not hold in three cases.

Very high volume. If you process millions of documents a month, the token bill is a real line item and model choice, caching and batch pricing deserve the same attention as the code.

Long-running agents and heavy reasoning. A feature that lets a model think at length or work through many tool calls per request can cost dollars per task. Measure before you promise a price to your own customers.

When the right answer is no AI. If the task is a fixed rule, a form or a report, a plain automation is cheaper to build and has no model to migrate. A good supplier tells you this before quoting an AI feature.

What to ask before you pay for an AI feature

Ask the supplier to separate the three lines: estimated monthly model cost on your volume, the one-off build, and what keeping it working will take. Ask which past examples the feature will be tested on, who approves its output, and what happens when the provider retires the model. If the quote is mostly about the model, it is looking at the cheapest part of the work.

Our offer for AI agents and AI features starts with one task written down, tested on your past examples, a person approving the result, and running costs estimated separately from the build.

Sources