Costing an AI agent:the honest ROI model

Most AI agent proposals understate cost and overstate autonomy. The honest model has four terms: hours returned, run cost, evaluation overhead and failure cost. If you cannot write those four numbers on one page, you are not ready to build.

Hours returned is the easy one. Time a person doing the job for a week. Multiply by loaded cost. That is the ceiling. Everything else is leakage from that ceiling.

Run cost is more than tokens

Tokens are visible and usually small. The rest of the run cost is retrieval infrastructure, logging, tracing, tool-call retries, staging environments and the engineer who keeps the evaluation harness green. Budget a platform slice even for a single workflow. Agents that share a gateway are cheaper than agents that each invent one.

Evaluation is a product, not a sprint leftover

Before an agent touches production it needs a gold set: real inputs, expected actions, expected side effects. We score task completion, factuality, policy violations and latency. Autonomy is a privilege earned when those scores clear a threshold for a defined window — typically two weeks of supervised runs.

Skipping evaluation is how you get a polite agent that books the wrong meeting, writes the wrong SKU or emails the wrong customer. The incident cost of one of those dwarfs a year of tokens.

The decision table

A worked example

Lead qualification for a 12-person sales team. Manual handling: 9.4 hours per seat per week at £45 loaded. Ceiling: ~£22k/month. Agent run cost including evaluation: £2.1k/month. Supervised review of 15% of conversations: £1.4k. Residual failure reserve: £1k. Net: roughly £17k/month returned after the system is stable. Payback on a £18k build is inside the first quarter — if, and only if, the gold set exists and checkpoints stay on anything that writes to the CRM as “closed”.

What we refuse to automate

We will not put an unsupervised agent on refunds, contracts, medical or legal advice, or any message that leaves the building without a named owner. That is not caution for its own sake. It is the difference between a system finance will fund and a demo that gets rolled back after the first angry email.

If you want a number for your own workflow, send the job description and the tools it must touch. The diagnosis is a spreadsheet, not a slogan.

The cultural risk is treating agents as headcount replacement in the first conversation with staff. That is how programmes die in the second month. Frame the job as hours returned to advisory work, then prove it with a timesheet before and after. If you cannot show the hours, you do not have ROI; you have a demo.

Tool contracts deserve more design time than prompts. A prompt can be changed in an afternoon. A tool that can delete a CRM record cannot. Least privilege, typed inputs, idempotent writes and an audit log are the difference between an agent and a liability. We will not connect an agent to a system that cannot reverse a write.

Supervised mode is not a delay tactic. It is how you build the gold set from live traffic instead of synthetic examples. Two weeks of “the agent proposes, a human confirms” produces better evaluation data than a month of workshopping hypotheticals. Autonomy is a threshold you cross, not a launch toggle.

Finance will ask about model lock-in. Stay model-agnostic behind a gateway. Route classification to small models, reasoning to frontier models, and keep the tool layer stable when the vendor landscape moves. The ROI model should survive a price change in tokens; if it does not, tokens were never the point.

Finally, write the kill criteria before you build. If completion accuracy stays below the threshold for three consecutive weeks, the workflow returns to human. That sentence in the statement of work is what makes autonomy safe enough for a CRO to sign.

Operators should also write a one-page source-of-truth for claims: founding year, headcount, markets, flagship services, contact points and any number they are willing to see quoted. Every public page, schema block and off-site profile is then a copy of that page. When the number changes, change it once. Drift is how generative systems lose confidence in you.

Measurement cadences matter because these programmes fail quietly. A monthly prompt panel, a weekly schema validation, a quarterly contradiction audit against LinkedIn and directories — that is enough. Daily dashboards of vanity metrics are how teams convince themselves they are shipping. They are not a substitute for a panel that names who got cited.

Procurement will ask for guarantees. There are none for rankings, snippets or citations, and any partner who offers them is selling theatre. What you can contract is leading work: the fact layer, the panel, the evaluation harness, the hooks-per-week, the JS budget. Buy the work that causes the outcome, not the outcome as a slogan.

If this reads like engineering, that is the point. Marketing that still depends on a human pushing it every Monday is a subscription to exhaustion. The studio that treats GEO, AEO, agents, creative and the website as one operating system will keep the clients who have already noticed that the click is no longer the whole game.

A practical first week looks the same across these programmes. Export the current questions, claims and workflows. Circle the ones you cannot prove. Delete or rewrite them. Then pick one surface to make extractable: a homepage answer, a service FAQ that matches schema, a fact table, a gold set for one agent, or a Monday creative brief. Depth on one surface beats a thin layer on twelve.

Documentation is part of delivery. Clients keep the accounts, the code, the evaluation sets and the prompt panels. Lock-in is not a business model we use. If a system cannot be handed to an in-house team with a runbook, it is not finished, however good the demo looked on a Friday.

We work in English from DHA Phase 1, Lahore, overlapping UK, EU and North American hours, with WhatsApp and email as the two human lines. The diagnosis after a complete brief is written and lands within 48 working hours. That constraint is intentional. If we cannot describe the work in two days, we should not be selling it.

Use the takeaways at the top of this article as a checklist. If you cannot tick them with artefacts — a panel, a schema file, a budget gate, a gold set, a scoreboard — you do not have a programme yet. You have a slide. The rest of this library exists so the slide can be replaced with a system.

Ship the smallest complete version. A panel of thirty prompts is better than a plan for three hundred. A single production agent with a gold set is better than a roadmap of twelve. Six live hooks are better than a manifesto about velocity. A site that answers in the first paragraph and stays under budget is better than a concept film that never loads. Completeness at a small scale is how these systems start compounding. Incompleteness at a large scale is how they become another subscription to exhaustion.