How to work out what one AI conversation actually costs you

One AI conversation costs the sum of every model call it triggered, priced from the token counts in each response, plus the retrieval, tool vendor and retry spend attached to those calls. To answer finance, attribute that sum at the unit you price on: per customer, per feature, or per resolved task. The provider invoice cannot do this for you, because it aggregates all of it into one line.

The invoice is one number

The bill from the model provider arrives as a monthly total, split at best by API key and by model. The support assistant, the internal summarizer, the eval runs an engineer kicked off on Tuesday, and the nightly job that re-indexes the help center all share that total. When a finance lead asks what the AI feature costs per paying customer, nobody on the engineering side can point to a figure without a modelling exercise.

The gap has a specific cause. Cost is created per model call, at the moment the provider returns a usage object, and the tenant, end user and feature that caused the call are known only in the process that made it. If that process does not write cost and identity down together, the information is gone by the time the invoice arrives, and no amount of spreadsheet work recovers it.

Working out the figure means rebuilding the invoice from the bottom: price each call, roll calls into turns, turns into conversations, conversations into accounts, then divide by the outcome you sell. The steps below do that in order, and the worked example after them runs the arithmetic for a support assistant.

How much does each AI chat conversation cost me

1. Price each model call from its usage object

Every provider returns token counts with each response. The names differ (input_tokens and output_tokens, prompt_tokens and completion_tokens, with separate fields for cache reads and cache writes), but the shape is the same, and it is the only trustworthy source. Character-based estimates miss the cached portion entirely, and on a streaming response the usage arrives in the final chunk, which a client that stops reading early never sees.

The formula for one call:

call_cost = uncached_input_tokens * price_input
          + cache_read_tokens     * price_cache_read
          + cache_write_tokens    * price_cache_write
          + output_tokens         * price_output

Cached tokens get their own terms because their prices differ from plain input, in both directions: providers generally bill a cache read at a discount and a cache write at a premium over the plain input rate, and the premium and discount vary by provider and model. An agent with a long system prompt and a large tool set can have most of its input volume in the cache-read term, so folding it into plain input overstates cost several times over. How to structure prompts so that term stays large is covered in prompt caching.

2. Roll calls up to turns

A turn is one user message and everything the system did before replying. In a plain chat completion that is one call. In an agent with tools it is usually two or more: a call that ends in a tool request, the tool execution, then another call that reads the tool result and answers. A retrieval step, a guard model that classifies the message, or an advisor model that reviews the draft each adds a call.

Attach every call to the turn that caused it and sum. Two figures fall out that are worth tracking on their own: calls per turn, which rises when a tool returns something the model reads as a failure and it retries, and input tokens per turn, which rises through a conversation because each call resends the history. The fifth turn of a conversation typically costs more than the first even when the user's message is shorter.

3. Roll turns up to conversations, then to accounts

Sum turns to get a conversation, keyed by whatever your product calls a session. Then group conversations by tenant (the paying account) and, where it matters, by end user inside the tenant. This step is only possible if every call carried those identifiers, and the places that forget are predictable: a background job that summarizes a thread after it closes, a webhook handler that replays a failed turn, a scheduled digest. Each of those creates spend that lands under an empty tenant, and the empty-tenant bucket is the first thing to check when the roll-up does not reconcile with the invoice.

4. Add the parts the model invoice hides

The model provider's line is the largest cost and rarely the whole cost. Fold these in per conversation, or amortize them per conversation where they are batch costs:

• Retrieval. Embedding the user's query is a small model call; embedding the knowledge base is a batch cost to amortize over the conversations that use it. A vector database bills by storage and by query.
• Tool vendor calls. A web search, a geocoding lookup, a CRM read: each has a per-call price, and an agent that calls search four times in one turn pays it four times.
• Retries. Client libraries retry on rate limits and timeouts by default, and a retried call is billed in full. Count the retries your client made, not the responses your code saw.
• Failed and abandoned runs. A turn that hit a tool timeout after two model calls still cost two model calls. A conversation the user closed mid-answer still cost the output tokens generated before the stream was cut.
• Evals. Judge-model scoring and regression suites are model spend that never touches a customer. Amortize a month of eval spend across that month's conversations, or report it as a separate overhead line, but do not leave it inside the customer figure unlabelled.

5. Convert to cost per resolved task

Finance prices outcomes, so the last step divides by outcomes. Define what a resolved task is for the feature (a ticket closed without escalation, a draft accepted, a booking made), measure the share of conversations that reach it, and divide:

cost_per_resolved_task = total_conversation_cost / resolved_conversations

The denominator is the number of conversations that resolved; the numerator includes every conversation, resolved or not, because the unresolved ones cost money too. This is the figure that can sit next to a price, and it is the one that moves when the model gets better at the task even if per-conversation cost is unchanged. What to charge for it is a separate question from what it costs.

Worked example: a support assistant

The prices below are illustrative placeholders chosen to keep the arithmetic readable. They are not any vendor's current rate; substitute the figures from your provider's price sheet.

The assistant answers questions about a product's help center, with a system prompt and tool definitions totalling 6,000 tokens in a cached prefix. A typical conversation runs four turns. Each turn is two model calls (one that requests a search, one that answers from the results), and each call carries about 3,000 uncached tokens of history, retrieved passages and the user's message. The tool-request call emits about 100 output tokens and the answering call about 350.

Three things in that table tend to surprise people. The search vendor is a third of the total, so a tool-call budget matters as much as model choice. The cached prefix, at 48,000 tokens, is two thirds of all input volume but a sixth of the input cost, so anything that breaks the cache (a timestamp in the system prompt, tool definitions that change order between calls) multiplies that line by ten.

The embedding call for each query, at roughly 50 tokens, rounds to nothing, which is why it was left out. The knowledge-base indexing that makes retrieval possible is real spend, but it is a batch cost amortized across every conversation in the month rather than a line in any one of them.

If 70% of conversations resolve without escalating to a human, cost per resolved task is $0.0589 / 0.70, about $0.084. An account that runs 1,200 conversations a month costs $70.68 in variable spend before the amortized eval and indexing overhead, which is the number to put next to that account's plan price. Whether the account should be allowed to run 12,000 conversations at the same price is the subject of usage limits per customer.

Reconcile against the invoice

A bottom-up figure is only credible once it matches the top-down one. Sum every recorded call for the billing period and compare it with the provider's invoice line for the same key and model. A gap of a few percent is normal, since price tables change mid-month and some usage objects are lost on aborted streams.

A gap of 20% or more usually has one of three causes: calls made outside the instrumented path (a notebook, a script, a second service on the same key), retries the client library made without reporting them, or the empty-tenant bucket from step 3 growing quietly. Once the reconciliation holds, the per-customer and per-feature figures inherit its credibility. The follow-up questions then become tractable: which accounts are underwater, which feature is the outlier, and whether a cheaper model on the tool-request call would move the total, which is the territory of model routing and of reducing LLM cost without breaking the feature.

Where this gets easier

Runtype records cost per execution, per record and per batch as the agent runs, with cached and uncached token counts separated and the tenant and end-user identity of the request attached to each figure, so per-customer and per-feature cost is a query over recorded executions rather than a modelling exercise rebuilt from the invoice.