How to stop one customer's usage from destroying your margin

Limit AI usage per customer by resolving the tenant and end-user identity before any model call, metering spend against that identity rather than counting requests, and enforcing a budget with a defined behavior at the cap: degrade to a cheaper model, queue the work, or refuse with a message that says when it resets. Provider rate limits protect the provider's capacity, so this is a control you build yourself.

One trial account, one weekend

The shape of the incident is always similar. An enterprise prospect gets a trial workspace on Friday, someone on their side points a script at your assistant's API to "see how it handles our backlog", and by Monday the account has run a few tens of thousands of turns on your most capable model, each with several tool calls. The trial converts to a contract worth a fraction of what those two days cost you, or it does not convert.

Three mechanisms make this worse than it looks on a chart. The first is the noisy neighbor: your provider's rate limit is keyed to your API key, so one tenant burning through it means your other tenants see 429 errors from a problem they did not cause. The second is retry amplification: a client times out at 30 seconds, retries, and the original run is still executing server-side with its own tool calls, so every retry is a fresh run and the customer's script counts it as one request.

The third is the agent loop. A tool returns an empty array, the model reads that as a failed call and calls it again, and a turn that should have cost a few cents runs to whatever turn limit you set, or to none if you set none. That failure mode has its own page at why agents loop and how to bound them; the point here is that a budget alone does not stop it fast enough, because one run can spend a week's allowance in ten minutes.

How do I limit AI usage per customer

The controls below are ordered so each one bounds a quantity the next one depends on. Identity first, because nothing can be limited per customer until every request carries the customer. Then run-level caps, because a per-tenant budget is only meaningful if a single run cannot consume it. Then the budget, the behavior at the cap, and the alerting and display that turn a cap into a plan feature.

1. Put the budget where the identity is

Resolve tenant_id and end_user_id from the credential at the edge, before the request reaches anything that can call a model. A session cookie maps to a user in a workspace, an API key is issued scoped to one tenant, a Slack workspace id maps to a tenant, a scheduled job carries the tenant it was scheduled for. Attach both ids to the execution context and treat a request that reaches the runtime without them as a bug: fail closed and log it, rather than running it under an empty tenant where its cost disappears from every report.

Trial accounts are tenants with a different plan, never a shared "trial" bucket. The isolation half of this, keeping tenant A's data and memory out of tenant B's context, is covered separately in multi-tenant AI agents. For budgets, the requirement is narrower: every model call and every tool call must be attributable to exactly one tenant and, where the surface has one, one end user.

2. Separate rate limiting from cost limiting

These are different controls that catch different failures, and teams that build only one get surprised by the other.

A request limit is a poor proxy for spend because the cost of one request varies with the model, the context length and the number of tool calls; a two-request-per-second script on a flagship model with six tool calls per turn costs roughly a hundred times what the same script costs on a small model with none. A spend limit is a poor proxy for abuse because cost is known only after the run completes. Run the rate limit and the concurrency limit synchronously at admission, and run the spend limit against a meter that is settled per run.

3. Cap the shape of a single run

A tenant budget bounds the sum; run-level caps bound the unit. Without them, one looping run burns the whole period's allowance before the meter can react, and the customer's next request is refused for something they never asked for. Four caps cover most of it:

• A per-turn tool-call cap, so one model turn cannot call tools forever. Ten is a common default, and a well-behaved turn rarely needs more than four or five.
• A per-tool cap, so a search or fetch tool that returns nothing useful cannot be called twenty times in one turn.
• A per-run turn cap, so a multi-turn agent stops after a bounded number of model calls regardless of what it believes it still needs to do.
• A per-run ceiling on output tokens and wall-clock time, because a turn that is still generating after two minutes is almost never producing value.

{
  "run": {
    "maxTurns": 12,
    "maxToolCallsPerTurn": 8,
    "perTool": {
      "web_search": { "maxCalls": 3 },
      "fetch_url": { "maxCalls": 5 }
    },
    "maxOutputTokens": 2000,
    "wallClockMs": 120000
  }
}

With these in place you can compute a worst-case cost for one run (turns times the priciest model call plus tool costs), and that number is what the budget check reserves in the next step.

4. Set per-tenant and per-end-user budgets

Two levels. The tenant budget comes from the contract: decide what fraction of a plan's revenue you are willing to spend on inference, and set the limit from that. If a plan bills 500 a month and you want model spend under 15 percent of revenue, the tenant budget is 75 a month.

The end-user budget sits inside it so one seat cannot spend the whole company's allowance, and a daily figure works better than a monthly one here because it resets before anyone notices. Trial accounts get a total budget rather than a periodic one.

budgets:
  tenant:
    period: monthly
    limit_usd: 75
    soft_thresholds: [0.5, 0.8, 0.95]
    hard_action: degrade
  end_user:
    period: daily
    limit_usd: 2
    hard_action: refuse
  trial:
    period: total
    limit_usd: 20
    hard_action: refuse
rate:
  tenant:
    requests_per_minute: 120
    concurrent_runs: 4

Behind this policy the meter needs two operations, not one. At admission, reserve the worst-case cost of the run computed from the caps in step 3 and check used + reserved + estimate <= limit; at completion, settle the reservation to the actual recorded cost. A check that reads only settled spend lets four concurrent runs each pass with the budget nearly exhausted, because none of them has cost anything yet. Getting the actual figure right (cached versus uncached tokens, tool costs, retries) is its own job, covered in how to measure cost per AI conversation.

5. Decide what happens at the limit

The cap is a product decision, and the wrong default turns a paying customer's Tuesday into a support ticket. Three behaviors cover the cases:

• Degrade. Route to a cheaper model, drop optional tools, shorten retrieved context. This is the right default for a paying tenant past a soft threshold, because the customer keeps working and the margin recovers. The hub page on model routing covers how to make that switch without changing the agent's behavior in ways the customer notices, and routing to cheaper models covers which requests can take it.
• Queue. Defer background work such as scheduled summaries or batch enrichment to the next period, with a visible position. Never queue an interactive chat turn; a spinner that lasts until Monday is a refusal with worse manners.
• Refuse. The right answer for a trial that has hit its total, or an end user past a daily cap. The response must say what limit was hit, when it resets, and how to raise it.

on_request(tenant, user, estimate):
  if rate.exceeded(tenant) or concurrency.full(tenant):
    return 429 { reason: "rate_limited", retry_after: rate.reset_in(tenant) }

  for scope in [trial(tenant), end_user(tenant, user), tenant]:
    if spend.used(scope) + spend.reserved(scope) + estimate > scope.limit:
      match scope.hard_action:
        degrade -> policy = "economy"; break
        queue   -> return 202 { queued_until: scope.resets_at, position: queue.len(scope) }
        refuse  -> return 429 { reason: "budget_exhausted", scope: scope.name,
                                resets_at: scope.resets_at, remaining: 0 }

  reservation = spend.reserve(tenant, user, estimate)
  result = run(tenant, user, policy)
  spend.settle(reservation, result.actual_cost)
  return result with headers(remaining(tenant), remaining(user))

Check the narrowest scope first so an end user who is out of their daily budget gets the end-user message, not the tenant's. Whichever branch fires, record the decision on the run itself, so the customer's admin can later see that the answer they are asking about was produced under the economy policy.

6. Alert before the cap, not at it

A threshold alert at 80 percent is necessary and late. The alert that catches the weekend script is a pace alert: project end-of-period spend as used / elapsed_fraction and fire when the projection exceeds the limit. A tenant at 40 percent of budget on day six of thirty is on pace for 200 percent, and that is visible on Saturday morning, long before the 80 percent threshold trips on Sunday night.

Send both alerts to two audiences. Your own on-call needs the pace alert with the tenant id, the current model mix and the top end users, so they can tell a runaway script from a customer who is genuinely getting value. The customer's admin needs a plainer version: usage is running ahead of plan, here is who is driving it, here is where to raise the limit. A customer who hears about it from you before the cap reads the cap as a feature.

7. Show the customer their remaining budget

A limit the customer can see and raise is a plan feature. One they discover from a refusal is an outage. Return the remaining budget on every API response alongside the standard rate-limit headers, put a usage page in your product that breaks spend down by end user and by day, and give the tenant admin a control to set their own end-user caps below yours. The refusal message from step 5 then points at a page the customer has already seen.

The same records feed your own pricing. Once spend is settled per tenant, the gap between what a plan bills and what its tenants cost is a query, and the plans whose budgets are set too generously show up as the ones with negative margin rather than as a surprise at quarter end.

Where this gets easier

Runtype attaches tenant and end-user identity to every execution and records cost per execution, per record and per batch under that identity, with cached and uncached tokens separated, which is the meter any of the budgets above reads from. Three of the run-level caps in step 3 map to its per-turn maxToolCalls (default 10, maximum 100), loopConfig.maxTurns (1 to 100) and the optional per-run cost ceiling loopConfig.maxCost, and a tool that should not run unattended past a threshold can sit behind an approval gate with a timeout. The per-tenant budget check, the degrade, queue or refuse policy, and the customer-facing usage page are still yours to build on top of those cost records.