AI agent platform for customer-facing products

An AI agent platform is the software that runs LLM agents in production: it executes the model and tool loop, resolves credentials, exposes the agent on the surfaces users reach it from, records every step with its cost, and enforces limits on what the agent can spend and do. It works with the agent framework you already picked: a library gives you the loop, and the platform is the operational layer around it, kept consistent across every agent you ship.

The case this page covers is narrower than most definitions: agents a software company's customers use inside its product, rather than agents your own team runs. That case changes four things (identity, isolation, cost attribution and irreversible actions), and each has a page of its own linked from the section that names it.

What a platform has to do

The vendor and the architecture vary, but the same seven jobs come up in every production deployment. The right-hand column is what tends to happen without one.

Why running agents in production is hard

Each failure mode below has a mechanism, and the mechanism tells you which control prevents it.

The tool that gets called until the budget is gone

A search tool returns an empty array. The model reads that as a failed call rather than an empty result, rephrases, and calls again, and the eleventh call looks exactly like the first from inside the loop. Nothing in the model stops this; the bound sits outside it, as a per-turn cap on tool calls and a cap on turns per run. How to find and stop this pattern in a live trace is covered in debugging an agent stuck in a loop.

The credential in the prompt

An API key for a billing system gets pasted into a tool definition or a system prompt to make the demo work. From then on the key goes to the model provider on every turn, lands in every trace, and appears in any log a support engineer exports. The fix is structural: tools reference secrets by name, the runtime resolves the name at dispatch, and the model never sees the value.

The second surface

The agent starts as web chat. Sales asks for Slack, then a customer wants to call it from their own backend, and each gets its own copy of the prompt and tool list. Six weeks later a tone fix in web chat has not reached Slack, and the API version still calls a tool the others retired. The control is one agent definition that every surface renders, with the channel-specific parts (formatting, approval prompts, auth) held in the surface.

The turn that outlives the request

A turn that chains five tool calls with a slow HTTP tool in the middle takes longer than the load balancer's idle timeout. Without a run handle the client disconnects and the work is either lost or continues unwatched. A platform gives the turn an identity of its own that the caller can poll, watch under a lease, and cancel, with a wall-clock budget so an abandoned turn cannot run forever.

The prompt change nobody measured

Someone edits the system prompt on a Friday to fix one complaint, and the change quietly regresses a different kind of question. Support tickets arrive on Tuesday. Prevention is an eval suite with cases promoted from real executions, run before the change ships, so the regression is a red row rather than a ticket. Scoring answers that have no single right output is covered at AI agent evals.

How it works

A platform reduces the jobs above to a definition the runtime executes. The shape below is illustrative rather than any one vendor's schema, but every field in it corresponds to a control described on this page:

{
  "agent": {
    "model": "claude-sonnet-5",
    "systemPrompt": "You are the support assistant for Acme Billing...",
    "tools": [
      { "name": "lookup_invoice", "kind": "http", "auth": "{{secret:BILLING_API_KEY}}" },
      { "name": "issue_refund", "kind": "http", "requiresApproval": true }
    ],
    "limits": { "maxToolCalls": 10, "maxTurns": 20 },
    "tenancy": { "preset": "end-user-isolated" }
  },
  "surfaces": ["web-chat", "slack", "rest-api"]
}

The model and prompt are versioned as a unit, so a change is a draft that can be evaluated before it is published. Each tool carries its own auth reference and limits, with the secret resolved server-side at dispatch. The per-turn maxToolCalls bound stops the eleven-call loop, and requiresApproval on the refund tool turns an irreversible action into a request a human answers. The tenancy preset is evaluated before execution, so a request that cannot prove who it is for is rejected rather than run and filtered later.

A message from Slack and a request from a customer's backend both resolve to the same definition, the same identity check, the same tool limits and the same trace format, so observability and evals see one agent rather than three.

What changes when the agent is customer-facing and multi-tenant

An internal agent has a simple environment: one company, engineers as its users, one bill, and a human nearby who can undo a mistake. An agent inside your product serves thousands of tenants and strangers typing whatever they like, its bill needs owners, its actions land in customers' data, and four things change with it.

Identity: who is this request for?

Every request must carry a tenant and, where the product has one, an end user, along with how strongly that identity was proven. A tenant id asserted by a browser is a different thing from one verified against a signed proof, and the platform should grade the two differently. The strategies for carrying identity through web chat, Slack and API calls, and for deciding what a resource requires, are at multi-tenant AI agents.

Isolation: what can this request reach?

Memory, records, connected credentials and retrieved documents must be keyed by tenant and end user so that a recall for one is impossible for another. The failure here is rarely a bug in the store; it is a code path that queried without the key, or a tool that fetched from a shared index. An instruction planted in one tenant's document that asks the agent to read another tenant's data is the case to design against; the threat model is in AI agent security and the walk-through in the tenant isolation checklist.

Cost attribution: who pays for this turn?

Cost per execution has to carry the tenant at the moment it is incurred, because reconstructing it later from logs means trusting that every entry point set the right field. Once each figure has an owner, two decisions become possible: routing cheaper models to lower tiers or lower-stakes steps, and capping what one customer can spend. The routing side is at model routing, the cap side at per-customer AI usage limits, and reading the cost and latency of a single run is covered in AI agent observability.

Irreversible actions: what happens when the agent is wrong?

Reading an invoice is recoverable. Issuing a refund, sending an email under the customer's name, or deleting a record is not, and a model will eventually do one of those for a bad reason, from a hallucinated tool argument or an instruction planted in a web page it retrieved. The control is an approval gate on the tool, with a timeout so an unanswered request fails closed.

The "reason" an agent gives for the action is the agent's own claim, so a policy must decide on the tool name and its parameters, never on how persuasive the reason reads. Patterns for the gate, the approver's view and resuming after approval are at human-in-the-loop AI.

Build, buy, or keep the framework

A team can build the platform layer in-house on top of a framework, buy one, or keep the framework and add the missing controls piece by piece. The trade-offs are laid out in build vs buy for an AI platform, and a survey of the products that do run the agent is at the best AI agent platforms.

Framework choice is a separate question from platform choice. A team happy with LangGraph can keep it and still need the four controls above; a team that finds the graph model heavy for its case should read LangGraph alternatives before deciding. Decide up front how the feature will be judged, because an agent nobody measures cannot justify its bill; measuring AI feature adoption covers what to instrument on day one.

Where Runtype fits

Runtype is AI product infrastructure for software companies. An agent you already run comes into it in four ways that stack, and a full rebuild is the last of them.

• Register the agent. An external agent whose endpoint speaks Runtype's unified stream or A2A is called by Runtype, so it can be tested from the dashboard, embedded in the open-source Persona widget, added to a product as a capability and exposed on every surface. Tool calls and cost are recorded per run, and the agent code is unchanged.
• Send traces. An OpenTelemetry-instrumented loop exports OTLP over HTTP to https://api.runtype.com/v1/otel for the Runs view, the trace tree, token usage and a display-only cost estimate. Point exactly one instrumentation at the endpoint, because two doubles tokens and cost. Spans carrying the GenAI content attributes give the run a transcript that can become an eval case.
• Serve it tools. An MCP surface publishes a product's flows, agents, records, skills and tools as MCP tools, so your existing loop stays the orchestrator and calls Runtype for the parts worth centralizing.
• Rebuild when it earns it. Port one capability to a hosted flow or agent once an eval suite harvested from real runs can prove parity. It is then versioned, published and gated: runtype eval run returns a non-zero exit code in CI on a regression.

These layers are built for a software company's customers. Identity is a declared property of each resource, evaluated before execution, so a request that cannot prove which tenant and end user it is for is rejected before it runs:

{
  "config": {
    "tenancyStrategy": {
      "preset": "end-user-isolated",
      "assuranceFloor": { "endUser": "verified" }
    }
  }
}

Every trace and every cost figure is filed under that tenant and end user, and long-term memory, when enabled, is keyed per agent, tenant or end user.

The failure modes above map to controls: a per-turn maxToolCalls (default 10, maximum 100), a maxTurns cap and an optional per-run cost ceiling bound the eleven-call loop; a turn that outlives the request becomes a durable execution with a 30-minute budget; an irreversible tool waits on an approval gate with a default 5-minute timeout, and the agent's stated reason is recorded and never read as a control signal.

Tools reference credentials as {{secret:NAME}}, resolved server-side so the model never sees the value, and a prompt change is gated by eval suites scored by an LLM judge with human review. The runtime runs in the managed cloud or on your own infrastructure.

Whichever way a team comes in, the agent keeps its own code while the platform holds the identity, the surfaces, the limits and the record.