Human-in-the-loop AI: approval before anything irreversible

Human-in-the-loop AI is a design in which an AI agent must obtain a person's decision before it performs certain actions, and the pause, the decision and the outcome are all recorded. The actions that need a decision are chosen by their blast radius (what they change, for whom, and whether it can be undone), and the gate is enforced by the runtime that executes the tool call rather than by an instruction in the prompt. For an agent your customers use, that gate is the difference between a model that was asked to be careful and a system that cannot issue the refund without a signature.

Every agent framework lists human approval as a feature. Most implement it as a callback or a prompt convention that works in the demo and stops working when the model is swapped, the tool list grows past twenty, or a customer pastes a document that tells the agent to skip the check. The step-by-step version of what follows is in the guide to adding a human approval step to an agent.

Why human-in-the-loop AI is hard

The gate lives in the prompt

"Always ask before sending an email" in a system prompt is a request, and the model grants it on each turn with some probability. Compliance shifts with the model version, drops as the context fills with tool results, and can be reversed by text the agent reads: a web page, a PDF, a support ticket that says the user has pre-approved all actions. The tool call is issued by the model, so if nothing sits between issuing and executing, there is no gate. How injected instructions reach tool calls is covered in prompt injection through tools.

Everything is gated

The opposite failure is a policy that asks for approval on every tool call. Within a week the approver is clicking through without reading, and the one dangerous call in a hundred gets the same reflex approval as the ninety-nine lookups. A gate only works while approvals are rare enough to be read.

The run dies while it waits

Most agent loops run inside one request: a serverless function, an HTTP handler, a worker with a wall-clock limit. A decision that takes twenty minutes outlives all of them. If the runtime cannot persist the execution state (the conversation so far, the pending call, its parameters) and resume from that exact call later, the fallback is to re-run the turn after approval, and a re-run model call may choose different parameters than the ones the human approved.

The approver sees the justification instead of the action

A tempting approval UI shows the agent's explanation: "I need to email the customer their invoice." The explanation is written by the model, and under prompt injection it is written by the attacker. The approver must see the tool name and the exact parameters, with credentials redacted, and the justification only as a labelled claim. Any code that branches on the justification (an auto-approve rule, a risk score, a classifier) turns attacker-writable text into an authorization input.

Nobody is there to answer

Scheduled runs, batch jobs, webhooks and API calls with no live caller all reach the same tools, and a gate that fires there has no one to ask. The two acceptable behaviors are to refuse the run before it starts, naming the policy that blocked it, or to route the decision to a person somewhere else. Hanging until a timeout, or skipping the gate because no approver is attached, are the behaviors that ship by accident.

How it works

A gate that survives a model swap has six parts, each enforced by the thing that executes the tool.

Classify actions by blast radius

The unit of gating is the tool. Each tool the agent can call gets a class based on three questions: does it change state, does that state belong to someone outside the system, and can the change be undone.

If one tool covers both a five-dollar credit and a five-thousand-dollar refund, split it into two tools so the gate can tell them apart. The classification is a property of the tool definition, so it holds across every agent that uses the tool and every model that runs the agent.

Where the approval surfaces

The options for collecting the decision differ in how much they trust text. Explicit controls (an approve and deny button in a dashboard, native buttons in a Slack thread) return a decision with one meaning. Reply-based approval (someone texts YES from a phone) requires parsing, and anything past an exact keyword set goes through a classifier that can misread.

An API-hosted approval hands the pause event to your own application, which renders it and posts the decision back. In every case the decision must come from the conversation and participant that triggered the run, so a reply from another number or another user does not resolve it. And an entry point that cannot host a human (a schedule, a batch, a bare API call) must fail closed before the agent starts, with an error that names the approval policy.

Keep the run alive while it waits

When the model issues a gated tool call, the runtime stops before executing it, persists the execution (messages, the pending call with its id and parameters), emits a pause event, and returns a handle:

{
  "type": "approval_start",
  "executionId": "EXECUTION_ID",
  "approvalId": "APPROVAL_ID",
  "toolCallId": "TOOL_CALL_ID",
  "toolName": "issue_refund",
  "parameters": { "orderId": "ord_4421", "amountCents": 12900 },
  "timeout": 300000,
  "startedAt": "2026-09-03T14:02:11Z"
}

A decision posted against the approvalId resumes the execution at that same tool call, with the parameters that were displayed. Nothing is regenerated. A non-streaming caller gets the same information as a paused response and resumes with the handle.

What the approver sees

The approver sees the tool name, the parameters with protected values redacted, who or what triggered the run, and, if the agent supplied one, its stated reason labelled as the agent's claim. The reason improves the prompt a human reads; it never enters the decision logic, and the UI keeps name and parameters at least as prominent as the explanation.

Timeouts and defaults

Every gate has a timeout and expiry means deny. The right value depends on where the approver is: a few minutes when a dashboard or API caller is waiting on the run, an hour when a person will answer from a phone. After expiry the tool does not run and a late reply does not resume the old execution, because the state the approver saw may have changed. To keep approvals rare, the policy usually pairs the gate with a per-turn tool-call cap and lets an approver choose "always deny" for a tool they never want to see again.

The audit record

The record that answers "who approved what" needs, at minimum: the approval id and tool call id, the tool name and parameters exactly as displayed, the approver's identity, the decision and its timestamp, whether the request expired, and the executed call's result. All of it should hang off the execution trace, so a reviewer can walk from the customer's message to the gated call to the human decision without joining logs from three systems.

What changes when the agent is customer-facing and multi-tenant

Most writing on human-in-the-loop assumes the human is the operator: an engineer or an analyst watching an agent their own team runs. An agent embedded in your product and used by your customers changes who the human is, and that changes every part of the gate.

The approver is now one of three people, and often a different one per tool. An end user approves actions in their own account ("send this from my address?"), a tenant admin approves actions with organizational consequences such as refunds over a threshold, and your own staff approve platform-risk actions. Each tenant may also want a different policy: one gates every refund, another gates nothing under fifty dollars. The policy has to be configurable per customer without forking the agent.

Identity becomes part of the gate. An approval from tenant A must not authorize an action on tenant B's data, so the runtime has to know which tenant the run belongs to at the moment it pauses, not infer it from a metadata field later.

The surfaces get more restrictive. A chat widget embedded in a customer's page runs under a client token, and the browser is the untrusted party, so it cannot safely host approval of a server-side tool. The correct behavior is to refuse an approval-gated request on that surface with a specific error rather than leave a run pending that no one can resolve, while the same agent reached through Slack or an authenticated dashboard hosts the gate.

Prompt injection stops being a hypothetical. A customer-facing agent reads content customers supply: uploaded files, pasted emails, records from their integrations. The tool call and the agent's reason may both be shaped by that content, which is why the gate must be enforced in the runtime and the reason must never be a control signal. The broader threat model is in AI agent security, and the content-side controls that pair with an action gate are covered under LLM guardrails.

Where Runtype fits

Runtype is an AI agent platform that enforces the gate in the runtime executing the tool call, whatever the model does or the prompt says.

Registration is the first step for an agent you already run. Create an external agent whose endpoint speaks Runtype's unified stream or A2A, and Runtype calls it and exposes it on web chat, Slack, REST, SMS, iMessage, MCP and A2A.

An instrumented loop can send traces first, exporting OTLP to https://api.runtype.com/v1/otel for the Runs view and trace tree. An MCP surface points the other way, handing a product's flows, agents, records and tools to a loop that stays the orchestrator.

Porting the gated capability is where the enforced pause arrives. A Runtype-hosted agent or flow names the tools needing approval in tools.approval.require, or requires approval for all of them, and execution stops at the call, with a five-minute default timeout. Expiry denies.

The pause emits an approval_start event carrying the approval id, tool name and parameters, with protected values redacted. The run is durable while it waits, and a decision posted through dispatch resumes that same call. The agent's stated reason arrives as a reserved _approvalReason parameter, displayed as its own claim; nothing branches on it.

Where the decision is collected follows the surface: approve and deny controls on the dashboard and in Slack, an exact-keyword reply on SMS and iMessage. Schedules, batches and API requests with no live caller fail closed before the agent starts, and the client-token chat surface refuses an approval-gated request with APPROVAL_MODE_UNSUPPORTED (approvals on messaging surfaces).

Each resource declares a tenancy strategy (internal, tenant-isolated or end-user-isolated) with an assurance floor evaluated before execution, so the run that pauses already carries the tenant and end user it was admitted under. An approval raised for tenant A cannot authorize an action on tenant B's data, and execution traces record every tool call with its arguments and result. The stored decision carries the outcome, its timestamp and whether a person or an expired timeout produced it, without naming the individual approver.