Running LangGraph agents multi-tenant

Keep LangGraph. It owns the loop, the typed state schema and every edge between nodes, and serving several customer accounts does not ask you to give any of that up. What it leaves open is tenant identity flowing through retrieval and tools, spend attributed to an account, and the surfaces the agent is reached through.

This page is written for a team whose graph already runs and now has to serve accounts that must never see each other's data. Every change below sits at the edge of the graph or in the config passed into it. None of them rewrites a node.

What LangGraph does well

The state schema is the part most agent libraries skip. You declare a TypedDict or a Pydantic model, annotate a field with a reducer such as add_messages, and every node returns a partial update that LangGraph merges rather than overwrites. Two branches running in parallel that both append to messages produce one list instead of a race.

Persistence hangs off a super-step boundary rather than a decorator. After each super-step the compiled graph writes the whole state to the checkpointer under the thread_id you passed in config["configurable"], so crash recovery, a pause for a human, and a resume a week later are one mechanism with three names. InMemorySaver for tests, SqliteSaver locally and PostgresSaver in production expose the same interface.

Human review is a graph primitive rather than an integration. interrupt() stops a node mid-execution, surfaces a payload to whoever invoked the graph, and Command(resume=value) feeds the answer back. The node then replays from its start, which is a detail worth internalizing: anything before the interrupt() call runs a second time, so a side effect belongs in a node of its own.

The config argument is the plumbing the rest of this page uses. Anything under configurable reaches every node that declares a config: RunnableConfig parameter, and it travels into tools and nested subgraphs without becoming part of the state the model reads. Streaming modes (values, updates, messages, custom) then control what a caller observes, without a single change to the graph itself.

The seam

LangGraph's contract covers what happens between START and END. Identity, money and entry points sit outside that boundary, and the graph runs identically whether you have handled them or not.

• thread_id is a string, not a scope. The checkpointer looks up state by that string and nothing else. Two accounts that both name a thread support-42 share a conversation, and no layer underneath notices.
• The store namespace is yours to design. Cross-thread memory in BaseStore is keyed by a namespace tuple you supply. Passing ("memories", user_id) where user_id came out of a request body rather than a verified session is an entire isolation failure in one argument.
• Tools receive what the model produced. A tool whose signature includes tenant_id: str takes a tenant id the model chose. It will choose correctly almost every time, and the exception is a cross-account read.
• Usage is on the message, cost is not. Model responses carry token counts. Converting those into dollars per account per month means a price table you maintain, a durable write path, and a decision about what happens when one customer's usage triples overnight.
• Every entry point assembles its own config. The web app, the Slack handler, the nightly re-summarization job and an engineer's eval script are four callers of graph.invoke, and each is a place to forget the tenant.

None of that is a criticism of the design. A graph runtime that also owned identity would be a worse graph runtime, and the library's smallness is why the state machine is worth having. The work is deciding what lives above invoke and what lives inside a node, then keeping the two from blurring.

How to close it

Put tenant identity in config, never in state or a tool signature

Read the tenant from config inside the node that needs it, and let the tool functions take only the arguments a model is allowed to choose.

from typing import Annotated, TypedDict

from langchain_core.runnables import RunnableConfig
from langgraph.graph.message import add_messages


class State(TypedDict):
    messages: Annotated[list, add_messages]


def retrieve(state: State, config: RunnableConfig) -> dict:
    tenant_id = config["configurable"]["tenant_id"]
    hits = search_index(tenant_id=tenant_id, query=last_user_text(state))
    return {"messages": [tool_message(hits)]}

The caller composes that config once, from a verified session, in a single factory every entry point calls:

config = {
    "configurable": {
        "thread_id": f"{tenant_id}:{conversation_id}",
        "tenant_id": tenant_id,
        "end_user_id": end_user_id,
    },
    "metadata": {"tenant_id": tenant_id, "end_user_id": end_user_id, "surface": "web"},
    "tags": [f"tenant:{tenant_id}"],
}

graph.invoke({"messages": [("user", question)]}, config)

LangGraph 0.6 added a separate per-run context channel: pass context_schema=Context to StateGraph, declare a runtime: Runtime[Context] parameter on the node, read runtime.context, and call graph.invoke(state, context=Context(...)). That keeps configurable for framework concerns such as thread_id, and it deprecates config_schema. Either channel works here. What matters is that the value is assembled in one place from something you verified, and that no tool's parameter schema contains it.

Compose the checkpointer key, then validate it on the way in

A checkpointer is a key-value store over conversation history and the key is the string you hand it. Build it from the verified tenant plus an opaque conversation id, and treat a request whose prefix does not match the session as a 404 rather than an empty result, so a probe cannot distinguish "not yours" from "does not exist".

Leave checkpoint_ns alone while doing this. That field namespaces checkpoints for subgraphs inside a run: empty at the root graph, and a node_name:uuid segment per subgraph invocation. Putting your isolation in it means sharing a field the library also writes.

Where isolation has to survive an application bug rather than a convention, the Postgres checkpointer takes a connection you supply, so a schema or database per tenant is available. That trade costs you pooling and migrations across many schemas, which is reasonable for a handful of large accounts and unpleasant for thousands of small ones. The decisions it raises are enumerated in the tenant isolation checklist.

Start the store namespace with the thing you verified

BaseStore is the memory that outlives a thread, and its namespace is a tuple whose first element should be the tenant, not the subject the request asked about.

def remember(state: State, config: RunnableConfig, *, store: BaseStore) -> dict:
    tenant_id = config["configurable"]["tenant_id"]
    end_user_id = config["configurable"]["end_user_id"]
    store.put((tenant_id, "memories", end_user_id), str(uuid4()), {"text": last_text(state)})
    return {}

LangGraph injects that store keyword argument when the node annotates it as BaseStore, and a runtime: Runtime[Context] parameter reaches the same object as runtime.store. put takes the namespace tuple first, then the key, then the value.

A search against that store takes the same prefix, so a recall for one account cannot reach another's namespace however the query text is phrased. The failure mode this prevents is quiet: a memory written under a shared namespace surfaces months later as a sentence about another customer, inside an answer that reads as confident.

Attach the attributes before the first node runs

Per-tenant questions get answered from traces, so the attributes have to exist on the root span. The metadata and tags keys in the config above are the carrier LangChain propagates to child runs, which is why they belong in the same factory as configurable. LangGraph traces to LangSmith through environment variables, and LANGSMITH_OTEL_ENABLED switches that same tracing onto an OTLP exporter aimed at a collector of your choosing. LANGSMITH_OTEL_ONLY drops the parallel export to LangSmith, and the endpoint wants /v1/traces appended when traces are all you send.

export LANGSMITH_OTEL_ENABLED=true
export LANGSMITH_OTEL_ONLY=true
export OTEL_EXPORTER_OTLP_ENDPOINT="https://collector.example.com/v1/traces"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer ${OTLP_TOKEN}"

Decide the payload-capture setting at the same time. Prompts and tool arguments are where customer data lands, and a redaction rule written now is cheaper than one applied to a year of retained traces. Pointing an externally-run graph's spans at a platform that reads them is walked through in instrumenting an external agent.

Attribute cost where the tokens are produced

LangGraph does not compute cost, and the token counts come off the model response rather than the graph. LangChain chat models expose usage_metadata on the AIMessage, carrying input_tokens, output_tokens and a nested input_token_details whose cache_read field is populated by the providers that report it, OpenAI and Anthropic among them. A callback that fires at the end of every model call is the one place all of them pass through, whichever node produced them.

Record the model id, the input and output tokens split by cached and uncached, and the tenant and end-user ids from the config. A monthly figure per account falls out of that, and so does the more useful question of which loop in which graph produced a spike. A ledger reports afterward, though. Stopping a run mid-turn is a separate mechanism: a counter in state with a conditional edge to END, or recursion_limit, which bounds super-steps rather than dollars.

Decide what stays inside the graph

Most retrofits go wrong by moving too much. The graph is the part with the clearest tests and the most deliberate design, so leave it and place the operational concerns beside it.

If the framework choice is still open rather than settled, LangGraph alternatives compares it against Mastra, Pydantic AI and the provider SDKs. The category view of the layer this section keeps adding to is AI agent platform, and the tenancy-specific version is multi-tenant AI agents.

Where Runtype fits

Runtype is the tenancy, spend and surface layer a LangGraph service can sit behind, and the compiled graph keeps running as written. The move that matters most here is registering that graph as an external agent: give it an endpoint that speaks Runtype's unified stream or A2A, and Runtype calls the graph. Registered that way it can be embedded in the open-source Persona chat widget, added to a product as a capability, reached through web chat, Slack, a REST API, SMS, iMessage, MCP and A2A, and put on a schedule. Tool calls and cost are recorded per run, and an A2A endpoint can also serve as an eval suite target, where cases that replay recorded tool activity are skipped (bring your own agent).

Three other ways in stack around that, in whatever order the graph needs:

• Send traces. The spans LangGraph already exports go to https://api.runtype.com/v1/otel over OTLP and become runs with a trace tree, structured logs, token usage and a display-only cost estimate. Point exactly one instrumentation at the endpoint, since two doubles the tokens and the cost, and your provider keeps billing you directly.
• Serve the graph tools. An MCP surface publishes a product's flows, agents, records, skills and tools as MCP tools, so LangGraph stays the orchestrator and calls Runtype for the parts worth centralizing. No node changes and no cutover.
• Rebuild what earns it. Move one capability out of the LangGraph service and into a native flow or hosted agent once a suite harvested from real runs can prove parity, then gate it in CI with runtype eval run, which returns a non-zero exit code on a regression.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.runtype.com/v1/otel"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer ${RUNTYPE_API_KEY},x-runtype-agent-id=${RUNTYPE_AGENT_ID}"

The identity contract is the part that is hard to bolt onto a thread_id. Each resource declares a tenancy strategy of internal, tenant-isolated or end-user-isolated with an assurance floor of asserted or verified, evaluated before execution, so a request whose identity scope falls below the floor is rejected before it runs. Every trace and every cost figure is filed under the tenant and end user the request ran for, and long-term memory, when enabled, is keyed per agent, tenant or end user through {{_endUser.id}} (end-user identity).

The rest of the outside layer answers the items the table above pushed out of the graph. Approvals gate all tools or a named list with a timeout that defaults to 5 minutes, and the run resumes when the decision lands. Cost lands per execution, per record and per batch with cached and uncached tokens separated, and eval cases are promoted from recorded executions, judged by a model with human review of individual scores. The same product definition runs on managed cloud or self-hosted on your own infrastructure.