Agent orchestration platform: deterministic where it matters

An agent orchestration platform (often sold as an AI orchestration platform) runs multi-step AI work in production: it executes the deterministic parts of a process as code, hands specific bounded decisions to a model, and enforces the limits, state and identity around both. The distinguishing design choice is that flows (fixed steps) and agents (model-driven loops) are separate primitives that nest inside each other, so your team decides, step by step, where the model should choose and where your code should.

That choice is the whole subject of this page. A process that lets a model decide every step is easy to demo and hard to operate, because every run is a fresh plan and every failure is a new one. A process with no model in it cannot handle unstructured input at all. The work is drawing the line between the two and having a runtime that holds it.

Why orchestrating agents is hard

The failures below all come from the same source: the model was asked to make a decision that should have been fixed, or the code was written to make a decision that needed judgment.

The plan changes on every run

An agent given "process this refund" and a set of tools will, on one run, look up the order, check the policy and issue the refund. On the next it will issue the refund first and then discover the order was outside the return window, with nothing wrong in the prompt. A model choosing the order of side effects means the order is sampled, and a business process with sampled ordering has no invariant you can test against. Which shape to reach for first is settled in agent vs workflow.

The loop that has no reason to stop

A tool that returns an empty array on a miss reads, to the model, like a failed call, so it retries with a slightly different query, gets the same empty array and retries again. Without a cap enforced outside the model, the turn ends when the provider's context window fills or your credit card does. The equivalent inside a deterministic flow is a retry-until-good loop with no iteration bound, which fails in the same way with a more respectable stack trace. Mitigations are in long-running agents.

State that lives only in the conversation

The agent that runs for forty minutes across twelve tool calls holds everything it has learned in its message history. When the process restarts, or the model's context is trimmed, that state is gone and the agent either repeats work or invents what it forgot. A conversation is a fine place for a transcript and a poor place for working memory, a point expanded in agent memory and context window management.

Handoffs that lose the thread

Splitting work across a coordinator and several specialist agents feels like delegation and behaves like a game of telephone. Each handoff passes a summary the coordinator wrote, so the specialist works from a lossy copy of the situation, and its answer comes back as another summary. Token budgets multiply per hop, and a failure three agents deep is attributed to whichever agent last spoke. Multi-agent systems covers when the split is worth it.

Output nobody can parse

The step after a model call is usually code, and code wants a field, not a paragraph. A model asked for JSON returns JSON most of the time, wrapped in prose some of the time, and with a renamed key when the prompt changed. Every downstream step then carries a defensive parser, and a silent parse failure becomes a null that propagates two steps before anything notices. Making this reliable is the subject of structured output from an LLM.

How it works: flows and agents that nest

The platform gives you two primitives and a rule for combining them. A flow is a deterministic sequence of typed steps. Branches are conditions your code wrote, loops have a hard iteration cap, and a model call is one step with a declared output variable. Given the same inputs and the same model responses, the flow takes the same path.

An agent is a model, a system prompt and a tool set, run in a loop until the model stops calling tools or hits a limit. The model decides which tool to call and in what order, inside bounds the runtime enforces.

The rule is that each can contain the other, which is the useful meaning of the phrase agentic workflow. A flow can include a model step whose job is one bounded judgment. An agent can call a whole flow as a single tool, so "look up the order, check eligibility, compute the amount" becomes one deterministic action the model can invoke but cannot reorder. The result is a spine of code with model-driven decisions at the joints, and a way to move a decision from one side to the other without rewriting the process.

Here is the spine in practice. A flow classifies an inbound message with a model step that must return JSON, branches on a field of that output with a condition your code wrote, and revises a draft in a loop that is capped at three rounds whatever the reviewer says:

{
  "steps": [
    {
      "type": "prompt",
      "name": "Classify",
      "config": {
        "model": "claude-sonnet-5",
        "systemPrompt": "Classify the message. Return JSON: {\"intent\": \"refund\" | \"status\" | \"other\", \"orderId\": string | null}",
        "userPrompt": "{{userMessage}}",
        "responseFormat": "json",
        "outputVariable": "triage"
      }
    },
    {
      "type": "conditional",
      "name": "Route",
      "config": {
        "branches": [
          {
            "id": "refund",
            "name": "Refund request",
            "condition": "triage.intent === 'refund' && triage.orderId !== null",
            "steps": [
              {
                "type": "loop",
                "name": "Draft and review",
                "config": {
                  "steps": [
                    {
                      "type": "prompt",
                      "name": "Draft reply",
                      "config": {
                        "model": "claude-sonnet-5",
                        "userPrompt": "Draft a refund reply for order {{triage.orderId}}. Reviewer notes from the previous round, if any: {{review.notes}}",
                        "outputVariable": "draft"
                      }
                    },
                    {
                      "type": "prompt",
                      "name": "Review",
                      "config": {
                        "model": "claude-sonnet-5",
                        "userPrompt": "Review {{draft}} against the refund policy. Return JSON with verdict ('pass' or 'revise') and notes.",
                        "responseFormat": "json",
                        "outputVariable": "review"
                      }
                    }
                  ],
                  "until": "review.verdict === 'pass'",
                  "maxIterations": 3
                }
              }
            ]
          }
        ],
        "otherwiseSteps": []
      }
    }
  ]
}

Three details in that sample carry the argument. responseFormat: "json" makes the classifier's output a value the next step can test, so the branch condition is an expression over a field, not a regex over prose. The condition is JavaScript your team wrote and can unit test; the model never sees it. And maxIterations: 3 is a hard cap enforced by the runtime, so a reviewer that never says "pass" costs three rounds and then the flow continues, which you handle with a conditional after the loop rather than a plea in the prompt.

The other direction of nesting is an agent that calls the flow as a tool. The agent decides whether the situation calls for a refund lookup; the flow decides everything about how a refund lookup happens:

export const supportAgent = defineAgent({
  name: 'Support Agent',
  model: 'claude-sonnet-5',
  tools: {
    runtimeTools: [
      {
        toolType: 'flow',
        name: 'lookup_refund',
        description: 'Check refund eligibility and amount for an order',
        parametersSchema: {},
        config: { flowId: 'flow:Refund Lookup' },
      },
    ],
  },
})

Above the agent sit the runtime limits: turns per conversation, tool calls per turn, a timeout per tool and a wall-clock budget per turn. These are the difference between "the prompt says stop after three attempts" and "the third attempt is the last one". A schedule wraps either primitive when the trigger is time rather than a message; scheduled AI agents covers what a scheduled run needs that a chat turn does not, such as an owner for the failure and a record of what each run did.

What changes when the agent is customer-facing and multi-tenant

Every framework example assumes one team running one agent for itself. The moment the agent sits inside your product and your customers use it, three things change about orchestration specifically.

The first is that the spine has to carry identity. A flow step that updates a record needs to know which tenant's record, and it needs to know that from the runtime, not from a parameter the model filled in. If tenant scope is a string the model passes to a tool, a confused or injected model passes the wrong one. The deterministic steps are exactly where scope should be pinned, because they are the steps that touch data.

Second, limits become per-tenant policy rather than global defaults. One customer's agent hitting a tool cap should not degrade another's, and a customer who pays for a higher budget should get it without a redeploy. The turn budget, the tool-call cap and the memory the agent is allowed to keep are all things a tenant configuration has to be able to set.

Third, the entry points multiply. The same agent definition is reached from a chat widget, a Slack workspace, an API call from the customer's own backend and a nightly schedule, and each is a place where the tenant context can go missing. Orchestration that lives in the process behind one HTTP handler has to be repeated for every surface, and it is the repeated copy that drifts. The broader shape of that problem is the subject of the AI agent platform overview.

Where Runtype fits

Runtype is built around the two primitives on this page: flows are deterministic multi-step definitions with branching and validation (a loop step requires maxIterations, capped at 10, and loops do not nest), and agents are a model, a system prompt and a tool set run in a multi-turn loop. An orchestrator you already run comes in four ways that stack, and the first two leave your loop in charge.

• Serve it tools. An MCP surface publishes a product's flows, agents, records, skills and tools as MCP tools, so your graph stays the orchestrator and calls Runtype for the steps worth making deterministic.
• Register the agent. An external agent whose endpoint speaks Runtype's unified stream or A2A is called by Runtype, so it can be tested from the dashboard, added to a product as a capability, exposed on every surface and scheduled. Tool calls and cost are recorded per run.
• Send traces. An OpenTelemetry-instrumented loop exports OTLP over HTTP to https://api.runtype.com/v1/otel for the Runs view, the trace tree, token usage and a display-only cost estimate. Point exactly one instrumentation at it, because two doubles tokens and cost.
• Rebuild when it earns it. Move one capability into a flow once an eval suite harvested from real runs can prove parity. It is then versioned, published and gated: runtype eval run returns a non-zero exit code in CI on a regression.

The limits above are runtime-enforced on anything Runtype runs: maxTurns from 1 to 100, a per-turn maxToolCalls defaulting to 10 with a ceiling of 100, an optional per-run cost ceiling, a 30-second tool timeout and a 30-minute wall-clock budget per turn. Turns are durable with a run handle, and long-term memory is opt-in and keyed per agent, tenant or end user.

These layers are built for a software company's customers. Each resource declares a tenancy strategy of internal, tenant-isolated or end-user-isolated with an assurance floor of asserted or verified, evaluated before any step runs, and every trace and cost figure is filed under the tenant and end user the request ran for.

The decision about where a model chooses stays with your team, and the runtime is what makes the stop condition true.