Build or buy: what an AI platform layer actually has to do

Build the platform layer if your deployment constraints rule out every vendor, if you already own most of the surrounding infrastructure, or if the layer is itself the product you sell. Otherwise buy. The inventory below is about ten separate systems, and the build case is rarely lost on the first one. It is lost on the eighth, and in the maintenance after the engineer who wrote it changes teams.

This page is written for a team shipping agents inside a product their own customers use, across many tenants. That case sets the bar higher than an internal tool does, because identity, isolation and cost attribution stop being reports and start being requirements.

The two-week ceiling

The first agent takes two weeks. One prompt, one tool, one surface, one bill, and an engineer who can read the logs by scrolling. The fifth agent takes two weeks as well, and that repetition is the signal.

Nothing from the first four carried forward. Evals lived in a notebook on someone's laptop, tracing was a log line and a Postgres table nobody pruned, cost was a monthly provider export reconciled by hand, and approvals were a message to whoever was awake.

Someone then proposes an internal platform, usually with a slide arguing that the hard part is already done. The agent loop is not the hard part. It is a loop around a chat completion and a tool dispatcher, and a competent engineer writes a working one in an afternoon. What follows is the list of things that surround it.

The inventory

Each row is a job the layer has to do before it earns the name. The middle column is the smallest version that is honestly usable, not a stub. The right column is what keeps costing after launch, which is where in-house platforms usually get into trouble.

Two of these rows are genuinely small: a tool registry with HTTP tools and secret substitution, and a draft-versus-published pointer. The rest are not, and three in particular get estimated as if they were.

The three that get underestimated

Redaction has to run before the write

A trace is only useful if it contains tool arguments and results, and that is exactly where customer data lives: the invoice lookup that returns a full billing address, the support tool that returns a phone number, the retrieval step that pulls a document with a national ID in it. Redaction applied when a trace is rendered is a display filter, so the raw value is still in your store, still in your backups, and still in any export a support engineer takes. Doing it correctly means a policy resolved at dispatch, applied on the write path, with a per-field decision rather than a regex over the whole payload, and a way to set it differently per product because one customer's contract says one thing and another's says something else.

Cost per tenant is not a sum of tokens

The naive version multiplies total input tokens by an input price. That overcharges every cached prefix, and prompt caching is the main lever most teams pull to reduce spend, so the error grows where it matters most. Each provider reports cache reads and writes under a different field name and at a different price ratio, and a routing layer spreading traffic across providers has to normalize all of it before it can put a number next to a customer.

Attribution also has to happen at execution time. Reconstructing it later from logs means trusting that every entry point set the tenant field, and the one that forgot is always the background job.

A paused run is a distributed systems problem

An approval gate looks like a conditional. What it actually requires is that the agent's state at the moment of the pause is durable, that a resume rehydrates it correctly hours later on a different process, that a timeout fails closed rather than leaving a run pending forever, and that two approvers clicking at the same time produce one outcome. Add the approver view per surface, and the audit record of who approved what and on what evidence, and the conditional has become a subsystem. A related trap: whatever justification the agent offers for the action is text the model produced, so the approval decision has to be made on the tool name and its parameters, and a prompt-injected instruction can write a persuasive reason.

Should we build our own LLM platform or buy one

Three conditions make building the right call, and they are conditions rather than preferences.

Your deployment constraints rule out every vendor. An air-gapped network, a regulated environment with an approved-software list, or hardware you own end to end. Confirm this with whoever owns the constraint, because "we need to self-host" often turns out to be satisfiable by a vendor that deploys into your own cloud account. The trade-offs are covered in self-hosted AI deployment.

The platform is your product. If you sell agent infrastructure, or your differentiation is a runtime property nobody else offers, then this list is your roadmap and buying it would be buying your own competitor.

You already own most of it. A team with a mature internal service platform (durable workflow execution, a secrets service, a tracing backend with retention policy, an approvals system from another product line) is not building ten systems. It is building two or three adapters on top of nine that already exist and already have owners. That is a genuinely different project, and it is often the cheaper one.

Outside those three, the argument for building rests on control and cost. Control is real, and it is worth something on the day a vendor deprecates something you depend on. Cost is where the estimate goes wrong, because the build gets quoted and the maintenance does not. Everything in the right-hand column above is permanent work for a team that also has product commitments.

There is a fourth answer worth naming. Keep the framework you have and buy only the layer around it, which is what most teams end up doing. Framework choice and platform choice are separate decisions; agent framework comparison covers the first, and best AI agent platforms surveys products that do the second.

A scoring rubric

Score each job from the inventory on two axes, then compare the totals rather than arguing about the whole decision at once.

• Need (0 to 3). 0 means you do not need it now and can see why you would not need it in a year. 1 means a manual process is acceptable. 2 means you need it and are currently faking it. 3 means a customer commitment or a compliance requirement depends on it.
• Have (0 to 3). 0 means nothing exists. 1 means a prototype one person understands. 2 means a shared service another team already runs. 3 means a supported internal platform with an owner and an on-call rotation.

For each row, the gap is Need minus Have, floored at zero. Sum the gaps.

Two checks on the result. Score Need for the product you plan to ship in a year, since tenancy and approvals move from 0 to 3 the moment a customer's own users touch the agent; that shift is covered in what an AI agent platform has to do. Then ask who is on call for each row scored as Have. A row with no name against it is a 1, whatever the wiki says.

Where this gets easier

This inventory is the shape of Runtype, which is a reason to read it as a specification whether or not you buy anything: versioning with draft and published versions, eval suites with judge scoring and human review, per-step traces with cost separated into cached and uncached tokens, a tenancy strategy resolved before execution, approval gates with a timeout, a tool registry with server-side secret resolution, surfaces from one definition, and durable runs with a run handle and a wall-clock budget. If you build instead, these are the parts you will end up building, and the right-hand column of that table is the part to staff for.