The new primitives

Twenty years ago, launching an application meant assembling servers, storage, databases, networks, queues, logs, and a deployment pipeline by hand. The cloud abstracted that work, and modern SaaS followed.

Agentic software runs on the same foundation: an LLM call still executes on a server and is routed via a network, data still lives in databases, jobs still enter queues. That foundation is sinking below the waterline: a decade of cloud investment made it mostly bulletproof. Above it, new problems emerge: most layers of an LLM-backed application are non-deterministic, unreliable, and open to manipulation by anything the model reads. Compute, storage, and databases were the primitives of a cloud application. An application that understands, decides, and acts needs a different set.

"Agent stack" diagrams mix protocols, vendors, model features, and product patterns. This is how we think about it at Runtype:

Model inference and routing

Calling the right model for the task, reliably, at scale.

Bounded agent execution

The agent harness that points the arcane power of the LLM in a useful direction, allowing it to loop relentlessly towards a goal within appropriate policies.

Context management and durable state

All the systems that help curate the model's context window with application-relevant facts about the current state of the world, focusing the execution and producing a coherent experience.

Tools and approval boundaries

The interfaces that allow agents to take actions and interact with the outside world, and the strategies to manage the risks related to these capabilities.

Sandboxed code execution

Bridging agentic applications to the power of traditional software by allowing a safe space to execute agent-authored code.

Observability and evals

Visibility into what happened, a semantic definition of success, and a means to track alignment and divergence over time, enabling iterative improvement.

Surfaces

The interface the user actually touches, which can take a wider variety of forms thanks to the power of generative AI.

Some of these are established patterns with many solutions on the market, while others are still becoming standards and hardening into infrastructure. Together they form the new application layer for this new kind of software.

Models are a runtime, not a product

Inference is the obvious first primitive, but a model in isolation is not a product; it's closer to a runtime. The other primitives are necessary to turn a model's next-token prediction into a useful application.

Models are a strange form of compute: they can interpret ambiguous input, generate language and code, choose among tools, and work across modalities. No single model is best across those axes, so builders must route for quality, latency, cost, governance, and modality. Often the right component is not a frontier model at all, but a small classifier, an embedding, speech recognition, or plain code.

Fifty years of software practice assumes a program is deterministic, that its spec lives in code you can read, that a wrong answer looks different from a right one, and that a test suite can protect you from misbehavior in production. Models challenge these principles, and software teams need new skills and tools to harness them effectively. The same prompt run twice can yield different answers. Behavior is specified in English, and runtime state is a probability distribution that can't be meaningfully introspected. Anything a model reads can instruct it, so a support ticket or a web page can rewrite its task mid-run. Its failures arrive as fluent, confident text that parses cleanly and passes every schema check.

A harness puts a budget and a policy around a loop outside the model's control. A refund with known rules should stay a workflow; open-ended investigation can have a longer leash. The model may choose the next step. The product decides the size of the room.

Tools are interfaces and integration points, but also trust boundaries. The model sits inside your topology but must be treated as an untrusted entity. Valid tool calls can still represent undesirable or harmful actions.

Surfaces are more than just web UIs, and agents can actively adapt to users' preferences or needs—as discussed in The Changing Face of Software.

From capability to product

The cloud made servers, storage, and networks dependable enough to compose a product without rebuilding them. Agentic products need the same treatment one layer up.

This is the layer we are building at Runtype.

Models are capability. The new primitives are what turn that capability into a product.

Start building on Runtype.