How to deploy AI into a customer's environment when they will not send data out

Separate four things that "self-hosted" blurs together: where the application runs, where data rests, where inference happens, and what telemetry leaves. Most deals that sound like "run it in our VPC" are settled by the middle two. A full air gap with open-weight models on the customer's own GPUs costs the most and is rarely what was asked for.

The steps below work through the ladder in order, from the cheapest arrangement that satisfies a real requirement to the most expensive one. Each rung costs you something specific in capability, latency or support burden, and the goal is to stop climbing as soon as the customer's actual constraint is met.

Customer won't let data leave their environment for AI

The sentence arrives late in a security review, usually from someone who is not in the room where the architecture was decided. Behind it sits one of four constraints, and they are not equally expensive.

A regulatory boundary (data residency in a region, or a healthcare or public-sector rule) is often satisfied by inference in the right region under the right contract. A procurement rule about subprocessors is satisfied by removing a vendor from the path. A network policy that forbids outbound traffic from a data subnet is satisfied by a private endpoint. A genuine air gap, where no route to the internet exists at all, is the only one that forces open-weight models.

Ask which of those four it is before designing anything. The answers diverge immediately: three of them let you keep a hosted model service, and one does not. Your own architecture is the other half, because a system assuming a hosted control plane, a shared database and a provider API call has three separate things to move, and they need not move together.

The four deployment shapes

Only the last row forces a different model. The third row keeps a frontier model and moves the boundary, which is why it closes most deals that started as "we need to self-host". All three document private network access inside the customer's own account: Azure OpenAI private endpoints, Bedrock interface VPC endpoints over AWS PrivateLink, and Private Service Connect for the Gemini Enterprise Agent Platform (Vertex AI).

Step 1: write down what actually moves

List every piece of state your feature touches and mark each one: customer content (documents, tickets, transcripts), derived content (embeddings, summaries, extracted fields), configuration (prompts, tool definitions, agent versions), and operational data (traces, costs, eval results). Customer content and derived content are almost always in scope. Configuration and operational data are usually negotiable, and that gap is what makes a split deployment possible.

Step 2: split the control plane from the data plane

The control plane holds definitions and history: prompts, tool configurations, versions, eval suites, dashboards. The data plane executes: it reads customer data, calls the model, runs tools, and writes results. If the control plane can push a signed, versioned definition into the data plane and the data plane never pushes content back, you keep central authoring and per-customer deployment at the same time.

The seam has to be a definition, not a live dependency. If the runtime calls home on every turn to resolve a prompt, a customer network outage becomes your outage, and a network policy review will find the call and reject it. Ship the definition, cache it locally, and make the fetch a deploy-time operation with an offline fallback.

Step 3: pick the model path

This is the decision that determines everything else. Ranked by how much you keep:

1. Customer's own account with a first-party model service. Azure OpenAI in their subscription, Bedrock in their AWS account, or Vertex AI in their project, reached over a private endpoint so no traffic crosses the public internet. You keep frontier models, and the model provider is already a subprocessor the customer has approved through their cloud vendor.
2. Your provider account with a zero-retention agreement. The data leaves the customer boundary, so this only works when the objection was about retention and training rather than about network egress. Providers retain API inputs for an abuse-monitoring window by default, and the terms differ as of September 2026: OpenAI documents 30 days on its standard endpoints, with zero data retention available on approval; Anthropic's zero-retention arrangement excludes stateful features such as batches and files, and does not cover the models it requires 30-day retention for; Google publishes no window for the paid Gemini API beyond logging for a limited period.
3. Open-weight models on customer hardware. The only option under a true air gap. Budget for GPU capacity, a serving stack, and a real evaluation of the quality drop, which shows up first in multi-step tool calling and structured output rather than in prose quality.

Step 4: close the other egress paths

The model is not the only thing that talks to the internet. Web search, page fetching, an outbound webhook, a vector database as a service, an error tracker, a feature-flag service and a license check all make outbound calls, and any of them will fail a network review that the model call passed. Enumerate them from the code rather than from memory, then either bring each one inside the boundary or remove it from the deployed configuration.

Tools that call the customer's own systems are the easier half, and they get better in this arrangement: an agent already inside the VPC can reach an internal API that was never exposed publicly. Tools that call third parties need an explicit allowlist and a documented purpose, which is the same artifact a security review of an AI feature will ask for.

Step 5: decide what telemetry leaves, field by field

Traces are the part teams forget. A trace of an agent turn contains the system prompt, the user message, tool arguments, tool results and the model's output, which is to say it contains the customer data the whole arrangement exists to keep local. Shipping traces to your hosted dashboard by default undoes the deployment.

There are two workable patterns. Keep full traces inside the boundary and let the customer's own operators read them, exporting only metrics with content stripped at the source: step names, durations, token counts, error codes, model ids, cost. Or redact at the point of capture with a policy the customer reviews, so prompt and completion fields are replaced before they are written anywhere. Write the exported field list into the contract, because a customer who finds one of their prompts in your logs will treat it as an incident.

Step 6: agree the support model before signing

You will lose the ability to reproduce a bug. No production database to query, no trace to open, no ability to redeploy at two in the morning. Decide in advance who holds the break-glass credential, whether support access is screen-share only, how the customer packages and sends a redacted failure bundle, and what your response commitment means when every diagnostic step needs their engineer awake too.

Version drift is the other cost. Every customer-hosted deployment is a version you support until they upgrade, and they upgrade on their schedule. Pick an update mechanism (a container image tag, a signed artifact, a pull-based agent) and a supported-version window at the start, then price the deal with that window in it.

Step 7: keep one product definition

The failure that costs the most is the fork. A team promises the VPC deployment, copies the service into a second repository, strips the parts that need hosted infrastructure, and now ships every feature twice. Six months later the deployments have different prompt behavior and no one can say which is correct.

Avoid it by making the deployment target a configuration value rather than a code path. Same agent definition, same tool set, same eval suites; what varies is where the model call goes, where results are written, and which telemetry sinks are enabled. This is also what keeps a per-tenant configuration model intact when one tenant runs in their own cloud, which is the harder version of the problem described in multi-tenant AI agents. The broader set of choices around shipping the feature at all is covered in AI agent deployment.

Step 8: price it honestly

A customer-hosted deployment carries setup engineering, a per-deployment support cost that does not decrease with scale, and a version-support tail. If the deal cannot carry that, the right answer is single-tenant hosting in your own account with a data-residency commitment, offered as the alternative rather than as a refusal. This is one of the places where the build-versus-buy calculation shifts, since the operational surface you just took on is the same one weighed in build vs buy for an AI platform.

Where this gets easier

The work shrinks when the deployment target is a property of the same product definition rather than a second product. Runtype runs as managed cloud or self-hosted on the customer's own infrastructure (BYOC), with the agent, its tools and its flows exported as one resolved runtime definition, so the control plane keeps authoring, versioning and eval suites while inference and customer data stay wherever the deal requires. The export and its host dependencies are documented at self-hosting.