How to add an AI assistant to a product people already pay for

Add an AI assistant to an existing product by deciding four things before writing a prompt: the one job it does end to end, the data boundary and how the signed-in user's identity reaches every retrieval and tool call, where it lives in the interface, and what a user sees when it is wrong. Each of those is expensive to reverse after launch, while the model and the prompt are cheap to change at any point.

The mandate is "add AI", the work is everything around the model

The request usually arrives as a sentence from leadership or a competitor's launch post, and the first prototype takes an afternoon: a chat box, a system prompt describing the product, a retrieval call over the help center. It answers demo questions well. Then someone asks it about their own account, and the team discovers that the prototype has no idea who is asking.

That is where the real project starts. The assistant needs to authenticate against the sessions your product already issues, and it needs to see exactly the invoices, tickets or records this user is allowed to see, in a product where that permission model took years to build. It needs a home in an interface designed before it existed, and it needs a defined behavior for the moment it confidently tells a paying customer something false. None of those are model problems, and none of them get easier by switching models.

The customer-facing setting changes the stakes. An internal tool that hallucinates a field costs one employee ten minutes; an assistant that tells a customer their refund was issued when it was not costs an escalation, and one that surfaces a tenant's data to another tenant is an incident report. The plan below puts the decisions with that blast radius first.

How do I add an AI assistant to my existing SaaS product

The steps are sequenced by cost of reversal. Steps 1 and 2 are contracts the rest of the build depends on; steps 3 through 5 are product decisions that are cheaper to revisit but still worth making on paper before code.

1. Pick one job the assistant does end to end

A general chat box is judged against everything the product can do, and it fails that test in the first session because it cannot do most of it. A single job has an input, an output and a definition of done. "Draft the reply to this ticket using the customer's order history", "explain why this invoice total changed since last month", "turn this meeting note into three tasks in the right project": each one names the data it needs and the action it ends with.

Write the job as a sentence a support engineer could hand-execute, then list the data and actions that sentence touches. That list is the assistant's entire permission scope for version one. If the list includes a write (creating a task, issuing a credit), mark it now, because step 4 treats writes differently from reads.

2. Decide the data boundary and how identity flows into every call

The boundary question is which rows the assistant may read on behalf of this request, and the answer has to be the same one your existing authorization layer would give. The failure mode most teams hit on the first attempt is an assistant service that runs under its own service account with broad read access, receives a userId or tenantId as a parameter, and relies on either the prompt ("only show data for customer 4821") or a filter inside each tool handler to keep tenants apart. The service account sees everything, so one handler that forgets the filter, or a model that fills in a customer_id argument from the conversation, returns another tenant's data with no error anywhere.

What holds is to take identity from the verified session and never from the model or the request body. The tool the model sees has no identity parameter at all:

{
  "name": "list_open_invoices",
  "description": "Open invoices for the signed-in customer, newest first.",
  "parameters": {
    "type": "object",
    "properties": {
      "limit": { "type": "integer", "minimum": 1, "maximum": 20 }
    },
    "additionalProperties": false
  }
}

Its handler reads the identity from the request context your existing middleware already populated, and calls the same authorized path the rest of the product uses:

async function listOpenInvoices(args: { limit?: number }, ctx: RequestContext) {
  // ctx.session was verified by the same middleware that guards the REST API
  const { tenantId, userId } = ctx.session
  return invoices.listOpen({ tenantId, actor: userId, limit: args.limit ?? 10 })
}

Two properties of this shape matter. The model cannot ask for a different tenant, because the tool has no parameter for it, and a handler cannot forget the filter, because it never held a credential that could see past the tenant. If your data layer supports row-level security or per-tenant connections, run the assistant's queries through it rather than a shared role. When the assistant lives in a separate service, forward the user's session token or exchange it for a short-lived token scoped to that user, and treat a request without one as unauthenticated rather than falling back to a default account.

A quick way to audit the boundary is to list who is allowed to supply the tenant on each path:

Multi-tenant products have a longer version of this exercise in the tenant isolation checklist, covering memory, logs and retrieval indexes as well as tool calls.

3. Choose the surface: inline action, side panel or background

Where the assistant lives decides how much context it gets for free and how visible its mistakes are. Three shapes cover most products:

An inline action is the safest first surface because the user is present, the input is bounded, and the output replaces nothing until they accept it. A side panel is the right shape once the job needs a back-and-forth, and a background surface only after the inline version has shown its output is trustworthy without a person checking it. Whichever you choose, the identity from step 2 has to reach it the same way: a panel embedded in the app and an API endpoint a customer's backend calls are two entry points that both need the verified session. Teams that later serve the same assistant in Slack or over an API will find the sequencing in one agent, many channels.

4. Define the failure behavior before the first customer sees it

The assistant will be wrong, and the design question is what the customer experiences when it is. Decide it per class of action:

1. Read-only answers cite the records they were built from, with a link to each, so a wrong answer is checkable in one click and the user can correct the assistant instead of the assistant correcting them.
2. Writes are proposals. The assistant drafts the task, the credit or the reply; the user confirms; the confirmation, and never the model's tool call, performs the write. For higher-stakes writes, route the proposal to a person on your side rather than the customer, and give it a timeout so an unanswered proposal expires rather than lingering.
3. Every write has an undo window, and the undo is a product feature the user can find, not a support ticket.
4. A turn has a time budget. When it runs past it, the user sees a visible failure with a retry, never a spinner that resolves to nothing.
5. Every failed or corrected turn is recorded with its input, so the same case can be replayed after a prompt or model change.

Item 2 is the one most likely to be skipped under deadline pressure, and the one that separates an assistant customers keep enabled from one they ask to turn off. A wrong draft costs nothing; a wrong refund is a phone call.

5. Set the one metric that says whether it worked

Choose the metric before launch, from the job in step 1, and instrument its baseline in the product as it exists today. For a drafting job, the share of drafts sent with fewer than a set number of edits; for an explanation job, the share of sessions with no support contact from the same account the next day; for a task-creation job, the share of created tasks still open and unedited a week later. Message volume, session length and "engagement" measure how much the assistant is used, which rises with novelty and falls with disappointment, and neither direction says whether it did the job.

Pair the metric with two guardrails: cost per completed job, so an assistant that spends a dollar per ticket is caught early, and the rate of corrections or undos, so a quality drop after a model change is visible before the metric moves.

6. Ship it to a cohort, with a way back

Launch behind a per-tenant flag to accounts that opted in, with the assistant's version pinned so a prompt change ships as a deliberate release rather than a hot edit. Keep the pre-assistant path working for every job, so turning the flag off is a rollback and not a feature removal. The recorded failures from step 4 become the regression set you run before each release, and the metric from step 5 decides whether the cohort widens. What changes between a working demo and this stage is the subject of taking an AI feature from demo to production; the wider operational picture, from versioning to rollback, is at deploying AI agents to production.

Where this gets easier

Runtype supplies the surrounding layer: end-user identity, per-user data scoping, the embeddable widget and the API behind it. An end user's identity is proven per request through your existing identity provider (Identity Exchange verifies an OIDC token against your issuer's keys), and an agent with the end-user-isolated tenancy strategy rejects a request that lacks a verified identity before it runs, so memory, records and connected credentials are scoped to that user by the runtime rather than by a filter in each handler. The open-source Persona widget gives the side-panel surface a chat interface you embed in the app, the same agent is reachable over a REST API for the inline and background shapes, writes can be gated behind an approval with a timeout, and each execution's trace and cost carry the tenant and end user it ran for. The identity contract is documented at end-user identity; the work that remains is the assistant's behavior.