Langfuse, LangSmith and Braintrust compared

Keep Langfuse if you self-host or run several frameworks, LangSmith if your agent is a LangGraph graph, and Braintrust if your recurring question is whether a change made the agent better. All three record what your code tells them, and none of them enforce anything at the moment the agent runs.

This page is written for a team that already pays for one of the three and would rather not migrate. The overlap between them is wide enough that a second subscription rarely pays for itself, so the useful work is picking the right one to keep and being clear-eyed about the job that sits outside all three.

Comparison table

Cells describe each product as of September 2026, checked against each vendor's own documentation. Price mechanics are described as a shape, because plan pages change faster than this page will.

What Langfuse does well

Langfuse is the one whose source you can read and whose deployment you can own. The application ships from a public repository with an MIT-licensed core, and Langfuse Cloud runs that same codebase rather than a separate enterprise fork. The self-hosting docs are candid about the lag: new capabilities are proven on Cloud before they reach a self-hosted release, so Cloud can run ahead of the newest version you can install, and you choose when to upgrade. Traces are a tree of spans and generations, generations carry token counts and a cost derived from a model price table, and prompts are versioned with labels such as production and cached in the SDK, so a prompt edit ships without a deploy.

Framework spread is the part teams underrate. There is a drop-in OpenAI client, a LangChain callback handler, integrations for LlamaIndex, the Vercel AI SDK and LiteLLM, and an OTLP endpoint for anything with no integration at all. If part of your product calls a provider SDK directly and another part runs a graph, one trace format across both is worth more than any single feature. The head-to-head detail lives in Langfuse vs LangSmith, and the narrow reasons teams leave are in Langfuse alternatives.

What LangSmith does well

LangSmith knows what a LangGraph node is. Set LANGSMITH_TRACING=true with an API key and every chain, tool call and graph node reports with no decorator, and the run tree mirrors the graph: the state at each node, the branch taken at each conditional edge, the tokens spent per step. LangSmith Studio, renamed from LangGraph Studio, opens that same graph as a canvas where you can edit state mid-run and replay from any node. No other tracer models checkpoints and interrupts, because no other tracer ships alongside the library that defined them.

Two eval primitives are worth naming. Pairwise evaluators compare two experiments row by row and record a preference, which answers "is the new prompt better than the old one" without anyone having to invent an absolute threshold first. Online evaluators score a filtered sample of live traffic continuously and feed the same dashboards and alerts as the traces, so a quality regression surfaces in the same place as a latency one.

What Braintrust does well

Braintrust begins at the experiment rather than the trace. A run is a dataset, a task and a set of scorers; you change a prompt or a model, run it, and read a row-by-row diff against the previous experiment with the regressions listed first. Scorers are ordinary functions or LLM judges, and the MIT-licensed autoevals library ships common ones, including Factuality, AnswerRelevancy, Faithfulness and ContextRecall. The same eval runs in CI as bt eval evals/ --no-input --json, which exits non-zero when an eval throws, so a prompt change can fail a pull request the way a unit test does.

The playground is the other reason teams keep it: several prompts and models run side by side over the same dataset rows, and the configuration that wins gets promoted rather than retyped. Production logs feed that loop in both directions: rows are captured from logs into datasets, and online scoring runs the same scorers against live traffic as a project-level rule with its own sampling rate, recording each result as a score span on the trace, which keeps the offline number and the production number comparable. Logs sit in Brainstore, Braintrust's own engine over object storage, and it is the default rather than an option; putting that storage in your own account means the BYOC or self-hosted deployment, and exporting project logs to S3 or GCS is a separate Enterprise feature. For a team whose bottleneck is deciding whether a change helped, that loop is tighter than either of the other two.

The seam all three share

Every one of the three sits downstream of execution. Tenant and end-user identity reach the trace because your code attached them, and nothing on the vendor side rejects a run that arrives without them. The failure is silent by construction: no error is raised, the trace still renders, and the spend lands in an unattributed bucket you notice at invoice time.

Entry points are where that model erodes. The HTTP handler sets the tenant, and so does the Slack event handler, and then a queue worker retries a failed turn, a nightly job re-summarizes a thread, and an engineer runs a backfill script against production keys. Each of those is a separate place to remember a metadata field, and the ones added six months from now are written by someone who never read the original handler. Practical guidance on which fields to capture and why is in what to log for LLM calls.

Isolation is the second half, and it is a property of the store rather than the dashboard. Every tenant's traces live in one project, so showing a customer their own usage, or promising that tenant A's prompts never appear in a view of tenant B's account, is an access layer you build over the vendor API. Role-based access in all three grades your own staff: Langfuse assigns Owner, Admin, Member and Viewer across organizations and projects, LangSmith puts workspaces between those two levels, and Braintrust uses permission groups per project. The smallest partition any of them offers is the project, so one project per end customer is the shape their documentation points you at, and Langfuse states plainly that its Cloud is multi-tenant only.

Completeness has a limit too. A provider SDK that retries internally after a timeout can consume tokens on an attempt your trace never shows, unless the integration unwraps the retry, and cache-read tokens are priced differently from ordinary input tokens in a way a price table can lag. The model provider's own usage block is the authoritative number; the tracer's cost column is a good cross-check and a poor invoice. Background on the wider category is at LLM observability, and the agent-specific version at AI agent observability.

How to close it without switching tools

None of the steps below require leaving the tool you have. They move the parts that have to be enforced out of the tracer and into the code path that runs the agent, and leave the tracer doing what it is good at.

1. Make identity a required argument. Build one context object at every entry point and refuse to call the agent without it, so a missing tenant is a type error at the call site rather than an empty field in a dashboard six weeks later.

2. Stamp the same attribute names everywhere. Pick a set once, keep it stable across vendors, and set it on the root span rather than on individual model calls, so a filter written for one tool ports to the next.

{
  "tenant.id": "acme",
  "end_user.id": "u_8123",
  "surface": "slack",
  "agent.version": "2026-08-30.3",
  "entrypoint": "queue-retry"
}

1. Map that set to each vendor's carrier once, in one helper, rather than at each call site.

1. Record cost yourself at the call site, keyed by tenant, from the provider's usage block, and treat the tracer's figure as a reconciliation check. Separate cached from uncached tokens while you are there, because the two prices differ enough to change a per-account margin.

2. Put the limits where the loop runs. A cap on tool calls per turn, a wall-clock budget, and a spend ceiling checked before the next model call are all decisions made microseconds before the money is spent, and a tracer is not in that path.

3. Redact before export. Every prompt and completion you send crosses out of your boundary into a vendor store, and all three hooks run inside your process and are opt-in: Langfuse takes mask_otel_spans or the older mask on its client, LangSmith takes hide_inputs, hide_outputs, hide_metadata and an anonymizer on its client or the matching LANGSMITH_HIDE_* environment variables, and Braintrust takes one global setMaskingFunction covering input, output, expected, metadata and context. A per-tenant retention or residency promise is yours to configure and yours to test.

Where Runtype fits

Runtype is where the agent is built and run when it is a feature your own customers use, and it sits under a tracer rather than replacing one. Four ways in stack, and rebuilding is optional.

Send traces first. Standard OTLP to https://api.runtype.com/v1/otel gives you the Runs view, the trace tree, structured logs, token usage and a cost estimate. That estimate is for display, since an imported run is not a Runtype execution and your provider still bills you, so it is the cross-check this page already recommends rather than an invoice.

A run whose spans carry the GenAI content attributes has a transcript, and that transcript can be captured as an eval case. Point exactly one instrumentation at the endpoint, since two double the tokens and the cost. Keeping Langfuse, LangSmith or Braintrust as the system of record alongside is the expected case (reporting external telemetry).

Register the agent second: an external agent whose endpoint speaks Runtype's unified stream (runtype-stream) or A2A lets Runtype call the loop you already run. It then embeds in the open-source Persona widget, joins a product as a capability, answers on web chat, Slack, SMS, REST, MCP and A2A, and runs on a schedule, with tool calls and cost recorded per run. An A2A endpoint can also be an eval suite target.

Serve it tools third: an MCP surface publishes a product's flows, agents, records, skills and tools, so your loop keeps the orchestration and calls Runtype for what is worth centralizing. Rebuild last: port the capability that keeps breaking into a flow once real-run evals prove parity, then gate it with runtype eval run in CI, which exits non-zero on regression.

Identity is a declared property of each resource. A tenancy strategy of internal, tenant-isolated or end-user-isolated carries an assurance floor of asserted or verified, evaluated before execution, so a request arriving without a tenant is refused instead of landing in an unattributed bucket (end-user identity).

The limits from step five live where the loop runs: maxToolCalls per turn (default 10, maximum 100), loopConfig.maxTurns from 1 to 100, an optional per-run cost ceiling, and approval gates per tool with a five-minute default timeout. For a run that executes there, cost is recorded per execution, per record and per batch, long-term memory is keyed under the same tenancy strategy, and PII redaction and logging verbosity resolve per product, surface or agent at dispatch. What a runtime owns that a sink cannot is the subject of AI agent platform.