Decision comparison
Arize Phoenix vs Braintrust vs Langfuse
Settle the licence question first, because it does not depend on anything else and it removes an option outright. Phoenix is Elastic Licence 2.0 — source-available rather than OSI open source — and Braintrust is proprietary, so a stack that must be open source end to end is left with Langfuse, whose core is MIT even though GitHub cannot resolve the composite LICENSE file. After that the split is which end of the loop the product starts from. Langfuse starts at tracing and treats evaluation as one stage; Braintrust starts at evaluation and can fail a build on a score; Phoenix starts at whichever trace you are staring at right now, and costs nothing anywhere. All three cover tracing and evals, so nobody is missing a feature the others have — they are arranged differently, and the arrangement is the decision.
Direct comparison. These are reviewed substitutes bought for the same job, so the differences below are the ones that decide between them.
All 3 are LLM observability & evaluation.
Quick Comparison
| Decision factor | Arize Phoenix | Braintrust | Langfuse |
|---|---|---|---|
| Best for | Getting a trace in front of you immediately. `uvx arize-phoenix serve` is the whole installation, and it runs in a notebook, a container, or the free Phoenix Cloud tier. That makes it the tool that ends up in a repository before anyone has decided on production observability, which is a real advantage and also how it gets adopted without a licence review. | Teams trying to make evaluation a discipline rather than a dashboard. The product is arranged so a production failure becomes a dataset row, the dataset becomes an experiment, and a failing score blocks a release before it reaches production. That last step is an organisational fact, not a feature, and it is genuinely hard to assemble from parts. | Teams that want the whole loop in one product and need the option of keeping trace data inside their own network. Tracing in one tool, prompts in a second and evals in a third is a common and painful arrangement, because the trace that reveals the problem is not in the system that tests the fix. |
| Which end it starts from | A named workflow rather than a feature list: observe, annotate, hypothesise, experiment, measure. Annotations are a first-class step, which is the part most tools skip and the one that converts a pile of traces into labelled data. Instrumentation is OpenTelemetry through OpenInference, so spans you send here during development can go somewhere else in production without re-instrumenting. | Evaluation. Versioned datasets, experiments comparing prompts and models side by side, and scoring by an LLM judge, by code or by a human. Observability is the second pillar rather than the first: it records every agent trace and tool call so the next eval set has something to be made from. | Tracing. Hierarchical traces capture every model call, tool invocation and retrieval step as a tree rather than a flat log, filterable by user, session, cost, latency or custom metadata. Prompt management with one-click deployment and rollback, datasets, experiments and evaluation close the loop around it, and evaluators run against production data as well as fixed test sets. |
| Licence, precisely | Elastic Licence 2.0. That is source-available: you can read, run and modify it, but it restricts offering the software as a managed service and it is not OSI-approved. The project's own site says ELv2, and a stack with an open-source-only requirement cannot include it however the community framing reads. | Proprietary. There is no repository that is the product, so it carries no public adoption signal of its own. Autoevals, the vendor's scoring library, is MIT-licensed and usable without the platform — the cheapest way to test the scoring approach before buying. | A split. The core is MIT and an `ee/` directory is separately licensed for commercial features. GitHub cannot resolve that to a single SPDX identifier and reports it as unresolved, so an automated open-source check cannot confirm it without a human reading the file. If your procurement runs one, expect that conversation. |
| Where it can run | Locally, as a self-hosted container, on Kubernetes, or as Phoenix Cloud. Traces stay in your environment when you self-host. | Hosted, with a hybrid option: the Brainstore data plane can run on your own infrastructure while the control plane stays with the vendor. A real answer to data residency, and not the same answer as a fully self-hosted product. | Self-hosted from one container image — Docker Compose, Kubernetes via Helm, or Terraform on AWS, GCP and Azure — and the same software serves Langfuse Cloud. That symmetry is unusual: many open-core products self-host a reduced build. |
| What it costs | Nothing. There is no paid Phoenix tier, no seat charge and no feature held back. The vendor's commercial product is the separate Arize AX, whose free tier covers 25,000 spans a month and whose AX Pro is $50 per month with 50,000 spans. | Starter is $0/month with $10 of model credits, 1 GB of processed data, 10,000 scores and 14-day retention, then $4/GB and $2.50 per 1,000 scores. Pro is $249/month with $100 of credits, 5 GB, 50,000 scores and 30-day retention, then $3/GB, $1.50 per 1,000 scores and $0.50/GB/month for longer retention. Enterprise is quoted. Qualifying startups get 6 to 12 months of Pro free. | Self-hosting is free. Hobby is free with 50,000 units a month and 2 users; Core is $29/month, Pro is $199/month and Enterprise is $2,499/month, each including 100,000 units with unlimited users. Overage is $8 per 100,000 units, falling to $7 above 1 million, $6.50 above 10 million and $6 above 50 million. A Teams add-on is $300/month. |
| When it is the wrong choice | When a licence requirement covers this layer, which is the decision to settle before any other. Also when prompt deployment and rollback are a first-class need, which is thinner here than in a tool built around it. | When an open-source licence requirement covers this layer, when nothing may leave your network, or when the immediate problem is simply seeing requests and their cost. | When evaluation should be the centre of the workflow rather than one stage of it, and when nobody will operate the self-hosted deployment while Cloud is ruled out. |
Arize Phoenix
- Best for:
- Getting a trace in front of you immediately. `uvx arize-phoenix serve` is the whole installation, and it runs in a notebook, a container, or the free Phoenix Cloud tier. That makes it the tool that ends up in a repository before anyone has decided on production observability, which is a real advantage and also how it gets adopted without a licence review.
- Which end it starts from:
- A named workflow rather than a feature list: observe, annotate, hypothesise, experiment, measure. Annotations are a first-class step, which is the part most tools skip and the one that converts a pile of traces into labelled data. Instrumentation is OpenTelemetry through OpenInference, so spans you send here during development can go somewhere else in production without re-instrumenting.
- Licence, precisely:
- Elastic Licence 2.0. That is source-available: you can read, run and modify it, but it restricts offering the software as a managed service and it is not OSI-approved. The project's own site says ELv2, and a stack with an open-source-only requirement cannot include it however the community framing reads.
- Where it can run:
- Locally, as a self-hosted container, on Kubernetes, or as Phoenix Cloud. Traces stay in your environment when you self-host.
- What it costs:
- Nothing. There is no paid Phoenix tier, no seat charge and no feature held back. The vendor's commercial product is the separate Arize AX, whose free tier covers 25,000 spans a month and whose AX Pro is $50 per month with 50,000 spans.
- When it is the wrong choice:
- When a licence requirement covers this layer, which is the decision to settle before any other. Also when prompt deployment and rollback are a first-class need, which is thinner here than in a tool built around it.
Braintrust
- Best for:
- Teams trying to make evaluation a discipline rather than a dashboard. The product is arranged so a production failure becomes a dataset row, the dataset becomes an experiment, and a failing score blocks a release before it reaches production. That last step is an organisational fact, not a feature, and it is genuinely hard to assemble from parts.
- Which end it starts from:
- Evaluation. Versioned datasets, experiments comparing prompts and models side by side, and scoring by an LLM judge, by code or by a human. Observability is the second pillar rather than the first: it records every agent trace and tool call so the next eval set has something to be made from.
- Licence, precisely:
- Proprietary. There is no repository that is the product, so it carries no public adoption signal of its own. Autoevals, the vendor's scoring library, is MIT-licensed and usable without the platform — the cheapest way to test the scoring approach before buying.
- Where it can run:
- Hosted, with a hybrid option: the Brainstore data plane can run on your own infrastructure while the control plane stays with the vendor. A real answer to data residency, and not the same answer as a fully self-hosted product.
- What it costs:
- Starter is $0/month with $10 of model credits, 1 GB of processed data, 10,000 scores and 14-day retention, then $4/GB and $2.50 per 1,000 scores. Pro is $249/month with $100 of credits, 5 GB, 50,000 scores and 30-day retention, then $3/GB, $1.50 per 1,000 scores and $0.50/GB/month for longer retention. Enterprise is quoted. Qualifying startups get 6 to 12 months of Pro free.
- When it is the wrong choice:
- When an open-source licence requirement covers this layer, when nothing may leave your network, or when the immediate problem is simply seeing requests and their cost.
Langfuse
- Best for:
- Teams that want the whole loop in one product and need the option of keeping trace data inside their own network. Tracing in one tool, prompts in a second and evals in a third is a common and painful arrangement, because the trace that reveals the problem is not in the system that tests the fix.
- Which end it starts from:
- Tracing. Hierarchical traces capture every model call, tool invocation and retrieval step as a tree rather than a flat log, filterable by user, session, cost, latency or custom metadata. Prompt management with one-click deployment and rollback, datasets, experiments and evaluation close the loop around it, and evaluators run against production data as well as fixed test sets.
- Licence, precisely:
- A split. The core is MIT and an `ee/` directory is separately licensed for commercial features. GitHub cannot resolve that to a single SPDX identifier and reports it as unresolved, so an automated open-source check cannot confirm it without a human reading the file. If your procurement runs one, expect that conversation.
- Where it can run:
- Self-hosted from one container image — Docker Compose, Kubernetes via Helm, or Terraform on AWS, GCP and Azure — and the same software serves Langfuse Cloud. That symmetry is unusual: many open-core products self-host a reduced build.
- What it costs:
- Self-hosting is free. Hobby is free with 50,000 units a month and 2 users; Core is $29/month, Pro is $199/month and Enterprise is $2,499/month, each including 100,000 units with unlimited users. Overage is $8 per 100,000 units, falling to $7 above 1 million, $6.50 above 10 million and $6 above 50 million. A Teams add-on is $300/month.
- When it is the wrong choice:
- When evaluation should be the centre of the workflow rather than one stage of it, and when nobody will operate the self-hosted deployment while Cloud is ruled out.
Public signals
Verified factual signals only. Bars appear only for like-for-like metrics with five weekly assessments for every tool; missing evidence stays explicit. These signals do not establish enterprise adoption, product quality, or total cost.
| Metric | Arize Phoenix | Braintrust | Langfuse |
|---|---|---|---|
| Docker Hub pulls(Product adoption) | 5.6M | Not available | 17.8M |
| GitHub commits, 90d(Product adoption) | 1.2k | Not available | 2.0k |
| GitHub stars(Product adoption) | 11,000+ | Not available | 34,000+ |
| npm weekly downloads(Developer adoption) | 62.7k | 1.2M | 1.3M |
| PyPI weekly downloads(Product adoption) | 155.0k | Not available | Not available |
| GitHub commits, 90d(Developer adoption) | Not available | 2 | Not available |
| GitHub stars(Developer adoption) | Not available | 1,000+ | Not available |
| PyPI weekly downloads(Developer adoption) | Not available | 1.5M | 5.3M |
| Product Hunt comments(Community interest) | Not available | Not available | 159 |
| Product Hunt rating(Community interest) | Not available | Not available | 5.0/5 |
| Product Hunt reviews(Community interest) | Not available | Not available | 48 |
| Product Hunt votes(Community interest) | Not available | Not available | 771 |
As of September 21, 2026 — updated weekly.
Health & risk evidence
Observed public-source checks for mapped package versions and repositories.
Arize Phoenix
September 21, 2026Package vulnerabilities
npm · @arizeai/phoenix-client@7.11.0 · PyPI · arize-phoenix@20.14.0
0 vulnerabilities
across 2 packages
Repository security score
Not available
Braintrust
Package vulnerabilities
npm · braintrust@3.34.0 · PyPI · braintrust@0.41.0
0 vulnerabilities
across 2 packages
Repository security score
Not available
Langfuse
September 21, 2026Package vulnerabilities
npm · langfuse@3.39.2 · PyPI · langfuse@4.15.4
0 vulnerabilities
across 2 packages
Repository security score
Not available
Feature Comparison
| Feature | Arize Phoenix | Braintrust | Langfuse |
|---|---|---|---|
| Licence and deployment | |||
| Licence on the product itself | Elastic Licence 2.0 — source-available, not OSI-approved | Proprietary; Autoevals, the scoring library, is MIT | MIT core with a separately licensed ee/ directory; GitHub reports the file as unresolved |
| Satisfies an open-source-only requirement | No — ELv2 is source-available, which an automated licence check treats as failing | No — the platform is proprietary | The core is MIT, but a human has to read the LICENSE because the identifier is unresolved |
| Fully self-hosted | Yes — local, container or Kubernetes, at no cost | Partly: the Brainstore data plane runs on your infrastructure, the control plane does not | Yes — one container image, the same software as the cloud product |
| Hosted tier | Phoenix Cloud, free | The primary way it is used | Langfuse Cloud, from a free Hobby tier |
| What it records and scores | |||
| Trace structure | Every step: prompts, retrievals, tool calls, outputs | Every agent trace and tool call, searchable across millions of logs | Hierarchical trace trees filterable by user, session, cost, latency and custom metadata |
| Instrumentation | OpenTelemetry via OpenInference, so spans stay portable | Vendor SDKs in Python and JavaScript | OpenTelemetry-based, so existing OTel spans and common frameworks work |
| Evaluation | Evals scoring outputs, run against datasets built from traces | The centre of the product: versioned datasets, experiments, LLM-judge, code and human scoring | LLM-as-a-judge, heuristic functions and human review, run on production data as well as test sets |
| Human annotation | A first-class step in its named workflow, by human or LLM judge | Custom views and annotation across engineering and product | Human review as one of three evaluator types |
| Release gating in CI | Experiments measure a change; gating a build is left to you | Scored evals can block a release before it reaches production | Experiments score a candidate before it ships; gating a build is left to you |
| Prompt management | A Prompt IDE that iterates prompts against datasets | A prompt playground over your own datasets | Prompts separated from code with one-click deployment and rollback |
| Commercial and compliance | |||
| Free tier | The whole product is free, self-hosted or on Phoenix Cloud | Starter at $0/month with unlimited users, projects, datasets and experiments | Hobby with 50,000 units a month and 2 users, plus free self-hosting |
| First paid tier | None for Phoenix; Arize AX Pro is $50 per month with 50,000 spans | Pro at $249/month | Core at $29/month |
| How usage is metered | Spans, on the separate Arize AX product | Three meters at once: processed data, scores and model credits | Units, with graduated overage from $8 to $6 per 100,000 |
| Compliance posture | Self-hosting keeps traces in your environment | SOC 2 Type II, HIPAA, GDPR, granular per-project permissions and a hybrid data plane | SOC 2 Type II and ISO 27001, EU and US data regions, a HIPAA-ready region and GDPR |
Licence and deployment
Licence on the product itself
Satisfies an open-source-only requirement
Fully self-hosted
Hosted tier
What it records and scores
Trace structure
Instrumentation
Evaluation
Human annotation
Release gating in CI
Prompt management
Commercial and compliance
Free tier
First paid tier
How usage is metered
Compliance posture
Which to choose
Settle the licence question first, because it does not depend on anything else and it removes an option outright. Phoenix is Elastic Licence 2.0 — source-available rather than OSI open source — and Braintrust is proprietary, so a stack that must be open source end to end is left with Langfuse, whose core is MIT even though GitHub cannot resolve the composite LICENSE file. After that the split is which end of the loop the product starts from. Langfuse starts at tracing and treats evaluation as one stage; Braintrust starts at evaluation and can fail a build on a score; Phoenix starts at whichever trace you are staring at right now, and costs nothing anywhere. All three cover tracing and evals, so nobody is missing a feature the others have — they are arranged differently, and the arrangement is the decision.
Best-fit scenarios
Choose Langfuse if:
Choose Langfuse when the trace data may need to stay inside your network and you want the whole loop in one product. The self-hosted server is the same software as the cloud one, runs from a container image via Docker Compose, Helm or Terraform, and costs nothing — that combination is close to unique in this category and it is the strongest single reason to pick it. Hierarchical traces make an agent debuggable rather than merely logged, prompt deployment and rollback move prompt changes out of the release cycle, and evaluators run against production traffic as well as fixed test sets, which turns live traffic into a source of test cases. Accept two things. Billing is in units rather than traces and one agent run can produce many, so model a week of real traffic before choosing a tier. And the LICENSE is a split that automated open-source checks report as unresolved, so a procurement gate will need a human to read it.
Choose Braintrust if:
Choose Braintrust when the discipline you are establishing is evaluation, and particularly when you want a quality score to fail a build. Versioned datasets are what make a regression claim mean anything — scoring against a moving test set tells you nothing — and blocking a release on a score turns quality from a dashboard somebody checks into a gate. The compliance surface is the other reason: SOC 2 Type II, HIPAA, GDPR, granular per-project permissions and a hybrid data plane clear procurement bars that open-source projects answer one question at a time. The Starter tier is unusually complete at $0/month with unlimited users and projects. Accept that it is proprietary, that the hybrid data plane is not a full self-hosted build, and that three simultaneous meters make the bill harder to predict — the score meter in particular scales with your CI cadence, which is in tension with the product's own argument for running more evals.
Choose Arize Phoenix if:
Choose Phoenix when you want to see what an agent did in the next thirty seconds and no licence requirement covers this layer. One command gives you a running instance; it works in a notebook, a container or the free Phoenix Cloud tier; and the whole product costs nothing, with no paid Phoenix edition holding anything back. OpenTelemetry through OpenInference keeps the spans portable, so adopting it during development does not commit you to it in production. Its named loop — observe, annotate, hypothesise, experiment, measure — describes what debugging an agent actually involves, and annotations being a first-class step is what turns traces into labelled data. Accept the licence: Elastic Licence 2.0 is source-available, restricts offering the software as a managed service, and is not OSI-approved, so an automated check will fail it regardless of the community framing. Accept too that the free product is the on-ramp to Arize AX, a separate commercial platform, and that the roadmap incentive points that way over time.
These scenarios reflect the available product evidence. Your requirements, existing stack, and team expertise should guide the final decision.
Frequently Asked Questions
Which of these is actually open source?
Only Langfuse has an OSI licence on its core, and even then the repository LICENSE is a split — MIT outside an ee/ directory — that GitHub reports as unresolved, so an automated check cannot confirm it without a human reading the file. Phoenix is Elastic Licence 2.0, which is source-available rather than OSI-approved. Braintrust is proprietary. If your requirement is contractual rather than a preference, settle this before comparing features.
Can I run any of them entirely inside my own network?
Langfuse and Phoenix, yes, and both at no cost: Langfuse from one container image via Docker Compose, Helm or Terraform, Phoenix locally or on Kubernetes. Braintrust offers a hybrid deployment where the Brainstore data plane runs on your infrastructure while the control plane stays with the vendor, which answers most data-residency questions but not a strict policy that nothing leaves the network.
What does each one actually cost?
Phoenix is free everywhere, including its hosted tier; the paid product is the separate Arize AX, whose Pro tier is $50 per month with 50,000 spans. Langfuse self-hosts free, and its cloud runs Hobby free, Core at $29/month, Pro at $199/month and Enterprise at $2,499/month with $8 per 100,000 units of overage falling to $6 at volume. Braintrust runs Starter at $0/month, Pro at $249/month with $3/GB and $1.50 per 1,000 scores past the allowance, and a quoted Enterprise tier.
Do I need more than one of them?
Usually not, and running two is the arrangement most teams are trying to escape: the trace that reveals the problem ends up in a different system from the one that tests the fix. The exception is a team that adopts Phoenix during development for the speed of it and standardises on Langfuse or Braintrust in production. OpenTelemetry instrumentation makes that migration cheap, which is one of the better arguments for the OTel-based options.
Where does a gateway like LiteLLM or Helicone fit?
Beside these rather than instead of them. LiteLLM routes requests and enforces budgets and per-team keys, and logs outward to Langfuse or Phoenix; Helicone records requests and cost through a proxy integration but does not do versioned-dataset evaluation or prompt lifecycle. A gateway answers who may spend what; these three answer what the model actually did and whether it was any good.