Comparison

7 best open-source LLMOps and observability platforms in 2026

Find the right platform to manage prompts and tools, trace LLM calls, and improve response quality. Compare seven open-source and source-available tools, with hands-on findings, clear trade-offs, and licensing details.

The short answer

Choose an LLMOps platform based on what you need to control and improve. LLM gateways sit in the request path to control calls before they reach a provider. Observability tools record calls made by your existing client so you can trace behavior and evaluate responses. Use both when you need request controls and visibility into results.

AcruxCore is the only platform in this comparison with a versioned tool catalog whose calls the gateway executes. Each tool is a catalog entry with its own version history, so you can add, change, or roll back a tool without deploying application code. Shortlist it if you want that alongside prompt versioning and feedback-driven prompt optimization in one workflow. It is also the only platform here whose audit trail needs no paid plan: every change anyone makes is recorded with their name, on every plan and when self-hosted. For the largest community in this comparison, consider Langfuse. Already using MLflow? Its prompt registry, tracing, and gateway may meet your needs. For evaluation, explore Opik; for agent tracing, explore Laminar. AcruxCore has a smaller community and a flat team structure; check its access controls if you need an organization layer or multiple roles per member.

How to choose

How to choose an LLMOps platform: seven key questions

Do you need an LLM gateway, observability, or both?

An LLM gateway controls requests before they reach a provider. An observability tool records traces so you can inspect what happened. AcruxCore, MLflow’s gateway, and Helicone sit in the request path, where routing, caching, budget checks, and provider-key management can happen. Check support for each control. Langfuse, Phoenix, Opik, and Laminar collect traces without acting as gateways, so you can add visibility without introducing a new proxy.

Can you add and version the tools a model calls without deploying code?

This is the one question where AcruxCore stood alone against all six. In AcruxCore a tool is a catalog entry with its own version history, and the gateway executes the call, so the definition the model reads and the call that ran are the same record. You can add a tool, change it, or roll it back without deploying application code. No other platform in this comparison had both a versioned catalog and execution. Langfuse saved playground tool schemas without versioning or execution. Laminar stored schemas within playground entries. Phoenix, Opik, and Helicone had no tool catalog. MLflow cataloged MCP servers rather than individual tools, without per-tool history or execution in the workflow tested. If your tools live in your own code and you want them to stay there, this question will not decide it for you.

Can you update prompts without deploying code?

A versioned prompt registry lets you change prompts without redeploying code. Look for aliases, version comparisons, and the templating features you need. In our tests, AcruxCore and MLflow supported native Jinja2 conditionals and loops. Langfuse, Phoenix, Opik, and Helicone supported variable substitution but left branching logic to application code. Laminar’s playground had no prompt versions, aliases, or variables. Check the dated findings for current support.

What team structure and permissions do you need?

Match access controls to how your team shares projects. In the self-hosted versions we tested, AcruxCore offered members, invites, and roles in a flat team structure. Langfuse and Laminar added an organization or workspace above projects. Laminar offered three workspace roles; AcruxCore offered one. Phoenix, Opik, and MLflow had no team, member, or invite management; MLflow also had no login screen. Helicone used one organization tier with no role selector in its invite dialog. For separate business units, compare the hierarchy and permissions in Langfuse and Laminar.

Can you see who changed what, and when — without paying for it?

An audit log answers this, and it is the question where paid tiers show up. Ask it about the whole account rather than about prompts alone: a compliance review asks who revoked an API key or changed a member’s role as often as it asks who edited a prompt. In AcruxCore every prompt and every tool carries its own trail. Owners and admins also get one team-wide trail in the dashboard, covering 34 event types across seven areas. Those areas are prompts, tools, members and invites, API keys, gateway credentials and budgets, secrets, and trace settings. Filter the trail by area, by one event, or by the person, including someone who has since left the team. It is populated by default on every plan and when self-hosted, with no upgrade. Langfuse has an audit log in its UI, limited to its top paid plan for hosted and self-hosted use. For Phoenix, Opik, Helicone, MLflow and Laminar we found no audit log in the settings pages we checked. Export and a retention window are the two things AcruxCore’s trail does not have yet.

How do you turn feedback into better prompts?

Feedback only helps if it leads to a prompt change you can test. AcruxCore keeps end-user feedback and developer notes apart, and either one can become a dataset of test cases. Experiments run those cases against different prompt versions and models so you can compare the results. Automated rules score live traffic as it arrives. The optimizer proposes rewrites and tests them on the same cases as your production prompt. Promoting a better rewrite creates a numbered version and moves the alias, so your app picks it up without a deployment. Opik also focuses on evaluation. Compare how many of these steps each platform handles for you and how many stay in your own code.

What does the license include?

Check the license and which features require payment. Our recorded comparison lists Opik, Helicone, MLflow, Laminar, and AcruxCore as Apache 2.0, with no gated directory. Langfuse has an MIT-licensed core and an enterprise-licensed directory; its audit log was limited to the top paid plan for hosted and self-hosted use. Phoenix uses Elastic License 2.0: it is source-available, rather than OSI-approved open source. Check the dated licensing sources for terms that apply to your deployment.

The platforms

Seven LLMOps platforms compared

Competitors are listed by community size, followed by AcruxCore. The order does not rank performance. Each entry includes a verification date; the comparison matrix links to the platform’s own documentation.

Langfuse

32,617 GitHub stars · checked 2026-08-06

OTel-native tracing and prompt management, with an org/project hierarchy above the team.

Best forTeams that need an organisation layer above the project, and the largest community of the six by a wide margin.
LimitationsNot in the request path, so a budget, cache or virtual key has no call to act on. Variable substitution only, with no conditionals or loops. The audit log sits behind the $2,499/mo Enterprise plan, on hosted Langfuse as well as self-host. A separate enterprise licence governs the ee/ directory.
LicenseMIT on the core, but a separate Enterprise License governs the ee/ directory, so some features are gated. AcruxCore gates none.

MLflow

27,416 GitHub stars · checked 2026-08-08

The open-source ML/GenAI platform — prompt registry, tracing, evaluation, and its own request-path AI Gateway.

Best forTeams already running MLflow for classic ML, who want a prompt registry, tracing, full Jinja2 templating and a request-path gateway with per-endpoint PII and safety guardrails in a tool they already operate.
LimitationsNo auth, teams, members or roles at all in self-hosted OSS, and no login screen at all. Its MCP Registry catalogs whole servers rather than individual tools, so there is no per-tool version history and nothing that executes a call.
LicenseApache License 2.0, no gated directory found in the repo — the same terms as AcruxCore

Opik

21,169 GitHub stars · checked 2026-08-07

Evaluation-first observability from Comet — datasets, experiments, and online scoring rules.

Best forEvaluation-first work: datasets, experiments and online scoring rules, under the same Apache 2.0 terms as AcruxCore.
LimitationsNo team, member or invite concept anywhere in self-host, since members are a Cloud-tier feature. Nothing in the request path, and no tool catalog. Mustache substitution by default, with playground conditionals still an open feature request.
LicenseApache License 2.0, no gated directory found in the repo — the same terms as AcruxCore

Phoenix

10,923 GitHub stars · checked 2026-08-07

Local-first tracing and evaluation, notebook-native, from Arize.

Best forLocal-first tracing and evaluation on one machine, notebook-native, with nothing to set up and no account to create.
LimitationsElastic License 2.0 is source-available rather than OSI-approved; there is no team or user-management concept in local OSS; nothing sits in the request path; and there is no tool catalog.
LicenseElastic License 2.0 — source-available, not OSI-approved and not permissive

Helicone

6,044 GitHub stars · checked 2026-08-07

A request-path proxy with per-user metrics — now in maintenance mode after being acquired by Mintlify.

Best forA small request-path proxy in front of a native provider key, when per-user metrics are the main thing wanted.
LimitationsIn maintenance mode since the Mintlify acquisition, which is the first thing to weigh; a non-native (OpenRouter) key 501’d or misrouted in testing; the member-invite dialog has no role field at all; and there is no tool catalog.
LicenseApache-2.0, no ee/ split found — the same terms as AcruxCore
Community notesAcquired by Mintlify; per Helicone's own announcement, "services will remain live... in maintenance mode."

Laminar

3,230 GitHub stars · checked 2026-09-06

Rust and ClickHouse observability built for agent runs — SQL over spans, LLM-watched Signals, and a coding-agent-first CLI.

Best forAgent-run observability: SQL over spans, an LLM-watched Signals engine and PII redaction, with a workspace layer above the project and three roles.
LimitationsNo prompt registry at all, so there is nothing to version or template; nothing in the request path; the lite self-host stack ships Signals switched off with its span index uncreated; and it has the smallest community of the six.
LicenseApache License 2.0, no gated ee/ directory found in the repo at the commit checked — the same terms as AcruxCore
Community notesY Combinator S24; 29 contributors and release v0.2.3 as of the date checked. Ships a PII redaction toggle and an LLM-watched Signals engine AcruxCore has no equivalent for.

AcruxCore

Public mirror launched 2026-08-03 · checked 2026-08-07

One control plane where the gateway sits in the request path and the tool catalog’s calls are versioned and executed.

Unique in this comparisonA versioned tool catalog whose calls the gateway executes. Every other platform here either has no tool catalog, stores an unversioned schema that nothing runs, or catalogs whole MCP servers instead of tools.
Best forTeams that want the whole loop in one self-hostable product: versioned prompts, a request-path gateway, tracing, and a versioned tool catalog whose calls the gateway executes. Feedback then turns into datasets, standing eval rules and optimizer rewrites you promote as a normal prompt version. Every change anyone makes is recorded with their name on every plan, which is the one row here no other platform matches without a paid tier.
LimitationsA single flat team with no organisation layer and one role per member, no content or PII guardrails in the request path, and by far the smallest community of the seven.
LicenseApache License 2.0 — permissive and OSI-approved, with no enterprise-only directory

Methodology

How we tested the platforms

We tested the same example prompt across all seven platforms, including AcruxCore, and self-hosted each where supported. The linked reviews include screenshots from these hands-on tests.

Findings reflect the versions and dates tested. A feature marked as absent was not found in our tests or the sources linked in the comparison matrix. Features can change, so check the date and source before making a decision.

In our benchmark, AcruxCore’s gateway added about 42 ms compared with calling OpenAI directly, with a confidence interval of +17 ms to +81 ms. Reproducible scripts are available in the repository. The gateway latency benchmark includes the full distribution and tail latency results.

This comparison is published by AcruxCore. We highlight competitor strengths and link claims to sources so you can judge the fit for your team.

Compare features. Find your fit.

Check the comparison matrix for features, trade-offs, and sources. Explore the hands-on review of nine platforms for screenshots and test details. Considering AcruxCore? Read the FAQto see how it fits your workflow and when to consider an alternative.