17 posts tagged with "LLM tracing"

This post compares eight alternatives to Langfuse on self-hosting effort, license, price and the features that need an enterprise plan. For each Langfuse alternative, it lists what the tool adds and what it lacks.

This post compares eight LangSmith alternatives on license, self-hosting, price and features. For each alternative to LangSmith, it lists what the tool adds and what it lacks.

Open-source LLM observability tools describe self-hosting the same way. You clone the repo, run one docker compose up, and open the dashboard. Most self-host pages say nothing after that step.

The compose file behind that command fixes several things the self-host page does not list. The file sets how many containers you run and which databases come with them. Those containers set how much memory the server needs before the first trace arrives. The file's volume mounts decide whether your data survives a restart.

We ran the official self-host setup of seven tools in October 2026: Langfuse, Opik, Phoenix, MLflow, Laminar, Agenta and AcruxCore. Each ran alone on the same machine. Each received the same 1,000 traces.

This post compares AcruxCore, Langfuse and LangSmith on the things a developer checks before choosing one. We also ran one app on all three platforms, with the same code, data, prompts and model, and measured each step. The app is a small agent that reads a client's email, checks each request against a signed project scope, prices the extra work, and writes a reply.

This post compares Langfuse and LangSmith on the things a developer checks before choosing one. We also ran one app on both hosted platforms, with the same code, data, prompts and model, and measured each step. The app is a small agent that reads a client's email, checks each request against a signed project scope, prices the extra work, and writes a reply.

Laminar (lmnr) is a Y Combinator S24 project that calls itself observability purpose-built for AI agents — a Rust ingest server, ClickHouse for spans, Quickwit for search, and an OpenTelemetry SDK that instruments fifteen-plus agent frameworks from one line of code — Vercel AI SDK, Claude Agent SDK, OpenAI Agents SDK, LangChain DeepAgents, Mastra, Pydantic AI, Browser Use, Stagehand, Playwright, LiteLLM, OpenCode and OpenHands among them. It is the most differently shaped tool we have compared AcruxCore against. Its project sidebar has dashboards, traces, evaluations, datasets, labeling, a SQL editor, playgrounds, a debugger and settings — and no prompt registry anywhere. That is not an oversight; it is the product being about agent runs rather than about the calls inside them. Both projects are Apache 2.0, so this is a comparison of shape, not of who is open.

Helicone is a proxy-first LLM observability tool — you point your API base URL at it and it sits in the request path, which makes it architecturally closer to AcruxCore's own gateway than most competitors in this series. So we built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on self-hosted Helicone, then tried to run the identical sequence we've run on every platform in this series: create the prompt, send a live call, inspect the trace, build a dataset, and call it from a script. On this self-hosted build, three of those steps hit real, reproducible errors on Helicone's side before we ever got a trace. We're publishing those errors, not routing around them — that's the most useful thing we found.

Opik is Comet's open-source LLM-ops platform, and unlike some competitors we've covered it ships with genuinely no login wall when self-hosted — you docker compose up and you're working inside a real workspace immediately. We built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.

Arize Phoenix is the OTel-native tracing-and-eval project a lot of teams reach for first, so it deserves a real answer, not a table copied from two docs sites. We built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.

Langfuse is the LLM-ops platform we get compared to most often, and it deserves a real answer, not a table copied from two docs sites. So we built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.