Skip to main content

Hands-on with Langfuse: prompts, tracing, and datasets

Published · Updated · 10 min read
LLM-ops platform

The Langfuse walkthrough in four steps: organizations and projects, prompts and versioning with production and latest labels, reading a trace with session and tokens, and datasets and experiments, plus tracing from code on OpenTelemetry

Langfuse is one of the best-known open-source LLM engineering platforms. It covers prompt management, tracing (a record of what an LLM app did, step by step), evaluation and datasets. We logged into Langfuse's hosted EU cloud and went through the loop by hand: open a prompt, version it, read a trace, and run an experiment over a dataset.

  • When: July 2026. The screenshots are from that month, and the web app showed version v4.0.0-rc.2. One template test was added on 2 November 2026.
  • Where: Langfuse Cloud, EU region, Hobby plan, with an OpenAI key added to the project.
  • Python SDK version: not recorded.

The main comparison post puts the findings next to LangSmith, PromptLayer and AcruxCore. We build AcruxCore, one of the tools in that comparison.

1. Organizations and projects​

Langfuse groups work into organizations, and each organization holds one or more projects. A project is a separate space for traces, prompts and datasets, like a "workspace" in other tools. The account we used already had two organizations with three projects between them, and the steps below use a project called acruxcore-research.

2. Prompts and versioning​

Inside the acruxcore-research project, a prompt called support-greeting already existed with two versions. Langfuse's prompt editor takes plain text, or a "chat" mode for multi-message prompts, with {{variable}} placeholders. It detects the variables from the text (company and name here), so there is no separate declaration step.

Every prompt version is immutable. Editing creates a new version instead of overwriting the old one, close to Git's commit model. Version #1 ("Initial version of support greeting prompt") carries the production label; version #2 ("add a word-count constraint") is the newest edit and carries the latest label. Your live app can keep pointing at production while you experiment with latest, and only move the production label forward once you're happy with the new version.

Langfuse prompt labels: version 1 carries production and serves your live app, version 2 carries latest for experiments, both versions are immutable, and moving the production label forward makes version 2 live

Show screenshot: Langfuse prompt versions with production and latest labels

Langfuse prompt versions list showing v1 tagged production and v2 tagged latest, with detected company and name variables

Labels are how Langfuse marks which version is live. Besides production and latest, you can add labels with your own names, such as staging or candidate. Langfuse fills {{variable}} placeholders but does not run template logic: in a test on 2 November 2026, {% if %} text in a prompt came back unchanged after compiling. A text prompt can include another prompt with a reference tag such as @@@langfusePrompt:name=rate-card|label=production@@@, which Langfuse fills in when the prompt is read.

3. Reading a real trace​

Langfuse's Playground runs a prompt against a model once the project has an LLM provider connection (OpenAI, Anthropic, etc.). You add one under Settings → LLM Connections, with your own API key. With an OpenAI connection, the Playground returned a reply on the first run:

Show screenshot: completed run in the Langfuse Playground

Langfuse Playground showing a completed run with a real generated reply about the benefit of versioning prompts

In our July 2026 test, the Playground run did not create a trace. The Tracing table showed no new rows right after the run. Langfuse's Playground docs do not say either way, so check this on the current version before you rely on it:

Show screenshot: Langfuse Tracing table still empty after the Playground run

Langfuse Tracing table still showing no results immediately after a successful Playground run

Langfuse's traces come from its ingestion API, which any app can send events to. The SDKs use this API, and the project already had trace data from that path.

Show screenshot: Langfuse tracing table with generations and spans

Langfuse tracing table listing chat-completion generations and support-greeting-run spans, grouped under trace names

Opening one trace shows the full picture:

Show screenshot: Langfuse trace detail with session, cost and tokens

Langfuse trace detail panel showing session grouping, cost, token counts, and tags for a support-greeting-run trace

  • A tree. The top-level trace (support-greeting-run) contains a chat-completion generation, with its own duration and cost: 0.93 s and $0.000016 for the generation, and 1.01 s and $0.000016 for the whole trace.
  • Tokens: 37 prompt tokens and 18 completion tokens (55 in total), shown inline.
  • Session and user: badges for Session: demo-session-support-greeting and User ID: walkthrough-user sit in the trace header. Each one opens every other trace with that session or user.
  • Tags on the trace (support-greeting, walkthrough), and "Add to datasets" and "Annotate" actions on the trace page.

This is a span-based view, not a flat call log. For an agent with tool calls, the same tree has more branches.

4. Datasets and experiments​

Langfuse's Datasets are simple input/output pairs you can build up over time (manually, from production traces, or via the API). The project already had a support-greeting-eval-set dataset with 5 items, meant to check prompt behavior across different company/name inputs.

Show screenshot: Langfuse datasets list

Langfuse datasets list showing support-greeting-eval-set with 5 items and 0 experiments run

From a prompt version, "Run experiment" opens a multi-step wizard: pick the prompt version and model, pick the dataset, pick evaluators, fill in run details, then review. The model step needs the same LLM connection as the Playground. In a project without one, the wizard stops at that step:

Show screenshot: Langfuse Run Experiment dialog asking for an LLM API key

Langfuse Run Experiment dialog blocked at the model step with a No LLM API key set in project message

With the OpenAI connection in place, we ran an experiment against the support-greeting-eval-set dataset: pick the prompt version and the openai: gpt-4.1 model, review, run. The run completed in seconds, cost $0.00021, and took 1.55s in total for the dataset.

Show screenshot: completed Langfuse experiment run

Langfuse experiment run row showing completed status, item count, latency, and total cost

The results show the generated output next to each dataset row. For example, the input {name: "Theo", company: "Harbor Systems"} produced "Hello Theo! Welcome to Harbor Systems support. How can I assist you today?", which follows the prompt with that row's variables:

Show screenshot: Langfuse experiment outputs per dataset item

Langfuse experiment results table showing per-item real generated outputs for each dataset input

The wizard also has an evaluator step for automatic scoring, and each run is tracked on the dataset page.

Doing this from code​

Everything above used the web UI. In an app, you instrument calls with the Langfuse SDK instead. It wraps your existing OpenAI client, so every call is traced with no span code:

import os

from langfuse.openai import openai

client = openai.OpenAI(api_key=os.environ["OPENAI_API_KEY"])

response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Explain prompt versioning in one sentence."}
],
name="explain-prompt-versioning",
)

print(response.choices[0].message.content)

Running this (with LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_HOST also set as environment variables) printed a real answer:

Prompt versioning is the practice of creating, managing, and refining different
iterations of prompts used in AI models to optimize performance, consistency, and
relevance in responses over time.

The call appeared as a trace within seconds, with no trace() or span() calls in the script:

Show screenshot: Langfuse trace from the SDK script

Langfuse trace detail for a script-generated call, showing real cost, latency, and OpenTelemetry SDK metadata

The trace's metadata shows that Langfuse's Python SDK is built on OpenTelemetry (resourceAttributes.telemetry.sdk.name: opentelemetry), an open standard for tracing. Langfuse also documents an OTLP endpoint at /api/public/otel, so an app that already uses OpenTelemetry can send its spans there (checked on 2 November 2026).

Key point: Langfuse's trace view and Git-like prompt versions worked as soon as the project had an OpenAI key. The Playground and the experiment wizard both need that key, and in our test only SDK calls and experiment runs produced traces.

What's unique to Langfuse​

  • An organization → project hierarchy, for teams that run many separate workspaces.
  • Session and user badges on the trace header. Each badge opens every trace with that session or user, from the trace you are reading.
  • "Add to datasets" on the trace page, which turns a production trace into a dataset row in one click.
  • Immutable, Git-like prompt versions with commit messages, so the version list shows what changed in each save. This is a version history, not an audit log: Langfuse's audit log is on the Enterprise plan.

Where this leaves things​

Langfuse's strongest part in this test was the trace view: the observation tree shows cost and latency per node, with the session and user always in view. The prompt versioning model is simple and Git-like. The Playground and Experiments both need an LLM provider connection with your own key, and both worked once a key was in place.

FAQ​

How do you trace OpenAI calls with Langfuse?​

Import the OpenAI client from langfuse.openai and set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_HOST. Each call then appears as a trace with no span code.

How do Langfuse prompt labels work?​

A label points at one prompt version. production marks the live version, latest follows the newest version, and you can add labels with your own names, such as staging.

Does Langfuse support Jinja templates?​

No. Langfuse fills {{variable}} placeholders, but in our 2 November 2026 test {% if %} text came back unchanged. A text prompt can include another prompt with a reference tag instead.

Does a Langfuse Playground run create a trace?​

In our July 2026 test it did not: the Tracing table showed no new rows after a Playground run. SDK calls and experiment runs did produce traces.

Is Langfuse built on OpenTelemetry?​

Yes. Langfuse's Python SDK is built on OpenTelemetry, and Langfuse accepts OpenTelemetry spans on its /api/public/otel endpoint.

Further reading​