Get started with Studio

This page takes you from installing Studio to looking at your own agents’ runs, then shows what to do with it day to day. Studio needs no account and no server: runs are saved to a file next to your app, and loom studio shows them in your browser.

Set it up

Install the web app


pip install 'loomflow[studio]'

Recording is part of loomflow itself. The studio extra adds the small web server that loom studio runs.

Turn recording on

Call enable() once when your app starts, before your agents run:


import loomflow.studio
from loomflow import Agent
 
loomflow.studio.enable(project="support-bot")
 
agent = Agent(
    "You help customers with billing.",
    model="gpt-5-mini",
    name="billing",
)

Or turn it on without touching the code, with an environment variable:


export LOOMFLOW_STUDIO=1                  # or a file path
export LOOMFLOW_STUDIO_PROJECT=support-bot
python app.py                             # or loom serve, uvicorn...

Where the call goes depends on what you’re running:

You’re running	Put `enable()`
A script	At the top, before the first run.
A web app (FastAPI, `loom serve`)	At module level, or in the app’s startup (a FastAPI `lifespan`).
A notebook	In the first cell.
Something you can’t edit	Nowhere: set `LOOMFLOW_STUDIO=1` in its environment.

One switch covers the whole process: every Agent and Workflow, including agents created before you turned it on, sub-agents, team members and workflow steps. Nothing about the agents changes.

Name your agents


billing = Agent(
    "You help customers with billing.",
    model="gpt-5-mini",
    name="billing",   # keeps its history together
)

Studio keeps an agent’s history under its name, so editing its instructions records a new version of billing instead of a new agent. Names also label the agent in lists, timelines and diagrams. An agent without a name is listed by the first line of its instructions.

Run your app as usual


result = await agent.run(
    "Why was I charged twice?",
    user_id="alice",          # filter runs by user
    session_id="chat-42",     # group a conversation into a thread
)

Nothing else changes. Passing user_id lets you filter runs by user, and session_id groups the runs of one conversation into a thread. Runs are written on a background thread while your app keeps going.

Open Studio


loom studio                    # http://127.0.0.1:7600
loom studio --open             # and open it in your browser
loom studio --db path/to/studio.db --port 7700

Runs go to .loomflow/studio.db in the folder your app ran from, so run loom studio from that folder or pass --db. Leave the page open and reload it to see new runs. Add .loomflow/ to your .gitignore.

To try it before wiring up your own app, examples/34_studio.py records a small support bot offline, with no API key:


python examples/34_studio.py
loom studio --db .loomflow/studio-demo.db

Use it day to day

Find a run

The Traces page lists every run, newest first, with its agent and version, user, model, tokens, cost, time and score.

Type in the search box to search everything recorded: prompts, replies, tool arguments and results. Searching an order number finds the run that handled it.
Filter by agent, version, user, model, status or time range.
Tick Problems only to see failed and interrupted runs, and Not scored to see the runs nobody has reviewed yet.
On a trace, click its thread to see every run in that conversation.

Every filtered list is a plain URL, so you can bookmark it or send it to a teammate.

Read a trace

A trace is one run as a timeline: the agent, its turns, each model call and tool call, guardrail hits, and any sub-agents or workflow steps, nested the way they ran, with the architecture diagram under it. Click a step to see it on the right; on a wide screen that panel stays in view while you scroll a long timeline:

A model call: the messages sent, the reply and the tool calls it asked for, tokens, cost and finish reason.
A tool call: the arguments, the result or error, the permission decision, and whether a person approved it.
The agent run: input, output, turns, tokens, cost and how the run ended. An error or an interruption is shown at the top.

JSON gives you everything recorded for the trace as JSON.

See the architecture

Each agent’s page opens with a diagram of how it’s put together: the members of its team and how they delegate or hand off, a workflow’s steps, routes and parallel branches, and the tools each agent holds. There’s a tab per version.

On a trace, the same diagram highlights the agents, steps and tools that run used. Use it to check that a request took the route you expected, or to see which worker a supervisor picked.

See what changed between versions

Studio records a new version whenever an agent’s instructions, model, tools, team or settings change. The Agents page lists the versions with their runs, error rate, score and a note of what changed, such as “Tool added: refund_charge.” Click a version’s run count to see its runs.

So when an agent gets worse, the page shows which version it started with and what that version changed.

Score runs

On a trace’s page, the Scores bar under the run’s summary has Good and Bad buttons that score it in one click; Other score… takes any key, a number or a label, and a comment. Or score from your code, for example with a user’s thumbs-up, a check your code makes, or a judge model’s verdict:


loomflow.studio.score(result, "resolved", True)
loomflow.studio.score(result, "rating", 4, comment="right, but slow")

Scores show on the trace, in the traces list and per version on the Agents page. Eval and OnlineScorer scores are attached for you.

Turn bad runs into test cases

On a trace’s page, Add to dataset… in the Scores bar offers the run’s input and output. Edit the output into the answer the agent should have given, name a dataset, and add it. The Datasets page lists what you’ve collected; run an eval on one with loomflow.studio.dataset("billing-regressions"), as below.

Compare eval runs

Run an eval with experiment= to save it to Studio:


from loomflow.eval import Contains, EvalHarness
from loomflow.eval.dataset import Dataset
 
dataset = Dataset.from_jsonl("billing-faq.jsonl")
await EvalHarness(agent, [Contains()]).run(dataset, experiment="shorter prompt")

or from the command line:


loom eval billing-faq.jsonl --agent app:agent --metric contains --studio

To run it on a dataset you built in Studio, use dataset = loomflow.studio.dataset("billing-regressions"). Or let Studio run it: start it with loom studio --agent app:agent, then use Run eval on the dataset’s page.

On the Experiments page, pick two runs of the same dataset and Compare them. Each case shows the expected answer and both outputs, marked improved, regressed or unchanged, and the top of the page compares the scores, cost and time of the two runs. Scores and experiments has the details.

Settings for each environment

Where	What to use
Your laptop	`enable()` or `LOOMFLOW_STUDIO=1`, plus `keep_days=30` if you leave it on.
Tests	`LOOMFLOW_STUDIO=off` in the test environment, so a variable exported in your shell doesn’t record test runs.
CI evals	`loom eval ... --studio results.db`, and keep the file as a build artifact.
A staging server	`enable(sample_rate=0.1, redact="pii", keep_days=14)` to keep a tenth of the traces, scrub personal data and delete runs after two weeks, or `capture_content=False` to keep no text at all. Start the page with `loom studio --read-only` if nobody should change scores there.

Give each app its own project= (or LOOMFLOW_STUDIO_PROJECT); the sidebar switches between projects. with loomflow.studio.session(...) sets the project, tags or metadata for the runs inside a block, and enabled=False leaves them out. To stop recording, call loomflow.studio.disable() or set LOOMFLOW_STUDIO=off.

The Studio page has no login, and traces hold prompts, replies and tool payloads. Keep loom studio on 127.0.0.1, its default. Studio is for looking at runs on one machine; for monitoring a fleet, export OpenTelemetry to your tracing backend as well.

Troubleshooting

No runs show up. Check that recording is on in the process that ran your agents: loomflow.studio.is_enabled() returns True. Then check that the page reads the same file: its sidebar shows the path it’s reading. The default file is relative to the folder your app ran from, so pass loom studio --db the right path, or give enable() an absolute one.

The last runs of a script are missing. Studio writes everything when the process exits normally. If the process is killed, or you want the runs on disk right away (in a notebook, say), call loomflow.studio.flush() first; await agent.aclose() waits for the writes too.

loom studio says uvicorn isn’t installed. Install the extra: pip install 'loomflow[studio]'.

One agent appears as several. It has no name, and its first instruction line changed. Give it a name=.

score() raises RuntimeError. Studio isn’t on in that process.

The page says “Studio doesn’t answer to this host name”. You opened it under a name other than localhost or 127.0.0.1, for example through a proxy. Start it with loom studio --allow-host NAME.

A form says the page is out of date. Studio was restarted after the page loaded. Reload the page and try again.

The file keeps growing. Set keep_days= (or LOOMFLOW_STUDIO_KEEP_DAYS) to delete old runs, or run loomflow.studio.prune(30) once.

Example: evaluate a RAG agent: the whole loop on a real agent, from recording to comparing two eval runs.
What gets recorded: every switch, what a trace holds, privacy, sampling and storage.
Scores and experiments.
The web app: its pages and URLs, diagrams, and mounting it in your own app.