How to Use Brim's Result Trace
What Result Trace is
Result Trace shows you how Brim produced a specific value. For any generated value, you can see every LLM call that ran, what each call was asked to do, what it returned, and the logic steps (like aggregation) that turned those outputs into the final result.
Use it to:
- Understand why a value came out the way it did when troubleshooting
- Find which note, prompt or step caused an unexpected result
- Refine a variable's instructions with evidence instead of guesswork
The Result Trace for a value is recorded at generation time. If Result Trace was not on when a value was generated, you'll need to re-generate the value. Due to LLMs being nondeterministic, this sometimes produces a different result than the first time you generated the value.
Turning Result Trace on
Result Trace is off by default. There are three places to control it:
| Where | Who | What it controls |
|---|---|---|
| Brim Admin → LLM Config → New Project Defaults → Store Result Trace | Brim admins | Whether new projects start with Result Trace on |
| Manage Project → LLM Model Configuration → Result Trace | Project admins | Whether this project records traces. Click Save LLM Configuration after changing it. |
| Generation form → Advanced Settings → Generate with Result Trace | Anyone generating | Whether this batch of generations records traces. Defaults to the project setting. |
Turning Result Trace on applies to values generated afterward. Values generated while it was off have no trace unless they are generated again.
Opening a Result Trace
- Find the reasoning for a value. This is visible anytime you see the value, including:
- Label Generation > Generated Labels
- Label Review, in both the tables and the document view.
- Validation
- Click the teal ? icon in the row for the value you want to trace.
- Under the value's reasoning, click the Trace this result icon with the route on it.
The Result Trace page opens for that patient and variable. The header shows the patient, the variable name and when the value was generated.
Reading a Result Trace
The page has three parts, top to bottom: the result, a diagram of how it was generated, and a detail card for each step.
Result
The final value Brim generated. Click the teal ? icon next to it to show the reasoning behind the value.
How this result was generated
A diagram of every step, read left to right, from the source data to the result. A summary under the heading counts the notes, LLM calls and deterministic steps involved. Click any step to jump to its detail card.
The colors match the detail cards below:
| Color | Means |
|---|---|
| Light grey | A note the variable read |
| Light green | An input variable or structured variable |
| Amber | An LLM call |
| Grey | A deterministic step, such as aggregation or a condition check. No LLM is involved, so the same inputs always give the same result. |
| Teal | The final result |
Use Show steps for to filter the diagram and the cards to one note or to aggregation only. When a variable has more than a few notes, the diagram shows the first 5; click Show all to see the rest.
Detail cards
LLM call cards show:
- What this prompt does: a plain-language description of the prompt and its purpose
- Sends: what the call includes, such as the variable definition, the note text or an earlier call's output
- Open Note: a link to the source note, shown on calls that send note text. It opens in a new tab.
- LLM output: exactly what the model returned
Input variable and structured variable cards list each saved value the variable read.
Deterministic step cards explain the rule Brim applied and show the inputs and outcome.
Values without a trace
If a value was generated while Result Trace was off, its trace page shows No Result Trace for this Value. You have two options:
- Re-generate with Result Trace: regenerates only this value with tracing on. The previous value stays visible with a progress bar until the new one is ready. The new value overwrites the current result.
- Turn on Result Trace for this project: opens Manage Project in a new tab. This allows you to enable Result Trace for the full project so future generations are traced.
Generating again
On any trace page, Generate Again reruns the value. It reruns every LLM query, so the result may differ from the current one.
FAQ
Does Result Trace change my results? No. It records how a value was generated; it doesn't change the prompts or the logic.
Can I see the full prompt sent to the LLM? Result Trace shows a plain-language description of each prompt and what it sends, plus the model's full output. You can see an example of the prompt in a variable's Advanced Settings.