Viewing and Improving LLM Reasoning

Brim is designed to learn from early generations and feedback to improve abstractions over time.

One of its learning mechanisms is LLM Reasoning and feedback.

LLM reasoning is represented by a question mark icon. You can find LLM reasoning grouped with the generated label value:

• Under the Predicted Label for aggregated variables and dependent variables in the Label Generation view.
• Under the Predicted Label in the Patient Label View
• In the left rail of the Patient Document View, under the text snippets associated with a label.

If you click on the question mark, the section will expand to show an explanation from the LLM for how it calculated the Predicted Label given the input information.

Learning based on LLM Reasoning

Once you've expanded the LLM reasoning, you should see a wrench icon at the end of the reasoning.

Clicking this icon shows a popup asking for feedback on the LLM reasoning. Once you've added your feedback, you have the option to only submit it, or to use that feedback to Optimize the variable so it can perform better on future generations.