Advanced: Troubleshooting Variables
My variable is computing dates or time periods incorrectly
Brim includes an option for including today's date in the LLM prompt.
- If computing your variable requires knowledge of today's today, scroll down to the bottom of the Edit/Create variable form, and make sure the box next to "Include Today's Date in Prompt" is checked.
- Specify the format you want a time period in explicitly (number of hours, number of minutes, number of days) and whether you want the variable to round up or down.
My variable is pulling a lot of irrelevant evidence
The LLM tries to find relevant evidence, and without more information, it can find too much.
- Try explaining the difference between what you want included and excluded. You can do this in the Variable Instructions, or by labelling examples in Label Review and Optimizing the variable.
- Be as black and white as you can in the instruction. Try adding the instruction to "Ignore" the specific case(s) you want to exclude
I'm not getting any results when I generate
By default, Brim hides label instances that represent an "Empty" response. In Label Review > Variables > choose a variable, try clicking "Show Default value labels".
If "Show Default value labels" is on and you see values now:
- Verify that your data set has evidence for this variable.
- Try making the variable name more descriptive.
If "Show Default value labels" is on and you still don't see values:
- Verify that your data set is loaded
- In the Variable Settings, scroll down, click "Advanced Settings" and ensure that:
- The "Include variable in generation" box is checked.
- The "Restrict variable to specific documents" option is empty or includes correctly spelled document names you want to look through.
- Ensure that the patient you're looking at was included in your most recent label generation.
- Try generating again for this specific patient.
The LLM Reasoning for my variable is drawing the wrong conclusion
Vague or faulty LLM reasoning is often a consequence of the LLM not knowing how to behave in a specific situation:
- Explicitly tell it how to react if it is missing relevant information (e.g. "If there are no values for the input variable, return False.")
- Check that the variable is receiving the information it needs from the notes.