Troubleshooting Variables

My variable is pulling a lot of irrelevant evidence

The LLM tries to find relevant evidence, and without more information, it can find too much.

• Try explaining the difference between what you want included and excluded. You can do this in the Variable Instructions, or by labelling examples in Label Review and Optimizing the variable.
• Be as black and white as you can in the instruction. Try adding the instruction to "Ignore" the specific case(s) you want to exclude

I'm not getting any results when I generate

By default, Brim hides label instances that represent an "Empty" response. In Label Review > Variables > choose a variable, try clicking "Show Default value labels".

If "Show Default value labels" is on and you see values now:

• Verify that your data set has evidence for this variable.
• Try making the variable name more descriptive.

If "Show Default value labels" is on and you still don't see values:

• Verify that your data set is loaded
• In the Variable Settings, scroll down, click "Advanced Settings" and ensure that:
  • The "Include variable in generation" box is checked.
  • The "Restrict variable to specific documents" option is empty or includes correctly spelled document names you want to look through.
• Ensure that the patient you're looking at was included in your most recent label generation.
• Try generating again for this specific patient.

My variable is computing dates or time periods incorrectly

If the LLM needs to use today's date, it needs to be told today's date.

• Try telling it today's date in the prompt explicitly (eg "assume today's date is 01/02/2025") and regenerate to see if that improves the results.
• Specify the format you want a time period in explicitly (number of hours, number of minutes, number of days) and whether you want the variable to round up or down.

The LLM Reasoning for my variable is drawing the wrong conclusion

Vague or faulty LLM reasoning is often a consequence of the LLM not knowing how to behave in a specific situation:

• Explicitly tell it how to react if it is missing relevant information (e.g. "If there are no values for the input variable, return False.")
• Check that the variable is receiving the information it needs from the notes.