Using the Variable Scorecard
Understanding the Variable Scorecard
The Variable Scorecard helps you design clearer, more consistent, and more effective variables. It automatically evaluates each variable and gives specific, actionable feedback so you can quickly see where to improve.
Why Use the Variable Scorecard?
Good variable design is the foundation of reliable AI-guided chart abstraction. The Variable Scorecard helps you:
- Improve clarity: Identify when a variable’s instructions are too complex or ambiguous.
- Ensure consistency: Confirm that prompts, options, and definitions align across a variable.
- Focus effort: See which variables need attention, and prioritize the specific edits that will make your variable stronger.
- Build trust: Well-defined variables produce more stable, reproducible outputs, both for abstractors and AI models.
The Scorecard is especially useful when:
- You’re building a new variable and want to confirm it’s well-structured.
- You’re validating existing variables before sharing them across projects.
- You want to standardize variable quality across a team or institution.
How to Find the Variable Scorecard
- Go to Project Setup on your Brim Dashboard.
- Find the variable you want to review, and click "Edit".
You’ll see a top-level score, as well as a breakdown of your variable’s evaluation across four categories:
- Simplicity. Is the variable doing one thing, clearly?
- Semantics. Are instructions precise and complete?
- References. Are linked variables correctly referenced?
- Completeness. Are examples, names, and options consistent and examples included?
Each item includes a rating and, when applicable, a recommended fix describing how to address the issue.
What to Do if My Variable Has a Low Score
A red or yellow variable score doesn’t mean your variable is “bad”; it means there’s room to make it more robust and easier to interpret.
Here are a few examples of feedback your Variable Scorecard might give you, and how to improve:
| Category | Common Issue | How to Improve |
|---|---|---|
| Simplicity | Variable includes multiple concepts | Split into separate variables or simplify the logic |
| Semantics | Missing temporal context | Specify “use most recent value” or “current status only” |
| References | References included in "Input Variables", but missing from prompt | Explicitly include "{Variable Name}" in the instruction for your input variables. |
| Completeness | No examples | Add text examples to the instruction; ideally an example and a counter example at least. |
After making changes, you can recalculate your score by clicking "Recalculate Score" to view updated feedback. Make sure to save your variable when you're done.