Understanding Token Usage in Brim

Brim provides clear, transparent visibility into how your projects consume compute  both at a high level across your organization and in the moment as you generate labels. Whether you're managing costs across a large study or understanding the compute impact of a single label generation, Brim surfaces token usage where it matters most.

This guide walks through each section of the dashboard and how to use it to manage spend.

IMPORTANT: If your project was created before your Brim version began tracking input and output tokens separately, costs start in the first full month after the upgrade. The dashboard shows a note marking when cost tracking began.


Why Token Usage Matters

Large language models (LLMs) process text in “tokens,” and token usage directly affects compute cost.

The Token Usage Dashboard helps you:

  • Track usage over time across projects
  • Understand which variables drive the most compute
  • Compare models by both usage and cost
  • Forecast estimated expected spend
  • Identify opportunities to optimize workflows (e.g., via conditional generation, variable redesign)

Label Generation Token Counts helps you see per generation token visibility directly in the Generate Labels workflow within your project.

Token counts include LLM calls for label generation, variable creation and optimization, Variable Scorecard, and other features, and exclude generations that Brim has cached.


For the most accurate token counts, we recommend checking with your compute provider.


Where to Find the Token Usage Dashboard

You can find the Token Usage Dashboard in two places:

  1. Admins only:
    1. Settings > Admin
    2. Projects
    3. "View Usage Dashboard" button
    4. You can filter by any combination of projects or all projects.
  2. Project Owners:
    1. When the Project you are an Owner for is open, go to Settings > Token Usage
    2. You will see the current project selected by default, but can select all projects for which you are the Owner.

Cost Transparency

Admins can turn on Cost Transparency in Admin > LLM Config. To turn it on, every active model needs at least an input and output price. Click Edit Pricing on a model to add input, output, and cached prices per million tokens. When Cost Transparency is on, Brim shows estimated costs across the app using these prices. To learn more see "Admin LLM Configuration"


Filters & Settings

Use the panel on the left to customize the view.

Projects

Select a single project or view All Projects for which you have permissions combined.

Date Range

Use From and To to filter usage for a specific time period—such as a monthly billing cycle or a particular phase of your study. Keep in mind that the most recent month is likely partial.

Average Cost per Million Tokens

Enter your cost-per-million-tokens based on your LLM provider’s pricing.

Brim uses this value to estimate spend throughout the dashboard.

Click "Apply" to apply the filters to the data on the right.

You can reset filters at any time using Reset.


Note: When Cost Transparency is on, Average Cost per Million Tokens is removed and Brim uses the model prices your admin set.


High-Level Metrics

At the top of the dashboard, Brim summarizes key compute indicators:

Total Tokens

Your overall token usage for the selected projects and timeframe, with an estimated cost.

Average per Document

This is a rough estimate of how many tokens are processed for each document, with an estimated cost.

  • NOTE: This is a very basic calculation of Total Tokens in the timeframe / Total Documents uploaded to the project. This can make the average skewed if you haven't generated for every document.

Average per Patient

This is a rough estimate of how many tokens are processed for each patient, with an estimated cost.

  • NOTE: This is a very basic calculation of Total Tokens in the timeframe / Total Patients uploaded to the project. This can make the average skewed if you haven't generated for every patient.

When Cost Transparency is on, each box leads with estimated cost (for example, Total Estimated Cost) with tokens shown underneath. When it’s off, costs use the Average Cost per Million Tokens you enter. Costs and averages only include priced usage. If some tokens can't be priced (for example, models without pricing, embeddings, or providers that don't report input and output tokens), the dashboard notes how many below these boxes.


Token Usage Over Time

This time-series chart shows how your token usage trends month by month.

You can use this view to identify:

  • Onboarding spikes
  • Months where new variables or models were added
  • Effects of optimization or conditional logic
  • Unexpected anomalies in compute consumption

Hover over any point to see exact usage values.


Token Usage by Feature

This pie chart shows how your token usage splits across Brim features:

  • Generation: label generation runs
  • Optimizer: variable optimization
  • Other: AI variable creation, embeddings, the Brim Assistant, Variable Scorecard, and other features. This also includes all token usage from before your institution upgraded to version 2026.10.31.

Use this view to see whether generation or other work, like optimization, is driving your usage.


Tokens by Project

This graph is only visible if you have multiple projects selected in Filters.

See which projects consume the most compute, ranked from highest to lowest.

This view is especially valuable for:

  • Understanding which projects are resource-intensive
  • Spotting projects where optimization could yield large cost savings
  • Reporting usage to stakeholders by project or study team

Each bar shows token usage and estimated cost, using your admin’s model prices when Cost Transparency is on, or your Average Cost per Million Tokens when it’s off.


Top 20 Variables by Token Usage

This breakdown shows which variables drive the most token usage. Each bar is labeled with that variable's estimated cost.

Use this view to:

  • Identify high-cost variables
  • Detect variables that might benefit from conditional generation
  • Compare performance across variable types
  • Prioritize optimization work

Tokens by Model

This graph is only visible if you have token usage for more than 1 model.

A pie chart summarizes the distribution of token usage across model types (e.g., gpt-4o-mini, gpt-5o-mini, gpt-4o).

This helps answer questions like:

  • Are premium models driving most of your spend?
  • How much compute is attributable to lightweight vs. heavyweight models?
  • Would switching models materially change cost?

Estimated Cost by Model

This graph is only visible if you have token usage for more than 1 model.

If Cost Transparency is off: you enter a price per model and click Update Graph.

If Cost Transparency is on: no inputs needed. Brim uses blended prices from LLM Config and shows each model's cost, share, and blended rate per million tokens in a table. The blended rate combines the input, output, and cached-token prices your admin set for each model.

You’ll see:

  • A per-model breakdown of estimated cost
  • A combined Total Estimated Cost
  • A visual pie chart showing which models drive the majority of spend


Token Usage by Input/Output

This shows the split between input and output tokens, overall and by model.

  • Output tokens usually cost more, so a higher output share can raise costs even at the same total.
  • The totals table shows input and output tokens with their share and estimated cost.
  • The by-model table shows each model's total tokens and its input/output split.
  • Tokens recorded without an input/output split aren't included in this view.



Token usage in Brim can also be seen in the Label Generation page. Brim offers an estimator for token generation as well as the token counts after you generate.

Token Estimates in Label Generation

Before you run a generation, Brim displays an Estimated Max Tokens figure to help you understand your potential token usage upfront. This estimate is calculated based on the number of variables in your project, the patients you've selected, and the average length of their documents, and it updates automatically as you adjust your patient selection.

Keep in mind: this is a maximum estimate, not a prediction of exact usage. In practice, your actual token consumption will often be lower, since each generation runs dynamically and real document lengths and filter conditions vary. Think of it as a ceiling, not a target.

If you've configured filters or overwrite settings, click Refine Estimate to get a tighter maximum that accounts for those specifics. Once refined, the estimate will reflect your variable filters, variable value filters, note ID filters, and overwrite settings. If you update any settings after refining, use Update Estimate to recalculate.

For a deeper look at how estimates are calculated, see Estimating Token Usage.

In addition to estimating the Max Tokens for a generation, Brim will:

  1. Show a yellow alert if your generation is expected to use more than 100 million tokens.
  2. Show a red warning if your generation is expected to use more tokens than the tokens remaining in your project token limit.
    1. In this case, Brim will also require a confirmation in order to start the generation.

Label Generation Token Counts

When generating labels, Brim displays the number of tokens consumed for each individual label generation. This gives you immediate, in-context feedback on compute usage as you work without needing to navigate to the dashboard. With Cost Transparency on, the completion banner shows the cost after tokens used, e.g. 65,871 tokens used (~$0.03).

You can use per-generation token counts to:

  • Understand the compute footprint of specific variables or prompts
  • Compare token usage across different label types
  • Spot unexpectedly high-cost generations before they scale across a full dataset
  • Inform decisions about prompt design and variable configuration
  • Estimate future costs of generation

This visibility is especially useful during the early stages of a project, when you're iterating on variable design and want to understand the cost implications of different approaches.


Estimated Cost in Generation History

When Cost Transparency is on, the Tokens column in Generation History shows an estimated cost for each generation, based on the model prices set in LLM Config.

Use estimated cost to:

  • See the estimated cost of a pilot run before scaling to your full cohort
  • Compare estimated cost across models or variable changes
  • Check how much a stopped generation used before it hit the project limit

Estimated cost is approximate. It excludes cached tokens and may not match your provider’s bill exactly. For the most accurate figures, check with your compute provider.



Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.