Grafana Variables: Build Once, Use Everywhere

The Hard-Coded Trap

Imagine you build a dashboard for the "checkout" service. The "login" team asks for one. You duplicate the dashboard, find-and-replace "checkout" with "login", and save. Now you have two dashboards. Next month you have 20 services and 20 dashboards. When you need to change the time window in the latency query, you update 20 dashboards.

This is the hard-coded trap. The solution is Grafana Variables.

How Variables Work

Variables are defined at the dashboard level and appear as dropdown menus at the top of the dashboard. When a user selects a value, Grafana injects it into every panel query that references the variable.

Step 1: Define the variable

In Dashboard Settings → Variables, create a variable named service with:
- Type: Query
- Data source: Prometheus
- Query: label_values(up, job)

This populates the dropdown with every unique value of the job label from the up metric — effectively, every service Prometheus is currently monitoring.

Step 2: Use the variable in panel queries

Panel queries reference the variable directly in a label matcher — for example, filtering the request rate to job="$service". When the user selects "login" from the dropdown, Grafana substitutes $service with login before sending the query to Prometheus. One dashboard now serves every team.

Multi-Value Variables

Enable "Multi-value" on the variable to allow selecting multiple services simultaneously. Grafana automatically formats the selected values as a regex — $service becomes checkout|login|payment when all three are selected, so a regex-match filter against job picks up all of them at once.

Common variable patterns: