Metric Filters: Logs to Metrics

Turning Text into Numbers

Logs Insights is powerful for investigation, but it requires a human to run a query. What if you want to automatically track how often a specific event occurs — and alert on it — without anyone having to remember to check?

That is what Metric Filters do. A Metric Filter is a persistent rule attached to a Log Group. Every time a new log event arrives that matches the filter pattern, CloudWatch increments (or sets) a custom CloudWatch metric. The result is a numerical time-series you can graph, alarm on, and include in dashboards — all derived automatically from your existing logs.

The Transformation Pipeline:

Application writes log line
        ↓
CloudWatch Logs ingests it into the Log Group
        ↓
Metric Filter pattern matcher evaluates the line
        ↓
  Match? → Emit metric value (e.g., 1) to custom namespace
  No match? → Discard (log line is still stored, just not counted)
        ↓
Custom metric appears in CloudWatch Metrics
        ↓
Alarm evaluates the metric on a schedule

Creating a Metric Filter:

Creating one is a single call against an existing log group: a filter name, a pattern to match, and a metric transformation specifying the target metric's name, namespace, the value to emit on a match, and a default value for periods with no matches.

The defaultValue=0 is critical. Without it, CloudWatch only emits a data point when the pattern matches. If there are no errors for 10 minutes, the metric has no data points — and an alarm evaluating "average over 5 minutes" will treat missing data as "insufficient data" rather than zero. Setting defaultValue=0 ensures the metric always has a value, making alarm behavior predictable.

Filter Pattern Syntax:

Metric Filter patterns are simpler than Logs Insights queries — they are pattern matchers, not a query language.

JSON Log Format (Recommended):
If your application logs structured JSON (e.g., {"level":"ERROR","service":"payment","latencyMs":1200}), Metric Filters can extract numeric values directly using the same $.field pattern syntax shown above, paired with a metric transformation.

You can even use the log field value as the metric value — for example, emitting $.latencyMs as the metric value to track latency as a metric derived from logs.

Limits to Know:
- Maximum 100 Metric Filters per Log Group.
- Metric Filters only process new log events after the filter is created. They do not backfill historical data.
- There is no cost for creating Metric Filters themselves — you pay standard CloudWatch custom metric pricing for the metrics they emit (approximately $0.30/metric/month for the first 10,000 metrics).