Get metric name in Prometheus Alert

Question

I am setting up a Prometheus alert that will alert on several metrics. It will look something like this:

metric_x > threshold or metric_y > threshold or metric_x > threshold

But if this alert goes off, I would like to include something in the description about which metric got alerted on. Is there a way to know which one of those 3 conditions cause the alarm to go off? Just as a way to add more details in the alarm description? Is the only way to do that is to have separate alarms or include the metric value in the description?

Answer 1

A metric name under the hood is just another label, which means you can insert it into an annotation. Here's how:

- record: foo
  expr: 1
- record: bar
  expr: 2
- alert: test
  expr: foo == 1 or bar == 2
  # You probably need just one of these
  labels:
    name: '{{ .Labels.__name__ }}'
  annotations:
    name: '{{ .Labels.__name__ }}'