Skip to content

Alerting

In one line

Alerting is the practice of automatically notifying the team when metrics cross thresholds or errors occur.

In simple words

Alerting is how you find out about problems before your users do. You define rules — e.g., "if error rate > 5% for 5 minutes, alert the team" — and when the condition is met, the system sends a notification.

Good alerting is a balance. Too few alerts and you miss critical issues. Too many alerts and you get alert fatigue — the team ignores them all. The best alerts are actionable: when an alert fires, there is a clear action to take.

Alerts are routed through an Alertmanager (like Prometheus Alertmanager) to the right destination — Slack for warnings, PagerDuty for critical issues at 3 AM, email for informational alerts. Alerts should include enough context (links to dashboards, logs) to help diagnose the problem.

Example

yaml
# Alertmanager routing
route:
  receiver: 'slack'
  routes:
    - match:
        severity: critical
      receiver: 'pagerduty'
receivers:
  - name: 'slack'
    slack_configs:
      - api_url: 'https://hooks.slack.com/...'
  - name: 'pagerduty'
    pagerduty_configs:
      - service_key: 'your-key'

This routes critical alerts to PagerDuty and all others to Slack.

How it works

  1. You define alert rules (e.g., error rate > 5% for 5 min).
  2. The monitoring system evaluates rules continuously.
  3. When a rule fires, it sends an alert to the Alertmanager.
  4. The Alertmanager routes the alert to the right channel (Slack, PagerDuty, email).

Common confusions

  • Confused with: Monitoring

    The difference: Monitoring collects and displays metrics; alerting is the action taken when metrics indicate a problem. Alerting is a part of monitoring.

Related terms

Decode error messages

Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.

View errors

← Back to glossary