Skip to content

Monitoring

Track 系统 health 使用 metrics, dashboards, and tools 如 Prometheus and Grafana.

你将学到

  • 为什么 monitoring 是关键 for DevOps
  • Metrics, logs, and traces — three pillars
  • Tools: Prometheus, Grafana, and Datadog
  • 构建 effective dashboards

概念讲解

Why Monitor?

Monitoring is the eyes and ears of your production system. Without it, you are flying blind — you won't know about problems until users complain, and you'll have no data to diagnose them. Good monitoring tells you what is happening right now, alerts you when something goes wrong, and provides historical data to understand trends and plan capacity. In DevOps, monitoring is not optional — it is a core practice that enables fast, safe deployments.

The Three Pillars

Observability rests on three pillars. Metrics are numeric measurements sampled over time — CPU usage, request latency, error rate. Logs are timestamped records of discrete events — a user logged in, a payment failed, a connection was refused. Traces follow a single request through all the services it touches, showing you where time is spent and where errors occur. Together, they give you a complete picture of system behavior.

Tools

Prometheus is an open-source metrics database that scrapes metrics from your applications. Grafana is a visualization tool that creates dashboards from Prometheus and many other data sources. Datadog is a commercial all-in-one platform. Jaeger and Zipkin handle distributed tracing. The ELK stack (Elasticsearch, Logstash, Kibana) is popular for log management. Most teams use a combination of these tools.

Effective Dashboards

A good dashboard answers a specific question: "是 checkout 工作?" or "是我们 meeting 我们的 latency SLO?" Start with high-level service health, then drill down to details. Use golden signals: latency, traffic, errors, and saturation. Avoid dashboard overload — too many panels make it hard to find what matters. Color-code consistently: green for healthy, red for problems. Make dashboards that a tired on-call engineer can understand at 3 AM.

Synthetic Monitoring

Beyond real-user metrics, synthetic monitoring runs scripted checks against your service from external locations. A synthetic check simulates a user journey — log in, add to cart, checkout — and alerts if any step fails. This catches issues before real users are affected, even during low-traffic periods. Combine synthetic checks with real-user monitoring for comprehensive coverage: synthetics for early detection, real-user data for understanding actual impact and performance across different geographies and device types.

示例代码

              # Prometheus scrape configuration
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'myapp'
    static_configs:
      - targets: ['app:9090']
    metrics_path: /metrics

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node-exporter:9100']

# Key metrics to track:
#   http_requests_total           - request count by route/code
#   http_request_duration_seconds - latency histogram
#   process_resident_memory_bytes - memory usage
#   node_cpu_seconds_total        - CPU usage
            

Prometheus scrapes metrics 从你的 applications at regular intervals. 配置定义哪个目标到 scrape and 如何经常. Grafana 然后 visualizes 这个数据.

动手试一试

常见错误

错误写法

Monitoring 一切但 alerting on 无

正确做法

Collecting metrics 是 useless 如果 nobody looks at 它们. 定义 alerting 规则 for metrics 那个 matter — 错误速率, latency, saturation. dashboard nobody 检查是 wasted effort.

错误写法

Alerting on symptoms 相反的 causes

正确做法

Alert on 用户-facing symptoms (错误速率, latency) 而不是 underlying causes (CPU at 80%). Symptom-based alerts 是更多 stable and directly indicate 用户 impact.

练习 Monitoring

2 个练习

练习 Monitoring

查阅不认识的术语

新手友好的术语表用大白话解释编程术语——变量、函数、DOM、Promise 等等。

查看术语表

动手做项目

通过引导式项目应用所学,含起始代码与解决方案。

查看项目

解读错误信息

用大白话解释常见错误——什么意思、为什么发生、如何修复。

查看错误