Logs, metrics, traces, profiles và correlation
Mỗi signal tối ưu cho loại câu hỏi khác; correlation dimensions nối chúng thành evidence chain.
| Signal | Mạnh ở | Giới hạn |
|---|---|---|
| Metrics | Trend, aggregate, alert, SLO | Mất per-event detail, cardinality bounded |
| Logs | Discrete event/context | Volume/cost/query consistency |
| Traces | Request path, latency breakdown | Sampling/cost, không phải audit |
| Profiles | CPU/allocation/lock hotspots | Code-focused, cần context timeline |
| Events | Deploy/config/failure transitions | Often ephemeral/rate-limited |
Correlation
service, environment, version, region/AZ, endpoint/operation, trace/span ID và bounded business workflow ID trong logs/traces. Không đưa user/order ID vào metric label. Deploy/config change events đặt trên timeline.
Telemetry pipeline
Instrumentation/agents → SDK/exporter → collector/agent/gateway → storage backends → query/dashboard/alert. Thiết kế backpressure, batching, retry, sampling, local buffering và failure isolation để telemetry outage không hạ application.
Observability vs monitoring
Monitoring checks known failure modes; observability supports novel questions. Cả hai cần semantic conventions, ownership và data quality. “Có log” không nghĩa query được hoặc không chứa PII.
Metrics tell me that a class of requests is unhealthy, traces show where time or errors accumulate across the path, and logs provide detailed event context. I correlate them using service, version, trace ID, and stable business identifiers without logging sensitive data.