3d541d818a
- Add prometheus-client to base requirements; sentry-sdk to prod - api/metrics.py: define HTTP latency histogram, request/error counters, in-flight gauge - api/middleware.py: extend SecurityAuditMiddleware to observe all four Prometheus collectors per request; low-cardinality path_template label via URL resolver - api/views.py: /metrics/ endpoint (gated by METRICS_ENABLED setting) - api/urls.py: wire /metrics/ route - config/settings/prod.py: METRICS_ENABLED flag; optional Sentry SDK init via SENTRY_DSN env var - ops/prometheus/alerts.yml: Prometheus alert rules for p95 latency SLO (≤500 ms), error rate SLO (<1%), availability, and saturation - ops/prometheus/prometheus.yml: scrape config for app + blackbox healthcheck probe - ops/scripts/backup.sh: pg_dump → S3 STANDARD_IA with retention metadata - ops/scripts/restore.sh: pg_restore from S3 or local file with interactive confirmation guard - ops/scripts/synthetic-check.sh: post-deploy smoke test (healthz, metrics gate, schema, 404 shape) - docs/TRA-249-observability-slos.md: SLO table, PromQL reference queries, alert routing - docs/TRA-249-backup-restore.md: RPO/RTO targets, drill procedure, restore validation steps - docs/TRA-249-release-checklist.md: pre/post-deploy checklist - docs/TRA-249-rollback-runbook.md: decision matrix, app rollback, migration revert, DB restore path Co-Authored-By: Paperclip <noreply@paperclip.ing>
36 lines
1.1 KiB
Python
36 lines
1.1 KiB
Python
"""
|
|
Prometheus metrics definitions for the Trainingssoftware API.
|
|
|
|
Collectors are registered once at app startup (see apps.py ready()).
|
|
The middleware records observations; the /metrics endpoint exposes them.
|
|
"""
|
|
from prometheus_client import Counter, Histogram, Gauge
|
|
|
|
# Request latency histogram — buckets tuned for web API (50ms … 10s)
|
|
HTTP_REQUEST_DURATION_SECONDS = Histogram(
|
|
"http_request_duration_seconds",
|
|
"HTTP request latency",
|
|
["method", "path_template", "status_code"],
|
|
buckets=(0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0),
|
|
)
|
|
|
|
# Total requests counter (separate from histogram for cheap rate queries)
|
|
HTTP_REQUESTS_TOTAL = Counter(
|
|
"http_requests_total",
|
|
"Total HTTP requests",
|
|
["method", "path_template", "status_code"],
|
|
)
|
|
|
|
# Error rate counter — 5xx responses only
|
|
HTTP_SERVER_ERRORS_TOTAL = Counter(
|
|
"http_server_errors_total",
|
|
"Total HTTP 5xx responses",
|
|
["method", "path_template", "status_code"],
|
|
)
|
|
|
|
# In-flight requests gauge
|
|
HTTP_REQUESTS_IN_FLIGHT = Gauge(
|
|
"http_requests_in_flight",
|
|
"Current number of in-flight HTTP requests",
|
|
)
|