3d541d818a
- Add prometheus-client to base requirements; sentry-sdk to prod - api/metrics.py: define HTTP latency histogram, request/error counters, in-flight gauge - api/middleware.py: extend SecurityAuditMiddleware to observe all four Prometheus collectors per request; low-cardinality path_template label via URL resolver - api/views.py: /metrics/ endpoint (gated by METRICS_ENABLED setting) - api/urls.py: wire /metrics/ route - config/settings/prod.py: METRICS_ENABLED flag; optional Sentry SDK init via SENTRY_DSN env var - ops/prometheus/alerts.yml: Prometheus alert rules for p95 latency SLO (≤500 ms), error rate SLO (<1%), availability, and saturation - ops/prometheus/prometheus.yml: scrape config for app + blackbox healthcheck probe - ops/scripts/backup.sh: pg_dump → S3 STANDARD_IA with retention metadata - ops/scripts/restore.sh: pg_restore from S3 or local file with interactive confirmation guard - ops/scripts/synthetic-check.sh: post-deploy smoke test (healthz, metrics gate, schema, 404 shape) - docs/TRA-249-observability-slos.md: SLO table, PromQL reference queries, alert routing - docs/TRA-249-backup-restore.md: RPO/RTO targets, drill procedure, restore validation steps - docs/TRA-249-release-checklist.md: pre/post-deploy checklist - docs/TRA-249-rollback-runbook.md: decision matrix, app rollback, migration revert, DB restore path Co-Authored-By: Paperclip <noreply@paperclip.ing>
68 lines
1.8 KiB
Bash
Executable File
68 lines
1.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# restore.sh — PostgreSQL restore for Trainingssoftware
|
|
#
|
|
# CAUTION: This drops and recreates the target database. Never run against
|
|
# production without explicit human confirmation and a pre-restore
|
|
# backup of the target.
|
|
#
|
|
# Usage:
|
|
# TARGET_DATABASE_URL=postgres://user:pass@host:5432/dbname_restore \
|
|
# ops/scripts/restore.sh s3://my-bucket/backups/trainingssoftware-20260101T120000Z.dump
|
|
#
|
|
# — or with a local file —
|
|
#
|
|
# TARGET_DATABASE_URL=postgres://... \
|
|
# ops/scripts/restore.sh /tmp/trainingssoftware-20260101T120000Z.dump
|
|
set -euo pipefail
|
|
|
|
SOURCE="${1:-}"
|
|
if [[ -z "${SOURCE}" ]]; then
|
|
echo "Usage: $0 <s3://bucket/key.dump | /local/path/file.dump>" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -z "${TARGET_DATABASE_URL:-}" ]]; then
|
|
echo "ERROR: TARGET_DATABASE_URL is not set." >&2
|
|
exit 1
|
|
fi
|
|
|
|
DUMP_FILE="${SOURCE}"
|
|
|
|
# Download from S3 if needed
|
|
if [[ "${SOURCE}" == s3://* ]]; then
|
|
DUMP_FILE="/tmp/restore-$(date -u +%Y%m%dT%H%M%SZ).dump"
|
|
echo "[restore] Downloading ${SOURCE} → ${DUMP_FILE}"
|
|
aws s3 cp "${SOURCE}" "${DUMP_FILE}"
|
|
fi
|
|
|
|
echo "[restore] Source dump: ${DUMP_FILE}"
|
|
echo "[restore] Target: ${TARGET_DATABASE_URL}"
|
|
echo ""
|
|
echo "WARNING: This will DROP all objects in the target database and restore"
|
|
echo " from the dump. Data in the target will be permanently lost."
|
|
echo ""
|
|
read -r -p "Type 'yes' to continue: " CONFIRM
|
|
if [[ "${CONFIRM}" != "yes" ]]; then
|
|
echo "[restore] Aborted."
|
|
exit 1
|
|
fi
|
|
|
|
echo "[restore] Restoring…"
|
|
pg_restore \
|
|
--format=custom \
|
|
--clean \
|
|
--if-exists \
|
|
--no-acl \
|
|
--no-owner \
|
|
--dbname="${TARGET_DATABASE_URL}" \
|
|
"${DUMP_FILE}"
|
|
|
|
echo "[restore] Restore complete."
|
|
|
|
# Cleanup temp download
|
|
if [[ "${SOURCE}" == s3://* ]]; then
|
|
rm -f "${DUMP_FILE}"
|
|
fi
|
|
|
|
echo "[restore] Done."
|