Optional local monitoring
Start the production-local application with its normal make prod-local-*
commands first. Monitoring is optional, uses private authenticated metric
listeners, and requires no cloud account or notification credentials.
Prometheus and Alertmanager UIs publish only on loopback; this is not an
internet-facing monitoring deployment or a route to a real responder.
Prepare private files from the bootstrap-generated trusted .env:
export HOOKWAY_LOCAL_DIRECTORY="$(pwd)/examples/production-local/.local"
export HOOKWAY_MONITOR_UID="$(id -u)" HOOKWAY_MONITOR_GID="$(id -g)"
umask 077
mkdir -p "$HOOKWAY_LOCAL_DIRECTORY/monitoring"
chmod 700 "$HOOKWAY_LOCAL_DIRECTORY/monitoring"
set -a
. "$HOOKWAY_LOCAL_DIRECTORY/.env"
set +a
printf '%s' "$HOOKWAY_API_KEY" > "$HOOKWAY_LOCAL_DIRECTORY/monitoring/core-token"
printf '%s' "$REPLAY_METRICS_TOKEN" > "$HOOKWAY_LOCAL_DIRECTORY/monitoring/replay-token"
chmod 600 "$HOOKWAY_LOCAL_DIRECTORY/monitoring/core-token" "$HOOKWAY_LOCAL_DIRECTORY/monitoring/replay-token"
docker compose --env-file "$HOOKWAY_LOCAL_DIRECTORY/.env" \
--project-name "$COMPOSE_PROJECT_NAME" \
-f examples/production-local/docker-compose.yaml \
-f examples/production-local/monitoring/compose.yaml \
--profile monitoring up -d --no-deps prometheus alertmanager alert-receiver
Use the same --project-name as the application (PROD_LOCAL_PROJECT if
overridden). Token files stay mode 600 and the scraper runs as their owner;
do not make secrets world-readable. Refresh files after credential rotation.
Run this recipe as a non-root local operator; do not set the monitoring UID to 0.
Docker volume paths resolve against the first Compose file, not this directory.
Inspect Prometheus targets/rules and
Alertmanager. The echo receiver acknowledges local
alert webhooks; it is not a paging service or a retained notification history.
For core-only mode copy prometheus.yaml into the ignored local directory,
remove the replay scrape job and override its config mount. Do not start replay
solely for monitoring or bypass metrics authentication. One replay job must
contain replicas of one logical database; separate databases need separate jobs.
Shared database gauges use max by (job, ...), not sums across replicas.
GC/freshness/database accessibility remain replica-local diagnostics.
These sample thresholds are starting points, not agreed SLOs. Review alert hold durations, GC cadence, storage limits and incident ownership in the observability runbook. Load the existing dashboards into an operator-managed Grafana if desired; Grafana is not required for application operation.
Monitoring data/silences use bounded temporary storage and are lost on container
recreation. Application volumes are unchanged. To remove only monitoring, use
the same Compose arguments with stop prometheus alertmanager alert-receiver,
then rm -f prometheus alertmanager alert-receiver. Do not use down -v.
Run make observability-check to check the supplied monitoring assets and local
notification tests. Docker is needed for notification tests; use
HOOKWAY_REQUIRE_INTEGRATION=1 to fail rather than skip if Docker is unavailable.