hernani/taler-monitoring
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

release 1.8.0: simplified mon pages + surface nmap + mail hosts

Browse source 
Public layout: only taler-monitoring-surface(+_err) for ecosystem
software/versions; nine landing stacks keep /monitoring(+_err).
Fold mail (firefly.gnunet.org, anastasis.taler-systems.com) and
mattermost into the surface job; deprecate separate mon pages/timers.
nmap OS fingerprinting; package-behind defaults to ERROR.
This commit is contained in:
Hernâni Marques 2026-07-19 00:29:49 +02:00
parent ae8646e8de
commit c1a506cf0e
No known key found for this signature in database
15 changed files with 263 additions and 154 deletions
Download patch file Download diff file Expand all files Collapse all files

24
host-agent/README.md
View file

@ -18,8 +18,9 @@ Thin wrappers only set stack defaults:
| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` |
| `run-fp-prod-monitoring.sh` | FP prod | `francpaysan` |
| `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh koopa-external** |
| `run-aptdeploy-monitoring.sh` | **aptdeploy** → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h timer, taler only) |
| `run-surface-monitoring.sh` | **surface** → `/taler-monitoring-surface*` | `hernani` @ koopa (hourly, taler only) |
| `run-aptdeploy-monitoring.sh` | optional apt-src tests (not a public mon-page type) | `hernani` @ koopa (4h) |
| `run-surface-monitoring.sh` | **surface** → only `/taler-monitoring-surface*` (ecosystem + mail + mattermost + versions) | `hernani` @ koopa (hourly) |
| `run-mail-monitoring.sh` / `run-mattermost-monitoring.sh` | **deprecated v1.8.0** — aliases of surface | disabled timers |
### apt-src deploy tests (hacktivism `/monitoring`)
@ -44,24 +45,19 @@ Thin wrappers only set stack defaults:
Checks: package table, `httpd --version`, libtalerutil/`ldd` on failure, `systemctl` for
`taler-merchant.target`, basic local `/config` probe. No nginx/HTTPS in containers.
### surface + aptdeploy HTML (taler.hacktivism.ch only)
### Public HTML layout (v1.8.0 simplified)
| URL | Role | Timer |
|-----|------|-------|
| `…/taler-monitoring-surface/` | ecosystem surface scan | 1h |
| `…/taler-monitoring-surface_err/` | errors + jump list | ″ |
| `…/taler-monitoring-aptdeploy/` | apt-src container deploy tests | 4h |
| `…/taler-monitoring-aptdeploy_err/` | errors + jump list | ″ |
| URL | Role |
|-----|------|
| `https://{bank,exchange,merchant}/monitoring/` (+ `_err`) | **Landing-stack** report — each of the **9** landing fronts (GOA 3 + FP stage 3 + FP prod 3) |
| `https://taler.hacktivism.ch/taler-monitoring-surface/` (+ `_err`) | **Only** ecosystem page: software/versions in the Taler universe (nmap OS, mail firefly+anastasis, mattermost, …) |
Not on bank/exchange. Staging:
`~/monitoring-sites-staging/taler.hacktivism.ch/taler-monitoring-{surface,aptdeploy}*`.
No separate public pages for mail/mattermost. Staging under `~/monitoring-sites-staging/`.
```bash
systemctl --user enable --now taler-monitoring-surface.timer
systemctl --user enable --now taler-monitoring-aptdeploy.timer
systemctl --user start taler-monitoring-surface.service
systemctl --user start taler-monitoring-aptdeploy.service
# Caddy: handles only in taler.hacktivism.ch block (root)
# mail/mattermost timers stay disabled (v1.8.0)
```
Env examples: **`env/*.env.example`**. FP: `run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` (same `run-host-report.sh` pipeline).

21
host-agent/install-host-agent.sh
View file

@ -60,8 +60,10 @@ install_local() {
install -m 644 "$ROOT/taler-monitoring-hacktivism.timer" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-surface.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-surface.timer" "$home_unit/"
# optional aptdeploy (not a public mon page layout — v1.8.0 simplified overview)
install -m 644 "$ROOT/taler-monitoring-aptdeploy.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-aptdeploy.timer" "$home_unit/"
# units still installed for migration; timers disabled (folded into surface)
install -m 644 "$ROOT/taler-monitoring-mattermost.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-mattermost.timer" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-mail.service" "$home_unit/"
@ -72,31 +74,28 @@ install_local() {
systemctl --user enable --now taler-monitoring-hacktivism.timer
systemctl --user enable --now taler-monitoring-surface.timer
systemctl --user enable --now taler-monitoring-aptdeploy.timer
systemctl --user enable --now taler-monitoring-mattermost.timer
systemctl --user enable --now taler-monitoring-mail.timer
# v1.8.0: mail + mattermost folded into surface — stop separate timers
systemctl --user disable --now taler-monitoring-mattermost.timer 2>/dev/null || true
systemctl --user disable --now taler-monitoring-mail.timer 2>/dev/null || true
# initial stamp so path exists
"$ROOT/touch-software-stamp.sh"
systemctl --user start taler-monitoring-hacktivism.service || true
systemctl --user start taler-monitoring-surface.service || true
systemctl --user start taler-monitoring-aptdeploy.service || true
systemctl --user start taler-monitoring-mattermost.service || true
systemctl --user start taler-monitoring-mail.service || true
echo "--- status ---"
systemctl --user status taler-monitoring-hacktivism.path --no-pager -l | head -15
systemctl --user status taler-monitoring-hacktivism.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-surface.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-aptdeploy.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-mattermost.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-mail.timer --no-pager -l | head -12
systemctl --user list-timers --all | grep taler-monitoring || true
echo "OK host-agent (hacktivism 4h + surface 1h + aptdeploy 4h + mattermost 1h + mail 4h)"
echo "OK host-agent (v1.8.0 simplified):"
echo " · hacktivism 4h → /monitoring on bank/exchange/taler (landing stacks)"
echo " · surface 1h → /taler-monitoring-surface* (ecosystem versions/software)"
echo " · aptdeploy 4h (optional tests; not a public mon-page type)"
echo " · mail/mattermost timers disabled (content in surface)"
echo "suite: $mon"
echo "manual: systemctl --user start taler-monitoring-hacktivism.service"
echo "manual surface: systemctl --user start taler-monitoring-surface.service"
echo "manual aptdeploy: systemctl --user start taler-monitoring-aptdeploy.service"
echo "manual mattermost: systemctl --user start taler-monitoring-mattermost.service"
echo "manual mail: systemctl --user start taler-monitoring-mail.service"
echo "after upgrades: $ROOT/touch-software-stamp.sh"
}

42
host-agent/run-mail-monitoring.sh
View file

@ -1,32 +1,20 @@
#!/usr/bin/env bash
# run-mail-monitoring.sh — mail MX/SMTP/IMAP (firefly + pixel, …) → public HTML
# Public only on taler.hacktivism.ch:
# https://taler.hacktivism.ch/taler-monitoring-mail/
# https://taler.hacktivism.ch/taler-monitoring-mail_err/
# DEPRECATED (v1.8.0): mail is part of taler-monitoring-surface.
# Kept as a thin alias so old timers/docs do not break.
#
# Prefer: run-surface-monitoring.sh
# Public page: https://taler.hacktivism.ch/taler-monitoring-surface/
#
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-mail-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-mail-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
export INSIDE_PODMAN=0
export LOCAL_STACK=0
export SKIP_SSH=1
export INSIDE_MODE="${INSIDE_MODE:-none}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
export PHASES="${PHASES:-mail monpages}"
echo "NOTE: run-mail-monitoring.sh is deprecated (v1.8.0) — folding into surface" >&2
export PHASES="${PHASES:-mail surface monpages}"
# force surface HTML paths (no /taler-monitoring-mail*)
export HTML_OK_DIR="taler-monitoring-surface"
export HTML_ERR_DIR="taler-monitoring-surface_err"
export HTML_URL_OK="/taler-monitoring-surface/"
export HTML_URL_ERR="/taler-monitoring-surface_err/"
export PAGE_LABEL="taler-monitoring-surface"
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
export HTML_OK_DIR="taler-monitoring-mail"
export HTML_ERR_DIR="taler-monitoring-mail_err"
export HTML_URL_OK="/taler-monitoring-mail/"
export HTML_URL_ERR="/taler-monitoring-mail_err/"
export PAGE_LABEL="taler-monitoring-mail"
export APT_DEPLOY_ENSURE=0
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
export MONPAGES_INVENTORY=job
exec bash "$AGENT_DIR/run-surface-monitoring.sh" "$@"

44
host-agent/run-mattermost-monitoring.sh
View file

@ -1,35 +1,19 @@
#!/usr/bin/env bash
# run-mattermost-monitoring.sh — Mattermost (mattermost.taler.net) → public HTML
# Public only on taler.hacktivism.ch:
# https://taler.hacktivism.ch/taler-monitoring-mattermost/
# https://taler.hacktivism.ch/taler-monitoring-mattermost_err/
# DEPRECATED (v1.8.0): mattermost is part of taler-monitoring-surface.
# Kept as a thin alias so old timers/docs do not break.
#
# Prefer: run-surface-monitoring.sh
# Public page: https://taler.hacktivism.ch/taler-monitoring-surface/
#
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-mattermost-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-mattermost-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
# remote public service — no SSH / podman
export INSIDE_PODMAN=0
export LOCAL_STACK=0
export SKIP_SSH=1
export INSIDE_MODE="${INSIDE_MODE:-none}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-300}"
# dedicated job: mattermost + public page check
export PHASES="${PHASES:-mattermost monpages}"
echo "NOTE: run-mattermost-monitoring.sh is deprecated (v1.8.0) — folding into surface" >&2
export PHASES="${PHASES:-mattermost surface monpages}"
export HTML_OK_DIR="taler-monitoring-surface"
export HTML_ERR_DIR="taler-monitoring-surface_err"
export HTML_URL_OK="/taler-monitoring-surface/"
export HTML_URL_ERR="/taler-monitoring-surface_err/"
export PAGE_LABEL="taler-monitoring-surface"
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
export HTML_OK_DIR="taler-monitoring-mattermost"
export HTML_ERR_DIR="taler-monitoring-mattermost_err"
export HTML_URL_OK="/taler-monitoring-mattermost/"
export HTML_URL_ERR="/taler-monitoring-mattermost_err/"
export PAGE_LABEL="taler-monitoring-mattermost"
export MATTERMOST_PUBLIC="${MATTERMOST_PUBLIC:-https://mattermost.taler.net}"
export APT_DEPLOY_ENSURE=0
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
export MONPAGES_INVENTORY=job
exec bash "$AGENT_DIR/run-surface-monitoring.sh" "$@"

36
host-agent/run-surface-monitoring.sh
View file

@ -1,26 +1,33 @@
#!/usr/bin/env bash
# run-surface-monitoring.sh — hourly remote-only ecosystem surface scan
# Public only on taler.hacktivism.ch:
# run-surface-monitoring.sh — ecosystem surface report (v1.8.0+ simplified)
#
# Public HTML only on taler.hacktivism.ch:
# https://taler.hacktivism.ch/taler-monitoring-surface/
# https://taler.hacktivism.ch/taler-monitoring-surface_err/
#
# This single page covers remote ecosystem inventory: host/port probes, nmap OS
# fingerprint, Mattermost, mail (firefly + anastasis), package/version signals.
# There are NO separate public pages for mail / mattermost / aptdeploy.
#
# Landing-stack reports stay on each of the 9 fronts as /monitoring(/_err).
#
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-surface-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-surface-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
# surface is remote-only — force flags (ignore shared env)
export INSIDE_PODMAN=0
export LOCAL_STACK=0
export SKIP_SSH=1
export INSIDE_MODE="${INSIDE_MODE:-none}"
# surface is primarily remote; versions may use host podman when available
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
export LOCAL_STACK="${LOCAL_STACK:-0}"
export SKIP_SSH="${SKIP_SSH:-1}"
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
# many hosts; allow up to 45 min wall (phase also has RUN_TIMEOUT)
export RUN_TIMEOUT="${RUN_TIMEOUT:-2400}"
# dedicated job: always surface only (never inherit PHASES from shared env)
export PHASES="${PHASES:-surface monpages}"
# Only merchant public front — not bank/exchange
# All ecosystem checks → one surface HTML report
export PHASES="${PHASES:-surface mattermost mail versions monpages}"
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
@ -31,11 +38,12 @@ export HTML_URL_OK="/taler-monitoring-surface/"
export HTML_URL_ERR="/taler-monitoring-surface_err/"
export PAGE_LABEL="taler-monitoring-surface"
# ecosystem catalog by default (not domain-only)
export SURFACE_SCOPE="${SURFACE_SCOPE:-ecosystem}"
export TALER_DOMAIN_FROM_CLI=0
# Do not run apt-deploy ensure for surface-only jobs
export APT_DEPLOY_ENSURE=0
# monpages job-only for this path (surface HTML); not the 9 landings
export MONPAGES_INVENTORY="${MONPAGES_INVENTORY:-job}"
# package behind → ERROR (v1.8.0)
export TALER_PKG_BEHIND="${TALER_PKG_BEHIND:-error}"
exec bash "$AGENT_DIR/run-host-report.sh" "$@"