From f38142cbd31b19c62204866ca8cd9176e7503bec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Wed, 22 Jul 2026 03:35:05 +0200 Subject: [PATCH] fix 1.23.5: per-job mon page publish (no full-tree rsync --delete) Shared /var/www/monitoring-sites is multi-agent. Full-tree --delete wiped sibling families (e.g. monitoring-max-ladder after main/surface runs). Publish only HTML_OK_DIR/HTML_ERR_DIR per host; apply-monitoring-live merges. --- VERSION | 2 +- VERSIONS.md | 1 + host-agent/README.md | 1 + host-agent/ROOT-APPLY-MONITORING.md | 10 ++++-- host-agent/apply-monitoring-live.sh | 7 ++-- host-agent/run-host-report.sh | 51 +++++++++++++++++++++++++---- site-gen/ROOT-ON-KOOPA.md | 2 +- 7 files changed, 61 insertions(+), 13 deletions(-) diff --git a/VERSION b/VERSION index 27ddcc1..ca8ec41 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.23.4 +1.23.5 diff --git a/VERSIONS.md b/VERSIONS.md index 1da8044..1506e64 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -17,6 +17,7 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre | Tag | Date (UTC) | Notes | |-----|------------|--------| +| **v1.23.5** | 2026-07-22 | **Bugfix / host-agent:** multi-agent shared `/var/www/monitoring-sites` — stop full-tree `rsync --delete` (wiped sibling pages e.g. `/monitoring-max-ladder/` after main/surface/aptdeploy runs). Publish **only this job’s** `HTML_OK_DIR`/`HTML_ERR_DIR` per host; `apply-monitoring-live.sh` merges without tree-delete. | | **v1.23.4** | 2026-07-19 | **Bugfix / mon:** shared **normalize_taler_uri** (strip taler:// host :443/:80); ladder/e2e/pay use it. **urls** scans bank+exchange+merchant intros + demo-withdraw/auto-account/stats for default ports (**ERROR** on GOA). Unit test tests/test_normalize_taler_uri.sh. | | **v1.23.3** | 2026-07-19 | **Bugfix:** payments stale because max-ladder never got coins — root cause on koopa was **taler-wallet-cli missing** (all mint-OK → SKIP_ACCEPT). Hard-fail ladder prereq if CLI/helper missing; settle keeps polling after transfer_done + short run-pending; pay wait default 180s; max-ladder wrapper sets PATH/WALLET_CLI. | | **v1.23.2** | 2026-07-19 | **Bugfix:** max-ladder host-agent forces **RUN_TIMEOUT=10800** (koopa env had 600 → run aborted mid-ladder); install starts oneshots with **--no-block**. | diff --git a/host-agent/README.md b/host-agent/README.md index cf8969c..dde3b1c 100644 --- a/host-agent/README.md +++ b/host-agent/README.md @@ -9,6 +9,7 @@ | Wall clock | `RUN_TIMEOUT=600` | | Log | line-buffered `run-*.log` (continuous write) | | HTML | always (first run / empty tree too) + commit URL; **bootstrap install stubs are monpages ERROR** | +| Publish | **per job page dirs only** (`HTML_OK_DIR` / `HTML_ERR_DIR` under each `MON_HOSTS` host) — **no full-tree `rsync --delete`** so shared www keeps sibling families (monitoring, max-ladder, surface, aptdeploy, …) | | Timeout notice | end of log + top jump on `/monitoring_err/` | **Rule:** public mon pages must show a **current suite tag/commit**, not install bootstrap or months-old trees. Timers exist so this keeps working without manual SSH. diff --git a/host-agent/ROOT-APPLY-MONITORING.md b/host-agent/ROOT-APPLY-MONITORING.md index 7d38b35..659a24b 100644 --- a/host-agent/ROOT-APPLY-MONITORING.md +++ b/host-agent/ROOT-APPLY-MONITORING.md @@ -35,11 +35,14 @@ merchant API as JSON **code 21**). That script: -1. `rsync` staging HTML → `/var/www/monitoring-sites/` (incl. surface + aptdeploy) +1. **Merge**-rsync staging HTML → `/var/www/monitoring-sites/` (no full-tree `--delete`; preserves sibling agent pages missing from staging) 2. Makes tree writable for `hernani` (group `caddy` + ACL) so later runs self-sync 3. Installs prepared Caddyfile → `/etc/caddy/Caddyfile`, validates, reloads Caddy 4. Smoke-curls all monitoring URLs (must **not** be JSON `code:21`) +**Per-job publish (v1.23.5+):** host-agent timers only update their own page dirs +(`HTML_OK_DIR` / `HTML_ERR_DIR`). They must not wipe other families under the same host. + ## Paths (only three families — suite name ≠ URL) There is **no** public page `/taler-monitoring`. Each family needs **bare and slash** @@ -60,7 +63,8 @@ HTML is generated as **hernani** into ## Manual equivalent ```bash -rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ +# merge only — no --delete (shared multi-agent tree) +rsync -a $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ chown -R hernani:caddy /var/www/monitoring-sites && chmod -R g+rwX /var/www/monitoring-sites sudo $KOOPA_CADDY_DIR/apply.sh # or apply-monitoring-live.sh ``` @@ -70,7 +74,7 @@ Expect **200** (HTML with sticky bar). Not merchant JSON `code:21`. ## After each monitoring run (if www is not hernani-writable) ```bash -rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ +rsync -a $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ chown -R caddy:caddy /var/www/monitoring-sites ``` diff --git a/host-agent/apply-monitoring-live.sh b/host-agent/apply-monitoring-live.sh index dfc4798..a8d723f 100755 --- a/host-agent/apply-monitoring-live.sh +++ b/host-agent/apply-monitoring-live.sh @@ -27,9 +27,12 @@ if [[ ! -d "$STAGING" ]]; then exit 1 fi -echo "== 1) HTML staging → www ==" +echo "== 1) HTML staging → www (merge, no full-tree --delete) ==" +# Shared www holds many agent families (monitoring, max-ladder, surface, …). +# Full-tree rsync --delete wiped sibling pages when staging was incomplete. +# Merge-update only: never remove dirs that staging does not list. mkdir -p "$WWW" -rsync -a --delete "$STAGING/" "$WWW/" +rsync -a --delay-updates "$STAGING/" "$WWW/" # allow hernani to update future runs without root chown -R hernani:caddy "$WWW" chmod -R g+rwX "$WWW" diff --git a/host-agent/run-host-report.sh b/host-agent/run-host-report.sh index 1f75caf..13df1ca 100755 --- a/host-agent/run-host-report.sh +++ b/host-agent/run-host-report.sh @@ -7,7 +7,7 @@ # 2) line-buffered run log (written continuously) # 3) RUN_TIMEOUT wall clock for taler-monitoring.sh (default 600s) # 4) always write HTML (first run / empty staging too) + Forgejo commit link -# 5) rsync to DEPLOY_WWW_ROOT (ERROR if not writable — v1.3.1) +# 5) publish this job's page dirs only to DEPLOY_WWW_ROOT (never full-tree --delete) # 6) monpages post-check: public FQDN must serve monitoring HTML # 7) exit code: STRICT_EXIT default 1 (v1.7.1) — unit fails if publish/public pages fail # @@ -454,16 +454,55 @@ for host in $MON_HOSTS; do echo "public URL (expected): https://${host}${HTML_URL_OK}" done -# Publish to live web root when possible (v1.3.1+: never silent-skip) +# Publish to live web root when possible (v1.3.1+: never silent-skip). +# Multi-agent shared tree (monitoring, max-ladder, surface, aptdeploy, …): +# NEVER rsync --delete the whole staging/www root — that wiped sibling pages. +# Publish only this job's HTML_OK_DIR / HTML_ERR_DIR under each MON_HOSTS entry. +publish_job_mon_pages() { + local host d src dst + local ok_dir="${HTML_OK_DIR:-monitoring}" + local err_dir="${HTML_ERR_DIR:-monitoring_err}" + for host in $MON_HOSTS; do + [ -n "$host" ] || continue + mkdir -p "$DEPLOY_WWW/$host" + for d in "$ok_dir" "$err_dir"; do + [ -n "$d" ] || continue + src="$HTML_BASE/$host/$d" + dst="$DEPLOY_WWW/$host/$d" + if [ -d "$src" ]; then + mkdir -p "$dst" + # --delete only within this page dir (stale files inside the page) + if ! rsync -a --delete --delay-updates "$src/" "$dst/"; then + echo "ERROR: rsync page failed: $src/ → $dst/" >&2 + return 1 + fi + echo " published $host/$d/" + else + # intentional absence (clean OK run removes _err) → drop only this page live + if [ -e "$dst" ]; then + rm -rf "$dst" + echo " removed $host/$d/ (absent in staging for this job)" + fi + fi + done + if [ -f "$HTML_BASE/$host/COMMIT" ]; then + cp -a "$HTML_BASE/$host/COMMIT" "$DEPLOY_WWW/$host/COMMIT" + fi + if [ -f "$HTML_BASE/$host/COMMIT_URL" ]; then + cp -a "$HTML_BASE/$host/COMMIT_URL" "$DEPLOY_WWW/$host/COMMIT_URL" + fi + done + return 0 +} + if [ -n "$DEPLOY_WWW" ]; then if [ -w "$DEPLOY_WWW" ] 2>/dev/null; then - # --delay-updates: write to temp names then rename (less partial-file exposure) - if rsync -a --delete --delay-updates "$HTML_BASE/" "$DEPLOY_WWW/"; then - echo "synced → $DEPLOY_WWW/ (delay-updates)" + if publish_job_mon_pages; then + echo "synced job pages → $DEPLOY_WWW/ (per-dir, no full-tree delete)" # brief settle so concurrent monpages curls do not hit mid-rename FS views sleep "${MONPAGES_PUBLISH_SETTLE_S:-0.3}" 2>/dev/null || sleep 1 else - echo "ERROR: rsync to DEPLOY_WWW_ROOT=$DEPLOY_WWW failed" >&2 + echo "ERROR: publish to DEPLOY_WWW_ROOT=$DEPLOY_WWW failed" >&2 ec=1 fi elif [ -e "$DEPLOY_WWW" ]; then diff --git a/site-gen/ROOT-ON-KOOPA.md b/site-gen/ROOT-ON-KOOPA.md index c0e4c1c..f3b8912 100644 --- a/site-gen/ROOT-ON-KOOPA.md +++ b/site-gen/ROOT-ON-KOOPA.md @@ -26,7 +26,7 @@ Caddyfile, reloads Caddy, and smokes mon URLs (including bare surface). ```bash install -d -o hernani -g caddy -m 755 /var/www/monitoring-sites -rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ +rsync -a $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/ chown -R hernani:caddy /var/www/monitoring-sites install -m 644 $KOOPA_CADDY_DIR/Caddyfile /etc/caddy/Caddyfile