hernani/taler-monitoring
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

release 1.9.0: monpages top/bottom content markers + surface required

Browse source 
Require meaningful monitoring HTML: sticky-bar/generated at top, footer
or taler-mon:bottom at end; ERROR if body is progress-only. Mid-run
(incomplete meta or progress without complete foot) relaxes bottom only.
GOA always checks taler-monitoring-surface. Generator emits taler-mon
markers and taler-mon-page complete|incomplete meta.
This commit is contained in:
Hernâni Marques 2026-07-19 00:32:35 +02:00
parent e7a3cd558f
commit 757090cbff
No known key found for this signature in database
GPG key ID: CB5738652768F7E9
6 changed files with 163 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -29,7 +29,7 @@ There are **no** first-class public pages for `/taler-monitoring-mail*`, `/taler
Related:
- **Phase `monpages`:** obligatory ERROR if expected pages are missing (not merchant JSON code 21).
- **Phase `monpages`:** obligatory **ERROR** if a page is missing, not suite HTML, or **content markers fail** (v1.9.0): top (`sticky-bar` / `taler-mon:top` / generated) and bottom (`footer` / `taler-mon:bottom:complete`). Mid-run pages (progress <100% or `incomplete` meta) may skip bottom ERROR. Body must contain real checks — progress bars alone → ERROR. On GOA, `/taler-monitoring-surface/` is always required.
- **Publish:** staging `~/monitoring-sites-staging` → `DEPLOY_WWW_ROOT` + reverse-proxy handles.
```bash
@ -403,11 +403,11 @@ FrancPaysan stacks auto-select **French** (`*lefrancpaysan*`). See `VERSIONS.md`
Phase **`monpages`** is **obligatory**: missing or non-suite HTML is **ERROR**.
Soft mode only with `MONPAGES_REQUIRE_PUBLIC=0` (escape hatch).
| Stack | What monpages expects (v1.8.0) |
| Stack | What monpages expects (v1.9.0) |
|-------|--------------------------------|
| **GOA** | `bank`/`exchange`/`taler` `/monitoring/` + `taler.hacktivism.ch/taler-monitoring-surface/` |
| **FrancPaysan** | That stack’s three mon hosts `/monitoring/` only |
| **Nine landings (model)** | Each of the 9 landing fronts has `/monitoring` + `/monitoring_err` (when a failed run exists) |
| **GOA** | 3× `/monitoring/` + **surface** (existence **and** top/bottom/content markers) |
| **FrancPaysan** | Stack mon hosts `/monitoring/` with same content rules |
| **Nine landings (model)** | Each landing front: `/monitoring` + `/monitoring_err` (err when a failed run exists) |
```bash
./taler-monitoring.sh -d hacktivism.ch monpages

2
TESTS.md
View file

@ -20,7 +20,7 @@ Every check line has a **global** run number and a **grouped** id:
| **versions** | `check_versions.sh` | `outside` `inside` `compare` |
| **aptdeploy** | `check_apt_deploy.sh` | `trixie` `trixie-testing` (koopa podman apt-src merchant smoke) |
| **surface** | `check_surface.sh` | **remote-only** ecosystem / `-d` domain inventory (not in default/all/full) |
| **monpages** | `check_monitoring_pages.sh` | public HTML via FQDN — **obligatory ERROR**; GOA: 3×`/monitoring/` + surface; FP: stack mon only (v1.8.0) |
| **monpages** | `check_monitoring_pages.sh` | public HTML via FQDN — **ERROR** if missing or markers fail (top+bottom+body); surface required on GOA (v1.9.0) |
| **surface** | `check_surface.sh` | ecosystem inventory + **nmap** OS fingerprint; mail/mattermost content reported via surface page |
| **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
| **server** | `check_server.sh` | (flat `server-NN` or host groups) |

2
VERSION
View file

@ -1 +1 @@
1.8.0
1.9.0

3
VERSIONS.md
View file

@ -17,10 +17,13 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre
| Tag | Date (UTC) | Notes |
|-----|------------|--------|
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
| **v1.8.0** | 2026-07-19 | **Simplified public pages:** only `/taler-monitoring-surface(+_err)` on `taler.hacktivism.ch` for ecosystem software/versions; **9 landing stacks** each keep `/monitoring(+_err)`. Mail (firefly.gnunet.org + anastasis.taler-systems.com) and Mattermost folded into surface job; separate mail/mattermost mon pages + timers deprecated. **nmap** OS fingerprinting on surface (`SURFACE_NMAP`); **ERROR** when packages are behind (`TALER_PKG_BEHIND=error` default) or OS fingerprint is EOL. |
| **v1.8.1** | *planned* | **L10n** packs: **fr-CH** (FrancPaysan) and **de-CH** (hacktivism / Taler CH German). |
| **v1.7.7** | 2026-07-18 | **domains:** stage mytops merchant FQDN is `stage.my.taler-ops.ch` (betel mon sites). |
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
| **v1.8.0** | 2026-07-18 | **monpages obligatory (ERROR):** GOA checks full suite inventory (bank/exchange/taler `/monitoring/` + surface/aptdeploy/mattermost/mail + on-disk discovery); FP only its own mon hosts; `MONPAGES_REQUIRE_PUBLIC=1` default (escape hatch `=0`). **Bugfix:** `with_timeout` no longer re-runs a phase when it exits non-zero. |
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
| **v1.8.0** | 2026-07-18 | **Bugfix:** monpages soft mode `MONPAGES_REQUIRE_PUBLIC=0` (FP wrappers default) — missing public HTML is WARN until vhost publish; generic publish hints. |
| **v1.7.4** | 2026-07-18 | **Bugfix:** monpages bare URLs (`/monitoring` without `/`) WARN by default (trailing slash remains ERROR); avoids false red when slash form is live. |
| **v1.7.3** | 2026-07-18 | Monitoring **pages themselves** as suite scope: README section + sticky “What this monitors” block (monpages / public HTML paths). |

152
check_monitoring_pages.sh
View file

@ -192,11 +192,15 @@ build_urls() {
case "$inv" in
job)
_job_urls
# v1.9.0: normal GOA runs always require surface page (content-checked)
if [ "$fam" = "goa" ]; then
printf 'https://taler.hacktivism.ch/taler-monitoring-surface/\n'
fi
;;
full|auto|*)
# Always include this job’s paths (host-agent post-check).
_job_urls
# Stack inventory: GOA = all suite mon sites; FP = only FP mon hosts.
# Stack inventory: GOA = landings + surface; FP = only FP mon hosts.
if [ "$fam" = "goa" ] || [ "$fam" = "fp" ]; then
_catalog_urls "$fam"
# On-disk discovery (err pages only exist after failed runs; extras welcome)
@ -225,7 +229,7 @@ is_monitoring_html() {
&& grep -qiE '404|not found|page introuvable' "$f" 2>/dev/null; then
return 1
fi
if grep -qE 'sticky-bar|version-link|taler-monitoring|class="sticky' "$f" 2>/dev/null; then
if grep -qE 'sticky-bar|version-link|taler-monitoring|class="sticky|taler-mon:top' "$f" 2>/dev/null; then
return 0
fi
if head -c 4096 "$f" | grep -qiE '<!DOCTYPE html|<html' \
@ -238,6 +242,113 @@ is_monitoring_html() {
return 1
}
# Highest progress percent in page body (0 if none). Pattern: ┌ 42%┐ or ┌100%┐
_progress_max_pct() {
local f="$1"
local m
m=$(grep -oE '┌[[:space:]]*[0-9]+%┐' "$f" 2>/dev/null \
| grep -oE '[0-9]+' | sort -n | tail -1 || true)
printf '%s' "${m:-0}"
}
# Mid-run: incomplete meta / redirect stub, or progress never reached 100% and no complete foot.
_page_mid_run() {
local f="$1"
local pct
if grep -qE 'name="taler-mon-page"[[:space:]]+content="incomplete"' "$f" 2>/dev/null \
|| grep -q 'taler-mon:bottom:incomplete' "$f" 2>/dev/null \
|| grep -qE 'http-equiv="refresh".*monitoring_err|mode-redirect' "$f" 2>/dev/null; then
return 0
fi
# finished page always has complete foot or classic footer
if grep -q 'taler-mon:bottom:complete' "$f" 2>/dev/null \
|| grep -qE 'id="mon-footer"' "$f" 2>/dev/null; then
return 1
fi
pct=$(_progress_max_pct "$f")
# max progress seen < 100% and no complete footer → still running
if [ "${pct:-0}" -gt 0 ] && [ "${pct:-0}" -lt 100 ]; then
if ! grep -qE 'Console-style render of taler-monitoring|Rendu console de taler-monitoring' "$f" 2>/dev/null; then
return 0
fi
fi
return 1
}
# Validate top/bottom markers + meaningful body (v1.9.0).
# Sets _MON_CONTENT_MSG; returns 0=ok, 1=hard fail, 2=soft/mid-run bottom skip
validate_mon_content() {
local f="$1" url="$2"
local top_ok=1 bot_ok=1 body_ok=1 mid=0 pct
_MON_CONTENT_MSG=""
# --- top markers (always required) ---
if ! head -c 12000 "$f" | grep -qE 'sticky-bar|id="status-bar"|taler-mon:top'; then
top_ok=0
_MON_CONTENT_MSG="missing top marker (sticky-bar / status-bar / taler-mon:top)"
elif ! head -c 16000 "$f" | grep -qE 'data-generated-iso|name="generated"|version-link|taler-mon-page'; then
top_ok=0
_MON_CONTENT_MSG="missing top meta (generated / version-link / taler-mon-page)"
fi
# --- meaningful body: real checks, not only progress bars ---
if grep -qE 'class="line (ok|error|warn|blocker|info)"|┌ OK|┌ ERROR|┌ WARN|┌ BLOCKER|┌ INFO|class="box"' "$f" 2>/dev/null; then
body_ok=1
elif grep -qE 'taler-mon:bottom:incomplete|mode-redirect|has failures' "$f" 2>/dev/null; then
body_ok=1 # redirect stub still has sticky + box
else
# only progress / empty?
if grep -qE '┌[[:space:]]*[0-9]+%┐|█|░' "$f" 2>/dev/null \
&& ! grep -qE '┌ OK|┌ ERROR|class="line ok"' "$f" 2>/dev/null; then
body_ok=0
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }only progress bars / no check lines"
elif ! grep -qE 'console|host-agent|taler-monitoring|monitoring' "$f" 2>/dev/null; then
body_ok=0
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }no meaningful suite content"
fi
fi
# --- bottom markers ---
if grep -q 'taler-mon:bottom:complete' "$f" 2>/dev/null \
|| tail -c 8000 "$f" | grep -qE 'id="mon-footer"|</footer>|Console-style render of taler-monitoring'; then
bot_ok=1
elif grep -q 'taler-mon:bottom:incomplete' "$f" 2>/dev/null; then
bot_ok=1 # incomplete is an explicit bottom marker
mid=1
else
bot_ok=0
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }missing bottom marker (footer / taler-mon:bottom)"
fi
if _page_mid_run "$f"; then
mid=1
fi
pct=$(_progress_max_pct "$f")
if [ "$top_ok" != "1" ]; then
_MON_CONTENT_MSG="TOP: $_MON_CONTENT_MSG"
return 1
fi
if [ "$body_ok" != "1" ]; then
_MON_CONTENT_MSG="BODY: $_MON_CONTENT_MSG"
return 1
fi
if [ "$bot_ok" != "1" ]; then
if [ "$mid" = "1" ]; then
_MON_CONTENT_MSG="BOTTOM soft (mid-run progress ${pct}%): $_MON_CONTENT_MSG"
return 2
fi
_MON_CONTENT_MSG="BOTTOM: $_MON_CONTENT_MSG"
return 1
fi
if [ "$mid" = "1" ] && [ "${pct:-0}" -gt 0 ] && [ "${pct:-0}" -lt 100 ]; then
_MON_CONTENT_MSG="mid-run progress ${pct}% (bottom markers relaxed)"
return 2
fi
_MON_CONTENT_MSG="top+body+bottom markers OK"
return 0
}
_is_bare_url() {
local url="$1"
case " ${MONPAGES_BARE_URLS:-} " in
@ -261,7 +372,7 @@ _mon_fail_or_soft() {
check_one() {
local url="$1"
local body code hint soft=0
local body code hint soft=0 vc
body=$(mktemp)
code=$(curl -skS -L --max-redirs 5 -m "${TIMEOUT}" -o "$body" -w '%{http_code}' "$url" 2>/dev/null || echo 000)
if _is_bare_url "$url" && [ "${MONPAGES_BARE_STRICT:-0}" != "1" ]; then
@ -290,17 +401,36 @@ check_one() {
rm -f "$body"
return 0
fi
if is_monitoring_html "$body"; then
if grep -qE 'sticky-bar|version-link' "$body" 2>/dev/null; then
ok "public mon page" "$url -> HTTP 200 monitoring HTML"
else
warn "public mon page" "$url -> HTTP 200 HTML but weak markers (stub/bootstrap?)"
fi
if ! is_monitoring_html "$body"; then
hint=$(head -c 120 "$body" | tr '\n' ' ' | tr -cd '[:print:] ')
_mon_fail_or_soft "public mon page not monitoring HTML" \
"$url -> HTTP 200 body!=suite (${hint}...)" || { rm -f "$body"; return 1; }
rm -f "$body"
return 0
fi
hint=$(head -c 120 "$body" | tr '\n' ' ' | tr -cd '[:print:] ')
_mon_fail_or_soft "public mon page not monitoring HTML" "$url -> HTTP 200 body!=suite (${hint}...)" || { rm -f "$body"; return 1; }
# v1.9.0: top + bottom markers + meaningful content
set +e
validate_mon_content "$body" "$url"
vc=$?
set -e
case "$vc" in
0)
ok "public mon page" "$url -> HTTP 200 · ${_MON_CONTENT_MSG}"
;;
2)
# mid-run: bottom may be incomplete — not ERROR
warn "public mon page mid-run" "$url -> ${_MON_CONTENT_MSG}"
ok "public mon page" "$url -> HTTP 200 suite HTML (mid-run bottom relaxed)"
;;
*)
if [ "$soft" = "1" ]; then
warn "public mon page content" "$url -> ${_MON_CONTENT_MSG}"
else
_mon_fail_or_soft "public mon page content" \
"$url -> ${_MON_CONTENT_MSG}" || { rm -f "$body"; return 1; }
fi
;;
esac
rm -f "$body"
return 0
}

14
site-gen/console_to_html.py
View file

@ -971,12 +971,15 @@ def build_html(
suite_version_url=suite_version_url,
)
kind = "surface" if "surface" in (page_label or "") else "monitoring"
return f"""<!DOCTYPE html>
<html lang="{html.escape(lang if "lang" in dir() else "en")}">
<head>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="generated" content="{html.escape(generated_iso)}"/>
<meta name="taler-mon-page" content="complete"/>
<meta name="taler-mon-kind" content="{html.escape(kind)}"/>
<title>{html.escape(title)}</title>
<style>
:root {{
@ -1042,16 +1045,18 @@ footer {{
</style>
</head>
<body>
<!-- taler-mon:top -->
{bar}
<main>
{err_nav}
<div class="console">
<div class="console" id="mon-console">
{"".join(body_lines)}
</div>
<footer>
<footer id="mon-footer">
{html.escape(footer_txt)}
</footer>
</main>
<!-- taler-mon:bottom:complete -->
{STICKY_JS}
</body>
</html>
@ -1105,6 +1110,7 @@ def build_redirect_html(
suite_version=suite_version,
suite_version_url=suite_version_url,
)
kind = "surface" if "surface" in (page_label or "") else "monitoring"
return f"""<!DOCTYPE html>
<html lang="{html.escape(lang if "lang" in dir() else "en")}">
<head>
@ -1112,6 +1118,8 @@ def build_redirect_html(
<meta http-equiv="refresh" content="2; url={html.escape(link)}"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="generated" content="{html.escape(generated_iso)}"/>
<meta name="taler-mon-page" content="incomplete"/>
<meta name="taler-mon-kind" content="{html.escape(kind)}"/>
<title>{html.escape(page_label)} · errors · {html.escape(hostname)}</title>
<style>
:root {{
@ -1129,6 +1137,7 @@ body {{ margin: 0; background: var(--bg); color: var(--fg); font-size: 13px; }}
</style>
</head>
<body>
<!-- taler-mon:top -->
{bar}
<div class="box">
<p><strong>{html.escape(page_label)}</strong> has failures.</p>
@ -1136,6 +1145,7 @@ body {{ margin: 0; background: var(--bg); color: var(--fg); font-size: 13px; }}
error index, and sticky status bar.</p>
<p style="color:#666;font-size:12px">generated {html.escape(generated_at)} · source {c}</p>
</div>
<!-- taler-mon:bottom:incomplete -->
{STICKY_JS}
</body>
</html>