diff --git a/README.md b/README.md index f035530..fd3674e 100644 --- a/README.md +++ b/README.md @@ -421,24 +421,7 @@ Soft mode only with `MONPAGES_REQUIRE_PUBLIC=0` (escape hatch). If pages are missing publicly: `sudo /home/hernani/koopa-caddy/apply-monitoring-live.sh` on koopa (FP: Infomaniak vhost for `/monitoring*`). -## Devtesting · fake-franken CHF (v1.13+) - -Phase **`devtesting`** (aliases: `franken`, `fake-franken`) exercises the -Taler Operations **rusty** host (`taler-devtesting` forced command): - -```bash -./taler-monitoring.sh -d stage.taler-ops.ch --currency CHF devtesting -``` - -Needs SSH to `devtesting@rusty.taler-ops.ch` (Host `taler-rusty-devtesting`). -Probes: CLI help, `geniban` (CH IBAN), `fake-incoming` CHF to exchange credit -IBAN. A nexus bounce *missing reserve public key* is **OK** for the plumbing -probe (no real wallet reserve required). - ## Version on monitoring pages (v1.3+ / sticky bar) -The sticky bar shows the suite **tag** (e.g. `v1.11.0`) and links to that tag on Forgejo. - -**v1.12.0:** Run header (**monitoring env context**: env_file, suite pin, domain/hosts/flags) is a collapsed `
` block above the console log — expand to inspect agent context. - -**v1.11.0:** Click the sticky **error** or **warning** count to filter the console log to only those lines (other log lines hide; sticky bar, stats, and top overviews stay). Same via `#filter-error` / `#filter-warn` (or `#first-error` / `#first-warn`). Click again or **Show all** to clear. +The sticky bar shows the suite **tag** (e.g. `v1.4.1`) and links to that tag on Forgejo. +Planned **v1.4.0**: fuller L10n (fr-CH for FrancPaysan, de-CH for hacktivism/Taler CH). diff --git a/TESTS.md b/TESTS.md index cd80328..7bde9f6 100644 --- a/TESTS.md +++ b/TESTS.md @@ -193,13 +193,3 @@ totals: 40 OK, 1 ERROR, 2 WARN, 5 INFO `NO_COLOR=1` or `CLICOLOR=0` disables boxes/colours (ASCII `[ OK ]` + `-- group --` headers). Always use `printf --` friendly plain headers when colour is off (leading `---` is not a printf option). - - -## devtesting · fake-franken (v1.13+) - -| ID prefix | What | -|-----------|------| -| **devtesting.cli-** | SSH to rusty · `taler-devtesting` CLI | -| **devtesting.geniban-** | synthetic CH IBAN | -| **devtesting.credit-payto-** | exchange credit IBAN for nexus | -| **devtesting.fake-incoming-** | CHF IN via nexus; bounce missing reserve pub = plumbing OK | diff --git a/VERSION b/VERSION index b50dd27..d90bb28 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.13.1 +1.10.11 diff --git a/VERSIONS.md b/VERSIONS.md index 7719546..5140f11 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -17,10 +17,6 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre | Tag | Date (UTC) | Notes | |-----|------------|--------| -| **v1.13.1** | 2026-07-19 | **Bugfix:** progress bar **global** done/total — phase `summary()` no longer snaps total to phase-local N (fixes 34/34 then 38/47); final `progress_finish` snaps once; header shows 0/N. | -| **v1.13.0** | 2026-07-19 | **Feature:** phase **devtesting** / **franken** — fake-franken CHF via `rusty.taler-ops.ch` (`taler-devtesting` geniban + fake-incoming); stage host-agent can include it. | -| **v1.12.0** | 2026-07-19 | **Feature:** collapsible **monitoring env context** (agent header: env_file, suite pin, domain/hosts/flags) — `
` default closed; summary shows agent · domain · suite. | -| **v1.11.0** | 2026-07-19 | **Feature:** sticky **error/warn filter** — click counts to show only error or only warn log lines; sticky bar + overviews stay; `#filter-error` / `#filter-warn` (+ `#first-*` still works). | | **v1.10.11** | 2026-07-19 | **Bugfix:** surface/aptdeploy wrappers **force** MON_HOSTS+PHASES+MONPAGES_INVENTORY (env cannot expand job to bank/exchange); wrap more mon knobs. | | **v1.10.10** | 2026-07-19 | **Bugfix:** monpages bare code 21 is WARN again by default (slash form remains ERROR); bare redir still needs live Caddy `redir *` — set MONPAGES_BARE_STRICT=1 to enforce. | | **v1.10.9** | 2026-07-19 | **Bugfix:** stats policy INFO no longer says bare “ERROR” (false #first-error on www.stats-01); extract_errors only from `┌ ERROR` badges. | diff --git a/check_devtesting.sh b/check_devtesting.sh deleted file mode 100755 index 575138c..0000000 --- a/check_devtesting.sh +++ /dev/null @@ -1,147 +0,0 @@ -#!/usr/bin/env bash -# check_devtesting.sh — fake-franken (CHF) via rusty.taler-ops.ch taler-devtesting -# -# Requires SSH to Host DEVTESTING_SSH (default: taler-rusty-devtesting → -# rusty.taler-ops.ch user devtesting, forced command taler-devtesting). -# -# Probes: -# 1) CLI reachable (fake-incoming --help) -# 2) geniban → Swiss IBAN -# 3) fake-incoming CHF amount to exchange credit IBAN (nexus IN) -# Bounce "missing reserve public key" = plumbing OK (no real wallet reserve) -# Full credit needs a real reserve pub as --subject (optional later) -# -# Env: -# DEVTESTING_SSH default taler-rusty-devtesting (or user@host) -# DEVTESTING_AMOUNT default CHF:1.00 -# DEVTESTING_CREDIT_PAYTO exchange IBAN payto (auto from EXCHANGE_PUBLIC/keys if empty) -# DEVTESTING_EXCHANGE_IBAN fallback CH6808573105529100001 (TOPS CHF exchange) -# DEVTESTING_SKIP=1 skip phase -# SKIP_SSH=1 skip (same as other SSH phases) -# -set -euo pipefail -ROOT=$(cd "$(dirname "$0")" && pwd) -# shellcheck source=lib.sh -source "$ROOT/lib.sh" - -set_area devtesting -section "devtesting · fake-franken CHF (rusty taler-devtesting)" - -# Do NOT honor SKIP_SSH here: stage/tops profiles set SKIP_SSH=1 for container -# inside checks, but rusty.taler-ops.ch is a separate SSH hop (devtesting key). -if [ "${DEVTESTING_SKIP:-0}" = "1" ]; then - info "skip" "DEVTESTING_SKIP=1" - exit 0 -fi - -: "${DEVTESTING_SSH:=taler-rusty-devtesting}" -: "${DEVTESTING_AMOUNT:=CHF:1.00}" -: "${DEVTESTING_EXCHANGE_IBAN:=CH6808573105529100001}" -: "${SSH_CONNECT_TIMEOUT:=8}" -: "${SSH_CMD_TIMEOUT:=45}" - -ssh_dt() { - # Prefer Host alias; fall back to explicit user@host if config missing - local target="$DEVTESTING_SSH" - if ! ssh -G "$target" >/dev/null 2>&1; then - target="devtesting@rusty.taler-ops.ch" - fi - if command -v timeout >/dev/null 2>&1; then - timeout "${SSH_CMD_TIMEOUT}" ssh \ - -o BatchMode=yes \ - -o ConnectTimeout="${SSH_CONNECT_TIMEOUT}" \ - -o StrictHostKeyChecking=accept-new \ - "$target" "$@" - else - ssh \ - -o BatchMode=yes \ - -o ConnectTimeout="${SSH_CONNECT_TIMEOUT}" \ - -o StrictHostKeyChecking=accept-new \ - "$target" "$@" - fi -} - -set_group cli -info "ssh" "DEVTESTING_SSH=${DEVTESTING_SSH} amount=${DEVTESTING_AMOUNT}" - -help_out=$(ssh_dt fake-incoming --help 2>&1) || true -if printf '%s' "$help_out" | grep -q 'fake-incoming'; then - ok "cli" "taler-devtesting fake-incoming reachable via ${DEVTESTING_SSH}" -else - fail "cli" "cannot run taler-devtesting on ${DEVTESTING_SSH}" \ - "$(printf '%s' "$help_out" | tr '\n' ' ' | head -c 200)" - exit 1 -fi - -set_group geniban -iban_out=$(ssh_dt geniban 2>&1) || true -iban=$(printf '%s\n' "$iban_out" | grep -E '^CH[0-9A-Z]+$' | tail -1 || true) -if [ -n "$iban" ]; then - ok "geniban" "synthetic IBAN $iban" -else - fail "geniban" "expected CH… IBAN" "$(printf '%s' "$iban_out" | tr '\n' ' ' | head -c 160)" - exit 1 -fi - -# Credit payto: rusty libeufin-nexus only accepts the *production* exchange IBAN -# configured there (DEVTESTING_EXCHANGE_IBAN). Do not use stage /keys first-account. -set_group credit-payto -credit_payto="${DEVTESTING_CREDIT_PAYTO:-}" -if [ -z "$credit_payto" ]; then - credit_payto="payto://iban/${DEVTESTING_EXCHANGE_IBAN}?receiver-name=Taler+Operations+AG" -fi -ok "credit-payto" "$credit_payto" - -# Subject: not a real reserve — nexus should bounce missing reserve public key -# (proves CHF fake-incoming path). Real reserve pub would credit wirewatch. -subject="mon-fake-franken-$(date -u +%Y%m%d%H%M%S)-$$" -debit_payto="payto://iban/${iban}?receiver-name=MonDevTest" - -set_group fake-incoming -info "fake-incoming" "amount=${DEVTESTING_AMOUNT} subject=${subject}" -fi_rc=0 -fi_out=$(ssh_dt fake-incoming \ - --amount "${DEVTESTING_AMOUNT}" \ - --subject "${subject}" \ - --credit-payto "${credit_payto}" \ - --debit-payto "${debit_payto}" 2>&1) || fi_rc=$? - -# Normalize multi-line for matching -fi_flat=$(printf '%s' "$fi_out" | tr '\n' ' ') - -if printf '%s' "$fi_flat" | grep -qiE 'Permission denied|Connection refused|Could not resolve|No such command'; then - fail "fake-incoming" "SSH/CLI failure" "$(printf '%s' "$fi_flat" | head -c 200)" - exit 1 -fi - -if printf '%s' "$fi_flat" | grep -qiE 'Creditor must be the exchange'; then - fail "fake-incoming" "wrong credit IBAN (not exchange)" "$(printf '%s' "$fi_flat" | head -c 220)" - exit 1 -fi - -if printf '%s' "$fi_flat" | grep -qiE 'Missing amount|unable to read secrets|CalledProcessError'; then - # secrets warn alone is soft if we still see IN line - if ! printf '%s' "$fi_flat" | grep -qE 'libeufin-nexus - IN .* CHF'; then - fail "fake-incoming" "nexus rejected request" "$(printf '%s' "$fi_flat" | head -c 220)" - exit 1 - fi -fi - -if printf '%s' "$fi_flat" | grep -qE 'libeufin-nexus - IN .* CHF:[0-9]'; then - if printf '%s' "$fi_flat" | grep -qiE 'bounced.*missing reserve public key'; then - ok "fake-incoming" "CHF IN accepted by nexus · bounced (no reserve pub) — fake-franken plumbing OK" - info "note" "use real reserve pub as --subject for wirewatch credit (not required for mon probe)" - elif printf '%s' "$fi_flat" | grep -qiE 'bounced'; then - warn "fake-incoming" "CHF IN bounced (other reason)" "$(printf '%s' "$fi_flat" | grep -oiE 'bounced[^ ]* [^ ]*' | head -c 160)" - else - ok "fake-incoming" "CHF IN recorded by nexus (no bounce line)" - fi -elif [ "$fi_rc" -eq 0 ] && printf '%s' "$fi_flat" | grep -qi 'Faking incoming CHF'; then - ok "fake-incoming" "CLI completed CHF fake-incoming" -else - fail "fake-incoming" "no CHF IN line from nexus" "rc=${fi_rc} $(printf '%s' "$fi_flat" | head -c 200)" - exit 1 -fi - -info "done" "devtesting fake-franken probe finished" -exit 0 diff --git a/host-agent/env/mytops-stage.env.example b/host-agent/env/mytops-stage.env.example deleted file mode 100644 index 5db18bf..0000000 --- a/host-agent/env/mytops-stage.env.example +++ /dev/null @@ -1,35 +0,0 @@ -# Copy to: ~/.config/taler-monitoring/env (user taler-monitoring @ betel) -# mytops CHF stage — NOT lifeline / production -# -SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git -SUITE_GIT_REF=main -SUITE_DIR=$HOME/src/taler-monitoring -TALER_DOMAIN=stage.taler-ops.ch -MON_HOSTS="stage.my.taler-ops.ch stage.taler-ops.ch" -HTML_OUT=$HOME/monitoring-sites-staging -DEPLOY_WWW_ROOT=/var/www/monitoring-sites -HTML_URL_OK=/monitoring/ -HTML_URL_ERR=/monitoring_err/ -HTML_OK_DIR=monitoring -HTML_ERR_DIR=monitoring_err -# urls + monpages + fake-franken CHF (rusty.taler-ops.ch) -PHASES="urls monpages devtesting" -SKIP_SSH=1 -LOCAL_STACK=0 -INSIDE_PODMAN=0 -CONTINUE_ON_ERROR=1 -RUN_TIMEOUT=600 -STRICT_EXIT=1 -MONPAGES_REQUIRE_PUBLIC=1 -MONPAGES_INVENTORY=job -TALER_MON_LANG=en -AGENT_LABEL=mytops-stage-host-agent -STATE_NAME=taler-monitoring-mytops-stage -SUITE_UPDATE_STRICT=1 -CHECK_BANK=0 -MERCHANT_REQUIRED=1 -CHECK_LANDING=0 -# rusty devtesting (SSH Host taler-rusty-devtesting → devtesting@rusty.taler-ops.ch) -DEVTESTING_SSH=taler-rusty-devtesting -DEVTESTING_AMOUNT=CHF:1.00 -# DEVTESTING_SKIP=1 diff --git a/i18n.sh b/i18n.sh index 5ca1d18..45dd1b7 100644 --- a/i18n.sh +++ b/i18n.sh @@ -55,9 +55,6 @@ i18n_text() { fi case "$s" in "numbered checks this run:") printf 'controles numerotes de cette execution :' ;; - "numbered checks:") printf 'controles numerotes :' ;; - "(global done/total)") printf '(fait/total global)' ;; - "global checks:") printf 'controles globaux :' ;; "failed — see list above"|"failed - see list above") printf 'echec — voir la liste ci-dessus' ;; "BLOCKERS · pay/withdraw cannot finish") printf 'BLOCAGES · paiement/retrait impossible' ;; "BLOCKERS (pay/withdraw cannot finish)") printf 'BLOCAGES (paiement/retrait impossible)' ;; diff --git a/lib.sh b/lib.sh index 92dc43b..7eb8e99 100755 --- a/lib.sh +++ b/lib.sh @@ -590,38 +590,20 @@ add_progress_total() { PROGRESS_TOTAL=$((PROGRESS_TOTAL + n)) _mon_state_save } -# When the global estimate was short, grow total just enough so done≤total. -# Never shrink. Mid-run headroom is small and monotonic (stable N in done/N). +# When estimate was short, grow total so we never print done>total (e.g. 224/205). +# Keeps ~10% headroom so the bar is not stuck at 100% while checks continue. _progress_rebalance() { [ "${PROGRESS_TOTAL:-0}" -gt 0 ] || return 0 [ "${PROGRESS_DONE:-0}" -gt 0 ] || return 0 if [ "$PROGRESS_DONE" -gt "$PROGRESS_TOTAL" ]; then - # + max(8, 5% of done) so we do not snap to 100% then jump again next check - local head=$((PROGRESS_DONE / 20)) - [ "$head" -lt 8 ] && head=8 + local head=$((PROGRESS_DONE / 10)) + [ "$head" -lt 12 ] && head=12 PROGRESS_TOTAL=$((PROGRESS_DONE + head)) - _mon_state_save - fi -} -# Snap total to actual done — only at whole-run end (not after each phase). -progress_finish() { - [ "${PROGRESS_OFF:-0}" = "1" ] && return 0 - [ "${PROGRESS_DONE:-0}" -gt 0 ] || return 0 - if [ "${PROGRESS_TOTAL:-0}" -ne "$PROGRESS_DONE" ]; then - PROGRESS_TOTAL=$PROGRESS_DONE - _mon_state_save - fi - _progress_bar_line "$PROGRESS_DONE" "$PROGRESS_TOTAL" - if [ "${GLOBAL_N:-0}" -gt 0 ]; then - printf -- '%s %s %d/%d (#001…#%03d)%s\n' \ - "$D" "$(i18n_text 'global checks:')" \ - "$PROGRESS_DONE" "$PROGRESS_TOTAL" "$GLOBAL_N" "$N" fi } _progress_bar_line() { - # Format (grows left → right; badge = percent): + # Format (grows left → right; badge = percent, not "PROG"): # ┌ 42% ┐ ████████░░░░░░░░░░░░░░░░ 31/74 - # done/total = global numbered checks so far / global expected total local done="$1" total="$2" width=28 pct=0 filled empty i bar pct_lab # Defensive: never display done > total if [ "$total" -gt 0 ] && [ "$done" -gt "$total" ]; then @@ -654,7 +636,7 @@ _progress_bar_line() { while [ "$i" -lt "$empty" ]; do bar="${bar}░"; i=$((i + 1)); done pct_lab=$(printf '%3d%%' "$pct") if [ "${BOX:-0}" = "1" ]; then - # badge = percent; bar grows ░→█ left to right; N/M = global check count + # badge = percent (clearer than "PROG"); bar grows ░→█ left to right printf '%s┌%s┐%s %s%s%s %s%d/%d%s\n' \ "$BG_PROG" "$pct_lab" "$N" \ "$C" "$bar" "$N" \ @@ -665,10 +647,10 @@ _progress_bar_line() { fi else if [ "${BOX:-0}" = "1" ]; then - printf '%s┌ … ┐%s %s%d/?%s (set PROGRESS_TOTAL= for global %%)\n' \ + printf '%s┌ … ┐%s %sdone=%d%s (set PROGRESS_TOTAL= for %%)\n' \ "$BG_PROG" "$N" "$D" "$done" "$N" else - printf -- '%s%d/?%s (set PROGRESS_TOTAL= for global %%)\n' "$D" "$done" "$N" + printf -- '%sdone=%d%s (set PROGRESS_TOTAL= for %%)\n' "$D" "$done" "$N" fi fi } @@ -829,29 +811,17 @@ section() { summary() { local blk_n=${#BLOCKERS[@]} echo "" - # Phase progress snapshot — keep GLOBAL total (do not snap N to phase-local done). - # Snapping here made mid-run bars like 34/34 then 38/47. Whole-run snap: progress_finish. - if [ "${PROGRESS_OFF:-0}" != "1" ] && [ "${PROGRESS_DONE:-0}" -gt 0 ]; then - if [ "${PROGRESS_FINAL:-0}" = "1" ]; then - # last phase / explicit final: done/total exact - if [ "$PROGRESS_TOTAL" -gt 0 ] && [ "$PROGRESS_DONE" -ne "$PROGRESS_TOTAL" ]; then - PROGRESS_TOTAL=$PROGRESS_DONE - _mon_state_save - fi + # final progress line (no _take_tid — plain printf) + if [ "${PROGRESS_OFF:-0}" != "1" ] && [ "$PROGRESS_DONE" -gt 0 ]; then + # Snap total to actual so last line is exact N/N 100% + if [ "$PROGRESS_TOTAL" -gt 0 ] && [ "$PROGRESS_DONE" -ne "$PROGRESS_TOTAL" ]; then + PROGRESS_TOTAL=$PROGRESS_DONE + _mon_state_save fi - _progress_rebalance _progress_bar_line "$PROGRESS_DONE" "$PROGRESS_TOTAL" fi if [ "$GLOBAL_N" -gt 0 ]; then - # Always show global position: current # / expected total (if known) - if [ "${PROGRESS_TOTAL:-0}" -gt 0 ]; then - printf -- '%s %s #001…#%03d · %d/%d %s%s\n' \ - "$D" "$(i18n_text 'numbered checks:')" "$GLOBAL_N" \ - "$PROGRESS_DONE" "$PROGRESS_TOTAL" \ - "$(i18n_text '(global done/total)')" "$N" - else - printf -- '%s %s #001…#%03d%s\n' "$D" "$(i18n_text 'numbered checks this run:')" "$GLOBAL_N" "$N" - fi + printf -- '%s %s #001…#%03d%s\n' "$D" "$(i18n_text 'numbered checks this run:')" "$GLOBAL_N" "$N" fi if [ "$blk_n" -gt 0 ]; then if [ "${BOX:-0}" = "1" ]; then diff --git a/secrets.env.example b/secrets.env.example index 9b3ce12..3ce731d 100644 --- a/secrets.env.example +++ b/secrets.env.example @@ -69,15 +69,6 @@ # AUTH401_ADMIN_USER=admin # FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets -# --- Devtesting / fake-franken CHF (rusty.taler-ops.ch) --- -# Phase: devtesting | franken | fake-franken -# SSH Host taler-rusty-devtesting (user devtesting, forced taler-devtesting CLI). -# DEVTESTING_SSH=taler-rusty-devtesting -# DEVTESTING_AMOUNT=CHF:1.00 -# DEVTESTING_CREDIT_PAYTO= # optional; auto from EXCHANGE_PUBLIC/keys or TOPS IBAN -# DEVTESTING_EXCHANGE_IBAN=CH6808573105529100001 -# DEVTESTING_SKIP=1 # skip phase - # --- Ladder (GOA withdraw ladder · explorer pool) --- # Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt). # Overrides only if needed: diff --git a/site-gen/caddy-monitoring-handles.snippet b/site-gen/caddy-monitoring-handles.snippet index 47b6725..f0b1a3a 100644 --- a/site-gen/caddy-monitoring-handles.snippet +++ b/site-gen/caddy-monitoring-handles.snippet @@ -1,27 +1,23 @@ # Host Caddy (root service). Place BEFORE catch-all reverse_proxy in each site block. # -# CRITICAL: bare paths (no trailing slash) must redirect to slash, or the request -# falls through to merchant/bank/exchange API → Taler JSON code 21. +# CRITICAL (v1.7.2): root must be the *host directory* (…/monitoring-sites/{host}), +# and the matcher must be a path *prefix* (`/monitoring*`), so that URI +# /monitoring/index.html maps to {root}/monitoring/index.html. # -# CORRECT (site-level named matcher — preferred): -# @mon_bare path /monitoring /monitoring_err \ -# /taler-monitoring-surface /taler-monitoring-surface_err \ -# /taler-monitoring-aptdeploy /taler-monitoring-aptdeploy_err -# redir @mon_bare {path}/ 302 +# WRONG (causes empty HTTP 404 from Caddy file_server): +# handle /monitoring/ { +# root * …/monitoring-sites/{host}/monitoring +# file_server +# } +# → looks for …/monitoring/monitoring/index.html # -# ALSO OK inside handle (matcher must be `*`): +# CRITICAL (v1.9.2): bare-path redir inside handle MUST use matcher `*`: # handle /taler-monitoring-surface { # redir * /taler-monitoring-surface/ 302 # } -# -# WRONG (Caddy footgun — causes merchant code 21 on bare URLs): -# redir /taler-monitoring-surface/ 302 -# → parsed as matcher=/taler-monitoring-surface/ to="302" -# → Location: 302, or no-op then fallthrough to reverse_proxy -# -# root must be the *host directory* (…/monitoring-sites/{host}), and the -# file_server matcher a path *prefix* (`/monitoring*`), so -# /monitoring/index.html maps to {root}/monitoring/index.html. +# WRONG: `redir /taler-monitoring-surface/ 302` is parsed as +# matcher=/taler-monitoring-surface/ to="302" → Location: 302, merchant code 21 +# (same footgun for /monitoring, aptdeploy, mail, mattermost bare paths). # # Disk layout: # /var/www/monitoring-sites/{host}/monitoring/index.html @@ -29,44 +25,49 @@ # # Live apply on koopa (absolute path — ~ as root is /root): # sudo /home/hernani/koopa-caddy/apply-monitoring-live.sh -# Runtime-only (no sudo; hernani admin API): -# curl -g -X POST http://[::1]:2019/load -H 'Content-Type: text/caddyfile' \ -# --data-binary @/home/hernani/koopa-caddy/Caddyfile # --- only inside taler.hacktivism.ch { ... } --- - @mon_bare path /monitoring /monitoring_err \ - /taler-monitoring-surface /taler-monitoring-surface_err \ - /taler-monitoring-aptdeploy /taler-monitoring-aptdeploy_err \ - /taler-monitoring-mail /taler-monitoring-mail_err \ - /taler-monitoring-mattermost /taler-monitoring-mattermost_err - redir @mon_bare {path}/ 302 - - handle /taler-monitoring-surface_err* { - root * /var/www/monitoring-sites/taler.hacktivism.ch - file_server + handle /taler-monitoring-surface { + redir * /taler-monitoring-surface/ 302 } handle /taler-monitoring-surface* { root * /var/www/monitoring-sites/taler.hacktivism.ch file_server } - handle /taler-monitoring-aptdeploy_err* { + handle /taler-monitoring-surface_err { + redir * /taler-monitoring-surface_err/ 302 + } + handle /taler-monitoring-surface_err* { root * /var/www/monitoring-sites/taler.hacktivism.ch file_server } + handle /taler-monitoring-aptdeploy { + redir * /taler-monitoring-aptdeploy/ 302 + } handle /taler-monitoring-aptdeploy* { root * /var/www/monitoring-sites/taler.hacktivism.ch file_server } + handle /taler-monitoring-aptdeploy_err { + redir * /taler-monitoring-aptdeploy_err/ 302 + } + handle /taler-monitoring-aptdeploy_err* { + root * /var/www/monitoring-sites/taler.hacktivism.ch + file_server + } # --- bank + exchange + taler (each site block; set root host dir) --- - @mon_bare path /monitoring /monitoring_err - redir @mon_bare {path}/ 302 - - handle /monitoring_err* { - root * /var/www/monitoring-sites/{host} - file_server + handle /monitoring { + redir * /monitoring/ 302 } handle /monitoring* { root * /var/www/monitoring-sites/{host} file_server } + handle /monitoring_err { + redir * /monitoring_err/ 302 + } + handle /monitoring_err* { + root * /var/www/monitoring-sites/{host} + file_server + } diff --git a/site-gen/console_to_html.py b/site-gen/console_to_html.py index 1860b4f..ba936ca 100755 --- a/site-gen/console_to_html.py +++ b/site-gen/console_to_html.py @@ -31,15 +31,6 @@ def ui(lang: str, key: str, **kwargs) -> str: "warn_many": "{n} warnings", "jump_first_error": "Jump to first error", "jump_first_warn": "Jump to first warning", - "filter_errors": "Filter: show all errors (hide other log lines)", - "filter_warns": "Filter: show all warnings (hide other log lines)", - "filter_clear": "Clear filter · show all lines", - "filter_active_err": "Filter: errors only", - "filter_active_warn": "Filter: warnings only", - "filter_show_all": "Show all", - "env_context": "Monitoring env context", - "env_context_title": "Expand: agent env, suite pin, domain hosts & flags (from run header)", - "env_context_empty": "No run header in log", "generated": "generated", "version": "version", @@ -92,7 +83,6 @@ def ui(lang: str, key: str, **kwargs) -> str: "stage_i3": "Phases: {phases} · suite auto-upgrade each timer run", "stage_i4": "Not GOA: no surface/aptdeploy/mail pages · no koopa-admin-log", "stage_i5": "Timer: taler-monitoring-mytops-stage.timer (4h) · user taler-monitoring", - "stage_i6": "devtesting: fake-franken CHF via rusty.taler-ops.ch (geniban + fake-incoming)", "pages_section_lead": "Monitoring pages themselves (suite):", "mm_title": "Mattermost chat health", "mm_summary": "mattermost.taler.net · SPA + /api/v4/system/ping · TLS", @@ -104,7 +94,7 @@ def ui(lang: str, key: str, **kwargs) -> str: "mail_i1": "Outside-in phase mail: MX, SMTP/IMAP ports and handshakes, SPF/DMARC", "mail_i2": "Mail hosts: firefly.gnunet.org (taler.net/gnunet.org) and anastasis.taler-systems.com", "mail_i3": "Reported on taler-monitoring-surface HTML (no separate /taler-monitoring-mail* page)", - "footer": "Console-style render of taler-monitoring output. Sticky bar: green = clean · yellow = warnings · red = errors. Click error/warning counts to filter the log. Monitoring env context (agent header) is collapsed by default. Commit pins the exact tree used for this run.", + "footer": "Console-style render of taler-monitoring output. Sticky bar: green = clean · yellow = warnings · red = errors. Commit pins the exact tree used for this run.", "redirect_fail": "has failures.", "redirect_see": "See {link} for the full console log, error index, and sticky status bar.", "ago_s": "{n}s ago", @@ -123,15 +113,6 @@ def ui(lang: str, key: str, **kwargs) -> str: "warn_many": "{n} avertissements", "jump_first_error": "Aller à la première erreur", "jump_first_warn": "Aller au premier avertissement", - "filter_errors": "Filtrer : toutes les erreurs (masquer le reste du journal)", - "filter_warns": "Filtrer : tous les avertissements (masquer le reste du journal)", - "filter_clear": "Effacer le filtre · tout afficher", - "filter_active_err": "Filtre : erreurs seulement", - "filter_active_warn": "Filtre : avertissements seulement", - "filter_show_all": "Tout afficher", - "env_context": "Contexte d'env. de monitoring", - "env_context_title": "Déplier : env agent, pin de suite, hôtes domaine et flags (en-tête d'exécution)", - "env_context_empty": "Pas d'en-tête d'exécution dans le journal", "generated": "généré", "version": "version", @@ -183,7 +164,6 @@ def ui(lang: str, key: str, **kwargs) -> str: "stage_i3": "Phases : {phases} · auto-upgrade suite à chaque minuterie", "stage_i4": "Pas GOA : pas de pages surface/aptdeploy/mail · pas de koopa-admin-log", "stage_i5": "Minuterie : taler-monitoring-mytops-stage.timer (4 h) · user taler-monitoring", - "stage_i6": "devtesting : fake-franken CHF via rusty.taler-ops.ch (geniban + fake-incoming)", "pages_i6": "Hôte de cette page : {host} · libellé : {label}", "pages_section_lead": "Les pages de monitoring elles-mêmes (suite) :", "mm_title": "Santé du chat Mattermost", @@ -196,7 +176,7 @@ def ui(lang: str, key: str, **kwargs) -> str: "mail_i1": "Phase outside-in mail : MX, ports SMTP/IMAP et handshakes, SPF/DMARC", "mail_i2": "Hôtes mail : firefly.gnunet.org (taler.net/gnunet.org) et anastasis.taler-systems.com", "mail_i3": "Rapporté dans le HTML taler-monitoring-surface (pas de page /taler-monitoring-mail*)", - "footer": "Rendu console de taler-monitoring. Barre collante : vert = OK · jaune = avertissements · rouge = erreurs. Compteurs = filtre. Contexte d'env. (en-tête agent) replié par défaut. Le commit fixe l'arbre exact de cette exécution.", +"footer": "Rendu console de taler-monitoring. Barre collante : vert = OK · jaune = avertissements · rouge = erreurs. Le commit fixe l'arbre exact de cette exécution.", "redirect_fail": "a des échecs.", "redirect_see": "Voir {link} pour le journal console complet, l'index d'erreurs et la barre de statut.", "ago_s": "il y a {n}s", @@ -614,7 +594,6 @@ def monitoring_scope( ui(lang, "stage_i3", phases=phase_txt), ui(lang, "stage_i4"), ui(lang, "stage_i5"), - ui(lang, "stage_i6"), ui(lang, "pages_i6", host=host, label=label), ] return { @@ -658,172 +637,6 @@ def monitoring_scope( } -def is_env_context_start(ln: str) -> bool: - """Host-agent / suite banner that opens the monitoring env header.""" - s = ln.strip() - if re.match(r"^={4,}\s*.+\s*={4,}$", s): - return True - if s.startswith("env_file="): - return True - return False - - -def is_env_context_line(ln: str) -> bool: - """ - Lines belonging to the pre-check monitoring env header - (agent banner, suite pin, target domain / flags / timeout). - """ - s = ln.rstrip("\n") - if not s.strip(): - return True # blank lines inside header block - t = s.strip() - if is_env_context_start(s): - return True - # agent header keys - if t.startswith( - ( - "env_file=", - "domain=", - "hosts=", - "flags ", - "suite ", - "using suite", - "commit=", - "target domain=", - "note:", - "WARN: Forgejo", - "WARN suite-update", - ) - ): - return True - if t.startswith(" dir=") or t.startswith("dir="): - return True - # indented target-domain block from taler-monitoring.sh - if re.match( - r"^\s+(bank|exchange|merchant|currency|phases|flags|access|run_timeout|progress)\b", - s, - ): - return True - # re-applying overlay chatter still part of startup - if "suite-overlay" in t.lower() and ( - t.startswith("WARN") or t.startswith(" ") or "re-applying" in t.lower() - ): - return True - return False - - -def is_env_context_end(ln: str) -> bool: - """First line that is clearly past the env header (phases / checks start).""" - s = ln.strip() - if not s: - return False - # box headers / section titles for real check phases - if s.startswith("╔") or s.startswith("║") or s.startswith("╚"): - return True - if s.startswith("┌") or s.startswith("└") or s.startswith("["): - return True - if re.match(r"^==\s+", s) or re.match(r"^--\s+", s): - return True - if "numbered checks" in s.lower(): - return True - if re.match(r"^\s*\d+%\s", s): # bare progress - return True - return False - - -def extract_env_context(lines: list[str]) -> tuple[list[str], list[str], str]: - """ - Split leading monitoring env header from the rest of the log. - Returns (env_lines, body_lines, hint_summary). - """ - if not lines: - return [], [], "" - # Find start: first env-context start within the first ~40 lines - start = None - for i, ln in enumerate(lines[:40]): - if is_env_context_start(ln): - start = i - break - if start is None: - # no banner — maybe only "target domain=" block at top (CLI without agent) - for i, ln in enumerate(lines[:15]): - if ln.strip().startswith("target domain=") or ln.strip().startswith( - "env_file=" - ): - start = i - break - if start is None: - return [], lines, "" - - end = start - for j in range(start, len(lines)): - ln = lines[j] - if j > start and is_env_context_end(ln): - end = j - break - if j > start and not is_env_context_line(ln) and ln.strip(): - # non-empty line that doesn't look like env → end before it - end = j - break - end = j + 1 - else: - end = len(lines) - - env = lines[start:end] - # trim trailing blanks from env - while env and not env[-1].strip(): - env.pop() - body = lines[:start] + lines[end:] - # hint: suite + domain + phases - hint_parts: list[str] = [] - for ln in env: - t = ln.strip() - if t.startswith("suite "): - hint_parts.append(t.split(" ")[0][:48]) - elif t.startswith("domain="): - hint_parts.append(t.replace("domain=", "").split("·")[0].strip()[:40]) - elif t.startswith("target domain="): - if not any("domain" in h for h in hint_parts): - hint_parts.append(t.split("=", 1)[1].strip()[:40]) - elif re.match(r"^={4,}", t): - # agent label between equals - m = re.match(r"^={4,}\s*(.+?)\s*={4,}$", t) - if m: - lab = m.group(1) - # drop timestamp prefix if present - lab2 = re.sub(r"^\d{4}-\d{2}-\d{2}T\S+\s+", "", lab) - hint_parts.insert(0, lab2[:40]) - hint = " · ".join(hint_parts[:3]) if hint_parts else f"{len(env)} lines" - return env, body, hint - - -def env_context_html( - env_lines: list[str], - *, - tid_to_slug: dict[str, str], - lang: str, - hint: str, -) -> str: - """Collapsible monitoring env context (default closed).""" - if not env_lines: - return "" - body = "".join(render_line(ln, tid_to_slug, None) for ln in env_lines) - label = html.escape(ui(lang, "env_context")) - title = html.escape(ui(lang, "env_context_title")) - hint_esc = html.escape(hint or "") - # native
: collapsed by default (no open attr) - return f"""
- - - {label} - {hint_esc} - -
-{body}
-
-""" - - def is_log_noise_line(ln: str) -> bool: """Drop git/update chatter that must not appear in public mon HTML.""" s = ln.strip() @@ -888,50 +701,26 @@ def sticky_bar_html( else: status_txt = ui(lang, "ok") - # Click error/warn stats → filter console to matching lines only (v1.11.0). - # Hash #filter-error / #filter-warn; #first-error / #first-warn also activate filter. - if n_err: + if first_err_href and n_err: err_lbl = ui(lang, "error_one", n=n_err) if n_err == 1 else ui(lang, "error_many", n=n_err) - err_href = first_err_href or "#filter-error" - # Prefer filter hash; keep first-error as secondary scroll target via JS err_stat = ( - f'{html.escape(err_lbl)}' + f'{html.escape(err_lbl)}' ) else: err_lbl = ui(lang, "error_many", n=n_err) err_stat = f'{html.escape(err_lbl)}' - if n_warn: + if first_warn_href and n_warn: warn_lbl = ui(lang, "warn_one", n=n_warn) if n_warn == 1 else ui(lang, "warn_many", n=n_warn) - warn_href = first_warn_href or "#filter-warn" warn_stat = ( - f'{html.escape(warn_lbl)}' + f'{html.escape(warn_lbl)}' ) else: warn_lbl = ui(lang, "warn_many", n=n_warn) warn_stat = f'{html.escape(warn_lbl)}' - filter_chip = ( - f'" - ) - # data attributes for JS i18n - filter_i18n = ( - f'data-i18n-filter-err="{html.escape(ui(lang, "filter_active_err"))}" ' - f'data-i18n-filter-warn="{html.escape(ui(lang, "filter_active_warn"))}" ' - f'data-i18n-filter-clear="{html.escape(ui(lang, "filter_clear"))}"' - ) - other = "" if link_other: other = ( @@ -969,7 +758,7 @@ def sticky_bar_html( return f"""