Compare commits
No commits in common. "v1.2.0" and "main" have entirely different histories.
74 changed files with 8777 additions and 2447 deletions
|
|
@ -5,9 +5,11 @@ stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring
|
|||
e2e/ladder). Ordered roughly by **how often they hurt automation** and how
|
||||
useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be.
|
||||
|
||||
Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md).
|
||||
Companions:
|
||||
- Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md)
|
||||
- Git / suite tree → [`VERSIONS.md`](./VERSIONS.md)
|
||||
|
||||
Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`.
|
||||
Scripts: `check_e2e.sh`, `check_amount_ladder.sh`, `taler-monitoring.sh`.
|
||||
|
||||
Legend:
|
||||
|
||||
|
|
@ -45,15 +47,82 @@ Legend:
|
|||
|
||||
---
|
||||
|
||||
## 3. `reserve_pub` / force-select is fragile (5114)
|
||||
## 3. `reserve_pub` / force-select is fragile (5114 + empty scrape)
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | ladder force-select, e2e confirm path, Android spinner issues |
|
||||
| **Area** | **cli** / **core** / **bank** |
|
||||
| **Problem** | Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). Automation scrapes accept/tx dumps for candidates. |
|
||||
| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. |
|
||||
| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. |
|
||||
| **Seen** | ladder force-select, e2e confirm path, Android spinner issues; **repro 2026-07-19 magikoopa** GOA ladder: `force-select skipped · no reserve_pub for this withdraw (WID=…); wallet may not have selected yet` |
|
||||
| **Area** | **cli** / **core** / **bank** / **ops** |
|
||||
| **Problem** | (a) Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). (b) Automation scrapes accept/tx dumps for candidates — when the dump has **no** pub at all, ladder WARNs and skips force-select (soft), then often **SKIP_CONFIRM**. (c) Empty scrape is **not** the same as “bank out of money”; do not treat as balance failure. (d) **Mon bug (fixed v1.20.2):** extract used `json.loads(file_from_first_brace)` on wallet dumps that append log lines after JSON → **zero** objects parsed; regex only scanned accept (which has no `reservePub`) while the pub lives under `transactions[].withdrawalDetails.reservePub`. |
|
||||
| **Workaround** | **Mon ≥1.20.2:** `JSONDecoder.raw_decode` + regex on accept **and** transactions; score by WID in `bankConfirmationUrl` / amount. Still: score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. Refresh `transactions` before each force-select try. Soft-skip confirm if still pending after polls. Fix wallet DB / helper first if accept/tx only print FATAL (see §15–16). |
|
||||
| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. Optional: bank integration already exposes `selected_reserve_pub` when wallet selected — prefer that over scraping wallet logs. |
|
||||
|
||||
### 3a. Ladder pin amounts hide mid-range behaviour
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | `LADDER_STEPS=3` plan collapses to pins only: `GOA:0`, `max-1`, `max` (no random mid rungs) |
|
||||
| **Area** | **ops** / **bank** / **cli** |
|
||||
| **Problem** | **Zero** withdraw often never produces a usable `reserve_pub` / coins (wallet 7006 or no denoms). **Absolute max** and sometimes **max-1** hit **mint HTTP 500 / code 5110** SQL `P0001` (ceiling probe). Reproducing “real” force-select needs a **mid** amount (e.g. `GOA:1` … exchange min denom scale), not only pins. |
|
||||
| **Workaround** | Use `LADDER_STEPS≥5` (or higher) so random mids exist; treat zero / absolute-max / max-1-5110 as probes (soft `CEILING_REJECT` in mon ≥1.20.2). |
|
||||
| **Wanted** | Ladder plan docs: pin vs mid semantics; CLI structured error for amount-too-large / zero. |
|
||||
|
||||
### 3b. We do **not** manage to withdraw **in time** (GOA) — bank path unreliable, not load
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Bottom line** | On live **hacktivism GOA**, automation **does not get spendable coins in time** for the pay phase (default wait **90 s**). Max-search may report a “best” amount and bank may even show `transfer_done`; pay still sees **`available=GOA:0`** with large **`pendingIncoming`**. Coins might arrive **later** (wirewatch / wallet) — **at least not within mon’s budget**. Recurring (several runs, 2026-07-19). |
|
||||
| **Area** | **bank** + wire/wallet settle — **not** mon overload, **not** “laptop too slow” |
|
||||
| **Not the cause: ecosystem load** | Outside-in check **2026-07-19 ~14:56 CEST**: bank/exchange/merchant all **HTTP 200**, config ~**50–150 ms**, exchange `/keys` ~**0.5 s**. Container **loadavg ~1.5–1.7**, RSS bank **~737 MiB** / exchange **~670 MiB**, bank probes **47 ms**, **`wirewatch_running: true`**. Stack is **healthy under moderate load** — failures are **application/SQL/protocol**, not “host on fire.” |
|
||||
| **Call site (mint)** | `POST ${BANK}/accounts/explorer/withdrawals` → often HTTP **500** `{ code: 5110, hint: "Unexpected sql error with state P0001" }` (**`BANK_UNMANAGED_EXCEPTION`**; Postgres **`P0001`** = `RAISE EXCEPTION`). Should often be clean **409** (insufficient funds); instead opaque **500**. |
|
||||
| **Config** | `/config` still allows ~**2⁵²** wire max; live mint fails **far below** that. Explorer **balance_explorer** on landing still **~4.5 Peta-GOA**. |
|
||||
|
||||
#### Analysis — where withdraw dies
|
||||
|
||||
| Stage | What we see | Manage to withdraw? |
|
||||
|-------|-------------|---------------------|
|
||||
| **Mint (high amounts)** | 5110/P0001 while hi falls Tera→Giga→Mega→Kilo with **lo=0** | **No** — withdrawal never created |
|
||||
| **Mint (some mid amounts)** | occasional **HTTP 200** + URI | Mint only — not done |
|
||||
| **Confirm** | force-select 200 then stuck **`selected`**, confirm **HTTP 500** → mon **`SKIP_CONFIRM`** (bounds unchanged) | **No** — bank never wires; **not** out-of-money (§3 / 5114 class) |
|
||||
| **Bank settle** | rare full path: confirm 204 + `transfer_done` | Bank side “OK_BANK” |
|
||||
| **Wallet coins** | almost always **`available=0`** within wait, large **`pendingIncoming`**, board **`done=0`** | **Not in time** for pay → **SKIP_BALANCE** / skip max-pay |
|
||||
|
||||
**Pay-wait board (same cumulative wallet), example:**
|
||||
|
||||
```text
|
||||
summary: tsv_ok=2 · wallet_wd=14 (pendingIncoming=14 done=0)
|
||||
timer 0s left — available still GOA:0
|
||||
pendingIncoming=GOA:335776
|
||||
```
|
||||
|
||||
Reading: only **2** TSV rows are full OK/OK_BANK, but the wallet holds **14** withdrawal txs still **pendingIncoming** and **none done** after the wait. Max-search also leaves many **mint-OK / confirm-failed** withdraws that **inflate pendingIncoming** and may never complete. After **90 s** + short `run-pending`: **still no spendable balance** → **we do not manage to withdraw in time** for pay (whether coins would appear later is unknown to the job).
|
||||
|
||||
#### What *did* work (sparse) — for contrast
|
||||
|
||||
| When / run | What worked | Amounts |
|
||||
|------------|-------------|---------|
|
||||
| Landing **2026-07-19 14:19–14:29 CEST** | Explorer singles **recorded** on bank intro stats | **~1.5–2 Giga-GOA**, **~145–202 Mega-GOA**, **~1.8–3.4 Mega-GOA** |
|
||||
| Max-search run A (same day) | Full bank path (mint→confirm→`transfer_done`) for **two** amounts only; mon best | **`GOA:424114`**, **`GOA:822256`** (~822 Kilo best) |
|
||||
| Max-search run B (later) | Even fewer full successes; many mint OK + **SKIP_CONFIRM**; mon best collapsed | e.g. full success around **`GOA:12276`** / **`GOA:17257`** (~**17 Kilo** best) |
|
||||
| Pay after either | Spendable **available** within **90 s** | **`GOA:0`** (not in time) |
|
||||
|
||||
So: **landing/history prove large withdraws were possible earlier the same day**; **automation does not manage a usable withdraw in time** (best amount drops run-to-run; coins not available inside mon’s wait).
|
||||
|
||||
#### Mon behaviour (workaround only)
|
||||
|
||||
- 5110 → soft **`CEILING_REJECT`** / max-search **`hi:=`**
|
||||
- Full OK only raises **`lo`**; **SKIP_CONFIRM** does not
|
||||
- Pay waits up to **`LADDER_PAY_WAIT_AVAILABLE_S`** (default 90) with withdraw board; then **SKIP_BALANCE** if still 0
|
||||
- **Does not fix** bank SQL mapping, stuck confirm, or wirewatch→coins lag
|
||||
|
||||
#### Wanted (bank / exchange ops)
|
||||
|
||||
1. Map mint **P0001** to structured errors (not **5110**).
|
||||
2. Confirm after force-select must not stick **selected** + HTTP 500.
|
||||
3. After `transfer_done`, coins must become wallet **available** (or document delay + API to wait).
|
||||
4. File bugs with: max-search “what worked” table + landing `recent_withdraws`/`balance_explorer` + load note (healthy) + pay board `tsv_ok` vs `wallet_wd`/`pendingIncoming`/`done`.
|
||||
|
||||
Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro stats, mon max-ladder logs 2026-07-19.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -64,7 +133,7 @@ Legend:
|
|||
| **Seen** | Android endless spinner; GOA `bank.hacktivism.ch` withdraw; fixed in app branch `fix/bank-withdraw-auto-exchange` |
|
||||
| **Area** | **core** / **cli** |
|
||||
| **Problem** | Withdraw URI names an exchange the wallet has never added → stuck loading / empty exchange list. CLI requires explicit `exchanges add` + `update` + `accept-tos` every fresh DB. |
|
||||
| **Workaround** | e2e always adds exchange before ATM ladder; Android needs ensureExchange-style fix. |
|
||||
| **Workaround** | e2e always adds exchange before bank withdraw ladder; Android needs ensureExchange-style fix. |
|
||||
| **Wanted** | CLI/core: on `accept-uri` withdraw, **auto-add** exchange from URI/bank details (`allowCompletion`), then surface ToS if needed in one command. |
|
||||
|
||||
---
|
||||
|
|
@ -76,8 +145,8 @@ Legend:
|
|||
| **Seen** | ladder/e2e strip ports; QR checks; mint URIs with `:443` |
|
||||
| **Area** | **cli** / **bank** / **doc** |
|
||||
| **Problem** | Some bank/wallet outputs include `taler://…host:443/…`. QR validators and some clients reject or double-normalize inconsistently. |
|
||||
| **Workaround** | `sed 's/:443\//\//g'` everywhere in automation. |
|
||||
| **Wanted** | Canonical form without default ports in **all** wallet-cli and bank integration outputs; document as invariant. |
|
||||
| **Workaround** | mon ≥1.23.4: `normalize_taler_uri` (lib.sh) on ladder/e2e/pay; bank landing `demo-withdraw-api.py` normalizes mint JSON; **urls** phase fails GOA/local if public landings still emit `:443`/`:80` in `taler://` (`www.taler-uri-ports-*`, unit `tests/test_normalize_taler_uri.sh`). |
|
||||
| **Wanted** | libeufin never emits default ports (source fix). |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -85,7 +154,7 @@ Legend:
|
|||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | e2e uses bank admin + ATM create; ladder/gui-chain reimplement explorer mint in Python |
|
||||
| **Seen** | e2e uses bank admin + bank withdraw create; ladder/gui-chain reimplement explorer mint in Python |
|
||||
| **Area** | **cli** / **ops** |
|
||||
| **Problem** | Automation always hand-rolls Basic auth → token → `POST …/withdrawals` → `taler_withdraw_uri`. Easy to get wrong (auth header, amount currency). |
|
||||
| **Workaround** | Shared Python in `goa-chain` / ladder / e2e; secrets files. |
|
||||
|
|
@ -189,6 +258,59 @@ Legend:
|
|||
|
||||
---
|
||||
|
||||
## 15. `--wallet-db=*.json` is rejected (“memory backend not supported”)
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | **2026-07-19 magikoopa** wallet-cli **1.6.8** / **1.6.11** bundle: any command with `--wallet-db=/tmp/foo.json` fails immediately with `Error: memory backend not supported` |
|
||||
| **Area** | **cli** / **ops** |
|
||||
| **Problem** | Current node host treats storage paths ending in **`.json` as unsupported memory backend** and throws before opening sqlite. Old notes / local tests that used `WALLET_DB=….json` break. Accept/transactions then never emit `reserve_pub` → ladder §3 empty-scrape WARNs. |
|
||||
| **Workaround** | Use a **non-`.json` path**, e.g. `--wallet-db=$SCRATCH/wallet.sqlite3` (ladder already prefers `wallet.sqlite3`). Never pass `….json` as wallet-db. |
|
||||
| **Wanted** | Clearer error: `wallet-db path must not end in .json; use a directory or .sqlite3 path`. Document legal path forms in `--help`. |
|
||||
|
||||
---
|
||||
|
||||
## 16. `taler-helper-sqlite3` requires Python ≥ 3.11 on PATH
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | magikoopa: helper is a **Python** script (`#!/usr/bin/env python3`); with Apple `/usr/bin/python3` **3.9.6** → `FATAL: python version >=3.11 required but running on 3.9.6` in accept/tx logs; wallet may still exit 0 on some paths while stdout is FATAL-only |
|
||||
| **Area** | **ops** / **cli** |
|
||||
| **Problem** | Wallet spawns `taler-helper-sqlite3` from **PATH**. Wrong `python3` ⇒ sqlite backend fails ⇒ no real wallet state ⇒ no `reserve_pub` for force-select. |
|
||||
| **Workaround** | Put **Python ≥3.11** (or 3.12+) first on PATH for mon jobs; ensure `command -v taler-helper-sqlite3` works; smoke: `taler-helper-sqlite3` starts without FATAL. Prefer Homebrew / user `python3.11+` over macOS system 3.9 for agent PATH. |
|
||||
| **Wanted** | Helper shebang or wrapper that finds a suitable python; wallet preflight error if helper HELLO fails (not silent empty logs). |
|
||||
|
||||
---
|
||||
|
||||
## 17. Portable wall-clock for scripts (no GNU `timeout` assumed)
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | macOS agent shells: `timeout: command not found`; e2e already tries `timeout` then `gtimeout` then `perl -e alarm` |
|
||||
| **Area** | **ops** / **doc** |
|
||||
| **Problem** | GNU coreutils `timeout` is not on stock macOS. Hard-wiring it breaks portable repro and CI on Darwin. |
|
||||
| **Workaround** | Prefer **python3** `subprocess.Popen(…).wait(timeout=…)` for outer budgets; for wallet-cli inner calls keep e2e order: `timeout` → `gtimeout` → perl alarm. Document both. |
|
||||
| **Wanted** | One suite helper `run_with_budget SECS cmd…` in `lib.sh` used by ladder/e2e/smoke. |
|
||||
|
||||
---
|
||||
|
||||
## 18. How to withdraw (operator checklist, GOA laptop)
|
||||
|
||||
Portable path that works when §§15–16 are fixed:
|
||||
|
||||
1. **PATH:** `taler-helper-sqlite3` + **python3 ≥ 3.11** ahead of system 3.9.
|
||||
2. **Wallet DB:** `WDB=$TMP/wallet.sqlite3` — **not** `*.json`.
|
||||
3. **Exchange:** `exchanges add` → `update -f` → `accept-tos` for `https://exchange.hacktivism.ch/`.
|
||||
4. **Bank mint (explorer):** token with explorer password from `SECRETS_ROOT` / `EXP_PW`; `POST …/accounts/explorer/withdrawals` with e.g. `{"amount":"GOA:1"}` (avoid pure **0** and absolute **max** pins for happy-path tests).
|
||||
5. **Strip** `:443` from `taler_withdraw_uri`.
|
||||
6. **`withdraw accept-uri --exchange URL URI`**.
|
||||
7. **Bank:** poll `…/taler-integration/withdrawal-operation/{WID}` → when `selected`, confirm; if stuck `pending`, force-select with scraped `reserve_pub` (§3).
|
||||
8. **Settle:** poll wallet balance / bank `transfer_done` — **no** `run-until-done` (§1).
|
||||
|
||||
If step 6’s stdout is only `FATAL: python version…` or `memory backend not supported`, **stop** and fix §15–16 before debugging force-select.
|
||||
|
||||
---
|
||||
|
||||
## Priority shortlist for `taler-wallet-cli` / core
|
||||
|
||||
If only a few changes land, these unlock the most automation:
|
||||
|
|
@ -198,6 +320,7 @@ If only a few changes land, these unlock the most automation:
|
|||
3. **Auto-add exchange + ToS on accept-uri** (§4, §8)
|
||||
4. **Canonical URIs without :443** (§5)
|
||||
5. **Emit reserve_pub for last withdraw** (§3)
|
||||
6. **Clear wallet-db path rules + helper preflight** (§15, §16)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -210,7 +333,12 @@ If only a few changes land, these unlock the most automation:
|
|||
| exchange | Explicit add/update/accept-tos |
|
||||
| ladder explorer | `read_secret` / EXP_PW_FILE |
|
||||
| stage maxima | bank `max_wire` + keys min denom |
|
||||
| force-select | multi-rpub try; soft-skip confirm |
|
||||
| force-select | multi-rpub try; soft-skip confirm; empty scrape → WARN not balance fail |
|
||||
| higher amounts / withdraw fail | **§3b: we do not manage to withdraw in time** on GOA (healthy load); mint 5110/P0001 + SKIP_CONFIRM + pendingIncoming; pay still available=0 after 90s; landing had Giga/Mega earlier same day |
|
||||
| pay vs balance | never order more than wallet **available** (soft `SKIP_BALANCE`); pendingIncoming does not count as spendable |
|
||||
| pay wait UX | board: OK/OK_BANK start order + live pendingIncoming/`done`; often `tsv_ok≪wallet_wd` and **done=0** after 90s → skip pay |
|
||||
| wallet-db path | Prefer `*.sqlite3` (never `*.json` as db path) |
|
||||
| helper / python | mon PATH must include helper + python≥3.11 |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,8 @@
|
|||
| Binary | Package (zypper) | Role |
|
||||
|--------|------------------|------|
|
||||
| `bash` | (base) | runner |
|
||||
| `curl` | `curl` | HTTPS/API probes |
|
||||
| `curl` | `curl` | HTTPS/API probes + **monpages** FQDN checks |
|
||||
| `nmap` | `nmap` | **surface** OS/service fingerprint (`SURFACE_NMAP=1`, v1.8.0+; OS detect may need root) |
|
||||
| `python3` | `python313-base` (provides `/usr/bin/python3`) | JSON, HTML site-gen, metrics |
|
||||
| `podman` | `podman` | **inside**: `podman exec` into taler-hacktivism* |
|
||||
| `git` | `git` | pin commit SHA in monitoring HTML footer |
|
||||
|
|
@ -40,9 +41,13 @@ command -v qrencode zbarimg
|
|||
| `jq` | nice-to-have; suite uses python3 for JSON |
|
||||
| `dig` / `bind-utils` | debug DNS; not required for default phases |
|
||||
| `ssh` | only for access mode **ssh** (laptop → `KOOPA_SSH` / stage `INSIDE_SSH`); not needed for **host-podman** |
|
||||
| `node` / `pnpm` + wallet-cli | **e2e** / ladder only (+ secrets) |
|
||||
| `node` + `taler-wallet-cli` (.mjs) | **e2e** / ladder only (+ secrets) |
|
||||
| **`taler-helper-sqlite3` on PATH** | **required for e2e** — wallet-core spawns it for withdraw accept-uri / pay; missing → ENOENT mid-ladder. Put next to wallet bindir (`~/.local/bin` on stagepaysan) |
|
||||
| `timeout` (coreutils) | preferred hard wall for `wcli` (suite falls back to perl only if missing) |
|
||||
| `secrets.env` / admin passwords | e2e only — never system packages |
|
||||
|
||||
**Preflight (agents / ops):** before trusting e2e on a host, smoke on the **same PATH** mon uses (`run-host-report` / systemd user): `command -v node taler-helper-sqlite3`, `node "$WALLET_CLI" --version`, short temp-DB `balance`. Do not wait for a full mon run to discover ENOENT.
|
||||
|
||||
### systemd user (no extra packages)
|
||||
|
||||
- `systemd` user session + **`loginctl enable-linger hernani`** (already **Linger=yes** on koopa)
|
||||
|
|
@ -55,7 +60,7 @@ command -v qrencode zbarimg
|
|||
|
||||
---
|
||||
|
||||
## firecuda-external (macOS, outside-only timer)
|
||||
## outside-runner (macOS, outside-only timer)
|
||||
|
||||
| Need | Notes |
|
||||
|------|--------|
|
||||
|
|
@ -65,6 +70,17 @@ command -v qrencode zbarimg
|
|||
|
||||
---
|
||||
|
||||
## Franc Paysan hosts (francpaysan / stagepaysan)
|
||||
## Franc Paysan stage host (stagepaysan)
|
||||
|
||||
Document package lists under **`francpaysan-admin-log`** when those host-agents are installed (same idea: curl, python3, podman if inside, qrencode/zbar for full urls).
|
||||
FP stage (`stagepaysan` via SSH alias `francpaysan-stage-user`): curl, python3, podman if inside, qrencode/zbar for full urls; suite at `~/src/taler-monitoring`.
|
||||
|
||||
**e2e on stagepaysan** also needs user-local:
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `~/.local/bin/node` | Node for wallet-cli |
|
||||
| `~/.local/bin/taler-wallet-cli.mjs` (or `WALLET_CLI=`) | wallet CLI |
|
||||
| `~/.local/bin/taler-helper-sqlite3` | **required** — accept-uri / coins |
|
||||
| `~/francpaysan-secrets/stage/*` | bank admin + merchant token (layout like laptop `francpaysan-secrets`) |
|
||||
|
||||
Host-agent PATH already puts `$HOME/.local/bin` first; still smoke-test under that PATH.
|
||||
|
|
|
|||
148
README.md
148
README.md
|
|
@ -1,15 +1,51 @@
|
|||
# taler-monitoring
|
||||
|
||||
Standalone repository: https://git.hacktivism.ch/hernani/taler-monitoring (tag **v1.0**).
|
||||
Checks for **GNU Taler** stacks (bank, exchange, merchant; optional surface inventory).
|
||||
|
||||
Formerly: `koopa-admin-log/scripts/taler-monitoring/`.
|
||||
| Mode | What |
|
||||
|------|------|
|
||||
| **Outside-only** | Public HTTPS from anywhere — no SSH, no host-agent. Phases such as `urls`, `monpages`, `surface`, … |
|
||||
| **Inside / local** | Where you operate the stack: podman/SSH (`inside`, `versions`, `e2e`, `ladder`, …) |
|
||||
| **Host-agent + HTML** | Optional: timers run the suite and publish sticky-bar HTML (see simplified page layout below) |
|
||||
|
||||
Outside-only example: `./taler-monitoring.sh -d demo.taler.net urls`
|
||||
Surface (ecosystem + mail + mattermost + versions): `./taler-monitoring.sh surface mattermost mail versions`
|
||||
Full local GOA example: `./taler-monitoring.sh -d hacktivism.ch urls inside versions`
|
||||
|
||||
Repo: https://git.hacktivism.ch/hernani/taler-monitoring · tree: `~/src/taler-monitoring`
|
||||
Version: `./taler-monitoring.sh --ver` (or `--version` / `-V`)
|
||||
|
||||
|
||||
## Monitoring pages — public URL families (only these three)
|
||||
|
||||
**`taler-monitoring` is the suite name (git / CLI), not a public URL path.**
|
||||
There is **no** mon page at `/taler-monitoring` or `/taler-monitoring/`.
|
||||
|
||||
Exactly **three** first-class public path families (each must work **bare and with trailing slash**; reverse-proxy should `redir` bare → slash). Error companions use `…_err` when the last run failed:
|
||||
|
||||
| Family | Paths (bare + slash) | Where | Purpose |
|
||||
|--------|----------------------|--------|---------|
|
||||
| **monitoring** | `/monitoring` · `/monitoring/` · optional `/monitoring_err` · `/monitoring_err/` | Per **landing** host (GOA bank/exchange/taler · FP · mytops stage, …) | Stack report for that host |
|
||||
| **taler-monitoring-surface** | `/taler-monitoring-surface` · `/…/` · optional `…_err` | **only** `taler.hacktivism.ch` | Ecosystem surface (ports, nmap, mail, mattermost, versions) |
|
||||
| **taler-monitoring-aptdeploy** | `/taler-monitoring-aptdeploy` · `/…/` · optional `…_err` | **only** `taler.hacktivism.ch` | apt-src merchant deploy tests (podman on koopa) |
|
||||
|
||||
**Not** public mon pages: `/taler-monitoring-mail*`, `/taler-monitoring-mattermost*`, bare `/taler-monitoring` — mail/mm are folded into **surface**.
|
||||
|
||||
Related:
|
||||
|
||||
- **Phase `monpages`:** obligatory **ERROR** if a listed page is missing, bootstrap stub, not suite HTML, or **content markers fail**. Bare paths that fall through to merchant **code 21** are ERROR (not “OK because slash works”).
|
||||
- **Publish:** staging `~/monitoring-sites-staging` → `DEPLOY_WWW_ROOT` + reverse-proxy handles for **only** the three families above.
|
||||
|
||||
```bash
|
||||
./taler-monitoring.sh monpages
|
||||
./taler-monitoring.sh -d hacktivism.ch monpages
|
||||
```
|
||||
|
||||
# taler-monitoring
|
||||
|
||||
Host packages (koopa / runners): **[DEPENDENCIES.md](./DEPENDENCIES.md)**.
|
||||
Host-agents (GOA + FP; shared report pipeline): **[host-agent/README.md](./host-agent/README.md)**.
|
||||
Public HTML site-gen (same timeout/log/HTML defaults): **[site-gen/README.md](./site-gen/README.md)**.
|
||||
FP install notes: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
|
||||
FP stage host-agent: `host-agent/run-fp-stage-monitoring.sh` + `host-agent/env/stagepaysan.env.example` (see `host-agent/README.md`).
|
||||
|
||||
Report for the **GOA** stack with boxed severity badges and **grouped test IDs**
|
||||
`area.group-NN` (e.g. `www.exchange-01`, `e2e.pay-03`):
|
||||
|
|
@ -25,9 +61,11 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs**
|
|||
IDs reset per **group** inside an area — so “too many www tests” become
|
||||
`www.exchange-*`, `www.bank-*`, `www.landing-*`, etc.
|
||||
Catalog: **[TESTS.md](./TESTS.md)**.
|
||||
Android / tooling: **[VERSIONS.md](./VERSIONS.md)**.
|
||||
Git / tags / suite tree: **[VERSIONS.md](./VERSIONS.md)**.
|
||||
Optional / deprecated tooling: **[meta/](./meta/)** (firecuda, legacy site-gen, old mail/mm units).
|
||||
|
||||
GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**.
|
||||
CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**.
|
||||
CLI / wallet-cli recurring issues: **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**.
|
||||
|
||||
```text
|
||||
┌ OK ┐ www.exchange-01 exchange /config · HTTP 200
|
||||
|
|
@ -38,13 +76,13 @@ End of each phase: totals + list of **BLOCKERS** and **ERRORS** (ids included).
|
|||
|
||||
## Secrets (local GOA e2e)
|
||||
|
||||
**Do not commit passwords.** Canonical trees: `~/src/koopa/koopa-admin-log` +
|
||||
**Do not commit passwords.** Suite tree: `~/src/taler-monitoring` · host ops (optional):
|
||||
sibling `~/src/koopa/koopa-admin-secrets`.
|
||||
|
||||
### Local file (recommended on laptop)
|
||||
|
||||
```bash
|
||||
cd ~/src/koopa/src/taler-monitoring
|
||||
cd ~/src/taler-monitoring
|
||||
cp secrets.env.example secrets.env # gitignored
|
||||
$EDITOR secrets.env
|
||||
./taler-monitoring.sh e2e
|
||||
|
|
@ -102,7 +140,7 @@ Each check prints **global** and **grouped** ids:
|
|||
# inside: low-priv SSH francpaysan-stage-user (User stagepaysan) → podman
|
||||
# containers stage-lfp-{bank,exchange-ansible,merchant} · ports 9032/9031/9030
|
||||
# stats: outside-in public …/intro/stats.json (age via generated_at_unix)
|
||||
# Admin credit for e2e: E2E_BANK_ADMIN_PASS=… or francpaysan-host secrets
|
||||
# Admin credit for e2e: E2E_BANK_ADMIN_PASS=… or stage host secrets
|
||||
# (bank-admin-password.txt under /mnt/data/stagepaysan/bank/secrets/)
|
||||
# Override: INSIDE_SSH=… INSIDE_PROFILE=stage-lfp
|
||||
|
||||
|
|
@ -165,8 +203,9 @@ Each monitored stack must say **what is bank, exchange, and merchant-backend**.
|
|||
Edit **[domains.conf](./domains.conf)** (or set `TALER_DOMAINS_CONF=`):
|
||||
|
||||
```text
|
||||
# name bank exchange merchant-backend currency local [canonical]
|
||||
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0
|
||||
# name bank exchange merchant currency local landing lang locale [canonical]
|
||||
taler-ops.ch bank.… exchange.… my.… CHF 0 0 en de-CH
|
||||
stage.lefrancpaysan.ch stage.bank.… stage.exchange.… stage.monnaie.… TESTPAYSAN 0 1 fr fr-CH
|
||||
```
|
||||
|
||||
| Field | Meaning |
|
||||
|
|
@ -178,8 +217,12 @@ taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0
|
|||
| **currency** | `GOA` / `KUDOS` / `CHF` / … |
|
||||
| **local** | `1` = koopa SSH stack; `0` = public only |
|
||||
| **landing** | `1` = check `/intro` landings + assets; `0` = skip (TOPS / mytops) |
|
||||
| **lang** | UI language **`en`** / **`fr`** / **`de`** when `TALER_MON_LANG_SET≠1`. Stock profiles use **en** or **fr** only — **`de` is never a default** (opt-in via `--lang de` or an explicit profile edit). |
|
||||
| **locale** | Regional l10n (BCP 47): **`fr-CH`** for FrancPaysan (Suisse romande); **`de-CH`** for other Swiss stacks. Aliases `ch-FR` / `ch-DE` accepted. Not the same as UI `lang`. |
|
||||
| **canonical** | optional label after alias (e.g. `my.taler-ops.ch` → `taler-ops.ch`) |
|
||||
|
||||
Workstation `TALER_MON_LANG=en` does **not** override a profile with `lang=fr`. Force English on FP only with `--lang en` / `TALER_MON_LANG_SET=1`.
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
|
|
@ -224,19 +267,21 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC.
|
|||
|
||||
| | Local (koopa) | Remote |
|
||||
|--|---------------|--------|
|
||||
| **ATM withdraw** | 20 · 50 · 100 · 200 · **4200** (paivana) | 10 · 20 · 50 |
|
||||
| **Bank withdraw** | 20 · 50 · 100 · 200 · **4200** (paivana) | 10 · 20 · 50 |
|
||||
| **Pay ladder** | 0.01 … 10 | 0.01 … 1 |
|
||||
| **Paivana** | HTTP paywall + template pay **GOA:4200** (`goa-shop` / `paivana`) | skipped |
|
||||
|
||||
```bash
|
||||
./taler-monitoring.sh e2e
|
||||
./taler-monitoring.sh -d taler.net urls e2e
|
||||
# Amount ladder (withdraw 0 → … → max, then pay): same script for GOA + stage
|
||||
./taler-monitoring.sh ladder # GOA / current domain
|
||||
# Amount ladder (any currency): classic 0 → … → max, then optional pay
|
||||
./taler-monitoring.sh ladder # classic (current domain)
|
||||
./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima
|
||||
# Max-search: highest withdrawable + highest payable (≤ available; GOA free template goa-free)
|
||||
./taler-monitoring.sh max-ladder
|
||||
# Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01).
|
||||
# Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=).
|
||||
# Override: LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 …
|
||||
# Override: LADDER_MODE=max LADDER_MAX_PROBES=16 LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 …
|
||||
# customize e2e:
|
||||
E2E_WITHDRAW_VALUES="20 50 100" E2E_PAY_VALUES="0.05 1 5" ./taler-monitoring.sh e2e
|
||||
E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # single fixed
|
||||
|
|
@ -261,7 +306,7 @@ E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # s
|
|||
| **versions** | `deb.taler.net` reachable (InRelease/Packages/pool `.deb`); containers can reach it + have apt source; installed Taler packages vs **trixie** |
|
||||
| **sanity** | bank · exchange · merchant sections (public + server) |
|
||||
| **e2e** | account → credit → withdraw → wallet → confirm → order → pay |
|
||||
| **ladder** | GOA withdraw: fixed **0** + low bands + **random high ranges** + fixed **max** |
|
||||
| **ladder** | amount ladder modules: classic 23-rung wd+pay, or max-search wd+pay (`max-ladder`) |
|
||||
|
||||
```bash
|
||||
# package suite (default trixie on deb.taler.net)
|
||||
|
|
@ -300,8 +345,8 @@ Amounts use exchange **`alt_unit_names`** (Kilo-GOA, Mega-GOA, …) with the bas
|
|||
**GOA value in parentheses**:
|
||||
|
||||
```text
|
||||
[INFO] coins after-ATM — amount_circ=2 Kilo-GOA (GOA:2000)
|
||||
[INFO] coins after-ATM — denoms_in_circ 1 Kilo-GOA (GOA:1000)×2 (=2 Kilo-GOA (GOA:2000))
|
||||
[INFO] coins after-bank-wd — amount_circ=2 Kilo-GOA (GOA:2000)
|
||||
[INFO] coins after-bank-wd — denoms_in_circ 1 Kilo-GOA (GOA:1000)×2 (=2 Kilo-GOA (GOA:2000))
|
||||
== ladder · rung 12 GOA:1000000 · 1 Mega-GOA (GOA:1000000) ==
|
||||
```
|
||||
|
||||
|
|
@ -332,13 +377,13 @@ snapshotted via SSH whenever the stack is under load:
|
|||
| Phase | When |
|
||||
|-------|------|
|
||||
| **inside** | end of container status |
|
||||
| **e2e** | before work · after ATM withdraws · after payments · after shop · e2e-end (+ delta) |
|
||||
| **e2e** | before work · after bank withdraws · after payments · after shop · e2e-end (+ delta) |
|
||||
| **ladder** | before first rung · after last rung (+ delta + timing overall) |
|
||||
|
||||
```bash
|
||||
# LAN down? force WAN DNAT host
|
||||
KOOPA_SSH=koopa-external ./taler-monitoring.sh inside
|
||||
# or rely on automatic fallback (default KOOPA_SSH_FALLBACKS=koopa-external)
|
||||
# or rely on automatic fallback (KOOPA_SSH_FALLBACKS from env)
|
||||
|
||||
METRICS_LOAD=0 ./taler-monitoring.sh e2e # skip host probes
|
||||
LADDER_LOAD=0 ./taler-monitoring.sh ladder
|
||||
|
|
@ -347,20 +392,71 @@ LADDER_LOAD=0 ./taler-monitoring.sh ladder
|
|||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `KOOPA_SSH` | `koopa` | primary SSH host |
|
||||
| `KOOPA_SSH_FALLBACKS` | `koopa-external` | tried if primary fails |
|
||||
| `KOOPA_SSH_FALLBACKS` | *(from env)* | tried if primary fails |
|
||||
| `METRICS_LOAD` | `1` | `0` = skip all host/container load probes |
|
||||
| `LADDER_LOAD` | `1` | `0` = skip load in ladder only |
|
||||
| `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws |
|
||||
| `LADDER_WITHDRAW_SCALE` | `1.5` | withdraw mids / pay mids ratio (funding headroom) |
|
||||
| `LADDER_STEPS` | `23` | rungs for both withdraw and pay (0 + mids + max−1 + max) |
|
||||
| `LADDER_MODE` | `classic` | default **classic** 23-rung wd+pay; `max` = max-search wd+pay (`max-ladder`) |
|
||||
| `LADDER_MAX_PROBES` | `32` | max-search probe budget; `0` = until converge / timeout |
|
||||
| `LADDER_HIGH_RUNGS` | `6` | max-search: first N probes random-high, then bisect |
|
||||
| `LADDER_MAX_TOL_REL` | `0.001` | max-search relative lo/hi gap to stop |
|
||||
| `LADDER_FREE_TEMPLATE` | `goa-free` | public free-amount pay template (hacktivism) |
|
||||
| `LADDER_USE_FREE_TEMPLATE` | `1` | create pays via template POST `{"amount":…}` |
|
||||
| `LADDER_WITHDRAW_SCALE` | `1.5` | classic: withdraw mids / pay mids ratio (funding headroom) |
|
||||
| `LADDER_STEPS` | `23` | classic: rungs for withdraw and pay (0 + mids + max−1 + max) |
|
||||
| `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python |
|
||||
|
||||
## Needs
|
||||
|
||||
- SSH `koopa` or `koopa-external` (inside / load / sanity server bits)
|
||||
- SSH via `KOOPA_SSH` / `KOOPA_SSH_FALLBACKS` (env) (inside / load / sanity server bits)
|
||||
- secrets under `koopa-admin-secrets/...` for e2e
|
||||
- `taler-wallet-cli` for e2e
|
||||
|
||||
## Languages (i18n)
|
||||
## Languages (i18n) — **v1.2+**
|
||||
|
||||
Default **English**. FrancPaysan stacks use **French** (`TALER_MON_LANG=fr`, auto for `*lefrancpaysan*`). See `VERSIONS.md` and `i18n.sh`.
|
||||
Default **English**. Select language with `--lang en|fr` or `TALER_MON_LANG`.
|
||||
FrancPaysan stacks auto-select **French** (`*lefrancpaysan*`). See `VERSIONS.md` and `i18n.sh`.
|
||||
|
||||
## Public monitoring pages via FQDN (v1.3.1+ / obligatory v1.7.6+ / layout v1.8.0+)
|
||||
|
||||
Phase **`monpages`** is **obligatory**: missing or non-suite HTML is **ERROR**.
|
||||
Soft mode only with `MONPAGES_REQUIRE_PUBLIC=0` (escape hatch).
|
||||
|
||||
| Stack | What monpages expects (v1.9.0) |
|
||||
|-------|--------------------------------|
|
||||
| **GOA** | 3× `/monitoring/` + **surface** (existence **and** top/bottom/content markers) |
|
||||
| **FrancPaysan** | Stack mon hosts `/monitoring/` with same content rules |
|
||||
| **Nine landings (model)** | Each landing front: `/monitoring` + `/monitoring_err` (err when a failed run exists) |
|
||||
|
||||
```bash
|
||||
./taler-monitoring.sh -d hacktivism.ch monpages
|
||||
./taler-monitoring.sh -d lefrancpaysan.ch monpages
|
||||
```
|
||||
|
||||
If pages are missing publicly: `sudo $APPLY_MONITORING_LIVE` on koopa
|
||||
(FP: Infomaniak vhost for `/monitoring*`).
|
||||
|
||||
## Devtesting · fake-franken CHF (v1.13+)
|
||||
|
||||
Phase **`devtesting`** (aliases: `franken`, `fake-franken`) exercises the
|
||||
Taler Operations **rusty** host (`taler-devtesting` forced command):
|
||||
|
||||
```bash
|
||||
./taler-monitoring.sh -d stage.taler-ops.ch --currency CHF devtesting
|
||||
```
|
||||
|
||||
Needs SSH to `devtesting@rusty.taler-ops.ch` (Host `taler-rusty-devtesting`).
|
||||
Probes: CLI help, `geniban` (CH IBAN / synthetic debit funding), baseline `fake-incoming`,
|
||||
optional **amount ladder** (`DEVTESTING_LADDER=1`, default **23** rungs up to **CHF:50000**).
|
||||
Each rung reuses geniban debit payto so volume covers the max rung without a real bank withdraw.
|
||||
`fake-incoming` CHF to exchange credit
|
||||
IBAN. A nexus bounce *missing reserve public key* is **OK** for the plumbing
|
||||
probe (no real wallet reserve required).
|
||||
|
||||
## Version on monitoring pages (v1.3+ / sticky bar)
|
||||
|
||||
The sticky bar shows the suite **tag** (e.g. `v1.11.0`) and links to that tag on Forgejo.
|
||||
|
||||
**v1.12.0:** Run header (**monitoring env context**: env_file, suite pin, domain/hosts/flags) is a collapsed `<details>` block above the console log — expand to inspect agent context.
|
||||
|
||||
**v1.11.0:** Click the sticky **error** or **warning** count to filter the console log to only those lines (other log lines hide; sticky bar, stats, and top overviews stay). Same via `#filter-error` / `#filter-warn` (or `#first-error` / `#first-warn`). Click again or **Show all** to clear.
|
||||
|
|
|
|||
34
TESTS.md
34
TESTS.md
|
|
@ -20,13 +20,15 @@ Every check line has a **global** run number and a **grouped** id:
|
|||
| **versions** | `check_versions.sh` | `outside` `inside` `compare` |
|
||||
| **aptdeploy** | `check_apt_deploy.sh` | `trixie` `trixie-testing` (koopa podman apt-src merchant smoke) |
|
||||
| **surface** | `check_surface.sh` | **remote-only** ecosystem / `-d` domain inventory (not in default/all/full) |
|
||||
| **monpages** | `check_monitoring_pages.sh` | public HTML via FQDN — **ERROR** if missing or markers fail (top+bottom+body); surface required on GOA (v1.9.0) |
|
||||
| **surface** | `check_surface.sh` | ecosystem inventory + **nmap** OS fingerprint; mail/mattermost content reported via surface page |
|
||||
| **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
|
||||
| **server** | `check_server.sh` | (flat `server-NN` or host groups) |
|
||||
| **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `atm` `settle` `pay` `shop` `paivana` `dig` `report` |
|
||||
| **ladder** | `check_goa_ladder.sh` | `plan` `load` `withdraw` `pay` `report` |
|
||||
| **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `bankwd` `settle` `pay` `shop` `paivana` `dig` `report` |
|
||||
| **ladder** | `check_amount_ladder.sh` + `ladder/lib_pay.sh` | `plan` `load` `withdraw` `pay` `report` — classic / max-search; phases `ladder` · `max-ladder` · `goa-ladder` (alias) |
|
||||
|
||||
**Why groups:** flat `www-001`…`www-080` was hard to talk about.
|
||||
`www.bank-04` / `e2e.atm-02` pin the failure to a logical block.
|
||||
`www.bank-04` / `e2e.bankwd-02` pin the failure to a logical block.
|
||||
|
||||
Usage in scripts:
|
||||
|
||||
|
|
@ -57,10 +59,13 @@ Numbering follows **executed** checks (early skip may shift later NN inside the
|
|||
| **www.webui-** | SPA fingerprints: **`/webui/version.txt`**, **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` |
|
||||
| **www.paivana-** | local GOA paywall front (redirect to template) |
|
||||
| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local |
|
||||
| **www.taler-uri-ports-** | **no default `:443`/`:80`** in public `taler://` URIs — bank/exchange/merchant `/intro/` HTML, `demo-withdraw.json`, `auto-account.json`, bank `stats.json` (ERROR on GOA/local). Unit: `tests/test_normalize_taler_uri.sh` |
|
||||
| **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` |
|
||||
|
||||
**QR rule:** `taler://withdraw/HOST/taler-integration/UUID` (no default `:443`/`:80`); `taler://pay/` / `pay-template/`; `payto://…` shape OK; decoded PNG must equal payload. Form errors on withdraw/pay/payto are **ERROR** on local/GOA.
|
||||
|
||||
**Default port rule:** public and automation URIs must never keep libeufin-style `host:443` / `host:80`. Consumers use `normalize_taler_uri` (lib.sh); bank landing API normalizes in `demo-withdraw-api.py`.
|
||||
|
||||
**Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. Local stack may require content needle (terms/privacy/FADP/GOA…).
|
||||
|
||||
**Performance rule:** Measured from the **monitoring runner** (public via Caddy), not container loopback. Latency (ms) on every perf line + **perf summary** (n / min / p50 / avg / max).
|
||||
|
|
@ -95,10 +100,10 @@ SSH: `KOOPA_SSH` (default `koopa`), then `KOOPA_SSH_FALLBACKS` when LAN is unrea
|
|||
| Group | Checks |
|
||||
|-------|--------|
|
||||
| **e2e.prereq-** | wallet-cli, currency, budgets, secrets, public reachability |
|
||||
| **e2e.load-** | host/container load snapshots (before / after ATM / after pay) |
|
||||
| **e2e.load-** | host/container load snapshots (before / after bank withdraw / after pay) |
|
||||
| **e2e.bank-** | account, credit, withdraw ops |
|
||||
| **e2e.wallet-** | exchange + ToS, wallet setup |
|
||||
| **e2e.atm-** | ATM withdraw ladder rungs |
|
||||
| **e2e.bankwd-** | Bank withdraw ladder rungs |
|
||||
| **e2e.settle-** | wallet settlement wait / spendable balance |
|
||||
| **e2e.pay-** | variable payments |
|
||||
| **e2e.shop-** | GOA shop product catalog pays |
|
||||
|
|
@ -107,7 +112,7 @@ SSH: `KOOPA_SSH` (default `koopa`), then `KOOPA_SSH_FALLBACKS` when LAN is unrea
|
|||
| **e2e.report-** | final tallies |
|
||||
|
||||
When filing an issue, quote the full id + label, e.g.
|
||||
`e2e.atm-03 ATM withdraw 5 GOA · bank confirmed, wallet empty`.
|
||||
`e2e.bankwd-03 Bank withdraw 5 GOA · bank confirmed, wallet empty`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -124,7 +129,7 @@ When filing an issue, quote the full id + label, e.g.
|
|||
| **aptdeploy.trixie-upgrade-** | upgrade-track `…-trixie-upgrade` (install once, then mytops-style upgrade) |
|
||||
| **aptdeploy.trixie-testing-upgrade-** | upgrade-track `…-trixie-testing-upgrade` |
|
||||
|
||||
Standalone: `host-agent/run-aptdeploy.sh` (on koopa local; else `ssh koopa-external`).
|
||||
Standalone: `host-agent/run-aptdeploy.sh` (on koopa local; else ssh `$DEPLOY_SSH` / `$APT_DEPLOY_SSH`).
|
||||
|
||||
### surface (remote-only public inventory)
|
||||
|
||||
|
|
@ -139,7 +144,7 @@ Standalone: `host-agent/run-aptdeploy.sh` (on koopa local; else `ssh koopa-exter
|
|||
- OSV CVE query when package+version known → **ERROR** on hits
|
||||
- Never SSH / never podman on the *targets*
|
||||
| **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component |
|
||||
| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder (GOA ceiling or stage TESTPAYSAN `max_wire`) |
|
||||
| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder classic or max-search (any currency; stage uses `max_wire`) |
|
||||
| **server-** | SSH host ports / processes (flat unless grouped later) |
|
||||
| **auth401.*** | merchant 401 / case matrix (WebUI APIs): **signup** `POST /instances` MIX id, **login** Basic user low vs MIX/UPPER, full **case** matrix, **password** bad/empty, **pwchange** `POST private/auth` + re-login, **admin-create** `POST /management/instances` (if admin token), **bearer**, **durable mon401**, **webui** SPA `toLowerCase` probe (`check_auth401.sh`) |
|
||||
|
||||
|
|
@ -162,7 +167,7 @@ Skip SPA block: `CHECK_WEBUI_SPA=0`.
|
|||
| **`full`** | `urls inside versions sanity server e2e ladder auth401` |
|
||||
|
||||
```bash
|
||||
cd ~/src/koopa/src/taler-monitoring
|
||||
cd ~/src/taler-monitoring
|
||||
# secrets.env with bank admin + merchant tokens (see secrets.env.example)
|
||||
./taler-monitoring.sh -d hacktivism.ch full
|
||||
# same, explicit:
|
||||
|
|
@ -191,3 +196,14 @@ totals: 40 OK, 1 ERROR, 2 WARN, 5 INFO
|
|||
|
||||
`NO_COLOR=1` or `CLICOLOR=0` disables boxes/colours (ASCII `[ OK ]` + `-- group --` headers).
|
||||
Always use `printf --` friendly plain headers when colour is off (leading `---` is not a printf option).
|
||||
|
||||
|
||||
## devtesting · fake-franken (v1.13+)
|
||||
|
||||
| ID prefix | What |
|
||||
|-----------|------|
|
||||
| **devtesting.cli-** | SSH to rusty · `taler-devtesting` CLI |
|
||||
| **devtesting.geniban-** | synthetic CH IBAN |
|
||||
| **devtesting.credit-payto-** | exchange credit IBAN for nexus |
|
||||
| **devtesting.fake-incoming-** / **baseline-** | CHF IN via nexus; bounce missing reserve pub = plumbing OK |
|
||||
| **devtesting.ladder-** | CHF fake-incoming amount ladder (default 23 rungs → CHF:50000; synthetic geniban debit each rung) |
|
||||
|
|
|
|||
2
VERSION
2
VERSION
|
|
@ -1 +1 @@
|
|||
1.2.0
|
||||
1.23.4
|
||||
|
|
|
|||
149
VERSIONS.md
149
VERSIONS.md
|
|
@ -3,34 +3,145 @@
|
|||
Release history for the standalone **taler-monitoring** suite
|
||||
(https://git.hacktivism.ch/hernani/taler-monitoring).
|
||||
|
||||
## Version scheme
|
||||
|
||||
Tags and `VERSION` use **MAJOR.FEATURE.FIX** (three components), introduced in **v1.4.0**:
|
||||
|
||||
| Part | Meaning |
|
||||
|------|---------|
|
||||
| **MAJOR** | Breaking changes to CLI, layout, or host-agent contract |
|
||||
| **FEATURE** | New capabilities (phases, HTML features, i18n packs, …) |
|
||||
| **FIX** | Bugfixes and docs that do not add a feature |
|
||||
|
||||
Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` prefix.
|
||||
|
||||
| Tag | Date (UTC) | Notes |
|
||||
|-----|------------|--------|
|
||||
| **v1.1** | 2026-07-18 | **i18n**: default UI/console language **English**; **French** for FrancPaysan (`TALER_MON_LANG=fr`, auto on `*lefrancpaysan*`). Sticky bar “what this monitors” EN/FR; badge/summary chrome + message translation via `i18n.sh`. |
|
||||
| **v1.0** | 2026-07-18 | Initial standalone release. Split from `koopa-admin-log` (`scripts/taler-monitoring/`). Host-agent HTML, surface, aptdeploy, sticky bar. |
|
||||
| **v1.23.4** | 2026-07-19 | **Bugfix / mon:** shared **normalize_taler_uri** (strip taler:// host :443/:80); ladder/e2e/pay use it. **urls** scans bank+exchange+merchant intros + demo-withdraw/auto-account/stats for default ports (**ERROR** on GOA). Unit test tests/test_normalize_taler_uri.sh. |
|
||||
| **v1.23.3** | 2026-07-19 | **Bugfix:** payments stale because max-ladder never got coins — root cause on koopa was **taler-wallet-cli missing** (all mint-OK → SKIP_ACCEPT). Hard-fail ladder prereq if CLI/helper missing; settle keeps polling after transfer_done + short run-pending; pay wait default 180s; max-ladder wrapper sets PATH/WALLET_CLI. |
|
||||
| **v1.23.2** | 2026-07-19 | **Bugfix:** max-ladder host-agent forces **RUN_TIMEOUT=10800** (koopa env had 600 → run aborted mid-ladder); install starts oneshots with **--no-block**. |
|
||||
| **v1.23.1** | 2026-07-19 | **Fix / scope:** only **`/monitoring-max-ladder/`** for GOA (drop classic `/monitoring-ladder`). Daily timer + install first-run. monpages catalog/Caddy bare redirs for max-ladder only. |
|
||||
| **v1.23.0** | 2026-07-19 | **Feature:** GOA daily mon page **`/monitoring-max-ladder/`** (max-search) on bank/exchange/taler.hacktivism.ch *(1.23.0 briefly also had classic ladder; removed in 1.23.1)*. |
|
||||
| **v1.22.1** | 2026-07-19 | **Bugfix (UX):** pay wait after withdraw no longer looks hung — shows **withdraw board in start order** (OK/OK_BANK from TSV) with live wallet status (`pendingIncoming` → `done`) + timer + aggregate available/pendingIncoming on the **same** wallet DB. Short `run-pending` nudge only (no run-until-done). Default wait 90s (`LADDER_PAY_WAIT_AVAILABLE_S`). |
|
||||
| **v1.22.0** | 2026-07-19 | **Feature:** modular amount ladder — `ladder/lib_pay.sh` (pay) + withdraw in `check_amount_ladder.sh`; `ladder/extract_rpubs.py`. **max-ladder** hunts highest **withdrawable** and highest **payable** (≤ wallet available). GOA pays use public free-amount template **`goa-free`** (`LADDER_FREE_TEMPLATE`). Classic still default 23-rung wd+pay. Phase alias `goa-ladder` → amount ladder (shim removed). |
|
||||
| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external extract (no stdin SyntaxError); force-select multi-rpub; soft `CEILING_REJECT` on mint 5110/P0001; crash-proof report. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. |
|
||||
| **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. *(Superseded notes folded into **v1.21.0** feature release.)* |
|
||||
| **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. |
|
||||
| **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. |
|
||||
| **v1.19.0** | 2026-07-19 | **Feature:** domains.conf **`lang`** column (en\|fr) is the global UI language default per stack; i18n + HTML read `TALER_DOMAIN_LANG` / conf (FP profiles `lang=fr`). |
|
||||
| **v1.18.9** | 2026-07-19 | **Bugfix:** UI lang — *lefrancpaysan*/*francpaysan* always default **fr** unless `TALER_MON_LANG_SET=1` / `--lang`; workstation `TALER_MON_LANG=en` no longer locks English (host-agent + console_to_html). |
|
||||
| **v1.18.8** | 2026-07-19 | **Bugfix:** e2e wallet — require `taler-helper-sqlite3` on PATH; use absolute `node` + `timeout` instead of fragile `perl -e alarm` for wcli; clearer accept-uri errors (no raw alarm-shift noise). |
|
||||
| **v1.18.7** | 2026-07-19 | **Bugfix:** mail/surface catalogs — TSA/Anastasis MX is **mail.anastasis.lu** (→ pixel), not non-resolving `anastasis.taler-systems.com`; add `anastasis.lu` + pixel host probes (fixes surface_err mail.*). **Also:** FP stage host-agent defaults include **e2e** + 1800s timeout; stagepaysan.env.example wires francpaysan-secrets + wallet-cli. |
|
||||
| **v1.18.6** | 2026-07-19 | **Bugfix:** TESTS.md table cell for `e2e.bankwd-` markdown typo from rename. |
|
||||
| **v1.18.5** | 2026-07-19 | **Bugfix:** TESTS.md e2e group list `atm` → `bankwd` (missed in 1.18.4). |
|
||||
| **v1.18.4** | 2026-07-19 | **Bugfix:** e2e group id **atm → bankwd** (`e2e.bankwd-*`); finish ATM wording cleanup so mon logs match **Bank withdraw** (no leftover “atm” phase label). Env `E2E_ATM_MAX` still accepted as alias. |
|
||||
| **v1.18.3** | 2026-07-19 | **Bugfix / wording:** e2e user-facing **ATM withdraw** → **Bank withdraw** (amounts still CHF-style cash-note ladder 20/50/100/200/… — “ATM” was only a metaphor for those note sizes, not a physical ATM). Env: `E2E_BANK_WITHDRAW_MAX` (legacy alias `E2E_ATM_MAX`). Internal group id `e2e.atm-*` kept for check-id stability. |
|
||||
| **v1.18.2** | 2026-07-19 | **Bugfix:** never publish / accept **stale mon HTML assets** — refuse pre-1.15.8 box-drawing sticky JS in site-gen + generated pages; monpages ERROR on public pages still carrying that JS or teed `SyntaxWarning: invalid escape sequence`; purge `site-gen/__pycache__` after suite reset; convert with `PYTHONWARNINGS=error::SyntaxWarning`. |
|
||||
| **v1.18.1** | 2026-07-19 | **Bugfix:** (1) mon HTML sticky/err counts honor French badges **ERREUR** / **AVERT** (were plain → 0 errors on FP pages); (2) `stage-lfp` + `INSIDE_PODMAN=1` uses **host-podman** (no empty `INSIDE_SSH` fail); (3) reaffirm filter JS has no box-glyph/`\\-` regex (charCodeAt only — kills SyntaxWarning “geheimsprache” noise). |
|
||||
| **v1.18.0** | 2026-07-19 | **Feature:** mattermost — client compatibility matrix: **mobile**, **desktop**, **vendor support** floors (default ≥10.11.0, ERROR) + **web** WARN on EOL; env `MATTERMOST_DESKTOP/SUPPORT_MIN_SERVER`, `MATTERMOST_CLIENT_CHECK`. |
|
||||
| **v1.17.0** | 2026-07-19 | **Feature:** mattermost phase — check **server version** (`X-Version-Id`) against **mobile store min** (default **≥10.11.0**); ERROR when too old (newest Android/iOS apps break); env `MATTERMOST_MOBILE_MIN_SERVER` / `MATTERMOST_MOBILE_CHECK=0`. |
|
||||
| **v1.16.0** | 2026-07-19 | **Feature:** mon sticky — blue **info** + green **ok** filter badges; suite **version tag under status pill** (not blue badge); filters `#filter-info` / `#filter-ok`. |
|
||||
| **v1.15.8** | 2026-07-19 | **Bugfix:** mon HTML filter JS — drop literal box-drawing / `\-` regex that caused Python SyntaxWarning (and weird glyphs in suite logs); use Unicode code-point loop instead. |
|
||||
| **v1.15.7** | 2026-07-19 | **Bugfix:** mon HTML — do not paint «perf/ladder summary» as SUMMARY header (restore 2 INFO lines); drop converter SyntaxWarning/noise from public log; «wrote» on stderr; host-agent PYTHONWARNINGS for convert. |
|
||||
| **v1.15.6** | 2026-07-19 | **Bugfix:** aptdeploy ERROR jump list is self-contained — systemd httpd failures include before/after state, Result/SubState, failed units, and journal snippets (no more vague «see systemd lines»); ldd/--version errors embed concrete missing libs / exit output. |
|
||||
| **v1.15.5** | 2026-07-19 | **Bugfix:** devtesting CHF ladder — `ssh` was draining ladder stdin (only rung 1/23 ran → progress 45/98 then snap 52/52); use `ssh -n` + `mapfile`; incomplete ladder is WARN not false complete. |
|
||||
| **v1.15.4** | 2026-07-19 | **Bugfix:** French sticky titles — page label *surveillance*, stack summary *public + interne*, SUMMARY verdict/totals via i18n (`tout est clair`, `totaux :`). |
|
||||
| **v1.15.3** | 2026-07-19 | **Bugfix:** console language auto-**fr** for `*lefrancpaysan*` domains again — env `TALER_MON_LANG=en` no longer locks English; only `--lang` / `TALER_MON_LANG_SET=1` is explicit. |
|
||||
| **v1.15.2** | 2026-07-19 | **Bugfix:** warn/error filter — drop blank log lines from HTML; compact console (no black voids above/mid/below); context skips SUMMARY chrome & pure box frames; tighter filter-gap. |
|
||||
| **v1.15.1** | 2026-07-19 | **Bugfix:** mon HTML — phase `section()` boxes no longer painted as SUMMARY (were blue/green `sum-header`); only real SUMMARY rows get solid severity backgrounds (OK/WARN/ERROR/INFO). |
|
||||
| **v1.15.0** | 2026-07-19 | **Feature:** CHF **fake-incoming amount ladder** in `devtesting` (`DEVTESTING_LADDER=1`, default **23** log-spaced rungs **CHF:0.01 → CHF:50000**); synthetic geniban debit funds each rung; baseline plumbing probe kept. |
|
||||
| **v1.14.0** | 2026-07-19 | **Feature:** SUMMARY box rows are **colour-distinct** (per-severity badge + tinted background + bold counts; verdict-coloured header); HTML mon pages style `sum-ok` / `sum-warn` / `sum-err` / … panels the same way. |
|
||||
| **v1.13.13** | 2026-07-19 | **Bugfix:** progress estimate for **devtesting** / fake-franken is 8 (happy-path numbered checks), not 10 — avoids mid-run 42/48 then end snap 46/46. |
|
||||
| **v1.13.12** | 2026-07-19 | **Bugfix:** SUMMARY chrome (`[ WARN ] 2`, `warnings only`, OK/ERROR counts without `#NNN`) is **meta**, not warn/error — sticky counts + warn filter no longer double SUMMARY rows; filter context skips blank lines. |
|
||||
| **v1.13.11** | 2026-07-19 | **Bugfix:** sticky error/warn filter keeps **±3 dimmed context lines** around each match (gray/grayscale) so phase/group context stays readable; dashed gap between distant clusters. |
|
||||
| **v1.13.10** | 2026-07-19 | **Bugfix:** multi-phase runs print **one global SUMMARY** at parent end (aggregated OK/ERROR/WARN/INFO/BLOCK via `TALER_MON_STATE`); no phase SUMMARY at all (not even last phase); mid-phase still lists ERRORS/BLOCKERS; solo `check_*.sh` unchanged. |
|
||||
| **v1.13.9** | 2026-07-19 | **Bugfix:** no mid-run SUMMARY / progress standings between phases (no more 34/52 + SUMMARY after urls while monpages still runs); full stand only on last phase or solo check script; `progress_finish` remains. |
|
||||
| **v1.13.8** | 2026-07-19 | **Bugfix:** monpages **TOP** meta/marker failures (missing generated/version-link/taler-mon-page, empty race body) are **WARN only** — no longer self-red the run; optional `MONPAGES_TOP_STRICT=1` restores ERROR. |
|
||||
| **v1.13.7** | 2026-07-19 | **Bugfix:** move optional/meta scripts under `meta/` (deprecated mail/mm units, firecuda outside-runner, legacy multi-host site-gen); production path stays host-agent + `site-gen/console_to_html.py`. |
|
||||
| **v1.13.6** | 2026-07-19 | **Bugfix:** machine paths and SSH hosts only via env (`~/.config/taler-monitoring/env` / `taler-monitoring-env`); suite loads env before defaults; no `/home/hernani` or bogus host hardcodes. |
|
||||
| **v1.13.5** | 2026-07-19 | **Bugfix:** progress bar — env-aware phase estimates (CHECK_BANK/LANDING, mon hosts); re-fit total after each phase (done+remaining); drop fake +12 for host-agent monpages post-check (fixes stage 42/139→46/46 snap). |
|
||||
| **v1.13.4** | 2026-07-19 | **Bugfix:** monpages race-proof — atomic HTML write; rsync `--delay-updates` + settle; fetch retries; larger head windows; pre-publish soft content (`MONPAGES_PRE_PUBLISH`); bootstrap markers. |
|
||||
| **v1.13.3** | 2026-07-19 | **Bugfix:** sticky **generated** time in **Europe/Zurich** (CEST/CET · MESZ/MEZ), not UTC `…Z`; ISO keeps offset for age JS. |
|
||||
| **v1.13.2** | 2026-07-19 | **Bugfix:** sticky version link uses **commit** URL (`/src/commit/<sha>`) not `/src/tag/v…` (tags not yet pushed → Forgejo fat 404); tags v1.11–v1.13.1 pushed to origin. |
|
||||
| **v1.13.1** | 2026-07-19 | **Bugfix:** progress bar **global** done/total — phase `summary()` no longer snaps total to phase-local N (fixes 34/34 then 38/47); final `progress_finish` snaps once; header shows 0/N. |
|
||||
| **v1.13.0** | 2026-07-19 | **Feature:** phase **devtesting** / **franken** — fake-franken CHF via `rusty.taler-ops.ch` (`taler-devtesting` geniban + fake-incoming); stage host-agent can include it. |
|
||||
| **v1.12.0** | 2026-07-19 | **Feature:** collapsible **monitoring env context** (agent header: env_file, suite pin, domain/hosts/flags) — `<details>` default closed; summary shows agent · domain · suite. |
|
||||
| **v1.11.0** | 2026-07-19 | **Feature:** sticky **error/warn filter** — click counts to show only error or only warn log lines; sticky bar + overviews stay; `#filter-error` / `#filter-warn` (+ `#first-*` still works). |
|
||||
| **v1.10.11** | 2026-07-19 | **Bugfix:** surface/aptdeploy wrappers **force** MON_HOSTS+PHASES+MONPAGES_INVENTORY (env cannot expand job to bank/exchange); wrap more mon knobs. |
|
||||
| **v1.10.10** | 2026-07-19 | **Bugfix:** monpages bare code 21 is WARN again by default (slash form remains ERROR); bare redir still needs live Caddy `redir *` — set MONPAGES_BARE_STRICT=1 to enforce. |
|
||||
| **v1.10.9** | 2026-07-19 | **Bugfix:** stats policy INFO no longer says bare “ERROR” (false #first-error on www.stats-01); extract_errors only from `┌ ERROR` badges. |
|
||||
| **v1.10.8** | 2026-07-19 | **Docs/clarity:** only three public mon URL families (`/monitoring`, `/taler-monitoring-surface`, `/taler-monitoring-aptdeploy`) each bare+slash(+_err); suite name ≠ path; no `/taler-monitoring` page. |
|
||||
| **v1.10.7** | 2026-07-19 | **Bugfix:** stop always-rsync of **stale suite-overlay** over Forgejo tip (broke stage sticky / new site-gen). Overlay only when suite missing phases. |
|
||||
| **v1.10.6** | 2026-07-19 | **Fix:** stage mon HTML quality — suite update out of public log; filter git noise; stage-specific sticky (not GOA); `CHECK_BANK=0` / `MERCHANT_REQUIRED` for mytops stage scope. |
|
||||
| **v1.10.5** | 2026-07-19 | **Fix:** aptdeploy monpages **job-only** (own `/taler-monitoring-aptdeploy*`); full GOA mon inventory + bare URL checks stay on **hacktivism** general run — no bank/exchange/surface monpages noise on aptdeploy HTML. |
|
||||
| **v1.10.4** | 2026-07-19 | **Fix:** suite **auto-upgrade strict** (`SUITE_UPDATE_STRICT=1` default — abort on fetch fail); monpages ERROR on **bootstrap** HTML; ship mytops-stage timer/service units; host-agent refuses stale tree. |
|
||||
| **v1.10.3** | 2026-07-19 | **Bugfix:** sticky ERROR only from check badges (`┌ ERROR`) / `ERROR monpages`; ignore git `HEAD is now at …` commit subjects that mention ERROR. |
|
||||
| **v1.10.2** | 2026-07-19 | **Bugfix:** sticky ERROR count — classify with word-boundary `\bERROR\b` (not substring); avoids false red from git “wrote-errors” commit lines and similar. |
|
||||
| **v1.10.1** | 2026-07-19 | **Bugfix:** HTML converter must not treat teed `wrote … (errors=N)` lines as ERROR (false sticky red after clean host-agent runs). |
|
||||
| **v1.10.0** | 2026-07-19 | **Feature:** aptdeploy **container overview board** — inventory + per-pod matrix (state, merchant, httpd, ldd, OK/ERROR) in log and HTML sticky/top (`#container-overview`). |
|
||||
| **v1.9.4** | 2026-07-19 | **Fix:** restore **taler-monitoring-aptdeploy(+_err)** as first-class GOA mon page (catalog + monpages path allow + surface docs). Mail/mattermost stay folded into surface. |
|
||||
| **v1.9.3** | 2026-07-19 | **Bugfix (suite):** monpages merchant **code 21** always ERROR (bare no longer soft-WARN); apply hints use absolute `/home/hernani/koopa-caddy/…`; ship `host-agent/apply-monitoring-live.sh` (smoke array, bare check); fix ROOT-ON-KOOPA wrong `redir /path/ 302` examples. |
|
||||
| **v1.9.2** | 2026-07-19 | **Bugfix:** Caddy bare-path redir footgun — inside `handle` use `redir * /path/ 302` (not `redir /path/ 302`, which becomes `Location: 302` and bare URLs fall through to merchant **code 21**). Snippet + ROOT-APPLY: absolute apply path (`/home/hernani/koopa-caddy/…`, not `~` as root). |
|
||||
| **v1.9.1** | 2026-07-19 | **Bugfix:** stale path confusions in repo — sticky `pages_i3` no longer lists mail/mattermost/aptdeploy mon pages; lib.sh/android-test drop `scripts/taler-monitoring/`; monpages/host-agent hints match v1.8 layout; clarify GIT vs GUI vs CLI `*AUTOMATION-NOTES.md` (three files by design). |
|
||||
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
|
||||
| **v1.8.0** | 2026-07-19 | **Simplified public pages:** only `/taler-monitoring-surface(+_err)` on `taler.hacktivism.ch` for ecosystem software/versions; **9 landing stacks** each keep `/monitoring(+_err)`. Mail (firefly.gnunet.org + anastasis.taler-systems.com) and Mattermost folded into surface job; separate mail/mattermost mon pages + timers deprecated. **nmap** OS fingerprinting on surface (`SURFACE_NMAP`); **ERROR** when packages are behind (`TALER_PKG_BEHIND=error` default) or OS fingerprint is EOL. |
|
||||
| **v1.8.1** | *planned* | **L10n** packs: **fr-CH** (FrancPaysan) and **de-CH** (hacktivism / Taler CH German). |
|
||||
| **v1.7.7** | 2026-07-18 | **domains:** stage mytops merchant FQDN is `stage.my.taler-ops.ch` (betel mon sites). |
|
||||
| **v1.9.1** | 2026-07-19 | **Bugfix:** stale path confusions in repo — sticky `pages_i3` no longer lists mail/mattermost/aptdeploy mon pages; lib.sh/android-test drop `scripts/taler-monitoring/`; monpages/host-agent hints match v1.8 layout; clarify GIT vs GUI vs CLI `*AUTOMATION-NOTES.md` (three files by design). |
|
||||
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
|
||||
| **v1.8.0** | 2026-07-18 | **monpages obligatory (ERROR):** GOA checks full suite inventory (bank/exchange/taler `/monitoring/` + surface/aptdeploy/mattermost/mail + on-disk discovery); FP only its own mon hosts; `MONPAGES_REQUIRE_PUBLIC=1` default (escape hatch `=0`). **Bugfix:** `with_timeout` no longer re-runs a phase when it exits non-zero. |
|
||||
| **v1.9.1** | 2026-07-19 | **Bugfix:** stale path confusions in repo — sticky `pages_i3` no longer lists mail/mattermost/aptdeploy mon pages; lib.sh/android-test drop `scripts/taler-monitoring/`; monpages/host-agent hints match v1.8 layout; clarify GIT vs GUI vs CLI `*AUTOMATION-NOTES.md` (three files by design). |
|
||||
| **v1.9.0** | 2026-07-19 | **monpages content:** ERROR if page missing or markers incomplete — top (sticky-bar/status-bar/generated) + bottom (footer/`taler-mon:bottom`); mid-run (progress <100% / incomplete meta) relaxes bottom only. Surface required on normal GOA runs with meaningful body (not progress-only). HTML emits `taler-mon:top` / `taler-mon:bottom:complete|incomplete`. |
|
||||
| **v1.8.0** | 2026-07-18 | **Bugfix:** monpages soft mode `MONPAGES_REQUIRE_PUBLIC=0` (FP wrappers default) — missing public HTML is WARN until vhost publish; generic publish hints. |
|
||||
| **v1.7.4** | 2026-07-18 | **Bugfix:** monpages bare URLs (`/monitoring` without `/`) WARN by default (trailing slash remains ERROR); avoids false red when slash form is live. |
|
||||
| **v1.7.3** | 2026-07-18 | Monitoring **pages themselves** as suite scope: README section + sticky “What this monitors” block (monpages / public HTML paths). |
|
||||
| **v1.7.2** | 2026-07-18 | **Bugfix:** Caddy static handles — `root` must be host dir + matcher `/monitoring*` (not leaf dir + `/monitoring/`), else file_server returns empty 404 after apply. |
|
||||
| **v1.7.1** | 2026-07-18 | Host-agent default `STRICT_EXIT=1`; monpages staging-vs-public diagnosis; re-htmlify after public check fails. |
|
||||
| **v1.7.0** | 2026-07-18 | CLI `--ver` / `--version` / `-V`: suite version, git commit, repo/tag/commit URLs. |
|
||||
| **v1.6.0** | 2026-07-18 | **Mail monitoring:** phase `mail` (firefly + pixel/TSA); MX/SMTP/IMAP/SPF/DMARC; `/taler-monitoring-mail*`. |
|
||||
| **v1.5.0** | 2026-07-18 | **Mattermost monitoring:** phase `mattermost` for `mattermost.taler.net`; `/taler-monitoring-mattermost*`. |
|
||||
| **v1.4.1** | 2026-07-18 | Generic docs (no client hostnames); canonical install path `~/src/taler-monitoring`. |
|
||||
| **v1.4.0** | 2026-07-18 | **MAJOR.FEATURE.FIX** version scheme formalized (three-component tags + `VERSION` file). |
|
||||
| **v1.3.4** | 2026-07-18 | Summary i18n, monpages code-21 detection, aptdeploy rsync path. |
|
||||
| **v1.3.3** | 2026-07-18 | GIT notes vs GUI notes; monpages in TESTS. |
|
||||
| **v1.3.2** | 2026-07-18 | Suite paths only `~/src/taler-monitoring` (no admin-log suite tree). |
|
||||
| **v1.3.1** | 2026-07-18 | monpages FQDN check + deploy no longer silent-skip. |
|
||||
| **v1.3.0** | 2026-07-18 | Sticky bar suite version + Forgejo tag link. |
|
||||
| **v1.2.0** | 2026-07-18 | Language selection: `--lang` / `TALER_MON_LANG` (en\|fr). |
|
||||
| **v1.1.0** | 2026-07-18 | i18n chrome: English default; French for FrancPaysan. |
|
||||
| **v1.0.0** | 2026-07-18 | Initial standalone release. |
|
||||
|
||||
## Language (v1.1+)
|
||||
## Sticky bar version (v1.3.0+)
|
||||
|
||||
| Context | Language |
|
||||
|---------|----------|
|
||||
| Default / GOA / hacktivism | **English** |
|
||||
| FrancPaysan prod + stage (`*lefrancpaysan*`) | **French** (auto or `TALER_MON_LANG=fr`) |
|
||||
Generated HTML shows the installed release tag (e.g. `v1.8.0`) and links to Forgejo:
|
||||
|
||||
```bash
|
||||
# force
|
||||
TALER_MON_LANG=fr ./taler-monitoring.sh -d lefrancpaysan.ch urls
|
||||
TALER_MON_LANG=en ./taler-monitoring.sh -d lefrancpaysan.ch urls
|
||||
```
|
||||
`https://git.hacktivism.ch/hernani/taler-monitoring/src/tag/<tag>`
|
||||
|
||||
Host-agent env examples:
|
||||
- `host-agent/env/francpaysan.env.example` → `TALER_MON_LANG=fr`
|
||||
- `host-agent/env/stagepaysan.env.example` → `TALER_MON_LANG=fr`
|
||||
- `host-agent/env/hacktivism.env.example` → `TALER_MON_LANG=en`
|
||||
Resolved via `git describe --tags` / `VERSION`.
|
||||
|
||||
## Public monitoring pages (v1.3.1+)
|
||||
|
||||
Publish under `DEPLOY_WWW_ROOT` and Caddy `handle`s. Missing handles → merchant **JSON code 21**.
|
||||
See koopa-admin-log issue **I-09** / `apply-monitoring-live.sh`.
|
||||
|
||||
| Check | How |
|
||||
|-------|-----|
|
||||
| Phase | `./taler-monitoring.sh monpages` |
|
||||
| CLI version | `./taler-monitoring.sh --ver` |
|
||||
|
||||
## Language (v1.2.0+)
|
||||
|
||||
| Method | Example |
|
||||
|--------|---------|
|
||||
| CLI | `./taler-monitoring.sh --lang fr …` |
|
||||
| Env | `TALER_MON_LANG=en` or `fr` |
|
||||
| Auto | `*lefrancpaysan*` → fr, else en |
|
||||
|
||||
Full locale packs (fr-CH / de-CH): **v1.8.0** (planned).
|
||||
|
||||
## Pin a release
|
||||
|
||||
```bash
|
||||
git clone https://git.hacktivism.ch/hernani/taler-monitoring.git ~/src/taler-monitoring
|
||||
cd ~/src/taler-monitoring && git checkout v1.1
|
||||
cd ~/src/taler-monitoring && git checkout v1.9.1
|
||||
```
|
||||
|
||||
Default host-agent tracks `origin/main`. Pin with `SUITE_GIT_REF=v1.1` in `~/.config/taler-monitoring/env`.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,10 @@
|
|||
# GUI automation notes (Android wallet)
|
||||
|
||||
|
||||
Companion docs:
|
||||
- CLI / wallet-cli: [`../CLI-AUTOMATION-NOTES.md`](../CLI-AUTOMATION-NOTES.md)
|
||||
- Git / tags / suite tree: [`../VERSIONS.md`](../VERSIONS.md)
|
||||
|
||||
Canonical notes for **graphical** Android wallet tests against GOA / stage.
|
||||
|
||||
---
|
||||
|
|
@ -87,7 +92,7 @@ Host-CLI-Settlement.
|
|||
- iOS / Dual-Platform → andere Repos/Flags, nicht diese Skripte.
|
||||
|
||||
```bash
|
||||
cd scripts/taler-monitoring/android-test
|
||||
cd ~/src/taler-monitoring/android-test
|
||||
|
||||
# Minimal hybrid (install + deep-link + light taps)
|
||||
STACK=goa ./run-android-pay-smoke.sh
|
||||
|
|
@ -111,7 +116,7 @@ GUI=1 ./run-android-build-and-smoke.sh
|
|||
## Platform capability flags (planned + defaults)
|
||||
|
||||
Automation can run on hosts that support **Android only**, **GUI tooling only**
|
||||
(conceptual), or **both**. Flags keep that explicit for CI and laptops.
|
||||
(conceptual), or **both**. Flags keep that explicit for CI and workstations.
|
||||
|
||||
### Proposed env flags
|
||||
|
||||
|
|
@ -147,7 +152,7 @@ with **`-no-window`** and still keep **graphics logic** in the guest:
|
|||
# or let smoke auto-start headless AVD when no phone is plugged in
|
||||
STACK=goa ./run-android-gui-smoke.sh
|
||||
|
||||
# laptop: show the emulator window
|
||||
# ops workstation: show the emulator window
|
||||
WINDOWED=1 ./start-android-emulator.sh --wait
|
||||
# or: EMULATOR_HEADLESS=0 EMULATOR_GPU=host ./start-android-emulator.sh --wait
|
||||
```
|
||||
|
|
|
|||
|
|
@ -104,7 +104,7 @@ Canonical notes (minimal variant, **flags**, Linux defaults):
|
|||
```bash
|
||||
# headless AVD only (default path for servers)
|
||||
./start-android-emulator.sh --wait
|
||||
# windowed (laptops)
|
||||
# windowed (workstations with display)
|
||||
WINDOWED=1 ./start-android-emulator.sh --wait
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -91,7 +91,7 @@ if [[ -z "${EXP_PW_FILE:-}" ]]; then
|
|||
fi
|
||||
# stage: try stagepaysan secret via ssh if missing
|
||||
if [[ ! -f "${EXP_PW_FILE:-}" && "$STACK" = "stage" ]]; then
|
||||
for host in francpaysan-stage-user francpaysan-host; do
|
||||
for host in ${INSIDE_SSH:+"$INSIDE_SSH"} ${FRANCPAYSAN_SSH:+"$FRANCPAYSAN_SSH"}; do
|
||||
tmp=$(mktemp)
|
||||
if ssh -o BatchMode=yes -o ConnectTimeout=10 "$host" \
|
||||
'tr -d "\n\r" </mnt/data/stagepaysan/bank/secrets/bank-explorer-password.txt' \
|
||||
|
|
|
|||
1577
check_amount_ladder.sh
Executable file
1577
check_amount_ladder.sh
Executable file
File diff suppressed because it is too large
Load diff
|
|
@ -9,6 +9,9 @@
|
|||
# koopa-taler-deploy-test-apt-src-trixie-upgrade
|
||||
# koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
|
||||
#
|
||||
# Emits a compact container board (board · key=value) for the HTML sticky/top
|
||||
# overview: which pods are active and which fail (httpd / ldd / …).
|
||||
#
|
||||
# Env:
|
||||
# APT_DEPLOY_CONTAINERS "name:suite:mode …" mode=fresh|upgrade (optional, default fresh)
|
||||
# APT_DEPLOY_SKIP=1
|
||||
|
|
@ -44,6 +47,58 @@ ok "podman" "$("$PODMAN_BIN" --version 2>/dev/null | head -1)"
|
|||
info "host" "$(hostname 2>/dev/null || echo unknown) · whoami=$(whoami)"
|
||||
|
||||
fail_any=0
|
||||
# board rows for end overview (human table + machine board · lines)
|
||||
BOARD_ROWS=()
|
||||
|
||||
_short_name() {
|
||||
# koopa-taler-deploy-test-apt-src-trixie-testing-upgrade → trixie-testing-upgrade
|
||||
local n="$1"
|
||||
n="${n#koopa-taler-deploy-test-apt-src-}"
|
||||
printf '%s' "$n"
|
||||
}
|
||||
|
||||
# Record one container for the top-of-page board (HTML parses "board · key=value").
|
||||
board_record() {
|
||||
local short="$1" name="$2" suite="$3" mode="$4" state="$5"
|
||||
local merchant="$6" httpd="$7" ldd="$8" verdict="$9" note="${10:-}"
|
||||
local kv
|
||||
kv="short=${short} name=${name} suite=${suite} mode=${mode} state=${state}"
|
||||
kv+=" merchant=${merchant} httpd=${httpd} ldd=${ldd} verdict=${verdict}"
|
||||
# note: no spaces (HTML/parser splits on whitespace)
|
||||
if [ -n "$note" ]; then
|
||||
note_safe=$(printf '%s' "$note" | tr ' ' '_' | tr -d '|')
|
||||
kv+=" note=${note_safe}"
|
||||
fi
|
||||
info "board" "$kv"
|
||||
BOARD_ROWS+=("$short|$suite|$mode|$state|$merchant|$httpd|$ldd|$verdict|$note")
|
||||
}
|
||||
|
||||
print_board_table() {
|
||||
section "aptdeploy · container overview (active / errors)"
|
||||
info "board-table" "| short | suite | mode | podman | merchant | httpd | ldd | verdict | note |"
|
||||
info "board-table" "|-------|-------|------|--------|----------|-------|-----|---------|------|"
|
||||
local r short suite mode state merchant httpd ldd verdict note
|
||||
local n_ok=0 n_err=0 n_miss=0
|
||||
for r in "${BOARD_ROWS[@]+"${BOARD_ROWS[@]}"}"; do
|
||||
IFS='|' read -r short suite mode state merchant httpd ldd verdict note <<<"$r"
|
||||
info "board-table" "| ${short} | ${suite} | ${mode} | ${state} | ${merchant} | ${httpd} | ${ldd} | ${verdict} | ${note} |"
|
||||
case "$verdict" in
|
||||
OK|ok) n_ok=$((n_ok + 1)) ;;
|
||||
MISSING|missing) n_miss=$((n_miss + 1)) ;;
|
||||
*) n_err=$((n_err + 1)) ;;
|
||||
esac
|
||||
done
|
||||
info "board-summary" "containers=${#BOARD_ROWS[@]} ok=${n_ok} error=${n_err} missing=${n_miss}"
|
||||
# one-line human skim (INFO/OK only — do not re-emit ERROR, details already counted)
|
||||
for r in "${BOARD_ROWS[@]+"${BOARD_ROWS[@]}"}"; do
|
||||
IFS='|' read -r short suite mode state merchant httpd ldd verdict note <<<"$r"
|
||||
if [ "$verdict" = "OK" ]; then
|
||||
ok "overview" "${short}: ${state} · merchant=${merchant} · httpd=${httpd} · ldd=${ldd}"
|
||||
else
|
||||
info "overview" "${short}: ${verdict} · state=${state} merchant=${merchant} httpd=${httpd} ldd=${ldd}${note:+ · ${note}}"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
print_pkg_table() {
|
||||
local name=$1
|
||||
|
|
@ -58,16 +113,21 @@ print_pkg_table() {
|
|||
done
|
||||
' 2>/dev/null || echo "| (query failed) | |")
|
||||
info "pkg-table" "$name"
|
||||
# emit as plain lines so HTML log keeps the table
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
info "pkg" "$line"
|
||||
done <<<"$rows"
|
||||
}
|
||||
|
||||
# merchant package version only (for board)
|
||||
_merchant_ver() {
|
||||
local name=$1
|
||||
"$PODMAN_BIN" exec "$name" dpkg-query -W -f='${Version}' taler-merchant 2>/dev/null || echo "?"
|
||||
}
|
||||
|
||||
check_ldd_deep() {
|
||||
local name=$1
|
||||
local out
|
||||
local out missing
|
||||
out=$("$PODMAN_BIN" exec "$name" bash -lc '
|
||||
set +e
|
||||
bin=$(command -v taler-merchant-httpd)
|
||||
|
|
@ -87,15 +147,48 @@ check_ldd_deep() {
|
|||
info "ldd" "$line"
|
||||
done <<<"$out"
|
||||
if echo "$out" | grep -q 'not found'; then
|
||||
err "ldd" "shared library missing in $name" "see ldd lines above"
|
||||
missing=$(echo "$out" | grep 'not found' | tr '\n' ' ' | head -c 280)
|
||||
err "ldd" "shared library missing in $name" "${missing:-not found (see ldd INFO above)}"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# Build a single-line diagnosis for err() so jump lists are self-contained
|
||||
# (operators should not need to hunt nearby INFO lines).
|
||||
_apt_systemd_diagnose() {
|
||||
local out="$1"
|
||||
local before after failed journal status_line result
|
||||
before=$(echo "$out" | sed -n 's/^httpd_active: //p' | tail -1)
|
||||
after=$(echo "$out" | sed -n 's/^after_start_httpd: //p' | tail -1)
|
||||
failed=$(echo "$out" | sed -n 's/^failed_taler: //p' | grep -v '^none$' | tr '\n' ';' | sed 's/;$//')
|
||||
[ -z "$failed" ] && failed=$(echo "$out" | grep -E '^\s*●\s+taler-' | head -3 | tr '\n' ';' | sed 's/;$//')
|
||||
result=$(echo "$out" | sed -n 's/^httpd_Result: //p' | tail -1)
|
||||
[ -z "$result" ] && result=$(echo "$out" | sed -n 's/^ *Result: //p' | head -1)
|
||||
status_line=$(echo "$out" | sed -n 's/^httpd_status_line: //p' | tail -1)
|
||||
journal=$(echo "$out" | sed -n 's/^httpd_journal: //p' | tr '\n' ' ' | head -c 220)
|
||||
# fallback: first non-meta journal-ish line
|
||||
if [ -z "$journal" ]; then
|
||||
journal=$(echo "$out" | grep -E 'taler-merchant-httpd\[[0-9]+\]|Failed|error|Error|NRestarts|Main PID' | head -3 | tr '\n' ' ' | head -c 220)
|
||||
fi
|
||||
local parts=()
|
||||
[ -n "$before" ] && parts+=("before=${before}")
|
||||
[ -n "$after" ] && parts+=("after_start=${after}")
|
||||
[ -n "$result" ] && parts+=("Result=${result}")
|
||||
[ -n "$status_line" ] && parts+=("status=${status_line}")
|
||||
[ -n "$failed" ] && [ "$failed" != "none" ] && parts+=("failed_units=${failed}")
|
||||
[ -n "$journal" ] && parts+=("journal=${journal}")
|
||||
if [ "${#parts[@]}" -eq 0 ]; then
|
||||
printf '%s' "no status/journal captured (podman exec empty?)"
|
||||
return
|
||||
fi
|
||||
local IFS=' · '
|
||||
printf '%s' "${parts[*]}"
|
||||
}
|
||||
|
||||
check_systemd() {
|
||||
local name=$1
|
||||
local out active_target active_httpd enabled
|
||||
local out active_httpd before_httpd detail
|
||||
out=$("$PODMAN_BIN" exec "$name" bash -lc '
|
||||
set +e
|
||||
echo "system: $(systemctl is-system-running 2>&1)"
|
||||
|
|
@ -103,31 +196,74 @@ check_systemd() {
|
|||
echo "target_active: $(systemctl is-active taler-merchant.target 2>&1)"
|
||||
echo "httpd_active: $(systemctl is-active taler-merchant-httpd.service 2>&1)"
|
||||
echo "httpd_enabled: $(systemctl is-enabled taler-merchant-httpd.service 2>&1)"
|
||||
systemctl start taler-merchant.target 2>&1 | tail -5
|
||||
sleep 2
|
||||
# unit properties help when is-active is only "activating"/"failed"
|
||||
echo "httpd_Result: $(systemctl show -p Result --value taler-merchant-httpd.service 2>&1)"
|
||||
echo "httpd_SubState: $(systemctl show -p SubState --value taler-merchant-httpd.service 2>&1)"
|
||||
echo "httpd_NRestarts: $(systemctl show -p NRestarts --value taler-merchant-httpd.service 2>&1)"
|
||||
echo "httpd_ExecMainStatus: $(systemctl show -p ExecMainStatus --value taler-merchant-httpd.service 2>&1)"
|
||||
echo "httpd_ExecMainCode: $(systemctl show -p ExecMainCode --value taler-merchant-httpd.service 2>&1)"
|
||||
# one-line status (Active: failed / activating (start) …)
|
||||
systemctl status taler-merchant-httpd.service --no-pager -l 2>&1 | head -12 | while IFS= read -r sl; do
|
||||
echo "httpd_status: $sl"
|
||||
done
|
||||
# compact single-line for err detail
|
||||
st1=$(systemctl status taler-merchant-httpd.service --no-pager -l 2>&1 | sed -n "s/^ *Active: //p" | head -1)
|
||||
echo "httpd_status_line: ${st1:-?}"
|
||||
systemctl start taler-merchant.target 2>&1 | tail -8 | while IFS= read -r sl; do
|
||||
echo "start_target: $sl"
|
||||
done
|
||||
# wait longer: "activating" often clears only after dbinit
|
||||
sleep 5
|
||||
echo "after_start_target: $(systemctl is-active taler-merchant.target 2>&1)"
|
||||
echo "after_start_httpd: $(systemctl is-active taler-merchant-httpd.service 2>&1)"
|
||||
systemctl --failed --no-legend 2>/dev/null | grep -i taler || echo "failed_taler: none"
|
||||
echo "after_httpd_Result: $(systemctl show -p Result --value taler-merchant-httpd.service 2>&1)"
|
||||
echo "after_httpd_SubState: $(systemctl show -p SubState --value taler-merchant-httpd.service 2>&1)"
|
||||
# journal: last failure reasons (self-contained for ERROR jump list)
|
||||
journalctl -u taler-merchant-httpd.service -n 12 --no-pager -o cat 2>/dev/null \
|
||||
| sed "/^$/d" | tail -8 | while IFS= read -r jl; do
|
||||
echo "httpd_journal: $jl"
|
||||
done
|
||||
journalctl -u taler-merchant-dbinit.service -n 6 --no-pager -o cat 2>/dev/null \
|
||||
| sed "/^$/d" | tail -4 | while IFS= read -r jl; do
|
||||
echo "dbinit_journal: $jl"
|
||||
done
|
||||
ft=$(systemctl --failed --no-legend 2>/dev/null | grep -i taler || true)
|
||||
if [ -n "$ft" ]; then
|
||||
echo "$ft" | while IFS= read -r fl; do echo "failed_taler: $fl"; done
|
||||
else
|
||||
echo "failed_taler: none"
|
||||
fi
|
||||
' 2>/dev/null || echo "systemd probe failed")
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
info "systemd" "$line"
|
||||
done <<<"$out"
|
||||
|
||||
before_httpd=$(echo "$out" | sed -n 's/^httpd_active: //p' | tail -1)
|
||||
active_httpd=$(echo "$out" | sed -n 's/^after_start_httpd: //p' | tail -1)
|
||||
# export for caller board
|
||||
APT_LAST_HTTPD="${active_httpd:-?}"
|
||||
if [ "$active_httpd" = "active" ]; then
|
||||
ok "systemd httpd" "taler-merchant-httpd active after start taler-merchant.target"
|
||||
return 0
|
||||
fi
|
||||
# soft if target not shipped as enabled in container — still ERROR for deploy smoke
|
||||
err "systemd httpd" "taler-merchant-httpd not active" "after_start_httpd=${active_httpd:-?} · see systemd lines"
|
||||
detail=$(_apt_systemd_diagnose "$out")
|
||||
# Prefer after_* Result if present
|
||||
local after_result after_sub
|
||||
after_result=$(echo "$out" | sed -n 's/^after_httpd_Result: //p' | tail -1)
|
||||
after_sub=$(echo "$out" | sed -n 's/^after_httpd_SubState: //p' | tail -1)
|
||||
[ -n "$after_result" ] && [ "$after_result" != "success" ] && detail="${detail} · after_Result=${after_result}"
|
||||
[ -n "$after_sub" ] && detail="${detail} · after_SubState=${after_sub}"
|
||||
# Human one-liner for jump list (no "see …" without content)
|
||||
err "systemd httpd" \
|
||||
"taler-merchant-httpd not active (container=${name})" \
|
||||
"${detail}"
|
||||
return 1
|
||||
}
|
||||
|
||||
check_merchant_basics() {
|
||||
local name=$1
|
||||
local out code
|
||||
# unix socket or curl localhost if httpd listens
|
||||
local out
|
||||
out=$("$PODMAN_BIN" exec "$name" bash -lc '
|
||||
set +e
|
||||
echo "=== binaries ==="
|
||||
|
|
@ -135,7 +271,6 @@ check_merchant_basics() {
|
|||
echo "=== --version ==="
|
||||
taler-merchant-httpd --version 2>&1
|
||||
echo "=== config probe ==="
|
||||
# try common paths without TLS
|
||||
for u in \
|
||||
http://127.0.0.1:9966/config \
|
||||
http://127.0.0.1:8081/config \
|
||||
|
|
@ -162,26 +297,24 @@ check_merchant_basics() {
|
|||
done <<<"$out"
|
||||
|
||||
if echo "$out" | grep -qi 'error while loading shared libraries'; then
|
||||
err "httpd" "shared library error on --version" "see basics/ldd"
|
||||
local lib_err
|
||||
lib_err=$(echo "$out" | grep -i 'error while loading shared libraries' | head -1 | tr '\n' ' ' | head -c 240)
|
||||
err "httpd" "shared library error on --version" "${lib_err:-see basics INFO lines}"
|
||||
return 1
|
||||
fi
|
||||
if ! echo "$out" | grep -q 'taler-merchant-httpd --version\|v\.\|taler-merchant-httpd'; then
|
||||
# version line varies; success if exit was ok — check via separate exec
|
||||
:
|
||||
fi
|
||||
|
||||
set +e
|
||||
"$PODMAN_BIN" exec "$name" taler-merchant-httpd --version >/dev/null 2>&1
|
||||
local ec=$?
|
||||
local ver_out ec
|
||||
ver_out=$("$PODMAN_BIN" exec "$name" taler-merchant-httpd --version 2>&1)
|
||||
ec=$?
|
||||
set -e
|
||||
if [ "$ec" -eq 0 ]; then
|
||||
ok "httpd --version" "exit 0"
|
||||
else
|
||||
err "httpd --version" "exit $ec"
|
||||
err "httpd --version" "exit $ec" "$(printf '%s' "$ver_out" | tr '\n' ' ' | head -c 240)"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# /config is nice-to-have if stack is fully configured
|
||||
if echo "$out" | grep -qE 'curl .* -> 200'; then
|
||||
ok "merchant /config" "HTTP 200 (local)"
|
||||
else
|
||||
|
|
@ -192,7 +325,10 @@ check_merchant_basics() {
|
|||
|
||||
check_one() {
|
||||
local name="$1" expect_suite="$2" mode="${3:-fresh}"
|
||||
local st suite_line pkgs
|
||||
local st short merchant="-" httpd="-" ldd="-" verdict="OK" note="" c_fail=0
|
||||
|
||||
short="$(_short_name "$name")"
|
||||
APT_LAST_HTTPD="-"
|
||||
|
||||
# group ids: aptdeploy.trixie-01 / aptdeploy.trixie-testing-upgrade-01
|
||||
set_group "${expect_suite}${mode:+-$mode}"
|
||||
|
|
@ -200,12 +336,14 @@ check_one() {
|
|||
|
||||
if ! "$PODMAN_BIN" container exists "$name" 2>/dev/null; then
|
||||
err "container" "$name missing" "run ensure-apt-deploy-test-containers.sh"
|
||||
board_record "$short" "$name" "$expect_suite" "$mode" "missing" "-" "-" "-" "MISSING" "ensure containers"
|
||||
fail_any=1
|
||||
return
|
||||
fi
|
||||
st=$("$PODMAN_BIN" inspect -f '{{.State.Status}}' "$name" 2>/dev/null || echo unknown)
|
||||
if [ "$st" != "running" ]; then
|
||||
err "container" "$name not running" "status=$st"
|
||||
board_record "$short" "$name" "$expect_suite" "$mode" "$st" "-" "-" "-" "ERROR" "not running"
|
||||
fail_any=1
|
||||
return
|
||||
fi
|
||||
|
|
@ -217,31 +355,48 @@ check_one() {
|
|||
ok "apt suite" "$expect_suite"
|
||||
else
|
||||
err "apt suite" "expected $expect_suite" "got: ${suite_line:-empty}"
|
||||
fail_any=1
|
||||
c_fail=1
|
||||
note="suite mismatch"
|
||||
fi
|
||||
|
||||
merchant="$(_merchant_ver "$name")"
|
||||
print_pkg_table "$name"
|
||||
|
||||
if ! check_merchant_basics "$name"; then
|
||||
fail_any=1
|
||||
check_ldd_deep "$name" || fail_any=1
|
||||
c_fail=1
|
||||
if ! check_ldd_deep "$name"; then
|
||||
ldd="FAIL"
|
||||
c_fail=1
|
||||
else
|
||||
ldd="OK"
|
||||
fi
|
||||
else
|
||||
# still show ldd summary as info when ok
|
||||
ldd_out=$("$PODMAN_BIN" exec "$name" bash -lc \
|
||||
'ldd "$(command -v taler-merchant-httpd)" 2>&1 | grep -E "libtalerutil|not found" || true' 2>/dev/null || true)
|
||||
if echo "$ldd_out" | grep -q 'not found'; then
|
||||
err "libtalerutil" "not found" "$ldd_out"
|
||||
check_ldd_deep "$name" || true
|
||||
fail_any=1
|
||||
ldd="FAIL"
|
||||
c_fail=1
|
||||
else
|
||||
ldd="OK"
|
||||
info "libtalerutil" "$(echo "$ldd_out" | tr '\n' ' ')"
|
||||
fi
|
||||
fi
|
||||
|
||||
if ! check_systemd "$name"; then
|
||||
fail_any=1
|
||||
check_ldd_deep "$name" || true
|
||||
c_fail=1
|
||||
httpd="${APT_LAST_HTTPD:-FAIL}"
|
||||
note="${note:+$note; }httpd=${httpd}"
|
||||
# deeper ldd if httpd failed
|
||||
if [ "$ldd" != "FAIL" ]; then
|
||||
check_ldd_deep "$name" || { ldd="FAIL"; c_fail=1; note="${note:+$note; }ldd"; }
|
||||
fi
|
||||
else
|
||||
httpd="active"
|
||||
fi
|
||||
# normalize httpd cell
|
||||
[ -z "$httpd" ] || [ "$httpd" = "-" ] && httpd="${APT_LAST_HTTPD:-?}"
|
||||
|
||||
# Disk free inside this deploy-test container (+ host once, first container only)
|
||||
if [ "${_APT_DEPLOY_HOST_DISK_DONE:-0}" != "1" ]; then
|
||||
|
|
@ -251,12 +406,39 @@ check_one() {
|
|||
fi
|
||||
set_group disk
|
||||
if ! mon_disk_check_podman "$name" "$PODMAN_BIN"; then
|
||||
fail_any=1
|
||||
c_fail=1
|
||||
note="${note:+$note; }disk"
|
||||
fi
|
||||
|
||||
if [ "$c_fail" -ne 0 ]; then
|
||||
verdict="ERROR"
|
||||
fail_any=1
|
||||
else
|
||||
verdict="OK"
|
||||
fi
|
||||
board_record "$short" "$name" "$expect_suite" "$mode" "running" "$merchant" "$httpd" "$ldd" "$verdict" "$note"
|
||||
}
|
||||
|
||||
# --- inventory first (what exists before checks) ---
|
||||
section "aptdeploy · inventory (podman)"
|
||||
info "inventory" "expected tracks: trixie/trixie-testing × fresh/upgrade (4 containers)"
|
||||
for entry in $LIST; do
|
||||
cname=${entry%%:*}
|
||||
rest=${entry#*:}
|
||||
case "$rest" in
|
||||
*:*) suite=${rest%%:*}; mode=${rest#*:} ;;
|
||||
*) suite=$rest; mode=fresh ;;
|
||||
esac
|
||||
short="$(_short_name "$cname")"
|
||||
if "$PODMAN_BIN" container exists "$cname" 2>/dev/null; then
|
||||
st=$("$PODMAN_BIN" inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || echo unknown)
|
||||
ok "inventory" "${short}: ${st} (${suite}/${mode})"
|
||||
else
|
||||
warn "inventory" "${short}: missing (${suite}/${mode})" "will ERROR in check"
|
||||
fi
|
||||
done
|
||||
|
||||
for entry in $LIST; do
|
||||
# name:suite or name:suite:mode
|
||||
cname=${entry%%:*}
|
||||
rest=${entry#*:}
|
||||
case "$rest" in
|
||||
|
|
@ -272,6 +454,9 @@ for entry in $LIST; do
|
|||
check_one "$cname" "$suite" "$mode"
|
||||
done
|
||||
|
||||
# Compact matrix + overview ERROR lines (jump targets for HTML)
|
||||
print_board_table
|
||||
|
||||
summary
|
||||
if [ "$fail_any" -ne 0 ]; then
|
||||
exit 1
|
||||
|
|
|
|||
337
check_devtesting.sh
Executable file
337
check_devtesting.sh
Executable file
|
|
@ -0,0 +1,337 @@
|
|||
#!/usr/bin/env bash
|
||||
# check_devtesting.sh — fake-franken (CHF) via rusty.taler-ops.ch taler-devtesting
|
||||
#
|
||||
# Requires SSH to Host DEVTESTING_SSH (default: taler-rusty-devtesting →
|
||||
# rusty.taler-ops.ch user devtesting, forced command taler-devtesting).
|
||||
#
|
||||
# Probes:
|
||||
# 1) CLI reachable (fake-incoming --help)
|
||||
# 2) geniban → Swiss IBAN (debit side — “enough money” is synthetic per rung)
|
||||
# 3) baseline fake-incoming CHF:1.00 (plumbing)
|
||||
# 4) optional amount ladder: DEVTESTING_LADDER=1 (default) — 23 rungs up to 50k CHF
|
||||
#
|
||||
# Env:
|
||||
# DEVTESTING_SSH default taler-rusty-devtesting (or user@host)
|
||||
# DEVTESTING_AMOUNT baseline amount (default CHF:1.00)
|
||||
# DEVTESTING_CREDIT_PAYTO exchange IBAN payto
|
||||
# DEVTESTING_EXCHANGE_IBAN fallback CH6808573105529100001 (TOPS CHF exchange)
|
||||
# DEVTESTING_LADDER 1 (default) run amount ladder; 0 = baseline only
|
||||
# DEVTESTING_LADDER_STEPS default 23
|
||||
# DEVTESTING_LADDER_MAX default 50000 (CHF major units)
|
||||
# DEVTESTING_LADDER_MIN default 0.01
|
||||
# DEVTESTING_LADDER_STOP_ON_FAIL 1 (default) stop ladder on hard fail; 0 = continue
|
||||
# DEVTESTING_SKIP=1 skip phase
|
||||
#
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
# shellcheck source=lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
|
||||
set_area devtesting
|
||||
section "devtesting · fake-franken CHF (rusty taler-devtesting)"
|
||||
|
||||
# Do NOT honor SKIP_SSH here: stage/tops profiles set SKIP_SSH=1 for container
|
||||
# inside checks, but rusty.taler-ops.ch is a separate SSH hop (devtesting key).
|
||||
if [ "${DEVTESTING_SKIP:-0}" = "1" ]; then
|
||||
info "skip" "DEVTESTING_SKIP=1"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
: "${DEVTESTING_SSH:=}"
|
||||
: "${DEVTESTING_AMOUNT:=CHF:1.00}"
|
||||
: "${DEVTESTING_EXCHANGE_IBAN:=CH6808573105529100001}"
|
||||
: "${DEVTESTING_LADDER:=1}"
|
||||
: "${DEVTESTING_LADDER_STEPS:=23}"
|
||||
: "${DEVTESTING_LADDER_MAX:=50000}"
|
||||
: "${DEVTESTING_LADDER_MIN:=0.01}"
|
||||
: "${DEVTESTING_LADDER_STOP_ON_FAIL:=1}"
|
||||
: "${SSH_CONNECT_TIMEOUT:=8}"
|
||||
: "${SSH_CMD_TIMEOUT:=60}"
|
||||
|
||||
ssh_dt() {
|
||||
# Prefer Host alias; fall back to explicit user@host if config missing.
|
||||
# Always -n: do not read local stdin (ladder while-read must not be drained by ssh).
|
||||
local target="$DEVTESTING_SSH"
|
||||
if ! ssh -G "$target" >/dev/null 2>&1; then
|
||||
target="devtesting@rusty.taler-ops.ch"
|
||||
fi
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout "${SSH_CMD_TIMEOUT}" ssh -n \
|
||||
-o BatchMode=yes \
|
||||
-o ConnectTimeout="${SSH_CONNECT_TIMEOUT}" \
|
||||
-o StrictHostKeyChecking=accept-new \
|
||||
"$target" "$@"
|
||||
else
|
||||
ssh -n \
|
||||
-o BatchMode=yes \
|
||||
-o ConnectTimeout="${SSH_CONNECT_TIMEOUT}" \
|
||||
-o StrictHostKeyChecking=accept-new \
|
||||
"$target" "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
# Run one fake-incoming. Sets globals: FI_RC FI_FLAT FI_STATUS
|
||||
# FI_STATUS: ok_bounce | ok_in | ok_cli | fail_ssh | fail_creditor | fail_nexus | fail_other
|
||||
run_fake_incoming() {
|
||||
local amount="$1" subject="$2" credit="$3" debit="$4"
|
||||
local fi_out
|
||||
FI_RC=0
|
||||
FI_FLAT=""
|
||||
FI_STATUS="fail_other"
|
||||
fi_out=$(ssh_dt fake-incoming \
|
||||
--amount "${amount}" \
|
||||
--subject "${subject}" \
|
||||
--credit-payto "${credit}" \
|
||||
--debit-payto "${debit}" 2>&1) || FI_RC=$?
|
||||
FI_FLAT=$(printf '%s' "$fi_out" | tr '\n' ' ')
|
||||
|
||||
if printf '%s' "$FI_FLAT" | grep -qiE 'Permission denied|Connection refused|Could not resolve|No such command'; then
|
||||
FI_STATUS=fail_ssh
|
||||
return 1
|
||||
fi
|
||||
if printf '%s' "$FI_FLAT" | grep -qiE 'Creditor must be the exchange'; then
|
||||
FI_STATUS=fail_creditor
|
||||
return 1
|
||||
fi
|
||||
if printf '%s' "$FI_FLAT" | grep -qiE 'Missing amount|unable to read secrets|CalledProcessError'; then
|
||||
if ! printf '%s' "$FI_FLAT" | grep -qE 'libeufin-nexus - IN .* CHF'; then
|
||||
FI_STATUS=fail_nexus
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
if printf '%s' "$FI_FLAT" | grep -qE 'libeufin-nexus - IN .* CHF:[0-9]'; then
|
||||
if printf '%s' "$FI_FLAT" | grep -qiE 'bounced.*missing reserve public key'; then
|
||||
FI_STATUS=ok_bounce
|
||||
return 0
|
||||
fi
|
||||
if printf '%s' "$FI_FLAT" | grep -qiE 'bounced'; then
|
||||
FI_STATUS=ok_bounce
|
||||
return 0
|
||||
fi
|
||||
FI_STATUS=ok_in
|
||||
return 0
|
||||
fi
|
||||
if [ "$FI_RC" -eq 0 ] && printf '%s' "$FI_FLAT" | grep -qi 'Faking incoming CHF'; then
|
||||
FI_STATUS=ok_cli
|
||||
return 0
|
||||
fi
|
||||
FI_STATUS=fail_other
|
||||
return 1
|
||||
}
|
||||
|
||||
set_group cli
|
||||
info "ssh" "DEVTESTING_SSH=${DEVTESTING_SSH} baseline=${DEVTESTING_AMOUNT} ladder=${DEVTESTING_LADDER} steps=${DEVTESTING_LADDER_STEPS} max=CHF:${DEVTESTING_LADDER_MAX}"
|
||||
|
||||
help_out=$(ssh_dt fake-incoming --help 2>&1) || true
|
||||
if printf '%s' "$help_out" | grep -q 'fake-incoming'; then
|
||||
ok "cli" "taler-devtesting fake-incoming reachable via ${DEVTESTING_SSH}"
|
||||
else
|
||||
fail "cli" "cannot run taler-devtesting on ${DEVTESTING_SSH}" \
|
||||
"$(printf '%s' "$help_out" | tr '\n' ' ' | head -c 200)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
set_group geniban
|
||||
iban_out=$(ssh_dt geniban 2>&1) || true
|
||||
iban=$(printf '%s\n' "$iban_out" | grep -E '^CH[0-9A-Z]+$' | tail -1 || true)
|
||||
if [ -n "$iban" ]; then
|
||||
ok "geniban" "synthetic IBAN $iban (debit / “withdraw” side for each rung)"
|
||||
else
|
||||
fail "geniban" "expected CH… IBAN" "$(printf '%s' "$iban_out" | tr '\n' ' ' | head -c 160)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Credit payto: rusty libeufin-nexus only accepts the *production* exchange IBAN
|
||||
# configured there (DEVTESTING_EXCHANGE_IBAN). Do not use stage /keys first-account.
|
||||
set_group credit-payto
|
||||
credit_payto="${DEVTESTING_CREDIT_PAYTO:-}"
|
||||
if [ -z "$credit_payto" ]; then
|
||||
credit_payto="payto://iban/${DEVTESTING_EXCHANGE_IBAN}?receiver-name=Taler+Operations+AG"
|
||||
fi
|
||||
ok "credit-payto" "$credit_payto"
|
||||
|
||||
debit_payto="payto://iban/${iban}?receiver-name=MonDevTest"
|
||||
ts_base=$(date -u +%Y%m%d%H%M%S)
|
||||
|
||||
# --- baseline plumbing probe ---
|
||||
set_group baseline
|
||||
subject="mon-fake-franken-${ts_base}-base-$$"
|
||||
info "fake-incoming" "amount=${DEVTESTING_AMOUNT} subject=${subject}"
|
||||
if run_fake_incoming "$DEVTESTING_AMOUNT" "$subject" "$credit_payto" "$debit_payto"; then
|
||||
case "$FI_STATUS" in
|
||||
ok_bounce)
|
||||
ok "fake-incoming" "CHF IN accepted by nexus · bounce/missing reserve pub — plumbing OK"
|
||||
info "note" "use real reserve pub as --subject for wirewatch credit (not required for mon probe)"
|
||||
;;
|
||||
ok_in) ok "fake-incoming" "CHF IN recorded by nexus (no bounce line)" ;;
|
||||
ok_cli) ok "fake-incoming" "CLI completed CHF fake-incoming" ;;
|
||||
*) ok "fake-incoming" "status=${FI_STATUS}" ;;
|
||||
esac
|
||||
else
|
||||
case "$FI_STATUS" in
|
||||
fail_ssh)
|
||||
fail "fake-incoming" "SSH/CLI failure" "$(printf '%s' "$FI_FLAT" | head -c 200)"
|
||||
exit 1
|
||||
;;
|
||||
fail_creditor)
|
||||
fail "fake-incoming" "wrong credit IBAN (not exchange)" "$(printf '%s' "$FI_FLAT" | head -c 220)"
|
||||
exit 1
|
||||
;;
|
||||
fail_nexus)
|
||||
fail "fake-incoming" "nexus rejected request" "$(printf '%s' "$FI_FLAT" | head -c 220)"
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
fail "fake-incoming" "no CHF IN line from nexus" "rc=${FI_RC} $(printf '%s' "$FI_FLAT" | head -c 200)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# --- CHF amount ladder (fake-incoming rungs) ---
|
||||
if [ "${DEVTESTING_LADDER}" = "1" ]; then
|
||||
set_group ladder
|
||||
section "devtesting · CHF fake-incoming ladder (→ ${DEVTESTING_LADDER_MAX})"
|
||||
|
||||
LADDER_FILE=$(mktemp "${TMPDIR:-/tmp}/devtesting-ladder.XXXXXX")
|
||||
python3 - <<'PY' "${DEVTESTING_LADDER_MAX}" "${DEVTESTING_LADDER_STEPS}" "${DEVTESTING_LADDER_MIN}" "$LADDER_FILE"
|
||||
import math, sys
|
||||
from decimal import Decimal, ROUND_HALF_UP
|
||||
from pathlib import Path
|
||||
|
||||
max_amt = Decimal(sys.argv[1])
|
||||
steps = max(1, int(sys.argv[2]))
|
||||
min_amt = Decimal(sys.argv[3])
|
||||
out = Path(sys.argv[4])
|
||||
if min_amt <= 0:
|
||||
min_amt = Decimal("0.01")
|
||||
if min_amt >= max_amt:
|
||||
min_amt = max_amt / Decimal(1000)
|
||||
# CHF: 2 decimal places
|
||||
q2 = Decimal("0.01")
|
||||
|
||||
def quant(v: Decimal) -> Decimal:
|
||||
if v < min_amt:
|
||||
v = min_amt
|
||||
return v.quantize(q2, rounding=ROUND_HALF_UP)
|
||||
|
||||
def fmt(v: Decimal) -> str:
|
||||
q = quant(v)
|
||||
if q == q.to_integral():
|
||||
return str(int(q))
|
||||
return format(q, "f")
|
||||
|
||||
if steps == 1:
|
||||
amts = [quant(max_amt)]
|
||||
elif steps == 2:
|
||||
amts = [quant(min_amt), quant(max_amt)]
|
||||
else:
|
||||
# Exactly `steps` log-spaced rungs from min → max (includes both ends).
|
||||
# Guarantees max rung = DEVTESTING_LADDER_MAX (“enough” volume at the top).
|
||||
amts = []
|
||||
lo, hi = float(min_amt), float(max_amt)
|
||||
if hi <= lo:
|
||||
hi = lo * 10
|
||||
for i in range(steps):
|
||||
if steps == 1:
|
||||
v = max_amt
|
||||
else:
|
||||
t = i / (steps - 1)
|
||||
v = Decimal(str(math.exp(math.log(lo) + t * (math.log(hi) - math.log(lo)))))
|
||||
amts.append(quant(v))
|
||||
amts[0] = quant(min_amt)
|
||||
amts[-1] = quant(max_amt)
|
||||
# enforce strictly increasing
|
||||
for i in range(1, len(amts)):
|
||||
if amts[i] <= amts[i - 1]:
|
||||
amts[i] = quant(amts[i - 1] + min_amt)
|
||||
amts[-1] = quant(max_amt)
|
||||
if amts[-1] <= amts[-2]:
|
||||
# last-but-one squeezed; leave max exact and drop collision by nudging previous
|
||||
amts[-2] = quant(max(amts[-3] + min_amt if len(amts) > 2 else min_amt, max_amt - Decimal(1)))
|
||||
if amts[-2] >= amts[-1]:
|
||||
amts[-2] = quant(amts[-1] - min_amt)
|
||||
|
||||
lines = ["CHF:%s" % fmt(a) for a in amts]
|
||||
out.write_text("\n".join(lines) + "\n")
|
||||
PY
|
||||
n_rungs_plan=$(grep -cE '^CHF:' "$LADDER_FILE" || echo 0)
|
||||
ladder_sum=$(python3 -c '
|
||||
from decimal import Decimal
|
||||
s=Decimal(0)
|
||||
for ln in open("'"$LADDER_FILE"'"):
|
||||
ln=ln.strip()
|
||||
if not ln: continue
|
||||
s += Decimal(ln.split(":",1)[-1])
|
||||
print(s)
|
||||
' 2>/dev/null || echo "?")
|
||||
|
||||
info "ladder plan" "steps=${n_rungs_plan} max=CHF:${DEVTESTING_LADDER_MAX} min=CHF:${DEVTESTING_LADDER_MIN} sum≈CHF:${ladder_sum} (synthetic debit geniban each rung)"
|
||||
info "ladder funding" "each rung uses geniban debit payto — no real bank withdraw; volume covers max rung CHF:${DEVTESTING_LADDER_MAX}"
|
||||
|
||||
LADDER_OK=0
|
||||
LADDER_FAIL=0
|
||||
LADDER_WARN=0
|
||||
rung=0
|
||||
# Load amounts into an array first — never `while read … done <file` around ssh:
|
||||
# without ssh -n, OpenSSH drains the ladder file as remote stdin (only rung 1 ran).
|
||||
mapfile -t LADDER_AMTS < <(grep -E '^CHF:' "$LADDER_FILE" || true)
|
||||
rm -f "$LADDER_FILE"
|
||||
n_rungs=${#LADDER_AMTS[@]}
|
||||
if [ "$n_rungs" -eq 0 ]; then
|
||||
fail "ladder" "no CHF: amounts generated"
|
||||
else
|
||||
for amt in "${LADDER_AMTS[@]}"; do
|
||||
[ -n "$amt" ] || continue
|
||||
rung=$((rung + 1))
|
||||
subject="mon-fake-ladder-${ts_base}-r${rung}-$$"
|
||||
info "rung ${rung}/${n_rungs}" "amount=${amt} subject=${subject}"
|
||||
if run_fake_incoming "$amt" "$subject" "$credit_payto" "$debit_payto"; then
|
||||
case "$FI_STATUS" in
|
||||
ok_bounce)
|
||||
ok "rung ${rung}" "${amt} · nexus IN + bounce (plumbing OK)"
|
||||
LADDER_OK=$((LADDER_OK + 1))
|
||||
;;
|
||||
ok_in)
|
||||
ok "rung ${rung}" "${amt} · nexus IN recorded"
|
||||
LADDER_OK=$((LADDER_OK + 1))
|
||||
;;
|
||||
ok_cli)
|
||||
ok "rung ${rung}" "${amt} · CLI ok"
|
||||
LADDER_OK=$((LADDER_OK + 1))
|
||||
;;
|
||||
*)
|
||||
warn "rung ${rung}" "${amt} · status=${FI_STATUS}"
|
||||
LADDER_WARN=$((LADDER_WARN + 1))
|
||||
;;
|
||||
esac
|
||||
else
|
||||
fail "rung ${rung}" "${amt} · ${FI_STATUS}" "$(printf '%s' "$FI_FLAT" | head -c 180)"
|
||||
LADDER_FAIL=$((LADDER_FAIL + 1))
|
||||
if [ "${DEVTESTING_LADDER_STOP_ON_FAIL}" = "1" ]; then
|
||||
warn "ladder stop" "DEVTESTING_LADDER_STOP_ON_FAIL=1 after rung ${rung}/${n_rungs}"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
info "ladder summary" "ok=${LADDER_OK} warn=${LADDER_WARN} fail=${LADDER_FAIL} / planned=${n_rungs} · max=CHF:${DEVTESTING_LADDER_MAX}"
|
||||
if [ "$n_rungs" -eq 0 ]; then
|
||||
:
|
||||
elif [ "$LADDER_FAIL" -gt 0 ] && [ "$LADDER_OK" -eq 0 ]; then
|
||||
fail "ladder" "all rungs failed"
|
||||
elif [ "$LADDER_FAIL" -gt 0 ]; then
|
||||
warn "ladder" "partial failures fail=${LADDER_FAIL} ok=${LADDER_OK}/${n_rungs}"
|
||||
elif [ "$LADDER_OK" -lt "$n_rungs" ]; then
|
||||
# e.g. stdin-eaten loop used to stop after 1 rung with ok=1 fail=0
|
||||
warn "ladder incomplete" "only ${LADDER_OK}/${n_rungs} rungs ran (ok) — not through max CHF:${DEVTESTING_LADDER_MAX}"
|
||||
else
|
||||
ok "ladder complete" "ok=${LADDER_OK}/${n_rungs} through max CHF:${DEVTESTING_LADDER_MAX}"
|
||||
fi
|
||||
else
|
||||
info "ladder" "skipped (DEVTESTING_LADDER=0)"
|
||||
fi
|
||||
|
||||
info "done" "devtesting fake-franken probe finished"
|
||||
exit 0
|
||||
264
check_e2e.sh
264
check_e2e.sh
|
|
@ -8,7 +8,7 @@ source "$ROOT/lib.sh"
|
|||
# shellcheck source=metrics.sh
|
||||
source "$ROOT/metrics.sh"
|
||||
|
||||
: "${E2E_TIMEOUT:=180}" # ATM ladder + wirewatch lag needs room (was 55 — too tight)
|
||||
: "${E2E_TIMEOUT:=180}" # Bank withdraw ladder + wirewatch lag needs room (was 55 — too tight)
|
||||
: "${E2E_PAY_SECS:=22}" # hard reserve for handle-uri + settle (do not starve pay)
|
||||
: "${E2E_SETTLE_ROUNDS:=18}"
|
||||
: "${E2E_SETTLE_SLEEP:=3}"
|
||||
|
|
@ -35,22 +35,22 @@ wcli_bal_snap() {
|
|||
fi
|
||||
fi
|
||||
local err="${out}.err"
|
||||
if command -v perl >/dev/null 2>&1; then
|
||||
if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then
|
||||
perl -e 'alarm shift; exec @ARGV' 10 \
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle balance \
|
||||
local _node="${NODE_BIN:-$(command -v node 2>/dev/null || true)}"
|
||||
[ -n "$_node" ] || { echo "(no node)" >"$out"; return 1; }
|
||||
if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout -k 2 10 "$_node" "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle balance \
|
||||
>"$out" 2>"$err" || true
|
||||
else
|
||||
perl -e 'alarm shift; exec @ARGV' 10 \
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults balance \
|
||||
"$_node" "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle balance \
|
||||
>"$out" 2>"$err" || true
|
||||
fi
|
||||
else
|
||||
if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle balance \
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout -k 2 10 "$_node" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults balance \
|
||||
>"$out" 2>"$err" || true
|
||||
else
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults balance \
|
||||
"$_node" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults balance \
|
||||
>"$out" 2>"$err" || true
|
||||
fi
|
||||
fi
|
||||
|
|
@ -226,18 +226,20 @@ except Exception: print("")' 2>/dev/null || true)
|
|||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Variable amounts
|
||||
# Withdraw = typical ATM notes only (Bankomat): 20 / 50 / 100 / 200 …
|
||||
# Withdraw = typical bank cash-note amounts (CHF-style: was called ATM): 20 / 50 / 100 / 200 …
|
||||
# Pay = several smaller purchase amounts
|
||||
# Env:
|
||||
# E2E_WITHDRAW_VALUES="20 50 100" # bare numbers → prefixed with CUR
|
||||
# E2E_PAY_VALUES="0.01 0.05 0.5 1 2 5"
|
||||
# E2E_VARIABLE=0 # single fixed WITHDRAW_AMT / PAY_AMT
|
||||
# E2E_ATM_MAX=N # try at most N ATM withdraws (budget)
|
||||
# E2E_BANK_WITHDRAW_MAX=N # try at most N bank withdraws (budget)
|
||||
# E2E_ATM_MAX=N # legacy alias of E2E_BANK_WITHDRAW_MAX
|
||||
# ---------------------------------------------------------------------------
|
||||
: "${E2E_VARIABLE:=1}"
|
||||
: "${E2E_ATM_MAX:=5}"
|
||||
: "${E2E_BANK_WITHDRAW_MAX:=${E2E_ATM_MAX:-5}}"
|
||||
E2E_ATM_MAX="${E2E_ATM_MAX:-$E2E_BANK_WITHDRAW_MAX}" # legacy alias
|
||||
if [ "$E2E_REMOTE" = "1" ]; then
|
||||
# remote: still ATM-shaped; TESTPAYSAN needs larger notes for shop pays
|
||||
# remote: still bank cash-note shaped; TESTPAYSAN needs larger notes for shop pays
|
||||
if [ "$CUR" = "TESTPAYSAN" ]; then
|
||||
: "${E2E_WITHDRAW_VALUES:=20 50}"
|
||||
# farmer shop template face values (oeufs-6=5, fromage=8.5, jus=6)
|
||||
|
|
@ -247,7 +249,7 @@ if [ "$E2E_REMOTE" = "1" ]; then
|
|||
: "${E2E_PAY_VALUES:=0.01 0.05 0.1 1}"
|
||||
fi
|
||||
else
|
||||
# local GOA: classic ATM notes + 4200 for paivana paywall template
|
||||
# local GOA: classic bank cash-note amounts + 4200 for paivana paywall template
|
||||
: "${E2E_WITHDRAW_VALUES:=20 50 100 200 4200}"
|
||||
: "${E2E_PAY_VALUES:=0.01 0.05 0.1 0.5 1 2 5 10}"
|
||||
fi
|
||||
|
|
@ -284,7 +286,7 @@ if [ "$E2E_VARIABLE" != "1" ]; then
|
|||
fi
|
||||
fi
|
||||
|
||||
# Credit = sum of planned ATM withdraws + small buffer for fees
|
||||
# Credit = sum of planned Bank withdraws + small buffer for fees
|
||||
CREDIT_AMT=$(python3 -c '
|
||||
import sys
|
||||
cur=sys.argv[1]
|
||||
|
|
@ -301,7 +303,7 @@ print("%s:%g" % (cur, s + buf))
|
|||
WITHDRAW_AMT=$(printf '%s' "$WITHDRAW_LIST" | awk '{print $1}')
|
||||
PAY_AMT=$(printf '%s' "$PAY_LIST" | awk '{print $1}')
|
||||
|
||||
# More wall time when running ATM ladder
|
||||
# More wall time when running Bank withdraw ladder
|
||||
if [ "$E2E_VARIABLE" = "1" ]; then
|
||||
n_w=$(printf '%s' "$WITHDRAW_LIST" | wc -w | tr -d ' ')
|
||||
n_p=$(printf '%s' "$PAY_LIST" | wc -w | tr -d ' ')
|
||||
|
|
@ -309,9 +311,9 @@ if [ "$E2E_VARIABLE" = "1" ]; then
|
|||
if [ "${E2E_TIMEOUT}" -lt "$need" ]; then
|
||||
E2E_TIMEOUT=$need
|
||||
fi
|
||||
# cap ATM attempts but always keep CUR:4200 if listed (paivana)
|
||||
# cap bank withdraw attempts but always keep CUR:4200 if listed (paivana)
|
||||
WITHDRAW_LIST=$(
|
||||
printf '%s' "$WITHDRAW_LIST" | tr ' ' '\n' | awk -v max="$E2E_ATM_MAX" -v big="${CUR}:4200" '
|
||||
printf '%s' "$WITHDRAW_LIST" | tr ' ' '\n' | awk -v max="$E2E_BANK_WITHDRAW_MAX" -v big="${CUR}:4200" '
|
||||
NF==0 { next }
|
||||
$0==big { has=1; next }
|
||||
{ small[++n]=$0 }
|
||||
|
|
@ -376,13 +378,35 @@ WALLET_CLI=$(find_wallet_cli) || {
|
|||
blocker "prereq" "taler-wallet-cli not found (set WALLET_CLI=)"
|
||||
exit 1
|
||||
}
|
||||
# wallet-core spawns `taler-helper-sqlite3` via PATH (not an absolute path).
|
||||
# Ensure operator wallet bindir + ~/.local/bin are first (systemd mon jobs).
|
||||
export PATH="${WALLET_CLI_BINDIR:+$WALLET_CLI_BINDIR:}${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
|
||||
NODE_BIN=$(command -v node 2>/dev/null || true)
|
||||
if [ -z "$NODE_BIN" ] || [ ! -x "$NODE_BIN" ]; then
|
||||
blocker "prereq" "node not found on PATH (install Node or set PATH; need node for wallet-cli)"
|
||||
exit 1
|
||||
fi
|
||||
if ! command -v taler-helper-sqlite3 >/dev/null 2>&1; then
|
||||
for _h in \
|
||||
"${WALLET_CLI_BINDIR:+$WALLET_CLI_BINDIR/taler-helper-sqlite3}" \
|
||||
"${HOME}/.local/bin/taler-helper-sqlite3" \
|
||||
/usr/bin/taler-helper-sqlite3
|
||||
do
|
||||
[ -n "$_h" ] && [ -x "$_h" ] && export PATH="$(dirname "$_h"):$PATH" && break
|
||||
done
|
||||
fi
|
||||
if ! command -v taler-helper-sqlite3 >/dev/null 2>&1; then
|
||||
blocker "prereq" "taler-helper-sqlite3 not on PATH (wallet accept-uri/pay will fail with spawn ENOENT) — install next to wallet-cli or set WALLET_CLI_BINDIR"
|
||||
exit 1
|
||||
fi
|
||||
ok "wallet-helper" "taler-helper-sqlite3=$(command -v taler-helper-sqlite3)"
|
||||
# Implementation version (--version) + wallet-core API ranges (version command)
|
||||
WALLET_IMPL_VER=$(
|
||||
node "$WALLET_CLI" --version 2>/dev/null | head -1 | tr -d '\r' || true
|
||||
"$NODE_BIN" "$WALLET_CLI" --version 2>/dev/null | head -1 | tr -d '\r' || true
|
||||
)
|
||||
WALLET_CORE_VER=$(
|
||||
_wver_db=$(mktemp "${TMPDIR:-/tmp}/wver.XXXXXX.sqlite3" 2>/dev/null || echo "${TMPDIR:-/tmp}/wver-$$.sqlite3")
|
||||
node "$WALLET_CLI" --wallet-db="$_wver_db" --no-throttle --skip-defaults version 2>/dev/null \
|
||||
"$NODE_BIN" "$WALLET_CLI" --wallet-db="$_wver_db" --no-throttle --skip-defaults version 2>/dev/null \
|
||||
| python3 -c '
|
||||
import json,re,sys
|
||||
raw=sys.stdin.read()
|
||||
|
|
@ -413,7 +437,7 @@ else
|
|||
fi
|
||||
info "e2e mode" "$([ "$E2E_REMOTE" = "1" ] && echo "remote/public domain (no SSH)" || echo "local koopa stack")"
|
||||
info "currency" "$CUR"
|
||||
info "ATM withdraw ladder" "$WITHDRAW_LIST (credit $CREDIT_AMT)"
|
||||
info "Bank withdraw ladder" "$WITHDRAW_LIST (credit $CREDIT_AMT)"
|
||||
info "pay ladder" "$PAY_LIST"
|
||||
info "e2e budget" "${E2E_TIMEOUT}s"
|
||||
|
||||
|
|
@ -426,62 +450,61 @@ else
|
|||
warn "alt_unit_names" "using built-in SI fallback"
|
||||
fi
|
||||
|
||||
# Host + bank/exchange/merchant RAM/CPU/load (SSH koopa or koopa-external)
|
||||
# Host + bank/exchange/merchant RAM/CPU/load (SSH via KOOPA_SSH from env)
|
||||
set_group load
|
||||
section "e2e · load snapshot (before withdraw/pay)"
|
||||
metrics_report_load "${METRICS_DIR}/load-before.json" "e2e-start" || true
|
||||
|
||||
# Local stack: koopa secrets. Remote: env first; TESTPAYSAN stage may load francpaysan host secrets.
|
||||
# Local stack: koopa secrets. Remote: env first; TESTPAYSAN stage may load stagepaysan secrets via FRANCPAYSAN_SSH/INSIDE_SSH.
|
||||
set_group prereq
|
||||
if [ "$E2E_REMOTE" = "1" ]; then
|
||||
ADMIN_PASS="${E2E_BANK_ADMIN_PASS:-${BANK_ADMIN_PASS:-}}"
|
||||
MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}"
|
||||
# Stage FrancPaysan: allow admin credit without committing passwords.
|
||||
# Order: already-set env → local secrets tree → SSH francpaysan-host (ops laptop).
|
||||
# Order: env → FRANCPAYSAN_SECRETS → SSH FRANCPAYSAN_SSH/INSIDE_SSH + remote secret paths.
|
||||
if [ -z "$ADMIN_PASS" ] && [ "${CUR:-${EXPECT_CURRENCY:-}}" = "TESTPAYSAN" ]; then
|
||||
_fp_try=""
|
||||
for _fp_try in \
|
||||
"${FRANCPAYSAN_SECRETS:-}/stage/bank-admin-password.txt" \
|
||||
"${HOME}/francpaysan-secrets/stage/bank-admin-password.txt" \
|
||||
"${HOME}/src/francpaysan-secrets/stage/bank-admin-password.txt" \
|
||||
"${HOME}/taler/src/francpaysan-secrets/stage/bank-admin-password.txt"
|
||||
do
|
||||
if [ -n "$_fp_try" ] && [ -f "$_fp_try" ]; then
|
||||
if [ -n "${FRANCPAYSAN_SECRETS:-}" ]; then
|
||||
_fp_try="${FRANCPAYSAN_SECRETS}/stage/bank-admin-password.txt"
|
||||
if [ -f "$_fp_try" ]; then
|
||||
ADMIN_PASS=$(tr -d '\n\r' <"$_fp_try")
|
||||
info "bank admin secret" "from $_fp_try"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
if [ -z "$ADMIN_PASS" ] && command -v ssh >/dev/null 2>&1; then
|
||||
for _fp_host in "${FRANCPAYSAN_SSH:-francpaysan-host}" francpaysan-host; do
|
||||
_remote_bank_sec="${FRANCPAYSAN_REMOTE_BANK_ADMIN_SECRET:-}"
|
||||
[ -z "$_remote_bank_sec" ] && [ -n "${FRANCPAYSAN_REMOTE_SECRETS_ROOT:-}" ] && \
|
||||
_remote_bank_sec="${FRANCPAYSAN_REMOTE_SECRETS_ROOT}/bank/secrets/bank-admin-password.txt"
|
||||
for _fp_host in ${FRANCPAYSAN_SSH:+"$FRANCPAYSAN_SSH"} ${INSIDE_SSH:+"$INSIDE_SSH"}; do
|
||||
[ -n "$_fp_host" ] && [ -n "$_remote_bank_sec" ] || continue
|
||||
ADMIN_PASS=$(ssh -o BatchMode=yes -o ConnectTimeout=8 "$_fp_host" \
|
||||
'sudo cat /mnt/data/stagepaysan/bank/secrets/bank-admin-password.txt 2>/dev/null' \
|
||||
"sudo cat ${_remote_bank_sec} 2>/dev/null" \
|
||||
2>/dev/null | tr -d '\n\r' || true)
|
||||
if [ -n "$ADMIN_PASS" ]; then
|
||||
info "bank admin secret" "from ${_fp_host}:/mnt/data/stagepaysan/bank/secrets/"
|
||||
info "bank admin secret" "from ${_fp_host} (remote path via env)"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
unset _fp_try _fp_host
|
||||
unset _fp_try _fp_host _remote_bank_sec
|
||||
fi
|
||||
# Stage merchant: public templates need no token; optional default-instance token
|
||||
if [ -z "$MPW" ] && [ "${CUR:-${EXPECT_CURRENCY:-}}" = "TESTPAYSAN" ]; then
|
||||
for _fp_try in \
|
||||
"${FRANCPAYSAN_SECRETS:-}/stage/default-instance-token.txt" \
|
||||
"${HOME}/francpaysan-secrets/stage/default-instance-token.txt"
|
||||
do
|
||||
if [ -n "$_fp_try" ] && [ -f "$_fp_try" ]; then
|
||||
MPW=$(tr -d '\n\r' <"$_fp_try")
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [ -z "$MPW" ] && command -v ssh >/dev/null 2>&1; then
|
||||
MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=8 "${FRANCPAYSAN_SSH:-francpaysan-host}" \
|
||||
'sudo cat /mnt/data/stagepaysan/merchant/secrets/default-instance-token.txt 2>/dev/null' \
|
||||
2>/dev/null | tr -d '\n\r' || true)
|
||||
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
|
||||
MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt")
|
||||
fi
|
||||
if [ -z "$MPW" ] && command -v ssh >/dev/null 2>&1; then
|
||||
_remote_mer_sec="${FRANCPAYSAN_REMOTE_MERCHANT_TOKEN:-}"
|
||||
[ -z "$_remote_mer_sec" ] && [ -n "${FRANCPAYSAN_REMOTE_SECRETS_ROOT:-}" ] && \
|
||||
_remote_mer_sec="${FRANCPAYSAN_REMOTE_SECRETS_ROOT}/merchant/secrets/default-instance-token.txt"
|
||||
_fp_host="${FRANCPAYSAN_SSH:-${INSIDE_SSH:-}}"
|
||||
if [ -n "$_fp_host" ] && [ -n "$_remote_mer_sec" ]; then
|
||||
MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=8 "$_fp_host" \
|
||||
"sudo cat ${_remote_mer_sec} 2>/dev/null" \
|
||||
2>/dev/null | tr -d '\n\r' || true)
|
||||
fi
|
||||
unset _fp_host _remote_mer_sec
|
||||
fi
|
||||
unset _fp_try
|
||||
fi
|
||||
else
|
||||
ADMIN_PASS="${E2E_BANK_ADMIN_PASS:-$(read_secret "taler-bank/bank-admin-password.txt" || true)}"
|
||||
|
|
@ -576,7 +599,7 @@ start_wallet_serve() {
|
|||
stop_wallet_serve
|
||||
rm -f "$WSOCK"
|
||||
# serve holds the DB open; clients use --wallet-connection only
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults \
|
||||
"$NODE_BIN" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults \
|
||||
advanced serve --unix-path "$WSOCK" \
|
||||
>"$SCRATCH/wallet-serve.log" 2>&1 &
|
||||
WSERVE_PID=$!
|
||||
|
|
@ -593,6 +616,25 @@ start_wallet_serve() {
|
|||
return 1
|
||||
}
|
||||
|
||||
# Run node wallet-cli with a hard wall clock. Prefer GNU/coreutils `timeout`
|
||||
# (clear errors). Fall back to perl alarm only if timeout is missing.
|
||||
# Always use absolute NODE_BIN + WALLET_CLI so systemd PATH quirks cannot
|
||||
# turn into opaque "alarm shift; exec @ARGV" noise in accept-uri output.
|
||||
_wcli_exec() {
|
||||
local cap="$1"
|
||||
shift
|
||||
local -a cmd=( "$NODE_BIN" "$WALLET_CLI" "$@" )
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout -k 2 "$cap" "${cmd[@]}"
|
||||
elif command -v gtimeout >/dev/null 2>&1; then
|
||||
gtimeout --kill-after=2 "$cap" "${cmd[@]}"
|
||||
elif command -v perl >/dev/null 2>&1; then
|
||||
perl -e 'alarm shift; exec @ARGV' "$cap" "${cmd[@]}"
|
||||
else
|
||||
"${cmd[@]}"
|
||||
fi
|
||||
}
|
||||
|
||||
# $1 = max seconds for this call (optional); rest = wallet-cli args
|
||||
# Prefer live serve socket so the shepherd stays up (alternative to run-until-done).
|
||||
wcli() {
|
||||
|
|
@ -604,22 +646,15 @@ wcli() {
|
|||
e2e_over && return 124
|
||||
local cap
|
||||
cap=$(e2e_left)
|
||||
[ "$cap" -gt "$maxc" ] && cap=$maxc
|
||||
[ "$cap" -lt 3 ] && return 124
|
||||
# e2e_left may print warnings on stdout — keep only first integer token
|
||||
cap=$(printf '%s' "$cap" | awk '/^[0-9]+$/{print; exit} {for(i=1;i<=NF;i++) if($i+0==$i){print $i; exit}}')
|
||||
[ -z "$cap" ] && cap=$maxc
|
||||
[ "$cap" -gt "$maxc" ] 2>/dev/null && cap=$maxc
|
||||
[ "$cap" -lt 3 ] 2>/dev/null && return 124
|
||||
if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then
|
||||
if command -v perl >/dev/null 2>&1; then
|
||||
perl -e 'alarm shift; exec @ARGV' "$cap" \
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
else
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
fi
|
||||
_wcli_exec "$cap" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
else
|
||||
if command -v perl >/dev/null 2>&1; then
|
||||
perl -e 'alarm shift; exec @ARGV' "$cap" \
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
else
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
fi
|
||||
_wcli_exec "$cap" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
|
|
@ -628,9 +663,10 @@ wcli_pay() {
|
|||
local cap=$E2E_PAY_SECS
|
||||
local left
|
||||
left=$(e2e_left)
|
||||
if [ "$left" -gt "$cap" ]; then
|
||||
left=$(printf '%s' "$left" | awk '/^[0-9]+$/{print; exit} {for(i=1;i<=NF;i++) if($i+0==$i){print $i; exit}}')
|
||||
if [ -n "$left" ] && [ "$left" -gt "$cap" ] 2>/dev/null; then
|
||||
cap=$E2E_PAY_SECS
|
||||
elif [ "$left" -ge 8 ]; then
|
||||
elif [ -n "$left" ] && [ "$left" -ge 8 ] 2>/dev/null; then
|
||||
cap=$left
|
||||
else
|
||||
# still try once with floor 12s so Alarm clock is not the blocker
|
||||
|
|
@ -638,19 +674,9 @@ wcli_pay() {
|
|||
fi
|
||||
info "pay wallet cap" "${cap}s"
|
||||
if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then
|
||||
if command -v perl >/dev/null 2>&1; then
|
||||
perl -e 'alarm shift; exec @ARGV' "$cap" \
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
else
|
||||
node "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
fi
|
||||
_wcli_exec "$cap" --wallet-connection="$WSOCK" --no-throttle "$@"
|
||||
else
|
||||
if command -v perl >/dev/null 2>&1; then
|
||||
perl -e 'alarm shift; exec @ARGV' "$cap" \
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
else
|
||||
node "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
fi
|
||||
_wcli_exec "$cap" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
|
|
@ -864,7 +890,7 @@ else
|
|||
warn "wallet accept ToS" "$(tail -c 80 "$SCRATCH/ex-tos.out" | tr '\n' ' ')"
|
||||
fi
|
||||
fi
|
||||
# Empty wallet baseline before ATM / pay load
|
||||
# Empty wallet baseline before bank withdraw / pay load
|
||||
metrics_report_coins "wallet-ready" || true
|
||||
|
||||
wd_status() {
|
||||
|
|
@ -902,15 +928,15 @@ fake_incoming_speedup() {
|
|||
fi
|
||||
}
|
||||
|
||||
# One ATM-style withdraw: create → accept → select → confirm → settle
|
||||
# One bank withdraw: create → accept → select → confirm → settle
|
||||
# returns 0 on wallet balance increase, 1 on soft fail (ladder continues)
|
||||
e2e_one_withdraw() {
|
||||
WITHDRAW_AMT="$1"
|
||||
local tag
|
||||
tag=$(printf '%s' "$WITHDRAW_AMT" | tr '.:' '__')
|
||||
section "e2e · ATM withdraw $WITHDRAW_AMT · $(format_amount_alt "$WITHDRAW_AMT")"
|
||||
e2e_over && { warn "ATM withdraw" "budget exhausted — skip $WITHDRAW_AMT"; return 1; }
|
||||
metrics_report_coins "before-ATM-${tag}" || true
|
||||
section "e2e · Bank withdraw $WITHDRAW_AMT · $(format_amount_alt "$WITHDRAW_AMT")"
|
||||
e2e_over && { warn "Bank withdraw" "budget exhausted — skip $WITHDRAW_AMT"; return 1; }
|
||||
metrics_report_coins "before-bank-wd-${tag}" || true
|
||||
|
||||
curl -sS -m 15 -o "$SCRATCH/wd-$tag.json" -H "Authorization: Bearer $UT" \
|
||||
-H 'Content-Type: application/json' \
|
||||
|
|
@ -918,23 +944,29 @@ e2e_one_withdraw() {
|
|||
"$BANK/accounts/${USER}/withdrawals"
|
||||
WID=$(python3 -c 'import json;d=json.load(open("'"$SCRATCH"'/wd-'"$tag"'.json"));print(d.get("withdrawal_id") or d.get("id") or "")' 2>/dev/null || true)
|
||||
URI=$(python3 -c 'import json;d=json.load(open("'"$SCRATCH"'/wd-'"$tag"'.json"));print(d.get("taler_withdraw_uri") or "")' 2>/dev/null || true)
|
||||
URI_CLEAN=$(python3 -c 'import json;u=json.load(open("'"$SCRATCH"'/wd-'"$tag"'.json")).get("taler_withdraw_uri") or "";print(u.replace(":443/","/"))' 2>/dev/null || true)
|
||||
URI_CLEAN=$(normalize_taler_uri "$URI")
|
||||
if [ -z "$WID" ] || [ -z "$URI_CLEAN" ]; then
|
||||
warn "ATM withdraw $WITHDRAW_AMT" "create failed — $(head -c 80 "$SCRATCH/wd-$tag.json" | tr '\n' ' ')"
|
||||
warn "Bank withdraw $WITHDRAW_AMT" "create failed — $(head -c 80 "$SCRATCH/wd-$tag.json" | tr '\n' ' ')"
|
||||
return 1
|
||||
fi
|
||||
ok "ATM withdrawal created $WITHDRAW_AMT ($WID)"
|
||||
ok "Bank withdrawal created $WITHDRAW_AMT ($WID)"
|
||||
cp "$SCRATCH/wd-$tag.json" "$SCRATCH/wd.json"
|
||||
|
||||
USE_URI="$URI"
|
||||
[ -z "$USE_URI" ] && USE_URI="$URI_CLEAN"
|
||||
# Always prefer stripped URI (libeufin often emits host:443)
|
||||
USE_URI="${URI_CLEAN:-$URI}"
|
||||
if wcli withdraw accept-uri --exchange "${EXCHANGE_PUBLIC}/" "$USE_URI" >"$SCRATCH/accept-$tag.out" 2>&1; then
|
||||
ok "wallet accept $WITHDRAW_AMT"
|
||||
elif [ "$USE_URI" != "$URI_CLEAN" ] && [ -n "$URI_CLEAN" ] \
|
||||
&& wcli withdraw accept-uri --exchange "${EXCHANGE_PUBLIC}/" "$URI_CLEAN" >"$SCRATCH/accept-$tag.out" 2>&1; then
|
||||
ok "wallet accept $WITHDRAW_AMT (no :443)"
|
||||
elif [ -n "$URI" ] && [ "$USE_URI" != "$URI" ] \
|
||||
&& wcli withdraw accept-uri --exchange "${EXCHANGE_PUBLIC}/" "$URI" >"$SCRATCH/accept-$tag.out" 2>&1; then
|
||||
ok "wallet accept $WITHDRAW_AMT (raw URI fallback)"
|
||||
else
|
||||
warn "ATM withdraw $WITHDRAW_AMT" "accept-uri failed — $(tail -c 100 "$SCRATCH/accept-$tag.out" | tr '\n' ' ')"
|
||||
# Prefer the real Error/ENOENT line over noise (perl -e source, stack frames)
|
||||
_acc_err=$(
|
||||
grep -E 'Error:|ENOENT|spawn taler-helper|code:|An error occurred' \
|
||||
"$SCRATCH/accept-$tag.out" 2>/dev/null | head -3 | tr '\n' ' ' | head -c 200
|
||||
)
|
||||
[ -z "$_acc_err" ] && _acc_err=$(tail -c 160 "$SCRATCH/accept-$tag.out" 2>/dev/null | tr '\n' ' ')
|
||||
warn "Bank withdraw $WITHDRAW_AMT" "accept-uri failed — ${_acc_err:-unknown}"
|
||||
return 1
|
||||
fi
|
||||
cp "$SCRATCH/accept-$tag.out" "$SCRATCH/accept.out"
|
||||
|
|
@ -976,7 +1008,7 @@ else:
|
|||
fi
|
||||
fi
|
||||
if [ "$st" != "selected" ] && [ "$st" != "confirmed" ]; then
|
||||
warn "ATM withdraw $WITHDRAW_AMT" "not selected (status=${st:-?})"
|
||||
warn "Bank withdraw $WITHDRAW_AMT" "not selected (status=${st:-?})"
|
||||
return 1
|
||||
fi
|
||||
if [ "$st" != "confirmed" ]; then
|
||||
|
|
@ -985,32 +1017,32 @@ else:
|
|||
"$BANK/accounts/${USER}/withdrawals/${WID}/confirm")
|
||||
case "$code" in
|
||||
200|204) ok "bank confirm $WITHDRAW_AMT" ;;
|
||||
*) warn "ATM withdraw $WITHDRAW_AMT" "confirm HTTP $code"; return 1 ;;
|
||||
*) warn "Bank withdraw $WITHDRAW_AMT" "confirm HTTP $code"; return 1 ;;
|
||||
esac
|
||||
fi
|
||||
fake_incoming_speedup
|
||||
|
||||
# Short per-ATM settle: poll balance + bank transfer_done only (never run-until-done)
|
||||
# Short per-withdraw settle: poll balance + bank transfer_done only (never run-until-done)
|
||||
local ok_bal=0 r av xfer
|
||||
for r in 1 2 3 4 5 6 8 10; do
|
||||
wcli_bal_snap "$SCRATCH/bal-$tag.out" || wcli balance >"$SCRATCH/bal-$tag.out" 2>&1 || true
|
||||
av=$(wallet_avail_num "$SCRATCH/bal-$tag.out")
|
||||
if python3 -c "import sys; sys.exit(0 if float(sys.argv[1]) > 0 else 1)" "$av" 2>/dev/null; then
|
||||
ok_bal=1
|
||||
ok "wallet funded after ATM $WITHDRAW_AMT" "avail=${CUR}:${av}"
|
||||
ok "wallet funded after bank withdraw $WITHDRAW_AMT" "avail=${CUR}:${av}"
|
||||
info "balance" "$(fmt_bal "$SCRATCH/bal-$tag.out")"
|
||||
break
|
||||
fi
|
||||
xfer=$(curl -sS -m 8 "$BANK/taler-integration/withdrawal-operation/${WID}" 2>/dev/null \
|
||||
| python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("transfer_done"), d.get("status"))' 2>/dev/null || echo "?")
|
||||
if echo "$xfer" | grep -qi True; then
|
||||
ok "ATM $WITHDRAW_AMT bank transfer_done" "wallet avail=${CUR}:${av} ($xfer) — no run-until-done"
|
||||
ok "Bank withdraw $WITHDRAW_AMT bank transfer_done" "wallet avail=${CUR}:${av} ($xfer) — no run-until-done"
|
||||
ok_bal=1
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
metrics_report_coins "after-ATM-${tag}" || true
|
||||
metrics_report_coins "after-bank-wd-${tag}" || true
|
||||
if [ "$ok_bal" = "1" ]; then
|
||||
metrics_record_flow withdrawn "$WITHDRAW_AMT" || true
|
||||
return 0
|
||||
|
|
@ -1018,10 +1050,10 @@ else:
|
|||
# Bank side often already confirmed — treat as timing lag, not hard fail
|
||||
st=$(wd_status)
|
||||
if [ "$st" = "confirmed" ]; then
|
||||
warn "ATM withdraw $WITHDRAW_AMT" "bank confirmed; wallet not funded yet (settlement timing — will recheck later)"
|
||||
warn "Bank withdraw $WITHDRAW_AMT" "bank confirmed; wallet not funded yet (settlement timing — will recheck later)"
|
||||
return 2
|
||||
fi
|
||||
warn "ATM withdraw $WITHDRAW_AMT" "no wallet balance yet (status=${st:-?})"
|
||||
warn "Bank withdraw $WITHDRAW_AMT" "no wallet balance yet (status=${st:-?})"
|
||||
return 1
|
||||
}
|
||||
|
||||
|
|
@ -1336,8 +1368,8 @@ sys.exit(1)
|
|||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
set_group atm
|
||||
section "e2e · ATM withdraw ladder"
|
||||
set_group bankwd
|
||||
section "e2e · Bank withdraw ladder"
|
||||
# ---------------------------------------------------------------------------
|
||||
WITHDRAW_OK=0
|
||||
WITHDRAW_OK_N=0
|
||||
|
|
@ -1345,7 +1377,7 @@ WITHDRAW_LAG_N=0
|
|||
WITHDRAW_FAIL_N=0
|
||||
WITHDRAW_REPORT=""
|
||||
for WITHDRAW_AMT in $WITHDRAW_LIST; do
|
||||
e2e_over && { warn "ATM ladder" "time budget low — stopping more ATM withdraws (not a protocol error)"; break; }
|
||||
e2e_over && { warn "Bank withdraw ladder" "time budget low — stopping more Bank withdraws (not a protocol error)"; break; }
|
||||
set +e
|
||||
e2e_one_withdraw "$WITHDRAW_AMT"
|
||||
wc=$?
|
||||
|
|
@ -1366,26 +1398,26 @@ for WITHDRAW_AMT in $WITHDRAW_LIST; do
|
|||
;;
|
||||
esac
|
||||
done
|
||||
info "ATM withdraw summary" "$WITHDRAW_REPORT (ok=$WITHDRAW_OK_N lag=$WITHDRAW_LAG_N fail=$WITHDRAW_FAIL_N)"
|
||||
info "Bank withdraw summary" "$WITHDRAW_REPORT (ok=$WITHDRAW_OK_N lag=$WITHDRAW_LAG_N fail=$WITHDRAW_FAIL_N)"
|
||||
set_group load
|
||||
section "e2e · load snapshot (after ATM withdraws)"
|
||||
section "e2e · load snapshot (after Bank withdraws)"
|
||||
metrics_report_load "${METRICS_DIR}/load-after-withdraw.json" "after-withdraw" || true
|
||||
cp -f "${METRICS_DIR}/load-after-withdraw.json" "${METRICS_DIR}/load-after.json" 2>/dev/null || true
|
||||
metrics_report_coins "after-ATM-ladder" || true
|
||||
metrics_report_coins "after-bank-wd-ladder" || true
|
||||
|
||||
# Settlement catch-up: bank may have confirmed while wallet was still empty
|
||||
set_group settle
|
||||
section "e2e · wallet settlement (timing)"
|
||||
if ! wait_wallet_balance 0 "${E2E_SETTLE_ROUNDS}" "${E2E_SETTLE_SLEEP}"; then
|
||||
if [ "$WITHDRAW_LAG_N" -gt 0 ] || [ "$WITHDRAW_OK_N" -gt 0 ]; then
|
||||
warn "settlement timing" "ATM path reached bank confirm (lag=$WITHDRAW_LAG_N ok=$WITHDRAW_OK_N) but wallet empty after wait — TIME lag, not protocol error"
|
||||
warn "settlement timing" "bank withdraw path reached bank confirm (lag=$WITHDRAW_LAG_N ok=$WITHDRAW_OK_N) but wallet empty after wait — TIME lag, not protocol error"
|
||||
fi
|
||||
fi
|
||||
av_now=$(wallet_avail_num)
|
||||
if python3 -c "import sys; sys.exit(0 if float(sys.argv[1]) > 0 else 1)" "$av_now" 2>/dev/null; then
|
||||
WITHDRAW_OK=1
|
||||
if [ "$WITHDRAW_OK_N" = "0" ]; then
|
||||
warn "settlement timing" "coins arrived after ATM ladder (${CUR}:${av_now}) — earlier 'FAIL' was timing lag"
|
||||
warn "settlement timing" "coins arrived after Bank withdraw ladder (${CUR}:${av_now}) — earlier 'FAIL' was timing lag"
|
||||
WITHDRAW_REPORT="${WITHDRAW_REPORT} → late-OK avail=${CUR}:${av_now}"
|
||||
fi
|
||||
ok "spendable balance for payments" "${CUR}:${av_now}"
|
||||
|
|
@ -1404,7 +1436,7 @@ else
|
|||
WITHDRAW_OK=1
|
||||
warn "settlement timing" "coins present after dig wait (${CUR}:${av_now}) — timing, not blocker"
|
||||
else
|
||||
blocker "withdraw-settle" "no spendable ${CUR} after ATM ladder + settle wait ($WITHDRAW_LIST)"
|
||||
blocker "withdraw-settle" "no spendable ${CUR} after Bank withdraw ladder + settle wait ($WITHDRAW_LIST)"
|
||||
section "e2e · report"
|
||||
info "WITHDRAW" "$WITHDRAW_REPORT — no coins after extended wait"
|
||||
info "PAY" "SKIPPED (no spendable balance after settle wait)"
|
||||
|
|
@ -1597,7 +1629,7 @@ set_group paivana
|
|||
section "e2e · paivana paywall (template GOA:4200)"
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public paywall: https://paivana.hacktivism.ch → 302 to merchant template "paivana".
|
||||
# Requires spendable ≥ 4200 GOA (ATM ladder includes GOA:4200 by default).
|
||||
# Requires spendable ≥ 4200 GOA (Bank withdraw ladder includes GOA:4200 by default).
|
||||
: "${PAIVANA_PUBLIC:=https://paivana.hacktivism.ch}"
|
||||
: "${E2E_PAIVANA:=1}"
|
||||
: "${E2E_PAIVANA_TEMPLATE:=paivana}"
|
||||
|
|
@ -1631,11 +1663,11 @@ else
|
|||
;;
|
||||
esac
|
||||
|
||||
# Ensure wallet can cover 4200 (extra ATM if ladder did not fund enough)
|
||||
# Ensure wallet can cover 4200 (extra bank withdraw if ladder did not fund enough)
|
||||
pay_need=$(python3 -c 'import sys; print(float(sys.argv[1].split(":",1)[-1]))' "$E2E_PAIVANA_AMOUNT" 2>/dev/null || echo 4200)
|
||||
av_now=$(wallet_avail_num)
|
||||
if ! python3 -c "import sys; sys.exit(0 if float(sys.argv[1]) + 1e-9 >= float(sys.argv[2]) else 1)" "$av_now" "$pay_need" 2>/dev/null; then
|
||||
info "paivana" "avail ${CUR}:${av_now} < ${pay_need} — ATM withdraw ${E2E_PAIVANA_AMOUNT}"
|
||||
info "paivana" "avail ${CUR}:${av_now} < ${pay_need} — Bank withdraw ${E2E_PAIVANA_AMOUNT}"
|
||||
e2e_over && warn "paivana" "budget low before 4200 withdraw"
|
||||
set +e
|
||||
e2e_one_withdraw "$E2E_PAIVANA_AMOUNT"
|
||||
|
|
@ -1688,7 +1720,7 @@ fi
|
|||
set_group report
|
||||
section "e2e · report"
|
||||
info "user" "$USER"
|
||||
info "ATM withdraws" "$WITHDRAW_REPORT"
|
||||
info "Bank withdraws" "$WITHDRAW_REPORT"
|
||||
info "payments" "$PAY_REPORT"
|
||||
info "shop" "${SHOP_REPORT:-(n/a)}"
|
||||
info "paivana" "${PAIVANA_REPORT:-(n/a)}"
|
||||
|
|
|
|||
1364
check_goa_ladder.sh
1364
check_goa_ladder.sh
File diff suppressed because it is too large
Load diff
|
|
@ -29,12 +29,14 @@ if [ -z "$PROFILE" ]; then
|
|||
fi
|
||||
fi
|
||||
|
||||
# Resolve host-podman vs ssh before first check ID
|
||||
# Resolve host-podman vs ssh before first check ID.
|
||||
# stage-lfp on the FP host itself uses local podman (INSIDE_PODMAN=1 /
|
||||
# INSIDE_MODE=local-podman); laptop → stage still uses INSIDE_SSH.
|
||||
_use_local_podman=0
|
||||
if [ "$PROFILE" != "stage-lfp" ]; then
|
||||
if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; then
|
||||
_use_local_podman=1
|
||||
elif command -v podman >/dev/null 2>&1 \
|
||||
if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; then
|
||||
_use_local_podman=1
|
||||
elif [ "$PROFILE" != "stage-lfp" ]; then
|
||||
if command -v podman >/dev/null 2>&1 \
|
||||
&& podman ps --format '{{.Names}}' 2>/dev/null | grep -qE 'taler-hacktivism'; then
|
||||
_use_local_podman=1
|
||||
fi
|
||||
|
|
@ -52,10 +54,10 @@ section "inside · collect (${PROFILE} · access=${INSIDE_ACCESS})"
|
|||
info "flags" "INSIDE_ACCESS=${INSIDE_ACCESS} INSIDE_PODMAN=${INSIDE_PODMAN:-0} INSIDE_MODE=${INSIDE_MODE:-} LOCAL_STACK=${LOCAL_STACK:-0} SKIP_SSH=${SKIP_SSH:-0} KOOPA_SSH=${KOOPA_SSH:-} INSIDE_SSH=${INSIDE_SSH:-}"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# stage-lfp: low-priv stagepaysan on francpaysan-host
|
||||
# stage-lfp: stagepaysan podman (host-local or SSH INSIDE_SSH from laptop)
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$PROFILE" = "stage-lfp" ]; then
|
||||
SSH_HOST="${INSIDE_SSH:-francpaysan-stage-user}"
|
||||
SSH_HOST="${INSIDE_SSH:-}"
|
||||
BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}"
|
||||
EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
|
||||
MER_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
|
||||
|
|
@ -69,15 +71,23 @@ if [ "$PROFILE" = "stage-lfp" ]; then
|
|||
STAGE_SSH_T="${INSIDE_SSH_TIMEOUT:-${SSH_CMD_TIMEOUT:-24}}"
|
||||
if [ "${STAGE_SSH_T}" -lt 24 ] 2>/dev/null; then STAGE_SSH_T=24; fi
|
||||
|
||||
if ! mon_ssh_ok "$SSH_HOST"; then
|
||||
err "ssh" "cannot reach ${SSH_HOST} (stagepaysan low-priv) — set INSIDE_SSH= or SKIP"
|
||||
if [ "$_use_local_podman" = "1" ]; then
|
||||
if ! command -v podman >/dev/null 2>&1; then
|
||||
err "host" "INSIDE_PODMAN/host-podman but podman missing"
|
||||
summary
|
||||
exit 1
|
||||
fi
|
||||
ok "host→container" "podman exec on this host (INSIDE_ACCESS=host-podman · no SSH)"
|
||||
elif ! mon_ssh_ok "$SSH_HOST"; then
|
||||
err "ssh" "cannot reach ${SSH_HOST:-?} (stagepaysan low-priv) — set INSIDE_SSH= or INSIDE_PODMAN=1"
|
||||
summary
|
||||
exit 1
|
||||
else
|
||||
ok "ssh ${SSH_HOST}" "stagepaysan (podman, no sudo)"
|
||||
fi
|
||||
ok "ssh ${SSH_HOST}" "stagepaysan (podman, no sudo)"
|
||||
|
||||
# Inject names/ports into remote (ssh bash -s does not inherit local env).
|
||||
RAW=$(
|
||||
# Inject names/ports (ssh bash -s does not inherit local env; local bash does).
|
||||
_stage_lfp_script() {
|
||||
{
|
||||
printf 'BANK_CTR=%q; EX_CTR=%q; MER_CTR=%q\n' "$BANK_CTR" "$EX_CTR" "$MER_CTR"
|
||||
printf 'BANK_PORT=%q; EX_PORT=%q; MER_PORT=%q\n' "$BANK_PORT" "$EX_PORT" "$MER_PORT"
|
||||
|
|
@ -173,11 +183,20 @@ fi
|
|||
|
||||
echo DONE
|
||||
REMOTE
|
||||
} | mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T}" || true
|
||||
)
|
||||
}
|
||||
}
|
||||
if [ "$_use_local_podman" = "1" ]; then
|
||||
RAW=$(_stage_lfp_script | bash || true)
|
||||
else
|
||||
RAW=$(_stage_lfp_script | mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T}" || true)
|
||||
fi
|
||||
|
||||
if [ -z "$RAW" ] || ! echo "$RAW" | grep -q '^E|'; then
|
||||
err "ssh" "stage remote timed out or empty (cap ${STAGE_SSH_T}s · host=${SSH_HOST})"
|
||||
if [ "$_use_local_podman" = "1" ]; then
|
||||
err "host" "stage collect timed out or empty (local podman)"
|
||||
else
|
||||
err "ssh" "stage remote timed out or empty (cap ${STAGE_SSH_T}s · host=${SSH_HOST})"
|
||||
fi
|
||||
summary
|
||||
exit 1
|
||||
fi
|
||||
|
|
@ -189,7 +208,7 @@ REMOTE
|
|||
IFS='|' read -r _ comp level key detail <<<"$line"
|
||||
case "$comp" in
|
||||
bank|exchange|merchant|caddy) _g="$comp" ;;
|
||||
*) _g="ssh" ;;
|
||||
*) _g="$INSIDE_ACCESS" ;;
|
||||
esac
|
||||
if [ "$_g" != "$_last_inside_grp" ]; then
|
||||
set_group "$_g"
|
||||
|
|
@ -242,7 +261,29 @@ REMOTE
|
|||
# Host load as stagepaysan (no container RSS from koopa metrics)
|
||||
set_group load
|
||||
section "inside · load (stagepaysan host)"
|
||||
LOAD_LINE=$(mon_ssh_bash "$SSH_HOST" 8 <<'EOF' || true
|
||||
_load_py() {
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
la=os.getloadavg()
|
||||
print("loadavg=%.2f,%.2f,%.2f" % la)
|
||||
try:
|
||||
with open("/proc/meminfo") as f:
|
||||
d={}
|
||||
for line in f:
|
||||
k,v=line.split(":")[0], line.split(":")[1].strip().split()[0]
|
||||
d[k]=int(v)
|
||||
total=d.get("MemTotal",0)/1024/1024
|
||||
avail=d.get("MemAvailable",0)/1024/1024
|
||||
used=total-avail
|
||||
print("mem_used=%.2fGiB avail=%.2fGiB total=%.2fGiB" % (used, avail, total))
|
||||
except Exception:
|
||||
print("mem=?")
|
||||
PY
|
||||
}
|
||||
if [ "$_use_local_podman" = "1" ]; then
|
||||
LOAD_LINE=$(_load_py || true)
|
||||
else
|
||||
LOAD_LINE=$(mon_ssh_bash "$SSH_HOST" 8 <<'EOF' || true
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
la=os.getloadavg()
|
||||
|
|
@ -262,6 +303,7 @@ except Exception:
|
|||
PY
|
||||
EOF
|
||||
)
|
||||
fi
|
||||
if [ -n "$LOAD_LINE" ]; then
|
||||
info "stage host" "$(echo "$LOAD_LINE" | tr '\n' ' ')"
|
||||
else
|
||||
|
|
@ -270,7 +312,8 @@ EOF
|
|||
|
||||
set_group disk
|
||||
section "inside · disk free space (stage host + containers)"
|
||||
_disk_raw=$(mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T:-24}" <<'DISK' || true
|
||||
_disk_script() {
|
||||
cat <<'DISK'
|
||||
set +e
|
||||
echo "###HOST###"
|
||||
df -Pk / /var /home /tmp /mnt/data 2>/dev/null || df -Pk
|
||||
|
|
@ -280,9 +323,16 @@ for c in $(podman ps --format '{{.Names}}' 2>/dev/null); do
|
|||
podman exec "$c" df -Pk / /var /tmp 2>/dev/null || podman exec "$c" df -Pk 2>/dev/null
|
||||
done
|
||||
DISK
|
||||
)
|
||||
}
|
||||
if [ "$_use_local_podman" = "1" ]; then
|
||||
_disk_raw=$(_disk_script | bash || true)
|
||||
_disk_label="host"
|
||||
else
|
||||
_disk_raw=$(_disk_script | mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T:-24}" || true)
|
||||
_disk_label="ssh:${SSH_HOST}"
|
||||
fi
|
||||
_host_df=$(printf '%s\n' "$_disk_raw" | sed -n '/^###HOST###$/,/^###CTRS###$/p' | sed '1d;$d')
|
||||
mon_disk_check_remote_text "ssh:${SSH_HOST}" "$_host_df" || true
|
||||
mon_disk_check_remote_text "$_disk_label" "$_host_df" || true
|
||||
_ctr=""; _buf=""
|
||||
while IFS= read -r _line || [ -n "$_line" ]; do
|
||||
case "$_line" in
|
||||
|
|
|
|||
292
check_mail.sh
Executable file
292
check_mail.sh
Executable file
|
|
@ -0,0 +1,292 @@
|
|||
#!/usr/bin/env bash
|
||||
# check_mail.sh — outside-in mail (MX / SMTP / IMAP / SPF / DMARC)
|
||||
#
|
||||
# Catalog: mail-catalog.conf (MAIL_CATALOG=… to override)
|
||||
# Covers firefly (taler.net, gnunet.org) and Anastasis/TSA
|
||||
# (mail.anastasis.lu → pixel.taler-systems.com; anastasis.lu).
|
||||
#
|
||||
# Outside-only. Phase: mail
|
||||
#
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
# shellcheck source=lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
|
||||
set_area mail
|
||||
section "mail · MX / SMTP / IMAP / SPF / DMARC (outside-in)"
|
||||
|
||||
CATALOG="${MAIL_CATALOG:-$ROOT/mail-catalog.conf}"
|
||||
PORT_TIMEOUT="${MAIL_PORT_TIMEOUT:-4}"
|
||||
SMTP_TIMEOUT="${MAIL_SMTP_TIMEOUT:-10}"
|
||||
|
||||
if [ ! -f "$CATALOG" ]; then
|
||||
fail "catalog" "missing $CATALOG"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# dig or host fallback
|
||||
dns_mx() {
|
||||
local d="$1"
|
||||
if command -v dig >/dev/null 2>&1; then
|
||||
dig +short +time=3 +tries=2 MX "$d" 2>/dev/null | awk '{print tolower($2)}' | sed 's/\.$//'
|
||||
else
|
||||
host -t MX "$d" 2>/dev/null | awk -F'in mail is |has address ' '/mail is/{print tolower($NF)}' | sed 's/\.$//'
|
||||
fi
|
||||
}
|
||||
|
||||
dns_a() {
|
||||
local h="$1"
|
||||
if command -v dig >/dev/null 2>&1; then
|
||||
dig +short +time=3 +tries=2 A "$h" 2>/dev/null | grep -E '^[0-9.]+$' || true
|
||||
dig +short +time=3 +tries=2 AAAA "$h" 2>/dev/null | grep -E ':' || true
|
||||
else
|
||||
getent ahosts "$h" 2>/dev/null | awk '{print $1}' | sort -u
|
||||
fi
|
||||
}
|
||||
|
||||
dns_txt() {
|
||||
local name="$1"
|
||||
if command -v dig >/dev/null 2>&1; then
|
||||
dig +short +time=3 +tries=2 TXT "$name" 2>/dev/null | tr -d '"'
|
||||
else
|
||||
host -t TXT "$name" 2>/dev/null | sed 's/.*"\(.*\)"/\1/'
|
||||
fi
|
||||
}
|
||||
|
||||
tcp_open() {
|
||||
local h="$1" p="$2"
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout "$PORT_TIMEOUT" bash -c "echo >/dev/tcp/${h}/${p}" 2>/dev/null
|
||||
else
|
||||
bash -c "echo >/dev/tcp/${h}/${p}" 2>/dev/null
|
||||
fi
|
||||
}
|
||||
|
||||
# SMTP: read banner first, then EHLO (avoids "protocol synchronization")
|
||||
smtp_probe() {
|
||||
local host="$1" port="$2"
|
||||
MAIL_HOST="$host" MAIL_PORT="$port" MAIL_TO="${SMTP_TIMEOUT}" python3 - <<'PY' 2>/dev/null
|
||||
import os, socket, sys
|
||||
host = os.environ["MAIL_HOST"]
|
||||
port = int(os.environ["MAIL_PORT"])
|
||||
to = float(os.environ.get("MAIL_TO", "10"))
|
||||
try:
|
||||
s = socket.create_connection((host, port), to)
|
||||
s.settimeout(to)
|
||||
banner = s.recv(1024).decode("utf-8", "replace").strip().split("\n")[0]
|
||||
if not banner.startswith("220"):
|
||||
print(f"bad_banner={banner[:80]}")
|
||||
sys.exit(1)
|
||||
s.sendall(b"EHLO taler-monitoring.invalid\r\n")
|
||||
data = b""
|
||||
while True:
|
||||
chunk = s.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
data += chunk
|
||||
if b"\n" in chunk and (data.endswith(b"\r\n") or len(data) > 8000):
|
||||
# multi-line 250-… ends with 250 space
|
||||
lines = data.decode("utf-8", "replace").splitlines()
|
||||
if any(l.startswith("250 ") for l in lines):
|
||||
break
|
||||
if any(l.startswith("5") for l in lines[:3]):
|
||||
break
|
||||
text = data.decode("utf-8", "replace")
|
||||
s.sendall(b"QUIT\r\n")
|
||||
try:
|
||||
s.recv(256)
|
||||
except Exception:
|
||||
pass
|
||||
s.close()
|
||||
starttls = "starttls" if "STARTTLS" in text.upper() else "no_starttls"
|
||||
print(f"banner={banner[:60]} · {starttls}")
|
||||
sys.exit(0)
|
||||
except Exception as e:
|
||||
print(f"err={e}")
|
||||
sys.exit(1)
|
||||
PY
|
||||
}
|
||||
|
||||
# IMAP unencrypted greeting on 143, or just TCP on 993 (TLS)
|
||||
imap_probe() {
|
||||
local host="$1" port="$2"
|
||||
if [ "$port" = "993" ] || [ "$port" = "995" ]; then
|
||||
if tcp_open "$host" "$port"; then
|
||||
echo "tcp_open tls_port=$port"
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
fi
|
||||
MAIL_HOST="$host" MAIL_PORT="$port" MAIL_TO="${SMTP_TIMEOUT}" python3 - <<'PY' 2>/dev/null
|
||||
import os, socket, sys
|
||||
host = os.environ["MAIL_HOST"]
|
||||
port = int(os.environ["MAIL_PORT"])
|
||||
to = float(os.environ.get("MAIL_TO", "10"))
|
||||
try:
|
||||
s = socket.create_connection((host, port), to)
|
||||
s.settimeout(to)
|
||||
banner = s.recv(1024).decode("utf-8", "replace").strip().split("\n")[0]
|
||||
s.sendall(b"a001 LOGOUT\r\n")
|
||||
try:
|
||||
s.recv(256)
|
||||
except Exception:
|
||||
pass
|
||||
s.close()
|
||||
if banner.upper().startswith("* OK") or "IMAP" in banner.upper():
|
||||
print(f"banner={banner[:70]}")
|
||||
sys.exit(0)
|
||||
print(f"unexpected={banner[:70]}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(f"err={e}")
|
||||
sys.exit(1)
|
||||
PY
|
||||
}
|
||||
|
||||
info "catalog" "$CATALOG"
|
||||
|
||||
# Track probed hosts to avoid duplicate SMTP checks
|
||||
declare -A HOST_DONE=()
|
||||
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
line=${line%%#*}
|
||||
line=$(echo "$line" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
|
||||
[ -z "$line" ] && continue
|
||||
# shellcheck disable=SC2086
|
||||
set -- $line
|
||||
domain=$1
|
||||
expect_mx=${2:-}
|
||||
mail_hosts=${3:-}
|
||||
smtp_ports=${4:-25,465,587}
|
||||
imap_ports=${5:-143,993}
|
||||
label=${6:-$domain}
|
||||
|
||||
set_group "$label"
|
||||
info "domain" "$domain · expected_mx=$expect_mx · hosts=$mail_hosts"
|
||||
|
||||
# --- MX ---
|
||||
mapfile -t mx_hosts < <(dns_mx "$domain" | sed '/^$/d')
|
||||
if [ "${#mx_hosts[@]}" -eq 0 ]; then
|
||||
fail "mx" "$domain has no MX records"
|
||||
else
|
||||
ok "mx" "$domain MX → ${mx_hosts[*]}"
|
||||
# expected MX match (suffix / exact)
|
||||
if [ -n "$expect_mx" ]; then
|
||||
matched=0
|
||||
IFS=',' read -ra want_list <<<"$expect_mx"
|
||||
for w in "${want_list[@]}"; do
|
||||
w=$(echo "$w" | tr '[:upper:]' '[:lower:]' | sed 's/\.$//')
|
||||
for m in "${mx_hosts[@]}"; do
|
||||
m=$(echo "$m" | tr '[:upper:]' '[:lower:]')
|
||||
case "$m" in
|
||||
"$w"|"$w".*|*."$w") matched=1; break ;;
|
||||
esac
|
||||
[ "$m" = "$w" ] && matched=1
|
||||
done
|
||||
[ "$matched" = "1" ] && break
|
||||
done
|
||||
if [ "$matched" = "1" ]; then
|
||||
ok "mx expected" "$domain MX matches catalog ($expect_mx)"
|
||||
else
|
||||
fail "mx expected" "$domain MX ${mx_hosts[*]} · want one of: $expect_mx"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- SPF / DMARC ---
|
||||
spf=$(dns_txt "$domain" | tr '\n' ' ')
|
||||
if echo "$spf" | grep -qi 'v=spf1'; then
|
||||
ok "spf" "$domain has SPF"
|
||||
else
|
||||
warn "spf" "$domain no SPF TXT (v=spf1)"
|
||||
fi
|
||||
dmarc=$(dns_txt "_dmarc.$domain" | tr '\n' ' ')
|
||||
if echo "$dmarc" | grep -qi 'v=dmarc1'; then
|
||||
ok "dmarc" "$domain has DMARC"
|
||||
else
|
||||
warn "dmarc" "$domain no DMARC at _dmarc.$domain"
|
||||
fi
|
||||
|
||||
# --- mail hosts: DNS + ports + banners ---
|
||||
IFS=',' read -ra hosts <<<"$mail_hosts"
|
||||
for h in "${hosts[@]}"; do
|
||||
h=$(echo "$h" | tr '[:upper:]' '[:lower:]' | sed 's/\.$//;s/^[[:space:]]*//;s/[[:space:]]*$//')
|
||||
[ -z "$h" ] && continue
|
||||
|
||||
addrs=$(dns_a "$h" | tr '\n' ' ')
|
||||
if [ -z "${addrs// }" ]; then
|
||||
fail "dns" "$h does not resolve (A/AAAA)"
|
||||
continue
|
||||
fi
|
||||
ok "dns" "$h → $addrs"
|
||||
|
||||
# skip full protocol re-probe if already done
|
||||
if [ "${HOST_DONE[$h]:-}" = "1" ]; then
|
||||
info "host" "$h already probed this run"
|
||||
continue
|
||||
fi
|
||||
HOST_DONE[$h]=1
|
||||
|
||||
IFS=',' read -ra sports <<<"$smtp_ports"
|
||||
for p in "${sports[@]}"; do
|
||||
p=${p// /}
|
||||
[ -z "$p" ] && continue
|
||||
if ! tcp_open "$h" "$p"; then
|
||||
# 587 optional on pixel
|
||||
case "$p" in
|
||||
587) warn "smtp port" "$h:$p closed/filtered (submission)" ;;
|
||||
*) fail "smtp port" "$h:$p closed/filtered" ;;
|
||||
esac
|
||||
continue
|
||||
fi
|
||||
case "$p" in
|
||||
465)
|
||||
ok "smtp port" "$h:$p open (SMTPS)"
|
||||
;;
|
||||
25|587)
|
||||
if detail=$(smtp_probe "$h" "$p"); then
|
||||
ok "smtp" "$h:$p $detail"
|
||||
else
|
||||
# port open but banner failed — still ERROR for 25
|
||||
if [ "$p" = "25" ]; then
|
||||
fail "smtp" "$h:$p open but SMTP handshake failed · $detail"
|
||||
else
|
||||
warn "smtp" "$h:$p open but handshake failed · $detail"
|
||||
fi
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
ok "smtp port" "$h:$p open"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
IFS=',' read -ra iports <<<"$imap_ports"
|
||||
for p in "${iports[@]}"; do
|
||||
p=${p// /}
|
||||
[ -z "$p" ] && continue
|
||||
if ! tcp_open "$h" "$p"; then
|
||||
fail "imap port" "$h:$p closed/filtered"
|
||||
continue
|
||||
fi
|
||||
case "$p" in
|
||||
993|995)
|
||||
ok "imap port" "$h:$p open (TLS)"
|
||||
;;
|
||||
143|110)
|
||||
if detail=$(imap_probe "$h" "$p"); then
|
||||
ok "imap" "$h:$p $detail"
|
||||
else
|
||||
warn "imap" "$h:$p open but greeting failed · $detail"
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
ok "imap port" "$h:$p open"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
done
|
||||
done <"$CATALOG"
|
||||
|
||||
info "hint" "firefly = taler.net/gnunet.org MX; pixel = taler-systems.com (mail.anastasis.lu / mail.taler-systems.com)"
|
||||
exit 0
|
||||
255
check_mattermost.sh
Executable file
255
check_mattermost.sh
Executable file
|
|
@ -0,0 +1,255 @@
|
|||
#!/usr/bin/env bash
|
||||
# check_mattermost.sh — outside-in checks for Taler Mattermost (chat)
|
||||
#
|
||||
# Default host: mattermost.taler.net
|
||||
# Override: MATTERMOST_PUBLIC=https://mattermost.example.org
|
||||
# MATTERMOST_HOST=mattermost.example.org
|
||||
#
|
||||
# Client compatibility (feature) — server min for current *official* clients:
|
||||
# Android / iOS store apps → default ≥ 10.11.0 (Play/App Store + mobile docs 2026)
|
||||
# Desktop (Win/Mac/Linux) → default ≥ 10.11.0 (current Desktop / ESR pairing)
|
||||
# Vendor support floor → default ≥ 10.11.0 (ESR; older = unsupported)
|
||||
# Web SPA → served by this server (always loads) but WARN if
|
||||
# server is below support floor (security/EOL).
|
||||
#
|
||||
# Env:
|
||||
# MATTERMOST_CLIENT_CHECK=1|0 master switch (default 1)
|
||||
# MATTERMOST_MOBILE_CHECK=1|0 alias / legacy (if set 0 and CLIENT unset → off)
|
||||
# MATTERMOST_MOBILE_MIN_SERVER default 10.11.0
|
||||
# MATTERMOST_DESKTOP_MIN_SERVER default 10.11.0
|
||||
# MATTERMOST_SUPPORT_MIN_SERVER default 10.11.0 (vendor-supported floor)
|
||||
#
|
||||
# Outside-only (no SSH). Phase name: mattermost
|
||||
#
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
# shellcheck source=lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
|
||||
set_area mattermost
|
||||
section "mattermost · public chat (outside-in)"
|
||||
|
||||
# Defaults: current official clients + vendor ESR floor (2026-07 docs).
|
||||
: "${MATTERMOST_MOBILE_MIN_SERVER:=10.11.0}"
|
||||
: "${MATTERMOST_DESKTOP_MIN_SERVER:=10.11.0}"
|
||||
: "${MATTERMOST_SUPPORT_MIN_SERVER:=10.11.0}"
|
||||
# Master switch: CLIENT_CHECK, else legacy MOBILE_CHECK, else on
|
||||
if [ -n "${MATTERMOST_CLIENT_CHECK:-}" ]; then
|
||||
:
|
||||
elif [ -n "${MATTERMOST_MOBILE_CHECK:-}" ]; then
|
||||
MATTERMOST_CLIENT_CHECK="${MATTERMOST_MOBILE_CHECK}"
|
||||
else
|
||||
MATTERMOST_CLIENT_CHECK=1
|
||||
fi
|
||||
: "${MATTERMOST_MOBILE_CHECK:=${MATTERMOST_CLIENT_CHECK}}"
|
||||
|
||||
if [ -n "${MATTERMOST_PUBLIC:-}" ]; then
|
||||
BASE="${MATTERMOST_PUBLIC%/}"
|
||||
elif [ -n "${MATTERMOST_HOST:-}" ]; then
|
||||
BASE="https://${MATTERMOST_HOST}"
|
||||
else
|
||||
BASE="https://mattermost.taler.net"
|
||||
fi
|
||||
HOST=${BASE#https://}
|
||||
HOST=${HOST#http://}
|
||||
HOST=${HOST%%/*}
|
||||
|
||||
info "target" "$BASE"
|
||||
info "client policy" "mobile≥${MATTERMOST_MOBILE_MIN_SERVER} · desktop≥${MATTERMOST_DESKTOP_MIN_SERVER} · support≥${MATTERMOST_SUPPORT_MIN_SERVER} (CLIENT_CHECK=${MATTERMOST_CLIENT_CHECK})"
|
||||
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
# GET with follow redirects (Mattermost SPA often 302 → /)
|
||||
mm_get() {
|
||||
local url="$1" out="$2"
|
||||
curl -skS -L --max-redirs 5 -m "${TIMEOUT:-12}" -o "$out" -w '%{http_code}' "$url" 2>/dev/null || echo 000
|
||||
}
|
||||
|
||||
# --- DNS / HTTPS landing ---
|
||||
set_group landing
|
||||
code=$(mm_get "$BASE/" "$tmp/root.html")
|
||||
if [ "$code" = "200" ] && head -c 512 "$tmp/root.html" | grep -qiE '<!doctype html|<html'; then
|
||||
ok "landing" "$BASE/ → HTTP $code HTML"
|
||||
else
|
||||
fail "landing" "$BASE/ → HTTP $code (want 200 HTML Mattermost SPA)"
|
||||
fi
|
||||
|
||||
# --- Official Mattermost health API + server version (X-Version-Id) ---
|
||||
set_group api
|
||||
# Capture body + headers (version is in X-Version-Id, not always in JSON)
|
||||
hdr_file="$tmp/ping.hdr"
|
||||
code=$(curl -skS -L --max-redirs 5 -m "${TIMEOUT:-12}" \
|
||||
-D "$hdr_file" -o "$tmp/ping.json" -w '%{http_code}' \
|
||||
"$BASE/api/v4/system/ping" 2>/dev/null || echo 000)
|
||||
|
||||
MM_SERVER_VER=""
|
||||
if [ -f "$hdr_file" ]; then
|
||||
# X-Version-Id: 9.5.3.8427860509.c8cabb… or 10.11.0.…
|
||||
MM_SERVER_VER=$(grep -i '^x-version-id:' "$hdr_file" | head -1 \
|
||||
| sed 's/^[Xx]-[Vv]ersion-[Ii]d:[[:space:]]*//' | tr -d '\r' \
|
||||
| sed 's/^\([0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\).*/\1/')
|
||||
fi
|
||||
|
||||
if [ "$code" != "200" ]; then
|
||||
fail "system/ping" "$BASE/api/v4/system/ping → HTTP $code (want 200 JSON)"
|
||||
else
|
||||
if python3 - "$tmp/ping.json" <<'PY' 2>/dev/null
|
||||
import json, sys
|
||||
with open(sys.argv[1]) as f:
|
||||
d = json.load(f)
|
||||
sys.exit(0 if isinstance(d, dict) else 2)
|
||||
PY
|
||||
then
|
||||
ok "system/ping" "$BASE/api/v4/system/ping → HTTP 200 JSON${MM_SERVER_VER:+ · server ${MM_SERVER_VER}}"
|
||||
else
|
||||
fail "system/ping" "$BASE/api/v4/system/ping → HTTP 200 but not valid JSON object"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Server version vs official client families (mobile / desktop / support / web) ---
|
||||
set_group clients
|
||||
mm_ver_ge() {
|
||||
# $1=server $2=need → 0 if server >= need
|
||||
python3 - "$1" "$2" <<'PY'
|
||||
import sys
|
||||
|
||||
def parse(v: str):
|
||||
parts = []
|
||||
for p in v.strip().split(".")[:3]:
|
||||
try:
|
||||
parts.append(int(p))
|
||||
except ValueError:
|
||||
parts.append(0)
|
||||
while len(parts) < 3:
|
||||
parts.append(0)
|
||||
return tuple(parts)
|
||||
|
||||
sys.exit(0 if parse(sys.argv[1]) >= parse(sys.argv[2]) else 1)
|
||||
PY
|
||||
}
|
||||
|
||||
# One policy line: id|min_env_default|severity(error|warn)|human label
|
||||
# severity: error → fail; warn → warn
|
||||
_mm_client_policies() {
|
||||
printf '%s\n' \
|
||||
"mobile|${MATTERMOST_MOBILE_MIN_SERVER}|error|Android/iOS store apps (Mattermost Mobile)" \
|
||||
"desktop|${MATTERMOST_DESKTOP_MIN_SERVER}|error|Desktop app (Windows/macOS/Linux official)" \
|
||||
"support|${MATTERMOST_SUPPORT_MIN_SERVER}|error|vendor-supported server floor (ESR/current)" \
|
||||
"web|${MATTERMOST_SUPPORT_MIN_SERVER}|warn|browser webapp (loads with server, but EOL server is insecure/unsupported)"
|
||||
}
|
||||
|
||||
if [ "${MATTERMOST_CLIENT_CHECK}" = "0" ]; then
|
||||
info "client compat" "skipped (MATTERMOST_CLIENT_CHECK=0 / MATTERMOST_MOBILE_CHECK=0)"
|
||||
elif [ -z "$MM_SERVER_VER" ]; then
|
||||
warn "server version" "could not parse X-Version-Id from /api/v4/system/ping — cannot judge client compatibility"
|
||||
else
|
||||
info "server version" "X-Version-Id → ${MM_SERVER_VER}"
|
||||
while IFS='|' read -r cid cmin csever clabel; do
|
||||
[ -n "$cid" ] || continue
|
||||
if mm_ver_ge "$MM_SERVER_VER" "$cmin"; then
|
||||
ok "${cid} compat" "server ${MM_SERVER_VER} ≥ ${cmin} · ${clabel} OK"
|
||||
else
|
||||
detail="server ${MM_SERVER_VER} < ${cmin} · ${clabel}"
|
||||
hint="upgrade Mattermost server · set MATTERMOST_${cid^^}_MIN_SERVER= or MATTERMOST_CLIENT_CHECK=0"
|
||||
# Fix hint for support/web keys
|
||||
case "$cid" in
|
||||
mobile) hint="upgrade server · MATTERMOST_MOBILE_MIN_SERVER= / MATTERMOST_CLIENT_CHECK=0" ;;
|
||||
desktop) hint="upgrade server · MATTERMOST_DESKTOP_MIN_SERVER= / MATTERMOST_CLIENT_CHECK=0" ;;
|
||||
support) hint="server below vendor support floor · MATTERMOST_SUPPORT_MIN_SERVER= / MATTERMOST_CLIENT_CHECK=0" ;;
|
||||
web) hint="web SPA is served by this old server (works) but platform is unsupported · upgrade recommended" ;;
|
||||
esac
|
||||
if [ "$csever" = "error" ]; then
|
||||
fail "${cid} compat" "$detail" "$hint"
|
||||
else
|
||||
warn "${cid} compat" "$detail · $hint"
|
||||
fi
|
||||
fi
|
||||
done < <(_mm_client_policies)
|
||||
fi
|
||||
|
||||
# Client config (optional): BuildDate / advertised Android/iOS min if server sets it
|
||||
set_group client-config
|
||||
code=$(mm_get "$BASE/api/v4/config/client?format=old" "$tmp/client.json")
|
||||
if [ "$code" = "200" ] && [ -s "$tmp/client.json" ]; then
|
||||
build=$(python3 - "$tmp/client.json" <<'PY' 2>/dev/null
|
||||
import json, sys
|
||||
with open(sys.argv[1]) as f:
|
||||
d = json.load(f)
|
||||
print(d.get("BuildDate") or d.get("Version") or "")
|
||||
PY
|
||||
)
|
||||
and_min=$(python3 - "$tmp/client.json" <<'PY' 2>/dev/null
|
||||
import json, sys
|
||||
with open(sys.argv[1]) as f:
|
||||
d = json.load(f)
|
||||
print((d.get("AndroidMinVersion") or "").strip())
|
||||
PY
|
||||
)
|
||||
ios_min=$(python3 - "$tmp/client.json" <<'PY' 2>/dev/null
|
||||
import json, sys
|
||||
with open(sys.argv[1]) as f:
|
||||
d = json.load(f)
|
||||
print((d.get("IosMinVersion") or d.get("IOSMinVersion") or "").strip())
|
||||
PY
|
||||
)
|
||||
if [ -n "$build" ]; then
|
||||
info "client config" "BuildDate/Version=${build}${and_min:+ · AndroidMinVersion=${and_min}}${ios_min:+ · IosMinVersion=${ios_min}}"
|
||||
else
|
||||
info "client config" "HTTP 200 (no BuildDate field)"
|
||||
fi
|
||||
if [ -n "$and_min" ] && [ "$and_min" != "0.0.0" ]; then
|
||||
info "android min (server)" "server advertises AndroidMinVersion=${and_min}"
|
||||
fi
|
||||
if [ -n "$ios_min" ] && [ "$ios_min" != "0.0.0" ]; then
|
||||
info "ios min (server)" "server advertises IosMinVersion=${ios_min}"
|
||||
fi
|
||||
else
|
||||
info "client config" "HTTP ${code:-?} — optional"
|
||||
fi
|
||||
|
||||
# --- Login SPA (must be served for users) ---
|
||||
set_group login
|
||||
code=$(mm_get "$BASE/login" "$tmp/login.html")
|
||||
if [ "$code" = "200" ] && head -c 512 "$tmp/login.html" | grep -qiE '<!doctype html|<html'; then
|
||||
ok "login" "$BASE/login → HTTP $code HTML"
|
||||
else
|
||||
fail "login" "$BASE/login → HTTP $code (want 200 HTML)"
|
||||
fi
|
||||
|
||||
# --- TLS certificate expiry (openssl s_client if available) ---
|
||||
set_group tls
|
||||
if command -v openssl >/dev/null 2>&1; then
|
||||
end=$(echo | openssl s_client -servername "$HOST" -connect "${HOST}:443" 2>/dev/null \
|
||||
| openssl x509 -noout -enddate 2>/dev/null | sed 's/notAfter=//')
|
||||
if [ -n "$end" ]; then
|
||||
# days remaining
|
||||
end_epoch=$(date -d "$end" +%s 2>/dev/null || date -j -f "%b %d %T %Y %Z" "$end" +%s 2>/dev/null || echo 0)
|
||||
now_epoch=$(date +%s)
|
||||
if [ "$end_epoch" -gt 0 ]; then
|
||||
days=$(( (end_epoch - now_epoch) / 86400 ))
|
||||
if [ "$days" -lt 0 ]; then
|
||||
fail "tls cert" "$HOST cert expired ($end)"
|
||||
elif [ "$days" -lt 14 ]; then
|
||||
warn "tls cert" "$HOST expires in ${days}d ($end)"
|
||||
elif [ "$days" -lt 30 ]; then
|
||||
warn "tls cert" "$HOST expires in ${days}d ($end)"
|
||||
else
|
||||
ok "tls cert" "$HOST valid · ${days}d left · $end"
|
||||
fi
|
||||
else
|
||||
info "tls cert" "$HOST notAfter=$end"
|
||||
fi
|
||||
else
|
||||
warn "tls cert" "$HOST could not read certificate"
|
||||
fi
|
||||
else
|
||||
info "tls cert" "openssl not available — skip expiry check"
|
||||
fi
|
||||
|
||||
info "hint" "surface catalog also lists $HOST; client floors: mobile/desktop/support ≥${MATTERMOST_MOBILE_MIN_SERVER}/${MATTERMOST_DESKTOP_MIN_SERVER}/${MATTERMOST_SUPPORT_MIN_SERVER}"
|
||||
# Non-zero when any fail/err so parent phase + mon pages mark ERROR
|
||||
if [ "${FAIL_N:-0}" -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
exit 0
|
||||
673
check_monitoring_pages.sh
Executable file
673
check_monitoring_pages.sh
Executable file
|
|
@ -0,0 +1,673 @@
|
|||
#!/usr/bin/env bash
|
||||
# check_monitoring_pages.sh — verify public monitoring HTML pages via FQDN
|
||||
#
|
||||
# Outside-in: curl https://<fqdn>/<path>/ and require a real HTML monitoring page.
|
||||
# Catches: merchant JSON code 21 (Caddy handle missing), WP 404, empty stubs, deploy skip.
|
||||
#
|
||||
# Policy (v1.7.6+):
|
||||
# • monpages is obligatory → missing/wrong pages are ERROR (exit 1).
|
||||
# • GOA / hacktivism: full inventory of suite monitoring sites (catalog + on-disk discovery).
|
||||
# • FP: only that stack’s own monitoring hosts/paths (never GOA URLs).
|
||||
# • Override soft mode only with MONPAGES_REQUIRE_PUBLIC=0 (emergency / staging).
|
||||
#
|
||||
# Env:
|
||||
# MON_HOSTS space-separated FQDNs (default from TALER_DOMAIN)
|
||||
# HTML_URL_OK path for OK page (default /monitoring/)
|
||||
# HTML_URL_ERR err path; checked if present on disk or MONPAGES_CHECK_ERR=1
|
||||
# MONITORING_PAGE_URLS optional full URL list (overrides inventory construction)
|
||||
# MONPAGES_EXTRA_URLS additional absolute URLs (space-separated)
|
||||
# MONPAGES_INVENTORY auto|full|job (default auto)
|
||||
# auto/full = stack inventory; job = only MON_HOSTS+paths
|
||||
# MONPAGES_REQUIRE_PUBLIC default 1 (ERROR). 0 = WARN only (escape hatch).
|
||||
# MONPAGES_STAGING_BASE / HTML_OUT / DEPLOY_WWW_ROOT trees for discovery
|
||||
# TIMEOUT curl timeout (from lib.sh)
|
||||
#
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
# shellcheck source=lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
|
||||
set_area monpages
|
||||
section "monpages · public monitoring HTML via FQDN (outside-in, obligatory)"
|
||||
|
||||
# Obligatory: public pages missing → ERROR. Soft only if explicitly disabled.
|
||||
: "${MONPAGES_REQUIRE_PUBLIC:=1}"
|
||||
: "${MONPAGES_INVENTORY:=auto}"
|
||||
|
||||
# Default hosts from domain (mirror run-host-report.sh)
|
||||
if [ -z "${MON_HOSTS:-}" ]; then
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
hacktivism.ch|koopa)
|
||||
MON_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch"
|
||||
;;
|
||||
stage.lefrancpaysan.ch|stage.*lefrancpaysan*)
|
||||
MON_HOSTS="stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch"
|
||||
;;
|
||||
lefrancpaysan.ch)
|
||||
MON_HOSTS="bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch"
|
||||
;;
|
||||
*)
|
||||
MON_HOSTS="${TALER_DOMAIN:-}"
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
HTML_URL_OK="${HTML_URL_OK:-/monitoring/}"
|
||||
HTML_URL_ERR="${HTML_URL_ERR:-/monitoring_err/}"
|
||||
# ensure leading + trailing slash
|
||||
_norm_path() {
|
||||
local p="$1"
|
||||
case "$p" in
|
||||
/*) ;;
|
||||
*) p="/$p" ;;
|
||||
esac
|
||||
case "$p" in
|
||||
*/) ;;
|
||||
*) p="${p}/" ;;
|
||||
esac
|
||||
printf '%s' "$p"
|
||||
}
|
||||
HTML_URL_OK="$(_norm_path "$HTML_URL_OK")"
|
||||
HTML_URL_ERR="$(_norm_path "$HTML_URL_ERR")"
|
||||
|
||||
# Stack family: goa | fp | other — FP never checks GOA and vice versa.
|
||||
_monpages_family() {
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
*lefrancpaysan*|*francpaysan*) printf 'fp' ;;
|
||||
hacktivism.ch|koopa) printf 'goa' ;;
|
||||
*)
|
||||
# Infer from MON_HOSTS if domain ambiguous
|
||||
case " ${MON_HOSTS:-} " in
|
||||
*lefrancpaysan*) printf 'fp' ;;
|
||||
*hacktivism*) printf 'goa' ;;
|
||||
*) printf 'other' ;;
|
||||
esac
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
_host_allowed() {
|
||||
local h="$1" fam="$2"
|
||||
case "$fam" in
|
||||
goa)
|
||||
case "$h" in *.hacktivism.ch) return 0 ;; *) return 1 ;; esac
|
||||
;;
|
||||
fp)
|
||||
case "$h" in *lefrancpaysan*) return 0 ;; *) return 1 ;; esac
|
||||
;;
|
||||
*) return 0 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Public mon URL families (suite name "taler-monitoring" ≠ a public path):
|
||||
# 1) /monitoring(+_err) — landing hosts (bare + slash)
|
||||
# 2) /monitoring-max-ladder(+_err) — GOA max-search amount ladder (daily, hacktivism)
|
||||
# 3) /taler-monitoring-surface(+_err) — taler.hacktivism.ch only
|
||||
# 4) /taler-monitoring-aptdeploy(+_err) — taler.hacktivism.ch only
|
||||
# No classic /monitoring-ladder. No /taler-monitoring page.
|
||||
|
||||
_path_allowed() {
|
||||
case "$1" in
|
||||
monitoring|monitoring_err|\
|
||||
monitoring-max-ladder|monitoring-max-ladder_err|\
|
||||
taler-monitoring-surface|taler-monitoring-surface_err|\
|
||||
taler-monitoring-aptdeploy|taler-monitoring-aptdeploy_err) return 0 ;;
|
||||
# explicit deny (never invent a mon page here)
|
||||
monitoring-ladder|monitoring-ladder_err|\
|
||||
taler-monitoring|taler-monitoring-mail|taler-monitoring-mattermost) return 1 ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Emit https://host/path/ for each …/host/path/index.html under root (max depth 3).
|
||||
_discover_tree() {
|
||||
local root="$1" fam="$2"
|
||||
local f rel host path
|
||||
[ -n "$root" ] && [ -d "$root" ] || return 0
|
||||
# layout: $root/<fqdn>/<url-path>/index.html
|
||||
while IFS= read -r -d '' f; do
|
||||
rel="${f#"${root%/}"/}"
|
||||
host="${rel%%/*}"
|
||||
path="${rel#*/}"
|
||||
path="${path%/index.html}"
|
||||
[ -n "$host" ] && [ -n "$path" ] || continue
|
||||
_host_allowed "$host" "$fam" || continue
|
||||
_path_allowed "$path" || continue
|
||||
printf 'https://%s/%s/\n' "$host" "$path"
|
||||
done < <(find "${root%/}" -mindepth 3 -maxdepth 3 -type f -name index.html -print0 2>/dev/null)
|
||||
}
|
||||
|
||||
# Landing stacks (GOA 3 + FP stage 3) with /monitoring/
|
||||
_landing_allowed() {
|
||||
printf '%s\n' \
|
||||
bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch \
|
||||
stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch
|
||||
}
|
||||
|
||||
# Suite catalog of OK pages that must exist for this family.
|
||||
_catalog_urls() {
|
||||
local fam="$1" h
|
||||
case "$fam" in
|
||||
goa)
|
||||
for h in bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch; do
|
||||
printf 'https://%s/monitoring/\n' "$h"
|
||||
printf 'https://%s/monitoring-max-ladder/\n' "$h"
|
||||
done
|
||||
# ecosystem surface (versions / software) + apt-src deploy mon page
|
||||
printf 'https://taler.hacktivism.ch/taler-monitoring-surface/\n'
|
||||
printf 'https://taler.hacktivism.ch/taler-monitoring-aptdeploy/\n'
|
||||
;;
|
||||
fp)
|
||||
for h in $MON_HOSTS; do
|
||||
[ -z "$h" ] && continue
|
||||
_host_allowed "$h" fp || continue
|
||||
printf 'https://%s/monitoring/\n' "$h"
|
||||
done
|
||||
;;
|
||||
all|nine)
|
||||
# GOA + FP stage landings + surface + aptdeploy (nine kept as alias for inventory)
|
||||
while IFS= read -r h; do
|
||||
[ -n "$h" ] || continue
|
||||
printf 'https://%s/monitoring/\n' "$h"
|
||||
case "$h" in
|
||||
*.hacktivism.ch)
|
||||
printf 'https://%s/monitoring-max-ladder/\n' "$h"
|
||||
;;
|
||||
esac
|
||||
done < <(_landing_allowed)
|
||||
printf 'https://taler.hacktivism.ch/taler-monitoring-surface/\n'
|
||||
printf 'https://taler.hacktivism.ch/taler-monitoring-aptdeploy/\n'
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Job-local URLs (this host-agent run’s path).
|
||||
_job_urls() {
|
||||
local h p
|
||||
for h in $MON_HOSTS; do
|
||||
[ -z "$h" ] && continue
|
||||
printf 'https://%s%s\n' "$h" "$HTML_URL_OK"
|
||||
if [ "${MONPAGES_CHECK_ERR:-0}" = "1" ]; then
|
||||
printf 'https://%s%s\n' "$h" "$HTML_URL_ERR"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
build_urls() {
|
||||
local fam inv
|
||||
fam="$(_monpages_family)"
|
||||
inv="${MONPAGES_INVENTORY:-auto}"
|
||||
|
||||
if [ -n "${MONITORING_PAGE_URLS:-}" ]; then
|
||||
# shellcheck disable=SC2086
|
||||
printf '%s\n' $MONITORING_PAGE_URLS
|
||||
else
|
||||
case "$inv" in
|
||||
job)
|
||||
# Only this host-agent job’s HTML paths (MON_HOSTS + HTML_URL_OK/_ERR).
|
||||
# Specialized timers (aptdeploy, surface) must not inherit GOA full catalog
|
||||
# (no bank/exchange /monitoring bare-URL noise on the aptdeploy page).
|
||||
_job_urls
|
||||
;;
|
||||
full|auto|*)
|
||||
# Always include this job’s paths (host-agent post-check).
|
||||
_job_urls
|
||||
# Stack inventory: GOA = landings + surface + aptdeploy; FP = only FP mon hosts.
|
||||
# Full inventory belongs on the general stack run (run-hacktivism-monitoring).
|
||||
if [ "$fam" = "goa" ] || [ "$fam" = "fp" ]; then
|
||||
_catalog_urls "$fam"
|
||||
# On-disk discovery (err pages only exist after failed runs; extras welcome)
|
||||
[ -n "${DEPLOY_WWW_ROOT:-}" ] && _discover_tree "${DEPLOY_WWW_ROOT}" "$fam"
|
||||
_discover_tree "${MONPAGES_STAGING_BASE:-${HTML_OUT:-${HTML_BASE:-$HOME/monitoring-sites-staging}}}" "$fam"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
if [ -n "${MONPAGES_EXTRA_URLS:-}" ]; then
|
||||
# shellcheck disable=SC2086
|
||||
printf '%s\n' $MONPAGES_EXTRA_URLS
|
||||
fi
|
||||
}
|
||||
|
||||
# Install stub only (no console log) — not a real mon page.
|
||||
is_bootstrap_html() {
|
||||
local f="$1"
|
||||
grep -qiE 'Monitoring page \(bootstrap\)|pill">STAGE</|install stub|bootstrap page' "$f" 2>/dev/null \
|
||||
&& ! grep -qE 'taler-mon:top|id="mon-console"|class="console"' "$f" 2>/dev/null
|
||||
}
|
||||
|
||||
# Stale mon HTML from pre-1.15.8 generators: box-drawing char-class regex in
|
||||
# sticky filter JS (Python SyntaxWarning: invalid escape sequence '\-').
|
||||
# Never accept these as "current" public assets (v1.18.2+).
|
||||
is_stale_generator_html() {
|
||||
local f="$1"
|
||||
# Old isLowValue() JS (literal box glyphs + \- class)
|
||||
if grep -qE 'if \(/\^\[[╔╗╚╝║═]' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
# Converter noise teed into old logs and re-rendered
|
||||
if grep -qiE 'SyntaxWarning: invalid escape sequence' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if grep -qiE 'console_to_html\.py:[0-9]+: SyntaxWarning' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# True if body looks like our monitoring HTML (not merchant API / WP 404 / bootstrap).
|
||||
is_monitoring_html() {
|
||||
local f="$1"
|
||||
if grep -qE '"code"[[:space:]]*:[[:space:]]*21' "$f" 2>/dev/null; then
|
||||
return 1
|
||||
fi
|
||||
if grep -qiE 'There is no endpoint defined for the URL' "$f" 2>/dev/null; then
|
||||
return 1
|
||||
fi
|
||||
if grep -qiE 'wp-content|wordpress' "$f" 2>/dev/null \
|
||||
&& grep -qiE '404|not found|page introuvable' "$f" 2>/dev/null; then
|
||||
return 1
|
||||
fi
|
||||
# Bootstrap sticky is not enough — need suite console output
|
||||
if is_bootstrap_html "$f"; then
|
||||
return 1
|
||||
fi
|
||||
if grep -qE 'sticky-bar|version-link|taler-monitoring|class="sticky|taler-mon:top' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if head -c 4096 "$f" | grep -qiE '<!DOCTYPE html|<html' \
|
||||
&& grep -qiE 'monitoring|mon ·|host-agent' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if head -c 512 "$f" | grep -qiE '<!DOCTYPE html|<html'; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Highest progress percent in page body (0 if none). Pattern: ┌ 42%┐ or ┌100%┐
|
||||
_progress_max_pct() {
|
||||
local f="$1"
|
||||
local m
|
||||
m=$(grep -oE '┌[[:space:]]*[0-9]+%┐' "$f" 2>/dev/null \
|
||||
| grep -oE '[0-9]+' | sort -n | tail -1 || true)
|
||||
printf '%s' "${m:-0}"
|
||||
}
|
||||
|
||||
# Mid-run: incomplete meta / redirect stub, or progress never reached 100% and no complete foot.
|
||||
_page_mid_run() {
|
||||
local f="$1"
|
||||
local pct
|
||||
if grep -qE 'name="taler-mon-page"[[:space:]]+content="incomplete"' "$f" 2>/dev/null \
|
||||
|| grep -q 'taler-mon:bottom:incomplete' "$f" 2>/dev/null \
|
||||
|| grep -qE 'http-equiv="refresh".*monitoring_err|mode-redirect' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
# finished page always has complete foot or classic footer
|
||||
if grep -q 'taler-mon:bottom:complete' "$f" 2>/dev/null \
|
||||
|| grep -qE 'id="mon-footer"' "$f" 2>/dev/null; then
|
||||
return 1
|
||||
fi
|
||||
pct=$(_progress_max_pct "$f")
|
||||
# max progress seen < 100% and no complete footer → still running
|
||||
if [ "${pct:-0}" -gt 0 ] && [ "${pct:-0}" -lt 100 ]; then
|
||||
if ! grep -qE 'Console-style render of taler-monitoring|Rendu console de taler-monitoring' "$f" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Validate top/bottom markers + meaningful body (v1.9.0).
|
||||
# Sets _MON_CONTENT_MSG; returns 0=ok, 1=hard fail, 2=soft/mid-run bottom skip
|
||||
#
|
||||
# Head windows are large (v1.13.4+): sticky CSS grew past old 12–16 KiB limits and
|
||||
# caused false TOP errors while publish races were also writing the file.
|
||||
validate_mon_content() {
|
||||
local f="$1" url="$2"
|
||||
local top_ok=1 bot_ok=1 body_ok=1 mid=0 pct
|
||||
local head_top="${MONPAGES_HEAD_TOP_BYTES:-49152}"
|
||||
local head_meta="${MONPAGES_HEAD_META_BYTES:-65536}"
|
||||
_MON_CONTENT_MSG=""
|
||||
|
||||
# Tiny / empty body = likely mid-publish race (partial rsync or curl)
|
||||
if [ ! -s "$f" ] || [ "$(wc -c <"$f" 2>/dev/null || echo 0)" -lt 200 ]; then
|
||||
top_ok=0
|
||||
_MON_CONTENT_MSG="empty or truncated body (publish race?)"
|
||||
# --- top markers (always required) ---
|
||||
elif ! head -c "$head_top" "$f" | grep -qE 'sticky-bar|id="status-bar"|taler-mon:top'; then
|
||||
# full-file fallback (small pages / unusual layout)
|
||||
if ! grep -qE 'sticky-bar|id="status-bar"|taler-mon:top' "$f" 2>/dev/null; then
|
||||
top_ok=0
|
||||
_MON_CONTENT_MSG="missing top marker (sticky-bar / status-bar / taler-mon:top)"
|
||||
fi
|
||||
elif ! head -c "$head_meta" "$f" | grep -qE 'data-generated-iso|name="generated"|version-link|taler-mon-page'; then
|
||||
if ! grep -qE 'data-generated-iso|name="generated"|version-link|taler-mon-page' "$f" 2>/dev/null; then
|
||||
top_ok=0
|
||||
_MON_CONTENT_MSG="missing top meta (generated / version-link / taler-mon-page)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- meaningful body: real checks, not only progress bars ---
|
||||
if grep -qE 'class="line (ok|error|warn|blocker|info)"|┌ OK|┌ ERROR|┌ WARN|┌ BLOCKER|┌ INFO|class="box"' "$f" 2>/dev/null; then
|
||||
body_ok=1
|
||||
elif grep -qE 'taler-mon:bottom:incomplete|mode-redirect|has failures' "$f" 2>/dev/null; then
|
||||
body_ok=1 # redirect stub still has sticky + box
|
||||
else
|
||||
# only progress / empty?
|
||||
if grep -qE '┌[[:space:]]*[0-9]+%┐|█|░' "$f" 2>/dev/null \
|
||||
&& ! grep -qE '┌ OK|┌ ERROR|class="line ok"' "$f" 2>/dev/null; then
|
||||
body_ok=0
|
||||
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }only progress bars / no check lines"
|
||||
elif ! grep -qE 'console|host-agent|taler-monitoring|monitoring' "$f" 2>/dev/null; then
|
||||
body_ok=0
|
||||
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }no meaningful suite content"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- bottom markers ---
|
||||
if grep -q 'taler-mon:bottom:complete' "$f" 2>/dev/null \
|
||||
|| tail -c 8000 "$f" | grep -qE 'id="mon-footer"|</footer>|Console-style render of taler-monitoring'; then
|
||||
bot_ok=1
|
||||
elif grep -q 'taler-mon:bottom:incomplete' "$f" 2>/dev/null; then
|
||||
bot_ok=1 # incomplete is an explicit bottom marker
|
||||
mid=1
|
||||
else
|
||||
bot_ok=0
|
||||
_MON_CONTENT_MSG="${_MON_CONTENT_MSG:+$_MON_CONTENT_MSG; }missing bottom marker (footer / taler-mon:bottom)"
|
||||
fi
|
||||
|
||||
if _page_mid_run "$f"; then
|
||||
mid=1
|
||||
fi
|
||||
pct=$(_progress_max_pct "$f")
|
||||
|
||||
if [ "$top_ok" != "1" ]; then
|
||||
# TOP missing is WARN-only (v1.13.8+): mid-run races and incomplete
|
||||
# /monitoring stubs used to self-red the next HTML. Hard ERROR still for
|
||||
# missing page / code 21 / non-suite body. Override: MONPAGES_TOP_STRICT=1.
|
||||
_MON_CONTENT_MSG="TOP: $_MON_CONTENT_MSG"
|
||||
if [ "${MONPAGES_TOP_STRICT:-0}" = "1" ]; then
|
||||
return 1
|
||||
fi
|
||||
return 2
|
||||
fi
|
||||
if [ "$body_ok" != "1" ]; then
|
||||
_MON_CONTENT_MSG="BODY: $_MON_CONTENT_MSG"
|
||||
return 1
|
||||
fi
|
||||
if [ "$bot_ok" != "1" ]; then
|
||||
if [ "$mid" = "1" ]; then
|
||||
_MON_CONTENT_MSG="BOTTOM soft (mid-run progress ${pct}%): $_MON_CONTENT_MSG"
|
||||
return 2
|
||||
fi
|
||||
_MON_CONTENT_MSG="BOTTOM: $_MON_CONTENT_MSG"
|
||||
return 1
|
||||
fi
|
||||
if [ "$mid" = "1" ] && [ "${pct:-0}" -gt 0 ] && [ "${pct:-0}" -lt 100 ]; then
|
||||
_MON_CONTENT_MSG="mid-run progress ${pct}% (bottom markers relaxed)"
|
||||
return 2
|
||||
fi
|
||||
_MON_CONTENT_MSG="top+body+bottom markers OK"
|
||||
return 0
|
||||
}
|
||||
|
||||
_is_bare_url() {
|
||||
local url="$1"
|
||||
case " ${MONPAGES_BARE_URLS:-} " in
|
||||
*" $url "*) return 0 ;;
|
||||
esac
|
||||
case "$url" in
|
||||
*/) return 1 ;;
|
||||
*) return 0 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
_mon_fail_or_soft() {
|
||||
local label="$1" detail="$2"
|
||||
if [ "${MONPAGES_REQUIRE_PUBLIC:-1}" != "1" ]; then
|
||||
warn "$label" "$detail (MONPAGES_REQUIRE_PUBLIC=0)"
|
||||
return 0
|
||||
fi
|
||||
fail "$label" "$detail"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Fetch + validate one URL. Retries content failures (publish/rsync races).
|
||||
# MONPAGES_PRE_PUBLISH=1 → content failures are WARN only (suite still regenerates HTML after).
|
||||
# MONPAGES_FETCH_RETRIES (default 3), MONPAGES_FETCH_SLEEP_S (default 0.5).
|
||||
check_one() {
|
||||
local url="$1"
|
||||
local body code hint soft=0 vc attempt max_try sleep_s pre_pub
|
||||
body=$(mktemp)
|
||||
if _is_bare_url "$url" && [ "${MONPAGES_BARE_STRICT:-0}" != "1" ]; then
|
||||
soft=1
|
||||
fi
|
||||
pre_pub=0
|
||||
[ "${MONPAGES_PRE_PUBLISH:-0}" = "1" ] && pre_pub=1
|
||||
max_try="${MONPAGES_FETCH_RETRIES:-3}"
|
||||
sleep_s="${MONPAGES_FETCH_SLEEP_S:-0.5}"
|
||||
[ "$max_try" -lt 1 ] && max_try=1
|
||||
|
||||
attempt=1
|
||||
while [ "$attempt" -le "$max_try" ]; do
|
||||
: >"$body"
|
||||
code=$(curl -skS -L --max-redirs 5 -m "${TIMEOUT}" -o "$body" -w '%{http_code}' "$url" 2>/dev/null || echo 000)
|
||||
|
||||
if grep -qE '"code"[[:space:]]*:[[:space:]]*21' "$body" 2>/dev/null \
|
||||
|| grep -qiE 'There is no endpoint defined for the URL' "$body" 2>/dev/null; then
|
||||
# Bare without trailing slash often hits merchant until Caddy has redir * /path/ 302.
|
||||
if [ "$soft" = "1" ]; then
|
||||
warn "public mon page bare URL" \
|
||||
"$url -> HTTP $code code 21 (prefer trailing slash; Caddy: redir * /path/ 302; set MONPAGES_BARE_STRICT=1 to ERROR)"
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
# hard 21: retry once more (rare brief outage) then fail
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
_mon_fail_or_soft "public mon page missing" \
|
||||
"$url -> HTTP $code merchant/API JSON code 21 (Caddy handle/redir; use redir * /path/ 302 inside handle${APPLY_MONITORING_LIVE:+; apply: sudo $APPLY_MONITORING_LIVE})" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ "$code" != "200" ]; then
|
||||
if [ "$soft" = "1" ]; then
|
||||
warn "public mon page bare URL" "$url -> HTTP $code (prefer trailing slash form)"
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
_mon_fail_or_soft "public mon page" "$url -> HTTP $code (want 200 HTML)" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if is_bootstrap_html "$body"; then
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
_mon_fail_or_soft "public mon page bootstrap stub" \
|
||||
"$url -> install bootstrap HTML still live (run host-agent; suite auto-update must replace stub)" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Stale generator JS / SyntaxWarning in public body (pre-1.15.8 assets)
|
||||
if is_stale_generator_html "$body"; then
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
_mon_fail_or_soft "public mon page stale generator" \
|
||||
"$url -> old mon HTML (box-drawing JS regex and/or SyntaxWarning in body) — re-run host-agent on suite ≥v1.15.8 / v1.18.2; do not leave pre-update assets live" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if ! is_monitoring_html "$body"; then
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
hint=$(head -c 120 "$body" | tr '\n' ' ' | tr -cd '[:print:] ')
|
||||
_mon_fail_or_soft "public mon page not monitoring HTML" \
|
||||
"$url -> HTTP 200 body!=suite (${hint}...)" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# v1.9.0: top + bottom markers + meaningful content
|
||||
set +e
|
||||
validate_mon_content "$body" "$url"
|
||||
vc=$?
|
||||
set -e
|
||||
case "$vc" in
|
||||
0)
|
||||
ok "public mon page" "$url -> HTTP 200 · ${_MON_CONTENT_MSG}"
|
||||
rm -f "$body"
|
||||
return 0
|
||||
;;
|
||||
2)
|
||||
# soft: TOP incomplete / mid-run bottom — WARN, never hard fail
|
||||
case "${_MON_CONTENT_MSG}" in
|
||||
TOP:*)
|
||||
warn "public mon page content" \
|
||||
"$url -> ${_MON_CONTENT_MSG} (WARN only; set MONPAGES_TOP_STRICT=1 to ERROR)"
|
||||
;;
|
||||
*)
|
||||
warn "public mon page mid-run" "$url -> ${_MON_CONTENT_MSG}"
|
||||
ok "public mon page" "$url -> HTTP 200 suite HTML (mid-run bottom relaxed)"
|
||||
;;
|
||||
esac
|
||||
rm -f "$body"
|
||||
return 0
|
||||
;;
|
||||
*)
|
||||
# Content race (truncated HTML while another agent publishes): retry
|
||||
if [ "$attempt" -lt "$max_try" ]; then
|
||||
attempt=$((attempt + 1))
|
||||
sleep "$sleep_s" 2>/dev/null || sleep 1
|
||||
continue
|
||||
fi
|
||||
# TOP failures should already be soft (return 2); belt-and-braces:
|
||||
case "${_MON_CONTENT_MSG}" in
|
||||
TOP:*)
|
||||
if [ "${MONPAGES_TOP_STRICT:-0}" != "1" ]; then
|
||||
warn "public mon page content" \
|
||||
"$url -> ${_MON_CONTENT_MSG} (WARN only; set MONPAGES_TOP_STRICT=1 to ERROR)"
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
# Pre-publish inventory in host-agent PHASES: WARN only (post-check is hard)
|
||||
if [ "$pre_pub" = "1" ] || [ "$soft" = "1" ]; then
|
||||
warn "public mon page content" \
|
||||
"$url -> ${_MON_CONTENT_MSG}${pre_pub:+ (pre-publish; post-check is authoritative)}"
|
||||
rm -f "$body"
|
||||
return 0
|
||||
fi
|
||||
_mon_fail_or_soft "public mon page content" \
|
||||
"$url -> ${_MON_CONTENT_MSG}" || { rm -f "$body"; return 1; }
|
||||
rm -f "$body"
|
||||
return 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
rm -f "$body"
|
||||
return 0
|
||||
}
|
||||
|
||||
URLS=()
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
# dedupe
|
||||
skip=0
|
||||
for e in "${URLS[@]+"${URLS[@]}"}"; do
|
||||
[ "$e" = "$line" ] && skip=1 && break
|
||||
done
|
||||
[ "$skip" = "1" ] && continue
|
||||
URLS+=("$line")
|
||||
done < <(build_urls | sort -u)
|
||||
|
||||
# Bare URLs without trailing slash: checked by default.
|
||||
# Merchant JSON code 21 is always ERROR (Caddy/proxy not serving mon path).
|
||||
# Other bare failures: WARN unless MONPAGES_BARE_STRICT=1 (then ERROR).
|
||||
# MONPAGES_CHECK_BARE=0 → skip bare entirely
|
||||
# MONPAGES_BARE_STRICT=1 → non-21 bare failures are ERROR too
|
||||
if [ "${MONPAGES_CHECK_BARE:-1}" = "1" ]; then
|
||||
_extra=()
|
||||
for u in "${URLS[@]}"; do
|
||||
case "$u" in
|
||||
*/) _bare=${u%/}; [ -n "$_bare" ] && _extra+=("$_bare") ;;
|
||||
esac
|
||||
done
|
||||
for u in "${_extra[@]+"${_extra[@]}"}"; do
|
||||
skip=0
|
||||
for e in "${URLS[@]+"${URLS[@]}"}"; do
|
||||
[ "$e" = "$u" ] && skip=1 && break
|
||||
done
|
||||
[ "$skip" = "1" ] && continue
|
||||
URLS+=("$u")
|
||||
MONPAGES_BARE_URLS="${MONPAGES_BARE_URLS:-} $u"
|
||||
done
|
||||
export MONPAGES_BARE_URLS
|
||||
fi
|
||||
|
||||
if [ "${#URLS[@]}" -eq 0 ]; then
|
||||
fail "monpages" "no URLs to check (set MON_HOSTS or MONITORING_PAGE_URLS)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
_fam="$(_monpages_family)"
|
||||
echo " family=${_fam} inventory=${MONPAGES_INVENTORY} require_public=${MONPAGES_REQUIRE_PUBLIC}"
|
||||
echo " checking ${#URLS[@]} URL(s) via FQDN..."
|
||||
ec=0
|
||||
for u in "${URLS[@]}"; do
|
||||
check_one "$u" || ec=1
|
||||
done
|
||||
|
||||
if [ "$ec" -ne 0 ]; then
|
||||
echo " hint: public monitoring HTML not served (404 / merchant code 21)"
|
||||
echo " publish staging HTML + configure reverse-proxy/Caddy handle for mon paths"
|
||||
echo " (set APPLY_MONITORING_LIVE in env; sudo "$APPLY_MONITORING_LIVE")"
|
||||
echo " bare code 21: Caddy redir inside handle must be: redir * /path/ 302 (not redir /path/ 302)"
|
||||
if [ "$_fam" = "fp" ]; then
|
||||
echo " FP: only FP mon hosts are checked — wire Infomaniak vhost for /monitoring*"
|
||||
fi
|
||||
if [ "$_fam" = "goa" ]; then
|
||||
echo " GOA: bank/exchange/taler /monitoring/ + surface + aptdeploy (v1.9.4+ layout)"
|
||||
fi
|
||||
# Staging vs public diagnosis (host-agent sets MONPAGES_STAGING_BASE / HTML_OUT)
|
||||
_stg="${MONPAGES_STAGING_BASE:-${HTML_OUT:-${HTML_BASE:-$HOME/monitoring-sites-staging}}}"
|
||||
_ok_dir="${HTML_OK_DIR:-monitoring}"
|
||||
if [ -n "${MON_HOSTS:-}" ] && [ -n "$_stg" ]; then
|
||||
for _h in $MON_HOSTS; do
|
||||
_f="$_stg/$_h/${_ok_dir}/index.html"
|
||||
if [ -f "$_f" ]; then
|
||||
echo " staging OK: $_f ($(wc -c <"$_f" | tr -d " ") bytes) — not published/served"
|
||||
else
|
||||
echo " staging missing: $_f"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
_www="${DEPLOY_WWW_ROOT:-}"
|
||||
if [ -e "$_www" ] && [ ! -w "$_www" ]; then
|
||||
echo " DEPLOY_WWW_ROOT=$_www not writable by $(id -un 2>/dev/null || echo user)"
|
||||
fi
|
||||
fi
|
||||
|
||||
exit "$ec"
|
||||
112
check_surface.sh
112
check_surface.sh
|
|
@ -32,6 +32,10 @@ PORT_TIMEOUT="${SURFACE_PORT_TIMEOUT:-2}"
|
|||
HTTP_TIMEOUT="${SURFACE_HTTP_TIMEOUT:-12}"
|
||||
# CVE via OSV (public, no key). Disable: SURFACE_CVE=0
|
||||
: "${SURFACE_CVE:=1}"
|
||||
# nmap OS / service fingerprint (v1.8.0+). Disable: SURFACE_NMAP=0
|
||||
: "${SURFACE_NMAP:=1}"
|
||||
# OS detection (-O) often needs root; falls back to -sV when denied
|
||||
: "${SURFACE_NMAP_OS:=1}"
|
||||
# Extra ports when -d domain mode
|
||||
DOMAIN_EXTRA_PORTS="${SURFACE_DOMAIN_PORTS:-22,80,443,8443}"
|
||||
|
||||
|
|
@ -496,6 +500,109 @@ cert_expiry_check() {
|
|||
return 0
|
||||
}
|
||||
|
||||
# nmap OS / service fingerprint (v1.8.0+).
|
||||
# ERROR when fingerprint indicates an OS that is EOL / should be upgraded.
|
||||
# Soft/info when nmap missing or unprivileged OS scan fails.
|
||||
nmap_fingerprint_host() {
|
||||
local h="$1"
|
||||
local xml out osline accuracy eol=0
|
||||
[ "${SURFACE_NMAP:-1}" = "1" ] || return 0
|
||||
if ! command -v nmap >/dev/null 2>&1; then
|
||||
warn "nmap" "$h · nmap not installed (apt install nmap) — skip OS fingerprint"
|
||||
return 0
|
||||
fi
|
||||
xml=$(mktemp)
|
||||
out=$(mktemp)
|
||||
# Prefer OS detection when allowed; always try service version light
|
||||
if [ "${SURFACE_NMAP_OS:-1}" = "1" ] && {
|
||||
nmap -Pn -n -T4 -O --osscan-guess --max-os-tries 1 --host-timeout 45s \
|
||||
-p 22,80,443,25,465,587,993 "$h" -oX "$xml" >"$out" 2>&1
|
||||
}; then
|
||||
:
|
||||
else
|
||||
nmap -Pn -n -T4 -sV --version-light --host-timeout 35s \
|
||||
-p 22,80,443,25,465,587,993 "$h" -oX "$xml" >"$out" 2>&1 || true
|
||||
fi
|
||||
# Parse best OS match from XML
|
||||
osline=$(
|
||||
python3 - "$xml" <<'PY' 2>/dev/null || true
|
||||
import sys, xml.etree.ElementTree as ET
|
||||
path = sys.argv[1]
|
||||
try:
|
||||
root = ET.parse(path).getroot()
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
best = None
|
||||
best_acc = -1
|
||||
for osm in root.findall(".//osmatch"):
|
||||
name = osm.get("name") or ""
|
||||
try:
|
||||
acc = int(osm.get("accuracy") or "0")
|
||||
except ValueError:
|
||||
acc = 0
|
||||
if name and acc >= best_acc:
|
||||
best_acc = acc
|
||||
best = name
|
||||
if best:
|
||||
print(f"{best_acc}|{best}")
|
||||
# also surface product/version from service table
|
||||
svcs = []
|
||||
for port in root.findall(".//port"):
|
||||
state = (port.find("state").get("state") if port.find("state") is not None else "")
|
||||
if state != "open":
|
||||
continue
|
||||
svc = port.find("service")
|
||||
if svc is None:
|
||||
continue
|
||||
prod = svc.get("product") or svc.get("name") or ""
|
||||
ver = svc.get("version") or ""
|
||||
if prod:
|
||||
svcs.append(f"{prod} {ver}".strip())
|
||||
if svcs:
|
||||
print("SVC|" + "; ".join(svcs[:8]))
|
||||
PY
|
||||
)
|
||||
rm -f "$xml" "$out"
|
||||
if [ -z "$osline" ]; then
|
||||
info "nmap" "$h · no OS/service fingerprint (need root for -O, or host filtered)"
|
||||
return 0
|
||||
fi
|
||||
local osname="" svcinfo=""
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
SVC\|*) svcinfo=${line#SVC|} ;;
|
||||
*\|*)
|
||||
accuracy=${line%%|*}
|
||||
osname=${line#*|}
|
||||
ok "nmap-os" "$h · OS guess ${accuracy}% · $osname"
|
||||
;;
|
||||
esac
|
||||
done <<<"$osline"
|
||||
[ -n "$svcinfo" ] && info "nmap-svc" "$h · $svcinfo"
|
||||
|
||||
# EOL / upgrade-required OS fingerprints → ERROR
|
||||
case "$(printf '%s' "$osname" | tr 'A-Z' 'a-z')" in
|
||||
*debian*6*|*debian*7*|*debian*8*|*debian*9*|*debian*10*|*debian*\"buster\"*|*debian*buster*)
|
||||
eol=1 ;;
|
||||
*ubuntu*12.04*|*ubuntu*14.04*|*ubuntu*16.04*|*ubuntu*18.04*|*ubuntu*20.04*)
|
||||
eol=1 ;;
|
||||
*centos*[5-7]*|*red\ hat\ enterprise*5*|*red\ hat\ enterprise*6*|*red\ hat\ enterprise*7*)
|
||||
eol=1 ;;
|
||||
*windows\ server\ 2008*|*windows\ server\ 2012*|*windows\ xp*|*windows\ 7*)
|
||||
eol=1 ;;
|
||||
*freebsd\ 1[0-2]*|*freebsd\ [0-9].*)
|
||||
eol=1 ;;
|
||||
esac
|
||||
if [ "$eol" = "1" ]; then
|
||||
err "nmap-upgrade" "$h OS fingerprint indicates EOL / upgrade required" "$osname"
|
||||
return 1
|
||||
fi
|
||||
if [ -n "$osname" ]; then
|
||||
info "nmap-upgrade" "$h OS fingerprint not on EOL list · $osname"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
scan_host() {
|
||||
local h="$1"
|
||||
local ports proto label expect
|
||||
|
|
@ -621,6 +728,9 @@ scan_host() {
|
|||
fi
|
||||
fi
|
||||
|
||||
# nmap OS / service fingerprint (v1.8.0+) — ERROR on EOL OS
|
||||
nmap_fingerprint_host "$h" || true
|
||||
|
||||
# Expected service must answer protocol (not just ping/port)
|
||||
# Note: ${array[*]:-x} is NOT valid default syntax (bash treats : as slice).
|
||||
local ports_txt="${open_ports[*]}"
|
||||
|
|
@ -667,7 +777,7 @@ else
|
|||
fi
|
||||
|
||||
info "inventory" "${#ORDERED_HOSTS[@]} hosts to probe (remote only · no SSH · no local podman on targets)"
|
||||
info "flags" "SURFACE_CVE=${SURFACE_CVE:-1} PORT_TIMEOUT=${PORT_TIMEOUT:-2} HTTP_TIMEOUT=${HTTP_TIMEOUT:-12}"
|
||||
info "flags" "SURFACE_CVE=${SURFACE_CVE:-1} SURFACE_NMAP=${SURFACE_NMAP:-1} SURFACE_NMAP_OS=${SURFACE_NMAP_OS:-1} PORT_TIMEOUT=${PORT_TIMEOUT:-2} HTTP_TIMEOUT=${HTTP_TIMEOUT:-12}"
|
||||
|
||||
for h in "${ORDERED_HOSTS[@]}"; do
|
||||
[ -n "$h" ] || continue
|
||||
|
|
|
|||
148
check_urls.sh
148
check_urls.sh
|
|
@ -259,7 +259,7 @@ set_group perf
|
|||
section "www · performance · public HTTPS latency (outside-in)"
|
||||
|
||||
# Latency thresholds (ms), outside-in from this runner.
|
||||
# ≥ PERF_WARN_MS → WARN. ≥ PERF_FAIL_MS → ERROR (suite fails).
|
||||
# ≥ PERF_WARN_MS → WARN. ≥ PERF_FAIL_MS → suite fail (hard).
|
||||
PERF_WARN_MS="${PERF_WARN_MS:-8000}"
|
||||
PERF_FAIL_MS="${PERF_FAIL_MS:-20000}"
|
||||
PERF_TSV="$tmp/perf.tsv"
|
||||
|
|
@ -303,14 +303,18 @@ check_perf() {
|
|||
esac
|
||||
}
|
||||
|
||||
# Bank first (wallet-critical paths before UI chrome)
|
||||
check_perf "perf bank /taler-integration/config" "$BANK_PUBLIC/taler-integration/config"
|
||||
check_perf "perf bank /config" "$BANK_PUBLIC/config"
|
||||
if [ "${CHECK_LANDING:-1}" = "1" ]; then
|
||||
check_perf "perf bank /intro/" "$BANK_PUBLIC/intro/"
|
||||
check_perf "perf bank /intro/stats.json" "$BANK_PUBLIC/intro/stats.json" 200
|
||||
# Bank first (wallet-critical paths before UI chrome) — skip when CHECK_BANK=0 (e.g. mytops stage mon)
|
||||
if [ "${CHECK_BANK:-1}" = "1" ]; then
|
||||
check_perf "perf bank /taler-integration/config" "$BANK_PUBLIC/taler-integration/config"
|
||||
check_perf "perf bank /config" "$BANK_PUBLIC/config"
|
||||
if [ "${CHECK_LANDING:-1}" = "1" ]; then
|
||||
check_perf "perf bank /intro/" "$BANK_PUBLIC/intro/"
|
||||
check_perf "perf bank /intro/stats.json" "$BANK_PUBLIC/intro/stats.json" 200
|
||||
fi
|
||||
check_perf "perf bank /webui/" "$BANK_PUBLIC/webui/" 200,301,302
|
||||
else
|
||||
info "perf bank" "skipped (CHECK_BANK=0)"
|
||||
fi
|
||||
check_perf "perf bank /webui/" "$BANK_PUBLIC/webui/" 200,301,302
|
||||
|
||||
# Exchange
|
||||
check_perf "perf exchange /config" "$EXCHANGE_PUBLIC/config"
|
||||
|
|
@ -576,7 +580,8 @@ PY
|
|||
if [ "${CHECK_LANDING:-1}" = "1" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
|
||||
set_group stats
|
||||
section "www · landing stats.json (freshness + display fields · outside-in)"
|
||||
info "stats policy" "STALE≥${STATS_STALE_SECS}s WARN · FAIL≥${STATS_FAIL_SECS}s ERROR (0=disable fail) · need generated_at_unix"
|
||||
# Avoid the bare word ERROR in INFO text (older HTML converters treated it as a check failure / first-error).
|
||||
info "stats policy" "STALE≥${STATS_STALE_SECS}s→WARN · FAIL≥${STATS_FAIL_SECS}s→suite-fail (0=off) · need generated_at_unix"
|
||||
check_landing_stats_json "bank" "$BANK_PUBLIC" || true
|
||||
check_landing_stats_json "exchange" "$EXCHANGE_PUBLIC" || true
|
||||
check_landing_stats_json "merchant" "$MERCHANT_PUBLIC" || true
|
||||
|
|
@ -719,13 +724,17 @@ esac
|
|||
|
||||
# --- bank ---
|
||||
set_group bank
|
||||
if [ "${LOCAL_STACK:-1}" = "0" ]; then
|
||||
if [ "${CHECK_BANK:-1}" != "1" ]; then
|
||||
info "bank" "skipped (CHECK_BANK=0 — not in scope for this mon job)"
|
||||
code=""
|
||||
elif [ "${LOCAL_STACK:-1}" = "0" ]; then
|
||||
check_url_soft "bank /config" 200 "$BANK_PUBLIC/config"
|
||||
code=$(http_body "$BANK_PUBLIC/config" "$tmp/bc.json")
|
||||
else
|
||||
check_url "bank /config" 200 "$BANK_PUBLIC/config"
|
||||
code=$(http_body "$BANK_PUBLIC/config" "$tmp/bc.json")
|
||||
fi
|
||||
code=$(http_body "$BANK_PUBLIC/config" "$tmp/bc.json")
|
||||
if [ "$code" = "200" ]; then
|
||||
if [ "${CHECK_BANK:-1}" = "1" ] && [ "$code" = "200" ]; then
|
||||
expect_currency "bank" "$tmp/bc.json"
|
||||
if json_has_alt_unit_names "$tmp/bc.json" >/tmp/alt-bank.$$ 2>&1; then
|
||||
ok "bank /config alt_unit_names" "$(tr '\n' '; ' </tmp/alt-bank.$$ | sed 's/; $//')"
|
||||
|
|
@ -773,25 +782,28 @@ if [ "$code" = "200" ]; then
|
|||
fi
|
||||
|
||||
# Bank legal docs (landing nginx via Caddy /terms* /privacy* or /intro/*)
|
||||
check_legal_doc "bank /terms" "$BANK_PUBLIC/terms" "terms|GOA|exploration|bank|FADP|revDSG"
|
||||
# Prefer /privacy; fall back to /intro/privacy.html for older deploys
|
||||
code=$(http_code "$BANK_PUBLIC/privacy")
|
||||
if [ "$code" = "200" ]; then
|
||||
check_legal_doc "bank /privacy" "$BANK_PUBLIC/privacy" "privacy|FADP|revDSG|data|GOA|bank"
|
||||
else
|
||||
if [ "${LOCAL_STACK:-1}" = "1" ]; then
|
||||
check_legal_doc "bank /privacy (or /intro/privacy.html)" \
|
||||
"$BANK_PUBLIC/intro/privacy.html" "privacy|FADP|revDSG|data|GOA|bank"
|
||||
# still report bare /privacy failure for local
|
||||
warn "bank /privacy" "HTTP $code — prefer Caddy handle /privacy* → landing"
|
||||
if [ "${CHECK_BANK:-1}" = "1" ]; then
|
||||
check_legal_doc "bank /terms" "$BANK_PUBLIC/terms" "terms|GOA|exploration|bank|FADP|revDSG"
|
||||
# Prefer /privacy; fall back to /intro/privacy.html for older deploys
|
||||
code=$(http_code "$BANK_PUBLIC/privacy")
|
||||
if [ "$code" = "200" ]; then
|
||||
check_legal_doc "bank /privacy" "$BANK_PUBLIC/privacy" "privacy|FADP|revDSG|data|GOA|bank"
|
||||
else
|
||||
check_url_soft "bank /privacy" 200 "$BANK_PUBLIC/privacy"
|
||||
if [ "${LOCAL_STACK:-1}" = "1" ]; then
|
||||
check_legal_doc "bank /privacy (or /intro/privacy.html)" \
|
||||
"$BANK_PUBLIC/intro/privacy.html" "privacy|FADP|revDSG|data|GOA|bank"
|
||||
# still report bare /privacy failure for local
|
||||
warn "bank /privacy" "HTTP $code — prefer Caddy handle /privacy* → landing"
|
||||
else
|
||||
check_url_soft "bank /privacy" 200 "$BANK_PUBLIC/privacy"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- merchant ---
|
||||
set_group merchant
|
||||
if [ "${LOCAL_STACK:-1}" = "0" ]; then
|
||||
# MERCHANT_REQUIRED=1: hard ERROR if /config fails even when LOCAL_STACK=0 (mytops stage)
|
||||
if [ "${LOCAL_STACK:-1}" = "0" ] && [ "${MERCHANT_REQUIRED:-0}" != "1" ]; then
|
||||
check_url_soft "merchant /config" 200 "$MERCHANT_PUBLIC/config"
|
||||
else
|
||||
check_url "merchant /config" 200 "$MERCHANT_PUBLIC/config"
|
||||
|
|
@ -1304,7 +1316,7 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
|
|||
fi
|
||||
;;
|
||||
404|405|501)
|
||||
# GOA bank landing mints demo-withdraw.json; stage TESTPAYSAN uses ATM/wallet guide only
|
||||
# GOA bank landing mints demo-withdraw.json; stage TESTPAYSAN uses bank withdraw/wallet guide only
|
||||
if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
|
||||
fail "landing bank demo-withdraw" "HTTP $dw_code (want 200)"
|
||||
else
|
||||
|
|
@ -1327,6 +1339,90 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
|
|||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# taler:// URIs must not carry default ports :443 / :80 (wallet / QR rule)
|
||||
# Scans public landings on this stack (bank + exchange + merchant intros,
|
||||
# bank mint JSON). Fail hard on GOA/local; soft warn elsewhere.
|
||||
# ---------------------------------------------------------------------------
|
||||
set_group landings
|
||||
section "www · taler:// default ports (:443/:80 must be absent)"
|
||||
_port_scan_fail=0
|
||||
_port_scan_ok=0
|
||||
_port_scan_one() {
|
||||
local label="$1" url="$2"
|
||||
local f code n detail
|
||||
f="$tmp/portscan-$$.body"
|
||||
code=$(http_body "$url" "$f")
|
||||
if [ "$code" != "200" ]; then
|
||||
if [ "${LOCAL_STACK:-1}" = "1" ] || [ "${EXPECT_CURRENCY:-}" = "GOA" ]; then
|
||||
# mint JSON is hard; HTML intros soft if missing
|
||||
case "$url" in
|
||||
*demo-withdraw.json|*auto-account.json)
|
||||
fail "taler-uri ports · $label" "HTTP $code · $url"
|
||||
_port_scan_fail=$((_port_scan_fail + 1))
|
||||
;;
|
||||
*)
|
||||
warn "taler-uri ports · $label" "HTTP $code · skip · $url"
|
||||
;;
|
||||
esac
|
||||
else
|
||||
info "taler-uri ports · $label" "HTTP $code · skip"
|
||||
fi
|
||||
rm -f "$f"
|
||||
return 0
|
||||
fi
|
||||
detail=$(python3 - "$f" <<'PY' 2>/dev/null || true
|
||||
import re, sys, json
|
||||
from pathlib import Path
|
||||
raw = Path(sys.argv[1]).read_text(errors="replace")
|
||||
# collect taler:// from JSON fields and raw HTML/JS
|
||||
uris = set(re.findall(r"taler://[A-Za-z0-9._~:/?#\[\]@!$&'()*+,;=%-]+", raw))
|
||||
# JSON-escaped sequences
|
||||
uris.update(re.findall(r"taler:\\/\\/[A-Za-z0-9._~:/?#\[\]@!$&'()*+,;=%\\-]+", raw))
|
||||
uris = {u.replace("\\/", "/") for u in uris}
|
||||
bad = []
|
||||
pat = re.compile(r"taler://\S+:(443|80)(/|$|\?|#)", re.I)
|
||||
for u in sorted(uris):
|
||||
if pat.search(u):
|
||||
bad.append(u[:120])
|
||||
if bad:
|
||||
print("BAD\t%d\t%s" % (len(bad), bad[0]))
|
||||
else:
|
||||
print("OK\t%d\t" % len(uris))
|
||||
PY
|
||||
)
|
||||
n_kind=${detail%%$'\t'*}
|
||||
rest=${detail#*$'\t'}
|
||||
n_count=${rest%%$'\t'*}
|
||||
sample=${rest#*$'\t'}
|
||||
if [ "$n_kind" = "BAD" ]; then
|
||||
if [ "${LOCAL_STACK:-1}" = "1" ] || [ "${EXPECT_CURRENCY:-}" = "GOA" ]; then
|
||||
fail "taler-uri ports · $label" "default :443/:80 still present · n=${n_count} e.g. ${sample}"
|
||||
_port_scan_fail=$((_port_scan_fail + 1))
|
||||
else
|
||||
warn "taler-uri ports · $label" "default port in URI · ${sample}"
|
||||
fi
|
||||
else
|
||||
ok "taler-uri ports · $label" "no default ports · taler_uris=${n_count:-0} · $url"
|
||||
_port_scan_ok=$((_port_scan_ok + 1))
|
||||
fi
|
||||
rm -f "$f"
|
||||
}
|
||||
if [ "${CHECK_LANDING:-1}" = "1" ]; then
|
||||
_port_scan_one "bank intro HTML" "${BANK_PUBLIC}/intro/"
|
||||
_port_scan_one "bank demo-withdraw.json" "${BANK_PUBLIC}/intro/demo-withdraw.json"
|
||||
_port_scan_one "bank auto-account.json" "${BANK_PUBLIC}/intro/auto-account.json"
|
||||
_port_scan_one "exchange intro HTML" "${EXCHANGE_PUBLIC}/intro/"
|
||||
_port_scan_one "merchant intro HTML" "${MERCHANT_PUBLIC}/intro/"
|
||||
# stats may embed demo withdraw URI
|
||||
_port_scan_one "bank stats.json" "${BANK_PUBLIC}/intro/stats.json"
|
||||
if [ "$_port_scan_fail" -eq 0 ]; then
|
||||
ok "taler-uri ports summary" "clean · ${_port_scan_ok} resource(s) checked (bank+exchange+merchant)"
|
||||
fi
|
||||
else
|
||||
info "taler-uri ports" "skipped (CHECK_LANDING=0)"
|
||||
fi
|
||||
|
||||
# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js +
|
||||
# shops.css + QR encoder (shop-ui modal needs /intro/qrcode.min.js or /qrcode.min.js).
|
||||
_ma=0
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@
|
|||
# TALER_APT_BASE=https://deb.taler.net/apt/debian
|
||||
# TALER_APT_INDEX=…/dists/trixie/main/binary-amd64/Packages
|
||||
# TALER_APT_TESTING_INDEX=…/dists/trixie-testing/…
|
||||
# TALER_PKG_BEHIND=warn|error (default: core packages ERROR if behind, else warn)
|
||||
# TALER_PKG_BEHIND=warn|error (default error as of v1.8.0 — any behind package is ERROR)
|
||||
# SKIP_SSH=1 skip container install checks
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
|
|
@ -27,7 +27,7 @@ APT_BASE="${APT_BASE%/}"
|
|||
INDEX="${TALER_APT_INDEX:-${APT_BASE}/dists/${SUITE}/main/binary-amd64/Packages}"
|
||||
INRELEASE="${TALER_APT_INRELEASE:-${APT_BASE}/dists/${SUITE}/InRelease}"
|
||||
TESTING_INDEX="${TALER_APT_TESTING_INDEX:-${APT_BASE}/dists/${SUITE}-testing/main/binary-amd64/Packages}"
|
||||
BEHIND_MODE="${TALER_PKG_BEHIND:-warn}"
|
||||
BEHIND_MODE="${TALER_PKG_BEHIND:-error}"
|
||||
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
|
@ -178,7 +178,7 @@ if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; t
|
|||
elif [ "${INSIDE_PROFILE:-}" = "stage-lfp" ] \
|
||||
|| [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ] \
|
||||
|| [ -n "${INSIDE_SSH:-}" ]; then
|
||||
_VERS_SSH_HOST="${INSIDE_SSH:-francpaysan-stage-user}"
|
||||
_VERS_SSH_HOST="${INSIDE_SSH:-}"
|
||||
_VERS_BANK="${INSIDE_BANK_CTR:-stage-lfp-bank}"
|
||||
_VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
|
||||
_VERS_MER="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
|
||||
|
|
|
|||
65
domains.conf
65
domains.conf
|
|
@ -1,46 +1,53 @@
|
|||
# taler-monitoring domain profiles
|
||||
#
|
||||
# When you add a stack, declare the three public endpoints here:
|
||||
# name bank exchange merchant-backend currency local landing [canonical]
|
||||
# name bank exchange merchant-backend currency local landing lang locale [canonical]
|
||||
#
|
||||
# - name: what -d / TALER_DOMAIN matches (aliases allowed as extra lines)
|
||||
# - bank / exchange / merchant: hostname or full https:// URL
|
||||
# - currency: GOA | KUDOS | CHF | … (empty = report only)
|
||||
# - currency: GOA | KUDOS | CHF | … (empty or "-" = report only)
|
||||
# - local: 1 = koopa SSH/inside/e2e stack; 0 = public-only
|
||||
# - landing: 1 = check /intro landings + assets; 0 = skip (no GOA-style landings)
|
||||
# - lang: UI language for mon console/HTML (en|fr|de). Source of truth when
|
||||
# TALER_MON_LANG_SET≠1. Workstation TALER_MON_LANG=en does not override.
|
||||
# **de is supported but must not be used as a profile default** (no row
|
||||
# sets lang=de unless an operator deliberately chooses it).
|
||||
# - locale: BCP 47 regional locale for l10n defaults (not the same as UI lang).
|
||||
# Use **fr-CH** (Suisse romande) for FrancPaysan; **de-CH** (Deutschschweiz)
|
||||
# for other Swiss / default stacks. Accepts also ch-FR / ch-DE as aliases.
|
||||
# - canonical: optional TALER_DOMAIN label after alias match
|
||||
#
|
||||
# CLI overrides after load: --bank --exchange --merchant --currency
|
||||
# Legacy lines without locale still parse (locale → de-CH except FP heuristic).
|
||||
# CLI: --lang en|fr|de · TALER_MON_LANG_SET=1 locks UI lang
|
||||
# Alternate file: TALER_DOMAINS_CONF=/path/to/domains.conf
|
||||
|
||||
# Local GOA (koopa) — landings matter
|
||||
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
|
||||
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
|
||||
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1
|
||||
# Local GOA (koopa) — landings matter · UI en · locale de-CH
|
||||
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
||||
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
||||
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH
|
||||
|
||||
# Public demo (KUDOS) — has intro-style pages; keep soft landing checks
|
||||
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 demo.taler.net
|
||||
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1
|
||||
# Public demo (KUDOS)
|
||||
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 en de-CH demo.taler.net
|
||||
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 en de-CH
|
||||
|
||||
# Public test stack (TESTKUDOS) — no GOA-style landings
|
||||
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0
|
||||
# Public test stack (TESTKUDOS)
|
||||
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0 en de-CH
|
||||
|
||||
# TOPS CHF — multi-tenant merchant my.taler-ops.ch; no public GOA landings
|
||||
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0
|
||||
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 taler-ops.ch
|
||||
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0
|
||||
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0 stage.taler-ops.ch
|
||||
# TOPS CHF
|
||||
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH
|
||||
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH taler-ops.ch
|
||||
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH
|
||||
stage.my.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
||||
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
||||
|
||||
# Franc Paysan (Infomaniak) — merchant host is monnaie.* (not taler./backend.)
|
||||
# Prod: no monetary launch yet; stage uses TESTPAYSAN (public HTTPS landings + shops)
|
||||
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0
|
||||
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
|
||||
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
|
||||
exchange.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
|
||||
# Franc Paysan — UI fr · regional locale fr-CH (Suisse romande)
|
||||
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH
|
||||
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
||||
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
||||
exchange.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
||||
|
||||
# Stage TESTPAYSAN — landings on / + farmer shops under stage.monnaie…/shops/
|
||||
# name bank exchange merchant currency local landing [canonical]
|
||||
stage.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1
|
||||
stage.bank.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
|
||||
stage.exchange.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
|
||||
stage.monnaie.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
|
||||
# Stage TESTPAYSAN — UI fr · locale fr-CH
|
||||
stage.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 fr fr-CH
|
||||
stage.bank.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 fr fr-CH stage.lefrancpaysan.ch
|
||||
stage.exchange.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 fr fr-CH stage.lefrancpaysan.ch
|
||||
stage.monnaie.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 fr fr-CH stage.lefrancpaysan.ch
|
||||
|
|
|
|||
|
|
@ -4,22 +4,42 @@
|
|||
|
||||
| Behaviour | Default |
|
||||
|-----------|---------|
|
||||
| Suite refresh | `update-suite.sh` → `origin/main` |
|
||||
| Suite refresh | **`update-suite.sh` every run** → `git fetch --tags` + hard reset to `origin/main` |
|
||||
| Update strict | `SUITE_UPDATE_STRICT=1` (default): **abort** if fetch/reset fails — no silent old tree |
|
||||
| Wall clock | `RUN_TIMEOUT=600` |
|
||||
| Log | line-buffered `run-*.log` (continuous write) |
|
||||
| HTML | always (first run / empty tree too) + commit URL |
|
||||
| HTML | always (first run / empty tree too) + commit URL; **bootstrap install stubs are monpages ERROR** |
|
||||
| Timeout notice | end of log + top jump on `/monitoring_err/` |
|
||||
|
||||
**Rule:** public mon pages must show a **current suite tag/commit**, not install bootstrap or months-old trees. Timers exist so this keeps working without manual SSH.
|
||||
|
||||
Thin wrappers only set stack defaults:
|
||||
|
||||
| Wrapper | Stack | Typical user |
|
||||
|---------|-------|----------------|
|
||||
| `run-hacktivism-monitoring.sh` | GOA `hacktivism.ch` | `hernani` @ koopa |
|
||||
| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` |
|
||||
| `run-fp-prod-monitoring.sh` | FP prod | `francpaysan` |
|
||||
| `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh koopa-external** |
|
||||
| `run-aptdeploy-monitoring.sh` | **aptdeploy** → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h timer, taler only) |
|
||||
| `run-surface-monitoring.sh` | **surface** → `/taler-monitoring-surface*` | `hernani` @ koopa (hourly, taler only) |
|
||||
| `run-host-report.sh` + `taler-monitoring-mytops-stage.{service,timer}` | mytops **stage / betel** only | `taler-monitoring` @ betel (4h) |
|
||||
| `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh $DEPLOY_SSH** |
|
||||
| `run-aptdeploy-monitoring.sh` | apt-src tests → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h) |
|
||||
| `run-surface-monitoring.sh` | **surface** → only `/taler-monitoring-surface*` (ecosystem + mail + mattermost + versions) | `hernani` @ koopa (hourly) |
|
||||
| `run-max-ladder-monitoring.sh` | GOA **max-search** ladder → `/monitoring-max-ladder/` on bank/exchange/taler | `hernani` @ koopa (**daily**) |
|
||||
|
||||
Deprecated mail/mattermost wrappers: **`../meta/deprecated/`** (not installed). No classic **`/monitoring-ladder`** page.
|
||||
|
||||
### GOA max-search ladder page (daily)
|
||||
|
||||
| URL | Phase | Timer |
|
||||
|-----|-------|-------|
|
||||
| `https://{bank,exchange,taler}.hacktivism.ch/monitoring-max-ladder/` | `max-ladder` | `taler-monitoring-max-ladder.timer` (~06:30 + jitter) |
|
||||
|
||||
```bash
|
||||
systemctl --user enable --now taler-monitoring-max-ladder.timer
|
||||
# first round now:
|
||||
systemctl --user start taler-monitoring-max-ladder.service
|
||||
```
|
||||
|
||||
Needs explorer secrets (`SECRETS_ROOT` / bank-explorer password). Wall ~3h (`RUN_TIMEOUT=10800`).
|
||||
|
||||
### apt-src deploy tests (hacktivism `/monitoring`)
|
||||
|
||||
|
|
@ -31,7 +51,7 @@ Thin wrappers only set stack defaults:
|
|||
| `koopa-taler-deploy-test-apt-src-trixie-testing-upgrade` | **upgrade** for testing suite |
|
||||
|
||||
```bash
|
||||
# from laptop / clementine → triggers on koopa-external
|
||||
# from ops workstation → triggers on koopa-external
|
||||
./host-agent/run-aptdeploy.sh
|
||||
./host-agent/run-aptdeploy.sh upgrade
|
||||
./host-agent/run-aptdeploy.sh fresh
|
||||
|
|
@ -44,27 +64,21 @@ Thin wrappers only set stack defaults:
|
|||
Checks: package table, `httpd --version`, libtalerutil/`ldd` on failure, `systemctl` for
|
||||
`taler-merchant.target`, basic local `/config` probe. No nginx/HTTPS in containers.
|
||||
|
||||
### surface + aptdeploy HTML (taler.hacktivism.ch only)
|
||||
### Public HTML layout (v1.8.0 simplified)
|
||||
|
||||
| URL | Role | Timer |
|
||||
|-----|------|-------|
|
||||
| `…/taler-monitoring-surface/` | ecosystem surface scan | 1h |
|
||||
| `…/taler-monitoring-surface_err/` | errors + jump list | ″ |
|
||||
| `…/taler-monitoring-aptdeploy/` | apt-src container deploy tests | 4h |
|
||||
| `…/taler-monitoring-aptdeploy_err/` | errors + jump list | ″ |
|
||||
| URL | Role |
|
||||
|-----|------|
|
||||
| `https://{bank,exchange,merchant}/monitoring/` (+ `_err`) | **Landing-stack** report — GOA 3 + FP **stage** 3 (+ mytops stage as configured) |
|
||||
| `https://taler.hacktivism.ch/taler-monitoring-surface/` (+ `_err`) | **Only** ecosystem page: software/versions in the Taler universe (nmap OS, mail firefly+anastasis, mattermost, …) |
|
||||
|
||||
Not on bank/exchange. Staging:
|
||||
`~/monitoring-sites-staging/taler.hacktivism.ch/taler-monitoring-{surface,aptdeploy}*`.
|
||||
No separate public pages for mail/mattermost. Staging under `~/monitoring-sites-staging/`.
|
||||
|
||||
```bash
|
||||
systemctl --user enable --now taler-monitoring-surface.timer
|
||||
systemctl --user enable --now taler-monitoring-aptdeploy.timer
|
||||
systemctl --user start taler-monitoring-surface.service
|
||||
systemctl --user start taler-monitoring-aptdeploy.service
|
||||
# Caddy: handles only in taler.hacktivism.ch block (root)
|
||||
```
|
||||
|
||||
Env examples: **`env/*.env.example`**. FP docs: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
|
||||
Env examples: **`env/*.env.example`**. FP stage: `run-fp-stage-monitoring.sh` (same `run-host-report.sh` pipeline).
|
||||
|
||||
## GOA (koopa) — host-podman
|
||||
|
||||
|
|
@ -74,7 +88,7 @@ Runs **as user `hernani`** (systemd user, linger). **Primary** mode:
|
|||
`podman exec` on this host into
|
||||
`taler-hacktivism-bank` / `taler-hacktivism-exchange-ansible` / `taler-hacktivism`
|
||||
(IDs `inside.host-*`, not `inside.ssh-*`).
|
||||
From a **laptop**, leave `INSIDE_PODMAN` off → access mode **`ssh`** via `KOOPA_SSH`
|
||||
From an **ops workstation**, leave `INSIDE_PODMAN` off → access mode **`ssh`** via `KOOPA_SSH`
|
||||
(`inside.ssh-*`).
|
||||
- Public **`urls` + `versions`** for the stack (`-d hacktivism.ch`).
|
||||
- Writes console HTML for all three sites under
|
||||
|
|
@ -89,7 +103,7 @@ Caddy (host root) can serve that tree at `/monitoring` — you configure Caddy.
|
|||
Every host-agent / aptdeploy run:
|
||||
|
||||
1. **`update-suite.sh`**: `git fetch` + **`SUITE_UPDATE_MODE=reset`** (default) → `origin/main`
|
||||
into `SUITE_DIR` (prefer `~/src/koopa-admin-log`).
|
||||
into `SUITE_DIR` (prefer `~/src/taler-monitoring`).
|
||||
2. **Re-exec** host-agent scripts from that tree so the running code matches `COMMIT_URL`.
|
||||
3. Local overrides only in **`~/.config/taler-monitoring/env`** — never overwritten by git.
|
||||
4. HTML footer = exact commit:
|
||||
|
|
@ -109,8 +123,8 @@ mkdir -p ~/.config/taler-monitoring
|
|||
cp ~/src/taler-monitoring/host-agent/env.example \
|
||||
~/.config/taler-monitoring/env
|
||||
|
||||
# laptop:
|
||||
./host-agent/install-host-agent.sh --remote koopa-external
|
||||
# ops workstation:
|
||||
./host-agent/install-host-agent.sh --remote $DEPLOY_SSH
|
||||
|
||||
# or on koopa:
|
||||
~/src/taler-monitoring/host-agent/install-host-agent.sh
|
||||
|
|
@ -120,7 +134,7 @@ Units:
|
|||
|
||||
| Unit | Role |
|
||||
|------|------|
|
||||
| `taler-monitoring-hacktivism.path` | fires on software stamp + landing/caddy config changes |
|
||||
| `taler-monitoring-hacktivism.path` | fires on software stamp (`touch-software-stamp.sh`) |
|
||||
| `taler-monitoring-hacktivism.timer` | every **4h** fallback |
|
||||
| `taler-monitoring-hacktivism.service` | oneshot → `/monitoring*` (urls inside versions) |
|
||||
| `taler-monitoring-surface.timer` | every **1h** |
|
||||
|
|
@ -144,16 +158,16 @@ journalctl --user -u taler-monitoring-hacktivism.service -n 50 --no-pager
|
|||
ls ~/monitoring-sites-staging/bank.hacktivism.ch/monitoring_err/
|
||||
```
|
||||
|
||||
## Relation to firecuda outside-only timer
|
||||
## Relation to optional outside-only timer
|
||||
|
||||
| | host-agent (koopa) | firecuda site-gen |
|
||||
| | host-agent (koopa) | outside-runner site-gen |
|
||||
|--|--------------------|-------------------|
|
||||
| Access | **podman on host** | public HTTPS only |
|
||||
| Phases | urls **inside** versions | urls versions |
|
||||
| Trigger | software path + 4h | *could* launchd 4h |
|
||||
| Primary for GOA | **yes** | **off** (optional only) |
|
||||
|
||||
**firecuda-external is not scheduled** right now (launchd disabled). Scripts under
|
||||
**outside-runner is not scheduled** right now (launchd disabled). Scripts under
|
||||
`site-gen/` can still one-shot a public run if needed; see `site-gen/README.md`.
|
||||
|
||||
No secrets required for default phases. `secrets.env` optional for e2e later.
|
||||
|
|
@ -181,3 +195,14 @@ zypper in qrencode zbar
|
|||
Without them, monitoring still runs; QR group WARNs and skips encode/decode.
|
||||
|
||||
Caddy `/var/www/monitoring-sites` is separate (you configured hernani write access).
|
||||
|
||||
|
||||
## Exit codes (v1.7.1+)
|
||||
|
||||
Host-agent jobs default to **`STRICT_EXIT=1`**: non-zero exit if suite checks fail, `DEPLOY_WWW_ROOT` is not writable, or public monpages (FQDN) fail.
|
||||
|
||||
Set `STRICT_EXIT=0` only if you want timers to stay green while still logging errors (not recommended for production).
|
||||
|
||||
## Monitoring pages (suite)
|
||||
|
||||
Public sticky-bar HTML under `/monitoring*` and `/taler-monitoring-*` is part of **taler-monitoring** (not a separate product). See main [README — Monitoring pages](../README.md#monitoring-pages-part-of-the-suite). Phase **monpages** is obligatory (**ERROR** if missing): GOA checks the full suite inventory; FP only FP mon hosts.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,11 @@
|
|||
# Root: make monitoring pages live on koopa (GOA)
|
||||
|
||||
## Detection (v1.3.1+)
|
||||
|
||||
Suite phase **`monpages`** and the host-agent **post-check** curl each public
|
||||
FQDN (`https://taler.hacktivism.ch/monitoring/` …). Merchant JSON **code 21**
|
||||
or non-HTML is an **ERROR** until this apply is done.
|
||||
|
||||
## Symptom if not applied
|
||||
|
||||
Public URLs return **merchant/bank JSON** like:
|
||||
|
|
@ -11,14 +17,22 @@ Public URLs return **merchant/bank JSON** like:
|
|||
or a **404 HTML** page from the app reverse_proxy.
|
||||
|
||||
Cause: live `/etc/caddy/Caddyfile` has **no** `handle /monitoring*` (falls through to
|
||||
API). Prepared config is only under **`/home/hernani/koopa-caddy/Caddyfile`**.
|
||||
API). Prepared config is only under **`$KOOPA_CADDY_DIR/Caddyfile`**.
|
||||
|
||||
## One-shot as root (recommended)
|
||||
|
||||
```bash
|
||||
sudo /home/hernani/koopa-caddy/apply-monitoring-live.sh
|
||||
# absolute path required: as root, ~ is /root (not /home/hernani)
|
||||
sudo $APPLY_MONITORING_LIVE
|
||||
# or already root:
|
||||
# $APPLY_MONITORING_LIVE
|
||||
```
|
||||
|
||||
**Caddy bare-path redir (v1.9.2+):** inside a `handle` block use
|
||||
`redir * /path/ 302` — never `redir /path/ 302` alone (Caddy treats the first
|
||||
token as matcher and sets `Location: 302`, so bare URLs fall through to the
|
||||
merchant API as JSON **code 21**).
|
||||
|
||||
That script:
|
||||
|
||||
1. `rsync` staging HTML → `/var/www/monitoring-sites/` (incl. surface + aptdeploy)
|
||||
|
|
@ -26,23 +40,29 @@ That script:
|
|||
3. Installs prepared Caddyfile → `/etc/caddy/Caddyfile`, validates, reloads Caddy
|
||||
4. Smoke-curls all monitoring URLs (must **not** be JSON `code:21`)
|
||||
|
||||
## Paths
|
||||
## Paths (only three families — suite name ≠ URL)
|
||||
|
||||
| Path | Host | Source timer |
|
||||
|------|------|----------------|
|
||||
| `/monitoring/` + `/monitoring_err/` | bank + exchange + taler | `taler-monitoring-hacktivism.timer` (4h) |
|
||||
| `/taler-monitoring-surface/` + `/_err/` | **taler.hacktivism.ch only** | `taler-monitoring-surface.timer` (1h) |
|
||||
| `/taler-monitoring-aptdeploy/` + `/_err/` | **taler.hacktivism.ch only** | `taler-monitoring-aptdeploy.timer` (4h) |
|
||||
There is **no** public page `/taler-monitoring`. Each family needs **bare and slash**
|
||||
(`redir * /path/ 302` inside Caddy `handle`):
|
||||
|
||||
| Family | Paths | Host | Source timer |
|
||||
|--------|-------|------|----------------|
|
||||
| **monitoring** | `/monitoring` · `/monitoring/` · `/monitoring_err` · `/monitoring_err/` | bank + exchange + taler (and other landings) | `taler-monitoring-hacktivism.timer` (4h) |
|
||||
| **monitoring-max-ladder** | `/monitoring-max-ladder` · `/` · `_err` | bank + exchange + taler (**GOA** max-search only) | `taler-monitoring-max-ladder.timer` (**daily**) |
|
||||
| **taler-monitoring-surface** | bare + slash + `_err` | **taler.hacktivism.ch only** | `taler-monitoring-surface.timer` (1h) |
|
||||
| **taler-monitoring-aptdeploy** | bare + slash + `_err` | **taler.hacktivism.ch only** | `taler-monitoring-aptdeploy.timer` (4h) |
|
||||
|
||||
No classic **`/monitoring-ladder`**. Max-ladder served via Caddy `handle /monitoring*` under `/var/www/monitoring-sites/{host}/`.
|
||||
|
||||
HTML is generated as **hernani** into
|
||||
`/home/hernani/monitoring-sites-staging/`.
|
||||
`$DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/`.
|
||||
|
||||
## Manual equivalent
|
||||
|
||||
```bash
|
||||
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
|
||||
rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/
|
||||
chown -R hernani:caddy /var/www/monitoring-sites && chmod -R g+rwX /var/www/monitoring-sites
|
||||
sudo /home/hernani/koopa-caddy/apply.sh # or apply-monitoring-live.sh
|
||||
sudo $KOOPA_CADDY_DIR/apply.sh # or apply-monitoring-live.sh
|
||||
```
|
||||
|
||||
Expect **200** (HTML with sticky bar). Not merchant JSON `code:21`.
|
||||
|
|
@ -50,7 +70,7 @@ Expect **200** (HTML with sticky bar). Not merchant JSON `code:21`.
|
|||
## After each monitoring run (if www is not hernani-writable)
|
||||
|
||||
```bash
|
||||
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
|
||||
rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/
|
||||
chown -R caddy:caddy /var/www/monitoring-sites
|
||||
```
|
||||
|
||||
|
|
|
|||
94
host-agent/apply-monitoring-live.sh
Executable file
94
host-agent/apply-monitoring-live.sh
Executable file
|
|
@ -0,0 +1,94 @@
|
|||
#!/usr/bin/env bash
|
||||
# ONE-SHOT as root: publish monitoring HTML + enable Caddy handles
|
||||
#
|
||||
# Suite copy (taler-monitoring host-agent). Live install path on koopa:
|
||||
# ${KOOPA_CADDY_DIR}/apply-monitoring-live.sh
|
||||
# Prefer absolute path ( ~ expands to /root when already root ):
|
||||
# sudo ${KOOPA_CADDY_DIR}/apply-monitoring-live.sh
|
||||
# ${KOOPA_CADDY_DIR}/apply-monitoring-live.sh # if already root
|
||||
#
|
||||
# After editing here: rsync to koopa ~/koopa-caddy/ before root apply.
|
||||
set -euo pipefail
|
||||
if [[ "$(id -u)" -ne 0 ]]; then
|
||||
exec sudo -E "$0" "$@"
|
||||
fi
|
||||
|
||||
STAGING="${DEPLOY_STAGING:-${HTML_OUT:-$HOME/monitoring-sites-staging}}"
|
||||
WWW="${DEPLOY_WWW_ROOT:?set DEPLOY_WWW_ROOT in env}"
|
||||
SRC="${KOOPA_CADDY_DIR:?set KOOPA_CADDY_DIR in env}/Caddyfile"
|
||||
DST=/etc/caddy/Caddyfile
|
||||
|
||||
if [[ ! -f "$SRC" ]]; then
|
||||
echo "ERROR: missing Caddyfile source: $SRC" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ ! -d "$STAGING" ]]; then
|
||||
echo "ERROR: missing staging HTML: $STAGING" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "== 1) HTML staging → www =="
|
||||
mkdir -p "$WWW"
|
||||
rsync -a --delete "$STAGING/" "$WWW/"
|
||||
# allow hernani to update future runs without root
|
||||
chown -R hernani:caddy "$WWW"
|
||||
chmod -R g+rwX "$WWW"
|
||||
find "$WWW" -type d -exec chmod g+s {} \;
|
||||
# ACL default if available
|
||||
if command -v setfacl >/dev/null 2>&1; then
|
||||
setfacl -R -m u:hernani:rwx,g:caddy:rwx "$WWW" || true
|
||||
setfacl -R -d -m u:hernani:rwx,g:caddy:rwx "$WWW" || true
|
||||
fi
|
||||
ls -la "$WWW" "$WWW/taler.hacktivism.ch" || true
|
||||
|
||||
echo "== 2) Caddyfile (backup + install) =="
|
||||
ts=$(date +%Y%m%d-%H%M%S)
|
||||
cp -a "$DST" "/etc/caddy/Caddyfile.bak-monitoring-${ts}"
|
||||
cp -a "$SRC" "$DST"
|
||||
chown root:caddy "$DST" 2>/dev/null || chown root:root "$DST"
|
||||
chmod 644 "$DST"
|
||||
caddy validate --config "$DST"
|
||||
systemctl reload caddy
|
||||
systemctl is-active caddy
|
||||
|
||||
echo "== 3) smoke =="
|
||||
# bash: every continued line except the last must end with \
|
||||
urls=(
|
||||
https://taler.hacktivism.ch/monitoring/
|
||||
https://bank.hacktivism.ch/monitoring/
|
||||
https://exchange.hacktivism.ch/monitoring/
|
||||
https://taler.hacktivism.ch/monitoring-max-ladder/
|
||||
https://bank.hacktivism.ch/monitoring-max-ladder/
|
||||
https://exchange.hacktivism.ch/monitoring-max-ladder/
|
||||
https://taler.hacktivism.ch/taler-monitoring-surface
|
||||
https://taler.hacktivism.ch/taler-monitoring-surface/
|
||||
https://taler.hacktivism.ch/taler-monitoring-surface_err/
|
||||
https://taler.hacktivism.ch/taler-monitoring-aptdeploy/
|
||||
https://taler.hacktivism.ch/taler-monitoring-aptdeploy_err/
|
||||
)
|
||||
smoke_fail=0
|
||||
for url in "${urls[@]}"; do
|
||||
code=$(curl -sS -o /tmp/monchk -w '%{http_code}' -L --max-redirs 3 -m 12 "$url" || echo ERR)
|
||||
ct=$(file -b /tmp/monchk 2>/dev/null | head -c 40)
|
||||
echo " $code $url ($ct)"
|
||||
if grep -qE '"code":[[:space:]]*21' /tmp/monchk 2>/dev/null; then
|
||||
echo " STILL merchant JSON code 21 — Caddy handles not active?"
|
||||
smoke_fail=1
|
||||
fi
|
||||
done
|
||||
# bare path must redirect (or land on HTML), not merchant JSON
|
||||
bare_code=$(curl -sS -o /tmp/monbare -w '%{http_code}' --max-redirs 0 -m 12 \
|
||||
https://taler.hacktivism.ch/taler-monitoring-surface || echo ERR)
|
||||
if grep -qE '"code":[[:space:]]*21' /tmp/monbare 2>/dev/null; then
|
||||
echo "ERROR: bare /taler-monitoring-surface still merchant code 21 (HTTP $bare_code)" >&2
|
||||
smoke_fail=1
|
||||
elif [[ "$bare_code" != "302" && "$bare_code" != "301" && "$bare_code" != "200" ]]; then
|
||||
echo "WARN: bare /taler-monitoring-surface HTTP $bare_code (want 302/301/200 HTML)" >&2
|
||||
fi
|
||||
echo " bare_no_follow=$bare_code https://taler.hacktivism.ch/taler-monitoring-surface"
|
||||
|
||||
if [[ "$smoke_fail" -ne 0 ]]; then
|
||||
echo "ERROR: smoke failed" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "OK apply-monitoring-live done"
|
||||
|
|
@ -11,7 +11,7 @@
|
|||
# koopa-taler-deploy-test-apt-src-trixie-upgrade
|
||||
# koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
|
||||
#
|
||||
# Run as hernani on koopa. From elsewhere use run-aptdeploy.sh (ssh koopa-external).
|
||||
# Run as hernani on koopa. From elsewhere use run-aptdeploy.sh (ssh $DEPLOY_SSH or $APT_DEPLOY_SSH).
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
|
|
@ -72,13 +72,14 @@ cleanup_legacy() {
|
|||
done < <("$PODMAN_BIN" images --format '{{.Repository}}:{{.Tag}}' 2>/dev/null | grep -E 'localhost/koopa-taler-.*-test-apt' || true)
|
||||
}
|
||||
|
||||
# Prefer single suite tree: if both ~/src/koopa-admin-log and ~/koopa-admin-log are
|
||||
# full git clones, keep the newer (by HEAD commit time) and replace the older with a symlink.
|
||||
# Prefer single suite tree: ~/src/taler-monitoring vs ~/taler-monitoring
|
||||
# (legacy koopa-admin-log paths are not used). Keep the newer git clone; symlink the other.
|
||||
dedupe_suite_repos() {
|
||||
local a="$HOME/src/koopa-admin-log" b="$HOME/koopa-admin-log"
|
||||
local a="$HOME/src/taler-monitoring" b="$HOME/taler-monitoring"
|
||||
local ta tb
|
||||
[ "${APT_DEPLOY_DEDUPE_REPOS:-1}" = "1" ] || return 0
|
||||
if [ -d "$a/.git" ] && [ -d "$b/.git" ] && [ ! -L "$a" ] && [ ! -L "$b" ]; then
|
||||
if [ -d "$a/.git" ] && [ -d "$b/.git" ] && [ ! -L "$a" ] && [ ! -L "$b" ] \
|
||||
&& [ "$(readlink -f "$a" 2>/dev/null || echo "$a")" != "$(readlink -f "$b" 2>/dev/null || echo "$b")" ]; then
|
||||
ta=$(git -C "$a" log -1 --format=%ct 2>/dev/null || echo 0)
|
||||
tb=$(git -C "$b" log -1 --format=%ct 2>/dev/null || echo 0)
|
||||
echo "======== dedupe suite repos (both are git clones) ========"
|
||||
|
|
@ -94,17 +95,11 @@ dedupe_suite_repos() {
|
|||
mkdir -p "$(dirname "$a")"
|
||||
ln -sfn "$b" "$a"
|
||||
fi
|
||||
elif [ -d "$a/scripts/taler-monitoring" ] && [ -d "$b/scripts/taler-monitoring" ] \
|
||||
&& [ ! -d "$a/.git" ] && [ -d "$b/.git" ] && [ ! -L "$a" ]; then
|
||||
echo "remove non-git tree $a · symlink → $b"
|
||||
rm -rf "$a"
|
||||
elif [ -d "$a/.git" ] && [ ! -e "$b" ]; then
|
||||
ln -sfn "$a" "$b"
|
||||
elif [ -d "$b/.git" ] && [ ! -e "$a" ]; then
|
||||
mkdir -p "$(dirname "$a")"
|
||||
ln -sfn "$b" "$a"
|
||||
elif [ -d "$b/scripts/taler-monitoring" ] && [ -d "$a/scripts/taler-monitoring" ] \
|
||||
&& [ ! -d "$b/.git" ] && [ -d "$a/.git" ] && [ ! -L "$b" ]; then
|
||||
echo "remove non-git tree $b · symlink → $a"
|
||||
rm -rf "$b"
|
||||
ln -sfn "$a" "$b"
|
||||
fi
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@
|
|||
# Full stack examples (same reporting generation for all):
|
||||
# env/hacktivism.env.example
|
||||
# env/stagepaysan.env.example
|
||||
# env/francpaysan.env.example
|
||||
#
|
||||
# SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
|
||||
# SUITE_GIT_REF=main
|
||||
|
|
|
|||
27
host-agent/env/francpaysan.env.example
vendored
27
host-agent/env/francpaysan.env.example
vendored
|
|
@ -1,27 +0,0 @@
|
|||
# Copy to: ~/.config/taler-monitoring/env (user francpaysan)
|
||||
# Never inside the git tree — update-suite never overwrites this file.
|
||||
#
|
||||
# Shared reporting generation (run-host-report.sh / run-fp-prod-monitoring.sh):
|
||||
# RUN_TIMEOUT, line-buffered log, always HTML, commit link on git.hacktivism.ch
|
||||
|
||||
TALER_DOMAIN=lefrancpaysan.ch
|
||||
INSIDE_PODMAN=1
|
||||
INSIDE_MODE=local-podman
|
||||
LOCAL_STACK=0
|
||||
CONTINUE_ON_ERROR=1
|
||||
PHASES="urls inside versions"
|
||||
RUN_TIMEOUT=600
|
||||
|
||||
HTML_OUT=$HOME/monitoring-sites-staging
|
||||
MON_HOSTS="bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch"
|
||||
|
||||
SUITE_DIR=$HOME/src/taler-monitoring
|
||||
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
|
||||
SUITE_GIT_REF=main
|
||||
SUITE_UPDATE_MODE=reset
|
||||
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
|
||||
|
||||
# DEPLOY_WWW_ROOT=/var/www/monitoring-sites
|
||||
|
||||
# UI language for console + HTML sticky bar (en|fr)
|
||||
TALER_MON_LANG=fr
|
||||
41
host-agent/env/mytops-stage.env.example
vendored
Normal file
41
host-agent/env/mytops-stage.env.example
vendored
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# Copy to: ~/.config/taler-monitoring/env (user taler-monitoring @ betel)
|
||||
# mytops CHF stage — NOT lifeline / production
|
||||
#
|
||||
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
|
||||
SUITE_GIT_REF=main
|
||||
SUITE_DIR=$HOME/src/taler-monitoring
|
||||
TALER_DOMAIN=stage.taler-ops.ch
|
||||
MON_HOSTS="stage.my.taler-ops.ch stage.taler-ops.ch"
|
||||
HTML_OUT=$HOME/monitoring-sites-staging
|
||||
DEPLOY_WWW_ROOT=/var/www/monitoring-sites
|
||||
HTML_URL_OK=/monitoring/
|
||||
HTML_URL_ERR=/monitoring_err/
|
||||
HTML_OK_DIR=monitoring
|
||||
HTML_ERR_DIR=monitoring_err
|
||||
# urls + monpages + fake-franken CHF (rusty.taler-ops.ch)
|
||||
PHASES="urls monpages devtesting"
|
||||
SKIP_SSH=1
|
||||
LOCAL_STACK=0
|
||||
INSIDE_PODMAN=0
|
||||
CONTINUE_ON_ERROR=1
|
||||
# ladder needs wall time (23× fake-incoming SSH)
|
||||
RUN_TIMEOUT=1800
|
||||
STRICT_EXIT=1
|
||||
MONPAGES_REQUIRE_PUBLIC=1
|
||||
MONPAGES_INVENTORY=job
|
||||
TALER_MON_LANG=en
|
||||
AGENT_LABEL=mytops-stage-host-agent
|
||||
STATE_NAME=taler-monitoring-mytops-stage
|
||||
SUITE_UPDATE_STRICT=1
|
||||
CHECK_BANK=0
|
||||
MERCHANT_REQUIRED=1
|
||||
CHECK_LANDING=0
|
||||
# rusty devtesting (SSH Host taler-rusty-devtesting → devtesting@rusty.taler-ops.ch)
|
||||
DEVTESTING_SSH=taler-rusty-devtesting
|
||||
DEVTESTING_AMOUNT=CHF:1.00
|
||||
# CHF fake-incoming amount ladder (23 rungs → 50k)
|
||||
DEVTESTING_LADDER=1
|
||||
DEVTESTING_LADDER_STEPS=23
|
||||
DEVTESTING_LADDER_MAX=50000
|
||||
DEVTESTING_LADDER_MIN=0.01
|
||||
# DEVTESTING_SKIP=1
|
||||
29
host-agent/env/stagepaysan.env.example
vendored
29
host-agent/env/stagepaysan.env.example
vendored
|
|
@ -1,8 +1,10 @@
|
|||
# Copy to: ~/.config/taler-monitoring/env (user stagepaysan)
|
||||
# Never inside the git tree — update-suite never overwrites this file.
|
||||
#
|
||||
# Shared reporting generation (run-host-report.sh / run-fp-stage-monitoring.sh):
|
||||
# RUN_TIMEOUT, line-buffered log, always HTML, commit link on git.hacktivism.ch
|
||||
# Secrets: use francpaysan-secrets layout (operator tree or host mirror):
|
||||
# $HOME/francpaysan-secrets/stage/bank-admin-password.txt
|
||||
# $HOME/francpaysan-secrets/stage/default-instance-token.txt
|
||||
# On stage host, symlink those to /mnt/data/stagepaysan/*/secrets/…
|
||||
|
||||
TALER_DOMAIN=stage.lefrancpaysan.ch
|
||||
INSIDE_PODMAN=1
|
||||
|
|
@ -10,21 +12,32 @@ INSIDE_MODE=local-podman
|
|||
INSIDE_PROFILE=stage-lfp
|
||||
LOCAL_STACK=0
|
||||
CONTINUE_ON_ERROR=1
|
||||
PHASES="urls inside versions"
|
||||
RUN_TIMEOUT=600
|
||||
PHASES="urls inside versions monpages e2e"
|
||||
RUN_TIMEOUT=1800
|
||||
E2E_TIMEOUT=900
|
||||
E2E_USE_TEMPLATES=1
|
||||
|
||||
HTML_OUT=$HOME/monitoring-sites-staging
|
||||
MON_HOSTS="stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch"
|
||||
DEPLOY_WWW_ROOT=/var/www/monitoring-sites
|
||||
|
||||
# Git clone of koopa-admin-log (repo root, not …/scripts/taler-monitoring)
|
||||
SUITE_DIR=$HOME/src/taler-monitoring
|
||||
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
|
||||
SUITE_GIT_REF=main
|
||||
SUITE_UPDATE_MODE=reset
|
||||
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
|
||||
|
||||
# Optional public docroot if this user can write it
|
||||
# DEPLOY_WWW_ROOT=/var/www/monitoring-sites
|
||||
FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets
|
||||
WALLET_CLI=$HOME/.local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs
|
||||
WALLET_CLI_CANDIDATES="$HOME/.local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs $HOME/.local/bin/taler-wallet-cli.mjs"
|
||||
|
||||
INSIDE_BANK_CTR=stage-lfp-bank
|
||||
INSIDE_EXCHANGE_CTR=stage-lfp-exchange-ansible
|
||||
INSIDE_MERCHANT_CTR=stage-lfp-merchant
|
||||
|
||||
# UI language for console + HTML sticky bar (en|fr)
|
||||
TALER_MON_LANG=fr
|
||||
MONPAGES_REQUIRE_PUBLIC=1
|
||||
MONPAGES_INVENTORY=job
|
||||
STRICT_EXIT=1
|
||||
AGENT_LABEL=fp-stage-host-agent
|
||||
STATE_NAME=taler-monitoring-stage-lfp
|
||||
|
|
|
|||
|
|
@ -4,18 +4,21 @@
|
|||
# # on koopa as hernani:
|
||||
# ~/src/taler-monitoring/host-agent/install-host-agent.sh
|
||||
# # or from laptop:
|
||||
# ./install-host-agent.sh --remote koopa-external
|
||||
# ./install-host-agent.sh --remote $DEPLOY_SSH
|
||||
#
|
||||
# Suite tree is ONLY ~/src/taler-monitoring (standalone repo). No koopa-admin-log.
|
||||
#
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
REMOTE="${1:-}"
|
||||
[ "${1:-}" = "--remote" ] && REMOTE="${2:-koopa-external}"
|
||||
[ "${1:-}" = "--remote" ] && REMOTE="${2:-${DEPLOY_SSH:-}}"
|
||||
|
||||
SUITE_URL="${SUITE_GIT_URL:-https://git.hacktivism.ch/hernani/taler-monitoring.git}"
|
||||
|
||||
install_local() {
|
||||
local mon home_unit
|
||||
mon=$(cd "$ROOT/.." && pwd)
|
||||
# Prefer ~/koopa-admin-log on live host
|
||||
if [ -d "$HOME/src/taler-monitoring" ]; then
|
||||
if [ -d "$HOME/src/taler-monitoring/.git" ] || [ -d "$HOME/src/taler-monitoring" ]; then
|
||||
mon="$HOME/src/taler-monitoring"
|
||||
fi
|
||||
home_unit="$HOME/.config/systemd/user"
|
||||
|
|
@ -31,75 +34,95 @@ install_local() {
|
|||
echo "created ~/.config/taler-monitoring/env"
|
||||
fi
|
||||
|
||||
# Prefer a real git clone for update-suite.sh
|
||||
if [ ! -d "$HOME/src/koopa-admin-log/.git" ] && [ ! -d "$HOME/koopa-admin-log/.git" ]; then
|
||||
if [ -d "$HOME/src/taler-monitoring" ] && [ ! -d "$HOME/koopa-admin-log/.git" ]; then
|
||||
echo "note: $HOME/koopa-admin-log has no .git — update-suite will clone to ~/src/koopa-admin-log"
|
||||
# Ensure suite git clone at canonical path
|
||||
if [ ! -d "$HOME/src/taler-monitoring/.git" ]; then
|
||||
if [ -d "$mon/.git" ] && [ "$mon" != "$HOME/src/taler-monitoring" ]; then
|
||||
mkdir -p "$HOME/src"
|
||||
ln -sfn "$mon" "$HOME/src/taler-monitoring"
|
||||
echo "symlink $HOME/src/taler-monitoring → $mon"
|
||||
else
|
||||
echo "clone $SUITE_URL → $HOME/src/taler-monitoring"
|
||||
git clone "$SUITE_URL" "$HOME/src/taler-monitoring" \
|
||||
|| echo "WARN: clone failed (auth/network) — using tree at $mon if present"
|
||||
fi
|
||||
git clone https://git.hacktivism.ch/hernani/taler-monitoring.git "$HOME/src/koopa-admin-log" 2>/dev/null \
|
||||
|| echo "WARN: clone failed (auth/network) — will use existing tree if any"
|
||||
fi
|
||||
mon="$HOME/src/taler-monitoring"
|
||||
# Convenience link (update-suite.sh also creates this)
|
||||
ln -sfn "$mon" "$HOME/taler-monitoring" 2>/dev/null || true
|
||||
|
||||
# Ensure suite is executable
|
||||
chmod +x "$mon"/taler-monitoring.sh "$mon"/check_*.sh 2>/dev/null || true
|
||||
chmod +x "$mon/host-agent"/*.sh 2>/dev/null || true
|
||||
chmod +x "$mon/site-gen"/*.sh "$mon/site-gen/console_to_html.py" 2>/dev/null || true
|
||||
|
||||
# Units reference %h/koopa-admin-log — prefer git clone or symlink
|
||||
if [ -d "$HOME/src/koopa-admin-log/.git" ] && [ ! -e "$HOME/koopa-admin-log" ]; then
|
||||
ln -sfn "$HOME/src/koopa-admin-log" "$HOME/koopa-admin-log"
|
||||
echo "symlink $HOME/koopa-admin-log → $HOME/src/koopa-admin-log"
|
||||
elif [ ! -e "$HOME/koopa-admin-log" ]; then
|
||||
admin_root=$(cd "$mon/../.." && pwd)
|
||||
ln -sfn "$admin_root" "$HOME/koopa-admin-log"
|
||||
echo "symlink $HOME/koopa-admin-log → $admin_root"
|
||||
fi
|
||||
|
||||
install -m 644 "$ROOT/taler-monitoring-hacktivism.service" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-hacktivism.path" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-hacktivism.timer" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-surface.service" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-surface.timer" "$home_unit/"
|
||||
# optional aptdeploy (not a public mon page layout — v1.8.0 simplified overview)
|
||||
install -m 644 "$ROOT/taler-monitoring-aptdeploy.service" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-aptdeploy.timer" "$home_unit/"
|
||||
|
||||
# Fix ExecStart if install path differs (units use %h/koopa-admin-log)
|
||||
# GOA max-search amount ladder page (daily only — no classic /monitoring-ladder)
|
||||
install -m 644 "$ROOT/taler-monitoring-max-ladder.service" "$home_unit/"
|
||||
install -m 644 "$ROOT/taler-monitoring-max-ladder.timer" "$home_unit/"
|
||||
chmod +x "$ROOT/run-max-ladder-monitoring.sh" 2>/dev/null || true
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now taler-monitoring-hacktivism.path
|
||||
systemctl --user enable --now taler-monitoring-hacktivism.timer
|
||||
systemctl --user enable --now taler-monitoring-surface.timer
|
||||
systemctl --user enable --now taler-monitoring-aptdeploy.timer
|
||||
systemctl --user enable --now taler-monitoring-max-ladder.timer
|
||||
# drop classic ladder mon page if previously installed
|
||||
systemctl --user disable --now taler-monitoring-ladder.timer 2>/dev/null || true
|
||||
systemctl --user stop taler-monitoring-ladder.service 2>/dev/null || true
|
||||
rm -f "$home_unit/taler-monitoring-ladder.service" \
|
||||
"$home_unit/taler-monitoring-ladder.timer" 2>/dev/null || true
|
||||
# Remove leftover mail/mattermost units if present (moved to meta/deprecated/)
|
||||
systemctl --user disable --now taler-monitoring-mattermost.timer 2>/dev/null || true
|
||||
systemctl --user disable --now taler-monitoring-mail.timer 2>/dev/null || true
|
||||
rm -f "$home_unit/taler-monitoring-mattermost.service" \
|
||||
"$home_unit/taler-monitoring-mattermost.timer" \
|
||||
"$home_unit/taler-monitoring-mail.service" \
|
||||
"$home_unit/taler-monitoring-mail.timer" 2>/dev/null || true
|
||||
systemctl --user daemon-reload 2>/dev/null || true
|
||||
# initial stamp so path exists
|
||||
"$ROOT/touch-software-stamp.sh"
|
||||
systemctl --user start taler-monitoring-hacktivism.service || true
|
||||
systemctl --user start taler-monitoring-surface.service || true
|
||||
systemctl --user start taler-monitoring-aptdeploy.service || true
|
||||
# oneshot jobs: --no-block so install does not wait hours
|
||||
systemctl --user start --no-block taler-monitoring-hacktivism.service || true
|
||||
systemctl --user start --no-block taler-monitoring-surface.service || true
|
||||
systemctl --user start --no-block taler-monitoring-aptdeploy.service || true
|
||||
# first max-ladder round immediately (daily thereafter)
|
||||
systemctl --user start --no-block taler-monitoring-max-ladder.service || true
|
||||
|
||||
echo "--- status ---"
|
||||
systemctl --user status taler-monitoring-hacktivism.path --no-pager -l | head -15
|
||||
systemctl --user status taler-monitoring-hacktivism.timer --no-pager -l | head -12
|
||||
systemctl --user status taler-monitoring-surface.timer --no-pager -l | head -12
|
||||
systemctl --user status taler-monitoring-aptdeploy.timer --no-pager -l | head -12
|
||||
systemctl --user status taler-monitoring-max-ladder.timer --no-pager -l | head -12
|
||||
systemctl --user list-timers --all | grep taler-monitoring || true
|
||||
echo "OK host-agent (hacktivism 4h + surface 1h + aptdeploy 4h)"
|
||||
echo "manual: systemctl --user start taler-monitoring-hacktivism.service"
|
||||
echo "manual surface: systemctl --user start taler-monitoring-surface.service"
|
||||
echo "manual aptdeploy: systemctl --user start taler-monitoring-aptdeploy.service"
|
||||
echo "OK host-agent:"
|
||||
echo " · hacktivism 4h → /monitoring on bank/exchange/taler (landing stacks)"
|
||||
echo " · max-ladder daily → /monitoring-max-ladder/ (GOA max-search, bank/exchange/taler)"
|
||||
echo " · surface 1h → /taler-monitoring-surface* (ecosystem versions/software)"
|
||||
echo " · aptdeploy 4h (optional tests)"
|
||||
echo " · no classic /monitoring-ladder page"
|
||||
echo " · mail/mattermost: not installed (meta/deprecated/; content in surface)"
|
||||
echo "suite: $mon"
|
||||
echo "manual max-ladder: systemctl --user start taler-monitoring-max-ladder.service"
|
||||
echo "after upgrades: $ROOT/touch-software-stamp.sh"
|
||||
}
|
||||
|
||||
if [ -n "$REMOTE" ] && [ "$REMOTE" != "--remote" ]; then
|
||||
echo "install on $REMOTE …"
|
||||
# ROOT = …/taler-monitoring/host-agent → suite root is parent
|
||||
rsync -az --delete \
|
||||
--exclude secrets.env \
|
||||
--exclude 'android-test/artifacts' \
|
||||
"$ROOT/../../" \
|
||||
--exclude '.git' \
|
||||
"$ROOT/../" \
|
||||
"${REMOTE}:src/taler-monitoring/"
|
||||
# ensure full tree for %h/koopa-admin-log if missing configs etc.
|
||||
ssh -o BatchMode=yes -o ConnectTimeout=20 "$REMOTE" \
|
||||
'test -d $HOME/koopa-admin-log || git clone https://git.hacktivism.ch/hernani/taler-monitoring.git $HOME/koopa-admin-log 2>/dev/null || true'
|
||||
rsync -az "$ROOT/" "${REMOTE}:src/taler-monitoring/host-agent/"
|
||||
rsync -az "$ROOT/../site-gen/" "${REMOTE}:src/taler-monitoring/site-gen/" 2>/dev/null || true
|
||||
ssh -o BatchMode=yes "$REMOTE" \
|
||||
'bash $HOME/src/taler-monitoring/host-agent/install-host-agent.sh'
|
||||
else
|
||||
|
|
|
|||
|
|
@ -20,10 +20,10 @@ export SKIP_SSH=0
|
|||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
# four containers + checks; allow long wall
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-1800}"
|
||||
# dedicated job: always aptdeploy only
|
||||
export PHASES=aptdeploy
|
||||
# Only merchant public front — not bank/exchange
|
||||
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
|
||||
# dedicated job: always aptdeploy only (force — do not inherit main-timer env)
|
||||
export PHASES="aptdeploy monpages"
|
||||
# Public HTML only on taler.hacktivism.ch (not bank/exchange landings)
|
||||
export MON_HOSTS="taler.hacktivism.ch"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
|
||||
|
|
@ -32,6 +32,8 @@ export HTML_ERR_DIR="taler-monitoring-aptdeploy_err"
|
|||
export HTML_URL_OK="/taler-monitoring-aptdeploy/"
|
||||
export HTML_URL_ERR="/taler-monitoring-aptdeploy_err/"
|
||||
export PAGE_LABEL="taler-monitoring-aptdeploy"
|
||||
# monpages: only this job’s paths on taler.hacktivism.ch
|
||||
export MONPAGES_INVENTORY="job"
|
||||
|
||||
# Ensure all four apt-src containers (fresh rebuild + upgrade track)
|
||||
if [ "${APT_DEPLOY_ENSURE:-1}" = "1" ] \
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ set -euo pipefail
|
|||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
BOOT_MON=$(cd "$AGENT_DIR/.." && pwd)
|
||||
MODE="${1:-all}"
|
||||
APT_DEPLOY_SSH="${APT_DEPLOY_SSH:-koopa-external}"
|
||||
APT_DEPLOY_SSH="${APT_DEPLOY_SSH:-${DEPLOY_SSH:-}}"
|
||||
OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay"
|
||||
|
||||
on_koopa() {
|
||||
|
|
@ -114,7 +114,7 @@ run_local() {
|
|||
run_remote() {
|
||||
echo "======== aptdeploy remote via ssh ${APT_DEPLOY_SSH} mode=$MODE ========"
|
||||
if [ "${APT_DEPLOY_RSYNC:-1}" = "1" ]; then
|
||||
echo "rsync mon suite → ${APT_DEPLOY_SSH}:src/src/taler-monitoring/"
|
||||
echo "rsync mon suite → ${APT_DEPLOY_SSH}:src/taler-monitoring/"
|
||||
rsync -az \
|
||||
--exclude secrets.env \
|
||||
--exclude 'android-test/artifacts' \
|
||||
|
|
@ -122,7 +122,7 @@ run_remote() {
|
|||
--exclude 'android-test/out*' \
|
||||
--exclude '__pycache__' \
|
||||
"$BOOT_MON/" \
|
||||
"${APT_DEPLOY_SSH}:src/src/taler-monitoring/" \
|
||||
"${APT_DEPLOY_SSH}:src/taler-monitoring/" \
|
||||
|| echo "WARN: rsync failed" >&2
|
||||
fi
|
||||
ssh -o BatchMode=yes -o ConnectTimeout=25 "$APT_DEPLOY_SSH" bash -s -- "$MODE" <<'EOS'
|
||||
|
|
|
|||
|
|
@ -1,24 +0,0 @@
|
|||
#!/usr/bin/env bash
|
||||
# FrancPaysan PROD host-agent (user francpaysan on francpaysan-host).
|
||||
# Shared reporting: run-host-report.sh (timeout, line-buffered log, first-run HTML).
|
||||
set -uo pipefail
|
||||
|
||||
# Language: en|fr (selectable). Default for this wrapper: fr
|
||||
export TALER_MON_LANG="${TALER_MON_LANG:-fr}"
|
||||
export TALER_MON_LANG_SET=1
|
||||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
|
||||
export AGENT_LABEL="${AGENT_LABEL:-fp-prod-host-agent}"
|
||||
export STATE_NAME="${STATE_NAME:-taler-monitoring-lfp-prod}"
|
||||
export TALER_DOMAIN="${TALER_DOMAIN:-lefrancpaysan.ch}"
|
||||
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
|
||||
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
|
||||
export LOCAL_STACK="${LOCAL_STACK:-0}"
|
||||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
|
||||
export PHASES="${PHASES:-urls inside versions}"
|
||||
export MON_HOSTS="${MON_HOSTS:-bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch}"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
|
||||
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env bash
|
||||
# FrancPaysan STAGE host-agent (user stagepaysan on francpaysan-host).
|
||||
# FrancPaysan STAGE host-agent (user stagepaysan via francpaysan-stage-user).
|
||||
# Shared reporting: run-host-report.sh (timeout, line-buffered log, first-run HTML).
|
||||
set -uo pipefail
|
||||
|
||||
|
|
@ -16,8 +16,10 @@ export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
|
|||
export INSIDE_PROFILE="${INSIDE_PROFILE:-stage-lfp}"
|
||||
export LOCAL_STACK="${LOCAL_STACK:-0}"
|
||||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
|
||||
export PHASES="${PHASES:-urls inside versions}"
|
||||
# monpages obligatory (ERROR): only FP stage mon hosts below — never GOA inventory
|
||||
export MONPAGES_REQUIRE_PUBLIC="${MONPAGES_REQUIRE_PUBLIC:-1}"
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-1800}"
|
||||
export PHASES="${PHASES:-urls inside versions monpages e2e}"
|
||||
export MON_HOSTS="${MON_HOSTS:-stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch}"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
|
|
|
|||
|
|
@ -16,12 +16,18 @@ export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
|
|||
export LOCAL_STACK="${LOCAL_STACK:-1}"
|
||||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
|
||||
# stack checks only — aptdeploy has its own HTML path + timer:
|
||||
# run-aptdeploy-monitoring.sh → /taler-monitoring-aptdeploy* (taler.hacktivism.ch)
|
||||
export PHASES="${PHASES:-urls inside versions}"
|
||||
# stack checks + full monpages inventory (landings + max-ladder + surface + aptdeploy).
|
||||
# Specialized pages also have their own timers; full outside-in inventory lives here.
|
||||
# run-max-ladder-monitoring.sh → /monitoring-max-ladder* (daily max-search, GOA)
|
||||
# run-aptdeploy-monitoring.sh → monpages job-only for /taler-monitoring-aptdeploy*
|
||||
# run-surface-monitoring.sh → monpages job-only for /taler-monitoring-surface*
|
||||
# No classic /monitoring-ladder page.
|
||||
export PHASES="${PHASES:-urls inside versions monpages}"
|
||||
export MON_HOSTS="${MON_HOSTS:-bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch}"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
export APT_DEPLOY_ENSURE=0
|
||||
# full GOA mon catalog (not job-only)
|
||||
export MONPAGES_INVENTORY="${MONPAGES_INVENTORY:-auto}"
|
||||
|
||||
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
#!/usr/bin/env bash
|
||||
# run-host-report.sh — global reporting generation for all stacks
|
||||
# (GOA/hacktivism, FP stage, FP prod, …).
|
||||
# (GOA/hacktivism, FP stage, mytops stage, …).
|
||||
#
|
||||
# Shared behaviour (always):
|
||||
# 1) update-suite → latest git.hacktivism.ch (local ~/.config never overwritten)
|
||||
# 2) line-buffered run log (written continuously)
|
||||
# 3) RUN_TIMEOUT wall clock for taler-monitoring.sh (default 600s)
|
||||
# 4) always write HTML (first run / empty staging too) + Forgejo commit link
|
||||
# 5) optional rsync to DEPLOY_WWW_ROOT if writable
|
||||
# 5) rsync to DEPLOY_WWW_ROOT (ERROR if not writable — v1.3.1)
|
||||
# 6) monpages post-check: public FQDN must serve monitoring HTML
|
||||
# 7) exit code: STRICT_EXIT default 1 (v1.7.1) — unit fails if publish/public pages fail
|
||||
#
|
||||
# Configure via env file (not in git):
|
||||
# ${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env
|
||||
|
|
@ -16,7 +18,6 @@
|
|||
# Thin wrappers only set stack defaults then exec this script:
|
||||
# run-hacktivism-monitoring.sh
|
||||
# run-fp-stage-monitoring.sh
|
||||
# run-fp-prod-monitoring.sh
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
|
|
@ -37,6 +38,9 @@ _wrap_keys=(
|
|||
RUN_TIMEOUT STATE_NAME AGENT_LABEL MON_HOSTS TALER_DOMAIN DEPLOY_WWW_ROOT
|
||||
INSIDE_PODMAN INSIDE_MODE LOCAL_STACK SKIP_SSH CONTINUE_ON_ERROR AUTH401_CONTINUE
|
||||
APT_DEPLOY_ENSURE SURFACE_SCOPE TALER_DOMAIN_FROM_CLI STRICT_EXIT SOURCE_SUITE_PATH
|
||||
MONPAGES_INVENTORY MONPAGES_CHECK_BARE MONPAGES_BARE_STRICT MONPAGES_REQUIRE_PUBLIC
|
||||
CHECK_BANK MERCHANT_REQUIRED CHECK_LANDING
|
||||
WALLET_CLI CLI_JS PATH LADDER_MODE LADDER_PAY LADDER_PAY_WAIT_AVAILABLE_S
|
||||
)
|
||||
for _k in "${_wrap_keys[@]}"; do
|
||||
if [ -n "${!_k+x}" ]; then
|
||||
|
|
@ -72,23 +76,19 @@ export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
|||
export AUTH401_CONTINUE="${AUTH401_CONTINUE:-${CONTINUE_ON_ERROR}}"
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
|
||||
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
|
||||
# UI language: selectable via TALER_MON_LANG / --lang (en|fr).
|
||||
# If unset, auto: FrancPaysan domains → fr, else en.
|
||||
if [ "${TALER_MON_LANG_SET:-0}" != "1" ] && [ -z "${TALER_MON_LANG:-}" ]; then
|
||||
case "$TALER_DOMAIN" in
|
||||
*lefrancpaysan*|*francpaysan*) export TALER_MON_LANG=fr ;;
|
||||
*) export TALER_MON_LANG=en ;;
|
||||
esac
|
||||
elif [ -n "${TALER_MON_LANG:-}" ]; then
|
||||
case "$TALER_MON_LANG" in
|
||||
# UI language: --lang / TALER_MON_LANG_SET=1 locks.
|
||||
# Otherwise domains.conf column `lang` → TALER_DOMAIN_LANG → TALER_MON_LANG
|
||||
# (applied in suite apply_taler_domain / i18n_init). Leave SET=0 so -d can re-apply.
|
||||
# Bare workstation TALER_MON_LANG=en must not override profile lang=fr.
|
||||
if [ "${TALER_MON_LANG_SET:-0}" = "1" ]; then
|
||||
case "${TALER_MON_LANG:-en}" in
|
||||
fr|FR|fra|french) export TALER_MON_LANG=fr ;;
|
||||
*) export TALER_MON_LANG=en ;;
|
||||
esac
|
||||
export TALER_MON_LANG_SET=1
|
||||
fi
|
||||
export TALER_MON_LANG
|
||||
|
||||
PHASES="${PHASES:-urls inside versions}"
|
||||
PHASES="${PHASES:-urls inside versions monpages}"
|
||||
HTML_BASE="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
MON_HOSTS="${MON_HOSTS:-}"
|
||||
SUITE_PATH="${SOURCE_SUITE_PATH:-.}"
|
||||
|
|
@ -103,6 +103,28 @@ HTML_ERR_DIR="${HTML_ERR_DIR:-monitoring_err}"
|
|||
HTML_URL_OK="${HTML_URL_OK:-/monitoring/}"
|
||||
HTML_URL_ERR="${HTML_URL_ERR:-/monitoring_err/}"
|
||||
PAGE_LABEL="${PAGE_LABEL:-monitoring}"
|
||||
SUITE_VERSION="${SUITE_VERSION:-}"
|
||||
SUITE_VERSION_URL="${SUITE_VERSION_URL:-}"
|
||||
if [ -z "$SUITE_VERSION" ] && [ -d "${ROOT:-$AGENT_MON}/.git" ]; then
|
||||
_vd="${ROOT:-$AGENT_MON}"
|
||||
SUITE_VERSION=$(git -C "$_vd" describe --tags --exact-match 2>/dev/null || git -C "$_vd" describe --tags --abbrev=0 2>/dev/null || true)
|
||||
fi
|
||||
if [ -z "$SUITE_VERSION" ] && [ -f "${ROOT:-$AGENT_MON}/VERSION" ]; then
|
||||
SUITE_VERSION="v$(tr -d '[:space:]' <"${ROOT:-$AGENT_MON}/VERSION" | sed 's/^v//')"
|
||||
fi
|
||||
[ -z "$SUITE_VERSION" ] && SUITE_VERSION="unknown"
|
||||
case "$SUITE_VERSION" in unknown|v*) ;; *) SUITE_VERSION="v$SUITE_VERSION" ;; esac
|
||||
# Prefer commit URL over /src/tag/… (tags lag push → sticky version 404 on Forgejo).
|
||||
if [ -z "$SUITE_VERSION_URL" ]; then
|
||||
_web="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-monitoring}"
|
||||
if [ -n "${COMMIT:-}" ] && [ "$COMMIT" != "unknown" ]; then
|
||||
SUITE_VERSION_URL="${_web}/src/commit/${COMMIT}"
|
||||
elif [ "$SUITE_VERSION" != "unknown" ]; then
|
||||
SUITE_VERSION_URL="${_web}/src/tag/${SUITE_VERSION}"
|
||||
fi
|
||||
fi
|
||||
export SUITE_VERSION SUITE_VERSION_URL
|
||||
|
||||
|
||||
# Sensible MON_HOSTS from domain if not set
|
||||
if [ -z "$MON_HOSTS" ]; then
|
||||
|
|
@ -126,7 +148,32 @@ mkdir -p "$LOG_DIR" "$HTML_BASE"
|
|||
|
||||
STAMP=$(date +%Y%m%d-%H%M%S)
|
||||
LOG="$LOG_DIR/run-${STAMP}.log"
|
||||
UPD_LOG="$LOG_DIR/suite-update-${STAMP}.log"
|
||||
: >"$LOG"
|
||||
|
||||
# --- suite auto-upgrade BEFORE the HTML log (git noise must not pollute mon pages) ---
|
||||
OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay"
|
||||
if [ -f "$AGENT_MON/taler-monitoring.sh" ]; then
|
||||
mkdir -p "$OVERLAY_DIR"
|
||||
rsync -a \
|
||||
--exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \
|
||||
--exclude 'android-test/out*' --exclude '__pycache__' \
|
||||
"$AGENT_MON/" "$OVERLAY_DIR/" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# shellcheck source=update-suite.sh
|
||||
if ! source "$AGENT_DIR/update-suite.sh" >"$UPD_LOG" 2>&1; then
|
||||
echo "ERROR: suite auto-update failed — refusing to run with stale tree" >&2
|
||||
echo " detail: $UPD_LOG" >&2
|
||||
tail -n 30 "$UPD_LOG" >&2 || true
|
||||
echo " fix git/network or set SUITE_UPDATE_STRICT=0 only as emergency" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ -d "${SUITE_DIR:-}/host-agent" ]; then
|
||||
AGENT_DIR="${SUITE_DIR}/host-agent"
|
||||
fi
|
||||
|
||||
# Now open the public HTML log (clean agent header + suite summary only)
|
||||
if command -v stdbuf >/dev/null 2>&1; then
|
||||
exec > >(stdbuf -oL -eL tee -a "$LOG") 2>&1
|
||||
else
|
||||
|
|
@ -138,20 +185,9 @@ echo "env_file=$ENV_FILE"
|
|||
echo "domain=$TALER_DOMAIN · phases=$PHASES · RUN_TIMEOUT=${RUN_TIMEOUT}s"
|
||||
echo "hosts=$MON_HOSTS"
|
||||
echo "flags INSIDE_PODMAN=${INSIDE_PODMAN:-} INSIDE_MODE=${INSIDE_MODE:-} LOCAL_STACK=${LOCAL_STACK:-} SKIP_SSH=${SKIP_SSH:-} INSIDE_PROFILE=${INSIDE_PROFILE:-}"
|
||||
|
||||
# --- refresh from Forgejo; re-apply suite-overlay if Forgejo lacks needed phases ---
|
||||
OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay"
|
||||
# preserve capable mon before reset wipes tracked files
|
||||
if [ -f "$AGENT_MON/taler-monitoring.sh" ]; then
|
||||
mkdir -p "$OVERLAY_DIR"
|
||||
rsync -a \
|
||||
--exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \
|
||||
--exclude 'android-test/out*' --exclude '__pycache__' \
|
||||
"$AGENT_MON/" "$OVERLAY_DIR/" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# shellcheck source=update-suite.sh
|
||||
source "$AGENT_DIR/update-suite.sh"
|
||||
# One-line suite pin (no git M/checkout noise)
|
||||
echo "suite ${SUITE_VERSION:-?} @ ${COMMIT_SHORT:-?} ${COMMIT_URL:-}"
|
||||
echo " dir=${SUITE_DIR:-}"
|
||||
|
||||
SUITE_MON="${SUITE_DIR}"
|
||||
ROOT="$SUITE_MON"
|
||||
|
|
@ -193,16 +229,8 @@ elif [ "$_need_overlay" = "1" ]; then
|
|||
ROOT="$AGENT_MON"
|
||||
else
|
||||
echo "using suite mon @ ${COMMIT_SHORT:-?} · $ROOT"
|
||||
fi
|
||||
# Always restore host-agent + site-gen from pre-reset snapshot when present
|
||||
# (systemd ExecStart points at koopa-admin-log which may be the reset suite tree).
|
||||
if [ -d "$OVERLAY_DIR/host-agent" ]; then
|
||||
mkdir -p "$SUITE_MON/host-agent" 2>/dev/null || true
|
||||
rsync -a "$OVERLAY_DIR/host-agent/" "$SUITE_MON/host-agent/" 2>/dev/null || true
|
||||
fi
|
||||
if [ -d "$OVERLAY_DIR/site-gen" ]; then
|
||||
mkdir -p "$SUITE_MON/site-gen"
|
||||
rsync -a "$OVERLAY_DIR/site-gen/" "$SUITE_MON/site-gen/" 2>/dev/null || true
|
||||
# Do NOT rsync suite-overlay over a fresh Forgejo tree — that reintroduced
|
||||
# stale site-gen/host-agent (bootstrap sticky, old monpages) on stage/betel.
|
||||
fi
|
||||
unset _need_overlay
|
||||
|
||||
|
|
@ -248,9 +276,17 @@ if [ -x "${ROOT:-}/taler-monitoring.sh" ]; then
|
|||
fi
|
||||
done
|
||||
|
||||
# monpages inside PHASES runs *before* htmlify/publish — content failures would
|
||||
# race with concurrent publishes or reflect last run's incomplete redirect stub.
|
||||
# Soft-content there; post-check after rsync stays hard (authoritative).
|
||||
case " $PHASES " in
|
||||
*" monpages "*|*" pages "*) export MONPAGES_PRE_PUBLISH=1 ;;
|
||||
*) unset MONPAGES_PRE_PUBLISH 2>/dev/null || true ;;
|
||||
esac
|
||||
# shellcheck disable=SC2086
|
||||
./taler-monitoring.sh -d "$TALER_DOMAIN" $PHASES
|
||||
ec=$?
|
||||
unset MONPAGES_PRE_PUBLISH 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# --- always generate HTML (first run / missing pages / every run) ---
|
||||
|
|
@ -261,26 +297,38 @@ write_bootstrap_html() {
|
|||
local curl="${COMMIT_URL:-}"
|
||||
local clink="$cshort"
|
||||
local gen iso
|
||||
gen=$(date -u +"%Y-%m-%d %H:%M:%SZ" 2>/dev/null || date)
|
||||
iso=$(date -u +"%Y-%m-%dT%H:%M:%SZ" 2>/dev/null || date -Iseconds)
|
||||
# Local wall clock (CEST/CET · MESZ/MEZ on CH/DE hosts), not UTC Z
|
||||
if TZ=Europe/Zurich date +"%Y-%m-%d %H:%M:%S %Z" >/dev/null 2>&1; then
|
||||
gen=$(TZ=Europe/Zurich date +"%Y-%m-%d %H:%M:%S %Z")
|
||||
iso=$(TZ=Europe/Zurich date +"%Y-%m-%dT%H:%M:%S%z" | sed 's/\([+-][0-9][0-9]\)\([0-9][0-9]\)$/\1:\2/')
|
||||
else
|
||||
gen=$(date +"%Y-%m-%d %H:%M:%S %Z" 2>/dev/null || date)
|
||||
iso=$(date -Iseconds 2>/dev/null || date)
|
||||
fi
|
||||
if [ -n "$curl" ]; then
|
||||
clink="<a href=\"$(printf '%s' "$curl" | sed 's/&/\&/g')\">$cshort</a>"
|
||||
fi
|
||||
cat >"$out" <<EOF
|
||||
local tmp
|
||||
tmp=$(mktemp "${out}.XXXXXX")
|
||||
cat >"$tmp" <<EOF
|
||||
<!DOCTYPE html>
|
||||
<html lang="en"><head><meta charset="utf-8"/>
|
||||
<meta name="generated" content="${iso}"/>
|
||||
<meta name="taler-mon-page" content="incomplete"/>
|
||||
<title>monitoring ${mode} · ${host}</title>
|
||||
<style>
|
||||
body{margin:0;background:#0c0c0c;color:#c8c8c8;font-family:ui-monospace,monospace}
|
||||
a{color:#61afef}.err{color:#ff5c5c}
|
||||
.sticky-bar{position:sticky;top:0;z-index:100;padding:10px 14px;background:#2a1010f2;border-bottom:2px solid #a33}
|
||||
.sticky-bar .pill{font-weight:800;color:#ff5c5c;margin-right:8px}
|
||||
.version-link{color:#61afef}
|
||||
main{padding:1.5rem}
|
||||
</style></head><body>
|
||||
<!-- taler-mon:top -->
|
||||
<div class="sticky-bar" id="status-bar">
|
||||
<span class="pill">BOOTSTRAP</span>
|
||||
${PAGE_LABEL:-monitoring} · ${host} ·
|
||||
<span class="version-link muted">bootstrap</span> ·
|
||||
<span data-generated-iso="${iso}">generated ${gen}</span>
|
||||
· source ${clink}
|
||||
</div>
|
||||
|
|
@ -288,8 +336,10 @@ main{padding:1.5rem}
|
|||
<p class="err">Bootstrap page (converter missing or first install). See host-agent log.</p>
|
||||
<pre style="white-space:pre-wrap;font-size:12px">$(tail -n 80 "$LOG" 2>/dev/null | sed 's/&/\&/g;s/</\</g' || true)</pre>
|
||||
</main>
|
||||
<!-- taler-mon:bottom:incomplete -->
|
||||
</body></html>
|
||||
EOF
|
||||
mv -f "$tmp" "$out"
|
||||
echo "bootstrap html → $out"
|
||||
}
|
||||
|
||||
|
|
@ -300,7 +350,29 @@ htmlify_host() {
|
|||
mkdir -p "$HTML_BASE/$host/${HTML_OK_DIR}" "$HTML_BASE/$host/${HTML_ERR_DIR}"
|
||||
|
||||
if [ -n "$SITE_GEN" ] && [ -f "$SITE_GEN/console_to_html.py" ]; then
|
||||
python3 "$SITE_GEN/console_to_html.py" \
|
||||
# Never convert with a pre-1.15.8 generator (old assets → SyntaxWarning + bad sticky JS).
|
||||
if grep -qE 'if \(/\^\[[╔╗╚╝║═]' "$SITE_GEN/console_to_html.py" 2>/dev/null; then
|
||||
echo "ERROR: refuse stale site-gen $SITE_GEN/console_to_html.py (box-drawing JS regex)" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
# SyntaxWarning = generator regression: fail convert (do not publish old-style pages).
|
||||
# Other stderr still goes to agent log file only, not the mon console stream.
|
||||
_mon_html_py() {
|
||||
local _out="" _i
|
||||
for _i in "$@"; do
|
||||
if [ "$_i" = "--out" ]; then _out=1; continue; fi
|
||||
if [ "$_out" = "1" ]; then _out="$_i"; break; fi
|
||||
done
|
||||
PYTHONWARNINGS=error::SyntaxWarning \
|
||||
python3 "$SITE_GEN/console_to_html.py" "$@" 2>>"${LOG_DIR:-/tmp}/console_to_html.err" || return 1
|
||||
# Never leave pre-1.15.8 sticky JS in a just-written page (stale log text is filtered in converter).
|
||||
if [ -n "$_out" ] && [ -f "$_out" ] && grep -qE 'if \(/\^\[[╔╗╚╝║═]' "$_out" 2>/dev/null; then
|
||||
echo "ERROR: generated mon HTML still has stale box-drawing JS: $_out" >&2
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
if ! _mon_html_py \
|
||||
--lang "${TALER_MON_LANG:-en}" \
|
||||
--log "$LOG" \
|
||||
--out "$mon_err" \
|
||||
|
|
@ -308,37 +380,55 @@ htmlify_host() {
|
|||
--mode err \
|
||||
--commit "${COMMIT:-}" \
|
||||
--commit-url "${COMMIT_URL:-}" \
|
||||
--suite-version "${SUITE_VERSION:-unknown}" \
|
||||
--suite-version-url "${SUITE_VERSION_URL:-}" \
|
||||
--suite-path "$SUITE_PATH" \
|
||||
--page-label "$PAGE_LABEL" \
|
||||
--path-err "$HTML_URL_ERR" \
|
||||
--link-other "$HTML_URL_OK"
|
||||
then
|
||||
echo "ERROR: mon HTML convert failed (err page) for $host — refuse publish of stale/broken assets" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
if [ "$ec" -eq 0 ]; then
|
||||
python3 "$SITE_GEN/console_to_html.py" \
|
||||
--lang "${TALER_MON_LANG:-en}" \
|
||||
if ! _mon_html_py \
|
||||
--lang "${TALER_MON_LANG:-en}" \
|
||||
--log "$LOG" \
|
||||
--out "$mon" \
|
||||
--hostname "$host" \
|
||||
--mode ok \
|
||||
--commit "${COMMIT:-}" \
|
||||
--commit-url "${COMMIT_URL:-}" \
|
||||
--suite-version "${SUITE_VERSION:-unknown}" \
|
||||
--suite-version-url "${SUITE_VERSION_URL:-}" \
|
||||
--suite-path "$SUITE_PATH" \
|
||||
--page-label "$PAGE_LABEL" \
|
||||
--path-err "$HTML_URL_ERR" \
|
||||
--link-other ""
|
||||
then
|
||||
echo "ERROR: mon HTML convert failed (ok page) for $host" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
rm -rf "$HTML_BASE/$host/${HTML_ERR_DIR}"
|
||||
echo "html $host → ${HTML_URL_OK} only (clean · ${COMMIT_SHORT:-?})"
|
||||
else
|
||||
python3 "$SITE_GEN/console_to_html.py" \
|
||||
--lang "${TALER_MON_LANG:-en}" \
|
||||
if ! _mon_html_py \
|
||||
--lang "${TALER_MON_LANG:-en}" \
|
||||
--log "$LOG" \
|
||||
--out "$mon" \
|
||||
--hostname "$host" \
|
||||
--mode redirect \
|
||||
--commit "${COMMIT:-}" \
|
||||
--commit-url "${COMMIT_URL:-}" \
|
||||
--suite-version "${SUITE_VERSION:-unknown}" \
|
||||
--suite-version-url "${SUITE_VERSION_URL:-}" \
|
||||
--suite-path "$SUITE_PATH" \
|
||||
--page-label "$PAGE_LABEL" \
|
||||
--path-err "$HTML_URL_ERR"
|
||||
then
|
||||
echo "ERROR: mon HTML convert failed (redirect page) for $host" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
echo "html $host → ${HTML_URL_OK} stub + ${HTML_URL_ERR} (ec=$ec · ${COMMIT_SHORT:-?})"
|
||||
fi
|
||||
else
|
||||
|
|
@ -361,17 +451,76 @@ htmlify_host() {
|
|||
}
|
||||
for host in $MON_HOSTS; do
|
||||
htmlify_host "$host"
|
||||
echo "public URL (expected): https://${host}${HTML_URL_OK}"
|
||||
done
|
||||
|
||||
if [ -n "$DEPLOY_WWW" ] && [ -w "$DEPLOY_WWW" ] 2>/dev/null; then
|
||||
rsync -a --delete "$HTML_BASE/" "$DEPLOY_WWW/" && \
|
||||
echo "synced → $DEPLOY_WWW/"
|
||||
# Publish to live web root when possible (v1.3.1+: never silent-skip)
|
||||
if [ -n "$DEPLOY_WWW" ]; then
|
||||
if [ -w "$DEPLOY_WWW" ] 2>/dev/null; then
|
||||
# --delay-updates: write to temp names then rename (less partial-file exposure)
|
||||
if rsync -a --delete --delay-updates "$HTML_BASE/" "$DEPLOY_WWW/"; then
|
||||
echo "synced → $DEPLOY_WWW/ (delay-updates)"
|
||||
# brief settle so concurrent monpages curls do not hit mid-rename FS views
|
||||
sleep "${MONPAGES_PUBLISH_SETTLE_S:-0.3}" 2>/dev/null || sleep 1
|
||||
else
|
||||
echo "ERROR: rsync to DEPLOY_WWW_ROOT=$DEPLOY_WWW failed" >&2
|
||||
ec=1
|
||||
fi
|
||||
elif [ -e "$DEPLOY_WWW" ]; then
|
||||
echo "ERROR: DEPLOY_WWW_ROOT=$DEPLOY_WWW exists but is not writable by $(id -un)" >&2
|
||||
echo " public FQDN pages will stay missing/stale — fix ownership or run:" >&2
|
||||
echo " sudo ${APPLY_MONITORING_LIVE}" >&2
|
||||
ec=1
|
||||
else
|
||||
echo "ERROR: DEPLOY_WWW_ROOT=$DEPLOY_WWW missing — public pages not published" >&2
|
||||
echo " create tree + Caddy handles: sudo ${APPLY_MONITORING_LIVE}" >&2
|
||||
ec=1
|
||||
fi
|
||||
else
|
||||
echo "WARN: DEPLOY_WWW_ROOT empty — skipping live publish (staging only: $HTML_BASE)" >&2
|
||||
fi
|
||||
|
||||
# Post-publish: monpages obligatory (v1.7.6+) — GOA full inventory / FP only FP hosts
|
||||
_monpages_failed=0
|
||||
if [ "${MONPAGES_POSTCHECK:-1}" = "1" ] && [ -x "$ROOT/check_monitoring_pages.sh" ]; then
|
||||
echo "--- monpages post-check (public FQDN, obligatory) ---"
|
||||
export MON_HOSTS HTML_URL_OK HTML_URL_ERR TALER_DOMAIN HTML_OUT HTML_BASE
|
||||
export MONPAGES_STAGING_BASE="${HTML_BASE}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW:-${DEPLOY_WWW_ROOT:-}}"
|
||||
export MONPAGES_REQUIRE_PUBLIC="${MONPAGES_REQUIRE_PUBLIC:-1}"
|
||||
# auto: GOA = all suite mon sites; FP = only FP mon hosts; job paths always included
|
||||
export MONPAGES_INVENTORY="${MONPAGES_INVENTORY:-auto}"
|
||||
# Hard check (not pre-publish soft)
|
||||
unset MONPAGES_PRE_PUBLISH 2>/dev/null || true
|
||||
export MONPAGES_PRE_PUBLISH=0
|
||||
if ! bash "$ROOT/check_monitoring_pages.sh"; then
|
||||
echo "ERROR: public monitoring page(s) missing or not suite HTML via FQDN" >&2
|
||||
echo " staging HTML_OUT=${HTML_BASE} · DEPLOY_WWW=${DEPLOY_WWW:-}" >&2
|
||||
ec=1
|
||||
_monpages_failed=1
|
||||
fi
|
||||
unset MONPAGES_PRE_PUBLISH 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# If public check failed after first HTML write, rewrite pages so sticky bar shows monpages ERRORs
|
||||
if [ "$_monpages_failed" = "1" ]; then
|
||||
echo "--- re-htmlify after monpages failure (include public outage in report) ---"
|
||||
# append marker to log so converter sees ERROR lines if monpages was post-only
|
||||
{
|
||||
echo ""
|
||||
echo "ERROR monpages: public FQDN monitoring HTML missing or merchant JSON code 21"
|
||||
echo "ERROR monpages: fix with sudo ${APPLY_MONITORING_LIVE} (www + Caddy handles)"
|
||||
} >>"$LOG"
|
||||
for host in $MON_HOSTS; do
|
||||
htmlify_host "$host"
|
||||
done
|
||||
fi
|
||||
|
||||
ls -1t "$LOG_DIR"/run-*.log 2>/dev/null | tail -n +30 | xargs rm -f 2>/dev/null || true
|
||||
|
||||
echo "======== done ec=$ec · log=$LOG · ${COMMIT_URL:-} ========"
|
||||
if [ "${STRICT_EXIT:-0}" = "1" ]; then
|
||||
# v1.7.1: default STRICT_EXIT=1 so systemd/timer reflects publish/public failures
|
||||
if [ "${STRICT_EXIT:-1}" = "1" ]; then
|
||||
exit "$ec"
|
||||
fi
|
||||
exit 0
|
||||
|
|
|
|||
48
host-agent/run-max-ladder-monitoring.sh
Executable file
48
host-agent/run-max-ladder-monitoring.sh
Executable file
|
|
@ -0,0 +1,48 @@
|
|||
#!/usr/bin/env bash
|
||||
# run-max-ladder-monitoring.sh — GOA max-search amount ladder → public HTML
|
||||
#
|
||||
# Public (landing hosts, same Caddy /monitoring* static root):
|
||||
# https://{bank,exchange,taler}.hacktivism.ch/monitoring-max-ladder/
|
||||
# https://{bank,exchange,taler}.hacktivism.ch/monitoring-max-ladder_err/
|
||||
#
|
||||
# Timer: once daily (taler-monitoring-max-ladder.timer). Long wall clock.
|
||||
#
|
||||
set -uo pipefail
|
||||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
|
||||
export AGENT_LABEL="${AGENT_LABEL:-max-ladder-host-agent}"
|
||||
export STATE_NAME="${STATE_NAME:-taler-max-ladder-monitoring}"
|
||||
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
|
||||
export TALER_MON_LANG="${TALER_MON_LANG:-en}"
|
||||
export TALER_MON_LANG_SET=1
|
||||
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
|
||||
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
|
||||
export LOCAL_STACK="${LOCAL_STACK:-1}"
|
||||
export SKIP_SSH="${SKIP_SSH:-0}"
|
||||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
# max-search probes + pay hunt — force high wall clock (koopa env often has RUN_TIMEOUT=600)
|
||||
export RUN_TIMEOUT="${MAX_LADDER_RUN_TIMEOUT:-10800}"
|
||||
|
||||
export PHASES="max-ladder monpages"
|
||||
export MON_HOSTS="${MON_HOSTS:-bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch}"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
|
||||
export HTML_OK_DIR="monitoring-max-ladder"
|
||||
export HTML_ERR_DIR="monitoring-max-ladder_err"
|
||||
export HTML_URL_OK="/monitoring-max-ladder/"
|
||||
export HTML_URL_ERR="/monitoring-max-ladder_err/"
|
||||
export PAGE_LABEL="monitoring-max-ladder"
|
||||
export MONPAGES_INVENTORY="job"
|
||||
|
||||
export LADDER_MODE=max
|
||||
export LADDER_PAY="${LADDER_PAY:-1}"
|
||||
export LADDER_PAY_WAIT_AVAILABLE_S="${LADDER_PAY_WAIT_AVAILABLE_S:-180}"
|
||||
export APT_DEPLOY_ENSURE=0
|
||||
# Prefer host wallet-cli install used by mon (find_wallet_cli / WALLET_CLI)
|
||||
export PATH="${HOME}/.local/bin:${PATH:-}"
|
||||
if [ -z "${WALLET_CLI:-}" ] && [ -f "${HOME}/.local/bin/taler-wallet-cli.mjs" ]; then
|
||||
export WALLET_CLI="${HOME}/.local/bin/taler-wallet-cli.mjs"
|
||||
fi
|
||||
|
||||
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
|
||||
|
|
@ -1,27 +1,36 @@
|
|||
#!/usr/bin/env bash
|
||||
# run-surface-monitoring.sh — hourly remote-only ecosystem surface scan
|
||||
# Public only on taler.hacktivism.ch:
|
||||
# run-surface-monitoring.sh — ecosystem surface report (v1.8.0+ simplified)
|
||||
#
|
||||
# Public HTML only on taler.hacktivism.ch:
|
||||
# https://taler.hacktivism.ch/taler-monitoring-surface/
|
||||
# https://taler.hacktivism.ch/taler-monitoring-surface_err/
|
||||
#
|
||||
# This page covers remote ecosystem inventory: host/port probes, nmap OS
|
||||
# fingerprint, Mattermost, mail (firefly + mail.anastasis.lu/pixel), package/version signals.
|
||||
# There are NO separate public pages for mail / mattermost (folded here).
|
||||
# Apt-src deploy tests keep their own page + timer:
|
||||
# run-aptdeploy-monitoring.sh → /taler-monitoring-aptdeploy(+_err)/
|
||||
#
|
||||
# Landing-stack reports stay on each of the 9 fronts as /monitoring(/_err).
|
||||
#
|
||||
set -uo pipefail
|
||||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
|
||||
export AGENT_LABEL="${AGENT_LABEL:-surface-host-agent}"
|
||||
export STATE_NAME="${STATE_NAME:-taler-surface-monitoring}"
|
||||
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
|
||||
# surface is remote-only — force flags (ignore shared env)
|
||||
export INSIDE_PODMAN=0
|
||||
export LOCAL_STACK=0
|
||||
export SKIP_SSH=1
|
||||
export INSIDE_MODE="${INSIDE_MODE:-none}"
|
||||
# surface is primarily remote; versions may use host podman when available
|
||||
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
|
||||
export LOCAL_STACK="${LOCAL_STACK:-0}"
|
||||
export SKIP_SSH="${SKIP_SSH:-1}"
|
||||
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
|
||||
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
|
||||
# many hosts; allow up to 45 min wall (phase also has RUN_TIMEOUT)
|
||||
export RUN_TIMEOUT="${RUN_TIMEOUT:-2400}"
|
||||
# dedicated job: always surface only (never inherit PHASES from shared env)
|
||||
export PHASES=surface
|
||||
# Only merchant public front — not bank/exchange
|
||||
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
|
||||
|
||||
# All ecosystem checks → one surface HTML report (force — do not inherit main-timer env)
|
||||
export PHASES="surface mattermost mail versions monpages"
|
||||
|
||||
export MON_HOSTS="taler.hacktivism.ch"
|
||||
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
|
||||
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
|
||||
|
|
@ -31,11 +40,12 @@ export HTML_URL_OK="/taler-monitoring-surface/"
|
|||
export HTML_URL_ERR="/taler-monitoring-surface_err/"
|
||||
export PAGE_LABEL="taler-monitoring-surface"
|
||||
|
||||
# ecosystem catalog by default (not domain-only)
|
||||
export SURFACE_SCOPE="${SURFACE_SCOPE:-ecosystem}"
|
||||
export TALER_DOMAIN_FROM_CLI=0
|
||||
|
||||
# Do not run apt-deploy ensure for surface-only jobs
|
||||
export APT_DEPLOY_ENSURE=0
|
||||
# monpages job-only for surface paths on taler.hacktivism.ch
|
||||
export MONPAGES_INVENTORY="job"
|
||||
# package behind → ERROR (v1.8.0)
|
||||
export TALER_PKG_BEHIND="${TALER_PKG_BEHIND:-error}"
|
||||
|
||||
exec bash "$AGENT_DIR/run-host-report.sh" "$@"
|
||||
|
|
|
|||
|
|
@ -4,13 +4,9 @@ Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
|||
|
||||
[Path]
|
||||
# Explicit stamp after package/image upgrades (touch-software-stamp.sh)
|
||||
# Landing/caddy ops live outside this suite (host admin-log) — touch the stamp there.
|
||||
PathChanged=%h/.local/state/taler-hacktivism-monitoring/software.stamp
|
||||
PathModified=%h/.local/state/taler-hacktivism-monitoring/software.stamp
|
||||
# Landing + caddy config in admin-log (when checked out under ~)
|
||||
PathChanged=%h/koopa-admin-log/configs/bank-landing
|
||||
PathChanged=%h/koopa-admin-log/configs/exchange-landing
|
||||
PathChanged=%h/koopa-admin-log/configs/merchant-landing
|
||||
PathChanged=%h/koopa-admin-log/configs/caddy
|
||||
# Debounce: wait for quiet period before firing
|
||||
TriggerLimitIntervalSec=120
|
||||
TriggerLimitBurst=3
|
||||
|
|
|
|||
19
host-agent/taler-monitoring-max-ladder.service
Normal file
19
host-agent/taler-monitoring-max-ladder.service
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
[Unit]
|
||||
Description=Taler monitoring GOA max-search amount ladder → /monitoring-max-ladder/
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
|
||||
Environment=CONTINUE_ON_ERROR=1
|
||||
Environment=RUN_TIMEOUT=10800
|
||||
EnvironmentFile=-%h/.config/taler-monitoring/env
|
||||
WorkingDirectory=%h/src/taler-monitoring
|
||||
ExecStart=%h/src/taler-monitoring/host-agent/run-max-ladder-monitoring.sh
|
||||
Nice=10
|
||||
TimeoutStartSec=4h
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
14
host-agent/taler-monitoring-max-ladder.timer
Normal file
14
host-agent/taler-monitoring-max-ladder.timer
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
[Unit]
|
||||
Description=Daily GOA max-search amount ladder mon page
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
|
||||
[Timer]
|
||||
# Once per day, after classic ladder window
|
||||
OnCalendar=*-*-* 06:30:00
|
||||
Persistent=true
|
||||
AccuracySec=15min
|
||||
RandomizedDelaySec=30min
|
||||
Unit=taler-monitoring-max-ladder.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
20
host-agent/taler-monitoring-mytops-stage.service
Normal file
20
host-agent/taler-monitoring-mytops-stage.service
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
[Unit]
|
||||
Description=taler-monitoring mytops stage (betel) — auto-update suite + urls/monpages
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
# Env: ~/.config/taler-monitoring/env (SUITE_GIT_*, MON_HOSTS, PHASES=urls monpages, …)
|
||||
EnvironmentFile=-%h/.config/taler-monitoring/env
|
||||
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
|
||||
Environment=SUITE_UPDATE_STRICT=1
|
||||
WorkingDirectory=%h/src/taler-monitoring
|
||||
# run-host-report sources update-suite.sh first (strict fetch/reset to main)
|
||||
ExecStart=%h/src/taler-monitoring/host-agent/run-host-report.sh
|
||||
Nice=10
|
||||
TimeoutStartSec=20min
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
11
host-agent/taler-monitoring-mytops-stage.timer
Normal file
11
host-agent/taler-monitoring-mytops-stage.timer
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
[Unit]
|
||||
Description=taler-monitoring mytops stage (betel) every 4h — suite auto-upgrade + mon HTML
|
||||
|
||||
[Timer]
|
||||
OnBootSec=5min
|
||||
OnUnitActiveSec=4h
|
||||
Persistent=true
|
||||
Unit=taler-monitoring-mytops-stage.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
|
|
@ -1,5 +1,18 @@
|
|||
#!/usr/bin/env bash
|
||||
# update-suite.sh — latest taler-monitoring from Forgejo (standalone repo root).
|
||||
# update-suite.sh — pull latest taler-monitoring from Forgejo (standalone repo root).
|
||||
#
|
||||
# Sourced by run-host-report.sh / wrappers. Every host-agent run must land on
|
||||
# current origin/$SUITE_GIT_REF (default main). Stale clones are not acceptable.
|
||||
#
|
||||
# Env:
|
||||
# SUITE_GIT_URL default https://git.hacktivism.ch/hernani/taler-monitoring.git
|
||||
# SUITE_GIT_REF default main
|
||||
# SUITE_DIR default ~/src/taler-monitoring
|
||||
# SUITE_UPDATE_MODE reset (default) | pull | keep
|
||||
# keep = no fetch/reset (local deploy / unpushed fixes)
|
||||
# SUITE_UPDATE_STRICT 1 (default) = fetch/reset failure aborts the agent run
|
||||
# 0 = warn and continue on old tree (emergency only)
|
||||
#
|
||||
set -uo pipefail
|
||||
CFG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring"
|
||||
ENV_FILE="${TALER_MONITORING_ENV:-$CFG_DIR/env}"
|
||||
|
|
@ -9,37 +22,132 @@ fi
|
|||
SUITE_GIT_URL="${SUITE_GIT_URL:-https://git.hacktivism.ch/hernani/taler-monitoring.git}"
|
||||
SUITE_GIT_REF="${SUITE_GIT_REF:-main}"
|
||||
SUITE_UPDATE_MODE="${SUITE_UPDATE_MODE:-reset}"
|
||||
SUITE_UPDATE_STRICT="${SUITE_UPDATE_STRICT:-1}"
|
||||
if [ -z "${SUITE_DIR:-}" ]; then
|
||||
if [ -d "$HOME/src/taler-monitoring/.git" ]; then SUITE_DIR="$HOME/src/taler-monitoring"
|
||||
elif [ -d "$HOME/taler-monitoring/.git" ]; then SUITE_DIR="$HOME/taler-monitoring"
|
||||
else SUITE_DIR="$HOME/src/taler-monitoring"; fi
|
||||
fi
|
||||
export SUITE_DIR
|
||||
|
||||
_upd_die() {
|
||||
echo "ERROR suite-update: $*" >&2
|
||||
if [ "${SUITE_UPDATE_STRICT}" = "1" ]; then
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
echo "WARN suite-update: continuing on existing tree (SUITE_UPDATE_STRICT=0)" >&2
|
||||
return 0
|
||||
}
|
||||
|
||||
mkdir -p "$(dirname "$SUITE_DIR")" "$CFG_DIR"
|
||||
if [ ! -d "$SUITE_DIR/.git" ]; then
|
||||
echo "clone $SUITE_GIT_URL → $SUITE_DIR"
|
||||
git clone --branch "$SUITE_GIT_REF" "$SUITE_GIT_URL" "$SUITE_DIR" || git clone "$SUITE_GIT_URL" "$SUITE_DIR"
|
||||
if ! git clone --branch "$SUITE_GIT_REF" "$SUITE_GIT_URL" "$SUITE_DIR" \
|
||||
&& ! git clone "$SUITE_GIT_URL" "$SUITE_DIR"; then
|
||||
_upd_die "git clone failed"
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
fi
|
||||
cd "$SUITE_DIR" || exit 1
|
||||
cd "$SUITE_DIR" || { _upd_die "cd $SUITE_DIR failed"; return 1 2>/dev/null || exit 1; }
|
||||
git remote set-url origin "$SUITE_GIT_URL" 2>/dev/null || true
|
||||
echo "fetch origin ($SUITE_GIT_REF) …"
|
||||
if ! git fetch --tags origin 2>&1; then
|
||||
echo "WARN: git fetch failed — using existing tree" >&2
|
||||
return 0 2>/dev/null || exit 0
|
||||
fi
|
||||
|
||||
_before=$(git rev-parse --short=12 HEAD 2>/dev/null || echo none)
|
||||
_before_ver=$(git describe --tags --exact-match 2>/dev/null \
|
||||
|| git describe --tags --abbrev=0 2>/dev/null \
|
||||
|| echo unknown)
|
||||
|
||||
case "$SUITE_UPDATE_MODE" in
|
||||
reset)
|
||||
git checkout -B "$SUITE_GIT_REF" "origin/$SUITE_GIT_REF" 2>/dev/null || git checkout -B "$SUITE_GIT_REF" FETCH_HEAD
|
||||
git reset --hard "origin/$SUITE_GIT_REF" 2>/dev/null || git reset --hard FETCH_HEAD
|
||||
keep|skip|off|none)
|
||||
echo "suite update skipped (SUITE_UPDATE_MODE=$SUITE_UPDATE_MODE) — keep local tree ${_before} (${_before_ver})"
|
||||
;;
|
||||
*)
|
||||
echo "fetch origin ($SUITE_GIT_REF) + tags …"
|
||||
if ! git fetch --tags --force --prune origin 2>&1; then
|
||||
_upd_die "git fetch failed — suite would stay at ${_before} (${_before_ver})"
|
||||
# fall through only if STRICT=0
|
||||
else
|
||||
case "$SUITE_UPDATE_MODE" in
|
||||
reset)
|
||||
if ! git checkout -B "$SUITE_GIT_REF" "origin/$SUITE_GIT_REF" 2>/dev/null \
|
||||
&& ! git checkout -B "$SUITE_GIT_REF" FETCH_HEAD; then
|
||||
_upd_die "git checkout $SUITE_GIT_REF failed"
|
||||
fi
|
||||
if ! git reset --hard "origin/$SUITE_GIT_REF" 2>/dev/null \
|
||||
&& ! git reset --hard FETCH_HEAD; then
|
||||
_upd_die "git reset --hard origin/$SUITE_GIT_REF failed"
|
||||
fi
|
||||
;;
|
||||
pull)
|
||||
if ! git pull --ff-only origin "$SUITE_GIT_REF" 2>&1; then
|
||||
_upd_die "git pull --ff-only failed"
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
if ! git pull --ff-only origin "$SUITE_GIT_REF" 2>&1; then
|
||||
_upd_die "git pull --ff-only failed (mode=$SUITE_UPDATE_MODE)"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
;;
|
||||
*) git pull --ff-only origin "$SUITE_GIT_REF" 2>&1 || true ;;
|
||||
esac
|
||||
|
||||
COMMIT=$(git rev-parse HEAD)
|
||||
COMMIT_SHORT=$(git rev-parse --short=12 HEAD)
|
||||
SOURCE_REPO_WEB="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-monitoring}"
|
||||
COMMIT_URL="${SOURCE_REPO_WEB}/src/commit/${COMMIT}"
|
||||
export COMMIT COMMIT_SHORT COMMIT_URL SOURCE_REPO_WEB
|
||||
|
||||
# Release tag for sticky bar (exact tag if HEAD is tagged, else nearest)
|
||||
SUITE_VERSION=""
|
||||
if command -v git >/dev/null 2>&1; then
|
||||
SUITE_VERSION=$(git describe --tags --exact-match 2>/dev/null || true)
|
||||
if [ -z "$SUITE_VERSION" ]; then
|
||||
SUITE_VERSION=$(git describe --tags --abbrev=0 2>/dev/null || true)
|
||||
fi
|
||||
fi
|
||||
if [ -z "$SUITE_VERSION" ] && [ -f "$SUITE_DIR/VERSION" ]; then
|
||||
SUITE_VERSION="v$(tr -d '[:space:]' <"$SUITE_DIR/VERSION" | sed 's/^v//')"
|
||||
fi
|
||||
[ -z "$SUITE_VERSION" ] && SUITE_VERSION="unknown"
|
||||
case "$SUITE_VERSION" in
|
||||
unknown|v*) ;;
|
||||
*) SUITE_VERSION="v${SUITE_VERSION}" ;;
|
||||
esac
|
||||
# Sticky version badge: prefer commit URL (exact tree of this run).
|
||||
# Tag URLs (/src/tag/vX.Y.Z) 404 when the tag is not yet pushed to Forgejo;
|
||||
# commit URLs work as soon as main (or the tree) is on the remote.
|
||||
SUITE_VERSION_URL=""
|
||||
if [ -n "$SOURCE_REPO_WEB" ] && [ -n "${COMMIT:-}" ] && [ "$COMMIT" != "unknown" ]; then
|
||||
SUITE_VERSION_URL="${SOURCE_REPO_WEB}/src/commit/${COMMIT}"
|
||||
elif [ -n "$SOURCE_REPO_WEB" ] && [ "$SUITE_VERSION" != "unknown" ]; then
|
||||
SUITE_VERSION_URL="${SOURCE_REPO_WEB}/src/tag/${SUITE_VERSION}"
|
||||
fi
|
||||
export SUITE_VERSION SUITE_VERSION_URL
|
||||
|
||||
if [ "$_before" != "$COMMIT_SHORT" ]; then
|
||||
echo " upgraded ${_before} (${_before_ver}) → ${COMMIT_SHORT} (${SUITE_VERSION})"
|
||||
else
|
||||
echo " already at ${COMMIT_SHORT} (${SUITE_VERSION})"
|
||||
fi
|
||||
echo " version=$SUITE_VERSION ${SUITE_VERSION_URL:+· $SUITE_VERSION_URL}"
|
||||
echo "suite @ $COMMIT_SHORT $COMMIT_URL"
|
||||
echo " dir=$SUITE_DIR"
|
||||
|
||||
# Sanity: must have suite entrypoint after update
|
||||
if [ ! -x "$SUITE_DIR/taler-monitoring.sh" ] && [ ! -f "$SUITE_DIR/taler-monitoring.sh" ]; then
|
||||
_upd_die "taler-monitoring.sh missing after update"
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
|
||||
ln -sfn "$SUITE_DIR" "$HOME/taler-monitoring" 2>/dev/null || true
|
||||
chmod +x taler-monitoring.sh check_*.sh host-agent/*.sh site-gen/*.sh site-gen/*.py 2>/dev/null || true
|
||||
# Drop bytecode so a stale site-gen never wins over the tree we just reset to.
|
||||
rm -rf "$SUITE_DIR/site-gen/__pycache__" 2>/dev/null || true
|
||||
# Refuse known-bad pre-1.15.8 generator (box-drawing JS regex → SyntaxWarning noise)
|
||||
if [ -f "$SUITE_DIR/site-gen/console_to_html.py" ] \
|
||||
&& grep -qE 'if \(/\^\[[╔╗╚╝║═]' "$SUITE_DIR/site-gen/console_to_html.py" 2>/dev/null; then
|
||||
_upd_die "site-gen/console_to_html.py still has pre-1.15.8 box-drawing JS regex — refuse stale generator"
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
return 0 2>/dev/null || true
|
||||
|
|
|
|||
251
i18n.sh
251
i18n.sh
|
|
@ -1,101 +1,202 @@
|
|||
#!/usr/bin/env bash
|
||||
# i18n.sh — language for taler-monitoring console + HTML chrome
|
||||
# i18n.sh — UI language + regional locale for taler-monitoring console + HTML
|
||||
#
|
||||
# TALER_MON_LANG=en|fr (default en)
|
||||
# Auto: domains *lefrancpaysan* → fr unless TALER_MON_LANG is set explicitly.
|
||||
# UI language (mon badges / sticky chrome):
|
||||
# TALER_MON_LANG=en|fr|de
|
||||
# TALER_MON_LANG_SET=1 → explicit (--lang); do not override.
|
||||
# Auto (SET≠1): TALER_DOMAIN_LANG from domains.conf `lang` column.
|
||||
# de is supported but never a stock profile default (no domains.conf row uses lang=de).
|
||||
#
|
||||
# Regional locale (l10n defaults, BCP 47 — not the same as UI lang):
|
||||
# TALER_MON_LOCALE / TALER_DOMAIN_LOCALE e.g. fr-CH, de-CH
|
||||
# From domains.conf `locale` column (fr-CH for FP, de-CH for others).
|
||||
# Aliases ch-FR / ch-DE accepted and normalized to fr-CH / de-CH.
|
||||
#
|
||||
# domains.conf is the source of truth for both lang and locale.
|
||||
|
||||
: "${TALER_MON_LANG:=}"
|
||||
: "${TALER_MON_LANG_SET:=0}"
|
||||
: "${TALER_DOMAIN_LANG:=}"
|
||||
: "${TALER_DOMAIN_LOCALE:=}"
|
||||
: "${TALER_MON_LOCALE:=}"
|
||||
|
||||
i18n_init() {
|
||||
if [ -n "${TALER_MON_LANG:-}" ]; then
|
||||
TALER_MON_LANG_SET=1
|
||||
fi
|
||||
if [ "${TALER_MON_LANG_SET}" != "1" ]; then
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
*lefrancpaysan*|*francpaysan*)
|
||||
TALER_MON_LANG=fr
|
||||
;;
|
||||
*)
|
||||
TALER_MON_LANG=en
|
||||
;;
|
||||
esac
|
||||
# Only TALER_MON_LANG_SET=1 is "explicit". Do not promote non-empty TALER_MON_LANG
|
||||
# to SET=1 — env files often ship LANG=en for the default GOA workstation.
|
||||
if [ "${TALER_MON_LANG_SET:-0}" != "1" ]; then
|
||||
if [ -n "${TALER_DOMAIN_LANG:-}" ]; then
|
||||
TALER_MON_LANG="$TALER_DOMAIN_LANG"
|
||||
else
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
*lefrancpaysan*|*francpaysan*)
|
||||
TALER_MON_LANG=fr
|
||||
;;
|
||||
*)
|
||||
if [ -z "${TALER_MON_LANG:-}" ]; then
|
||||
TALER_MON_LANG=en
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
fi
|
||||
case "${TALER_MON_LANG}" in
|
||||
fr|FR|fra|french) TALER_MON_LANG=fr ;;
|
||||
de|DE|deu|ger|german|deutsch) TALER_MON_LANG=de ;;
|
||||
*) TALER_MON_LANG=en ;;
|
||||
esac
|
||||
|
||||
# Regional locale from profile (or FP/other fallback)
|
||||
if [ -n "${TALER_DOMAIN_LOCALE:-}" ]; then
|
||||
TALER_MON_LOCALE="$TALER_DOMAIN_LOCALE"
|
||||
elif [ -z "${TALER_MON_LOCALE:-}" ]; then
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
*lefrancpaysan*|*francpaysan*) TALER_MON_LOCALE=fr-CH ;;
|
||||
*) TALER_MON_LOCALE=de-CH ;;
|
||||
esac
|
||||
fi
|
||||
# normalize aliases
|
||||
case "$(printf '%s' "$TALER_MON_LOCALE" | tr '[:upper:]' '[:lower:]' | tr '_' '-')" in
|
||||
fr-ch|ch-fr) TALER_MON_LOCALE=fr-CH ;;
|
||||
de-ch|ch-de) TALER_MON_LOCALE=de-CH ;;
|
||||
en-ch|ch-en) TALER_MON_LOCALE=en-CH ;;
|
||||
esac
|
||||
|
||||
export TALER_MON_LANG
|
||||
export TALER_MON_LANG_SET
|
||||
export TALER_DOMAIN_LANG
|
||||
export TALER_DOMAIN_LOCALE
|
||||
export TALER_MON_LOCALE
|
||||
}
|
||||
|
||||
i18n_tag() {
|
||||
local t="$1"
|
||||
if [ "${TALER_MON_LANG:-en}" != "fr" ]; then
|
||||
printf '%s' "$t"
|
||||
return 0
|
||||
fi
|
||||
case "$t" in
|
||||
OK) printf 'OK' ;;
|
||||
ERROR) printf 'ERREUR' ;;
|
||||
WARN) printf 'AVERT' ;;
|
||||
INFO) printf 'INFO' ;;
|
||||
BLOCKER) printf 'BLOCAGE' ;;
|
||||
BLOCK) printf 'BLOC' ;;
|
||||
SUMMARY) printf 'RESUME' ;;
|
||||
PROG|PROGRESS) printf 'PROG' ;;
|
||||
*) printf '%s' "$t" ;;
|
||||
case "${TALER_MON_LANG:-en}" in
|
||||
fr)
|
||||
case "$t" in
|
||||
OK) printf 'OK' ;;
|
||||
ERROR) printf 'ERREUR' ;;
|
||||
WARN) printf 'AVERT' ;;
|
||||
INFO) printf 'INFO' ;;
|
||||
BLOCKER) printf 'BLOCAGE' ;;
|
||||
BLOCK) printf 'BLOC' ;;
|
||||
SUMMARY) printf 'RESUME' ;;
|
||||
PROG|PROGRESS) printf 'PROG' ;;
|
||||
*) printf '%s' "$t" ;;
|
||||
esac
|
||||
;;
|
||||
de)
|
||||
case "$t" in
|
||||
OK) printf 'OK' ;;
|
||||
ERROR) printf 'FEHLER' ;;
|
||||
WARN) printf 'WARNUNG' ;;
|
||||
INFO) printf 'INFO' ;;
|
||||
BLOCKER) printf 'BLOCKER' ;;
|
||||
BLOCK) printf 'BLOCK' ;;
|
||||
SUMMARY) printf 'ZUSAMMENFASSUNG' ;;
|
||||
PROG|PROGRESS) printf 'PROG' ;;
|
||||
*) printf '%s' "$t" ;;
|
||||
esac
|
||||
;;
|
||||
*)
|
||||
printf '%s' "$t"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
i18n_text() {
|
||||
local s="$*"
|
||||
if [ "${TALER_MON_LANG:-en}" != "fr" ] || [ -z "$s" ]; then
|
||||
printf '%s' "$s"
|
||||
return 0
|
||||
fi
|
||||
case "$s" in
|
||||
"numbered checks this run:") printf 'controles numerotes de cette execution :' ;;
|
||||
"failed — see list above"|"failed - see list above") printf 'echec — voir la liste ci-dessus' ;;
|
||||
"BLOCKERS · pay/withdraw cannot finish") printf 'BLOCAGES · paiement/retrait impossible' ;;
|
||||
"BLOCKERS (pay/withdraw cannot finish)") printf 'BLOCAGES (paiement/retrait impossible)' ;;
|
||||
"ERRORS · failed checks") printf 'ERREURS · controles en echec' ;;
|
||||
"ERRORS (failed checks)") printf 'ERREURS (controles en echec)' ;;
|
||||
"pay/withdraw cannot finish") printf 'paiement/retrait impossible a terminer' ;;
|
||||
"SUMMARY") printf 'RESUME' ;;
|
||||
"totals:") printf 'totaux :' ;;
|
||||
"container") printf 'conteneur' ;;
|
||||
"disk") printf 'disque' ;;
|
||||
"package") printf 'paquet' ;;
|
||||
"version") printf 'version' ;;
|
||||
"reachability") printf 'accessibilite' ;;
|
||||
"server-header") printf 'en-tete Server' ;;
|
||||
"inventory") printf 'inventaire' ;;
|
||||
"flags") printf 'options' ;;
|
||||
"target domain") printf 'domaine cible' ;;
|
||||
"currency") printf 'devise' ;;
|
||||
"phases") printf 'phases' ;;
|
||||
"access") printf 'acces' ;;
|
||||
[ -z "$s" ] && { printf '%s' "$s"; return 0; }
|
||||
case "${TALER_MON_LANG:-en}" in
|
||||
fr)
|
||||
case "$s" in
|
||||
"numbered checks this run:") printf 'controles numerotes de cette execution :' ;;
|
||||
"numbered checks:") printf 'controles numerotes :' ;;
|
||||
"(global done/total)") printf '(fait/total global)' ;;
|
||||
"global checks:") printf 'controles globaux :' ;;
|
||||
"failed — see list above"|"failed - see list above") printf 'echec — voir la liste ci-dessus' ;;
|
||||
"BLOCKERS · pay/withdraw cannot finish") printf 'BLOCAGES · paiement/retrait impossible' ;;
|
||||
"BLOCKERS (pay/withdraw cannot finish)") printf 'BLOCAGES (paiement/retrait impossible)' ;;
|
||||
"ERRORS · failed checks") printf 'ERREURS · controles en echec' ;;
|
||||
"ERRORS (failed checks)") printf 'ERREURS (controles en echec)' ;;
|
||||
"pay/withdraw cannot finish") printf 'paiement/retrait impossible a terminer' ;;
|
||||
"SUMMARY") printf 'RESUME' ;;
|
||||
"totals:") printf 'totaux :' ;;
|
||||
"all clear") printf 'tout est clair' ;;
|
||||
"warnings only (no hard fail)") printf 'avertissements seulement (pas d'\''echec dur)' ;;
|
||||
"monitoring") printf 'surveillance' ;;
|
||||
"surface") printf 'surface' ;;
|
||||
"aptdeploy") printf 'aptdeploy' ;;
|
||||
"container") printf 'conteneur' ;;
|
||||
"disk") printf 'disque' ;;
|
||||
"package") printf 'paquet' ;;
|
||||
"version") printf 'version' ;;
|
||||
"reachability") printf 'accessibilite' ;;
|
||||
"server-header") printf 'en-tete Server' ;;
|
||||
"inventory") printf 'inventaire' ;;
|
||||
"flags") printf 'options' ;;
|
||||
"target domain") printf 'domaine cible' ;;
|
||||
"currency") printf 'devise' ;;
|
||||
"phases") printf 'phases' ;;
|
||||
"access") printf 'acces' ;;
|
||||
*)
|
||||
local out="$s"
|
||||
out=${out//shared library missing/bibliotheque partagee manquante}
|
||||
out=${out//not active/non actif}
|
||||
out=${out//does not resolve/ne resout pas}
|
||||
out=${out//from Server header/depuis en-tete Server}
|
||||
out=${out//public HTTPS/HTTPS public}
|
||||
out=${out//no SSH/sans SSH}
|
||||
out=${out//failed — see list above/echec — voir la liste ci-dessus}
|
||||
printf '%s' "$out"
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
de)
|
||||
case "$s" in
|
||||
"numbered checks this run:") printf 'Nummerierte Pruefungen dieser Ausfuehrung :' ;;
|
||||
"numbered checks:") printf 'Nummerierte Pruefungen :' ;;
|
||||
"(global done/total)") printf '(fertig/gesamt global)' ;;
|
||||
"global checks:") printf 'Globale Pruefungen :' ;;
|
||||
"failed — see list above"|"failed - see list above") printf 'fehlgeschlagen — siehe Liste oben' ;;
|
||||
"BLOCKERS · pay/withdraw cannot finish") printf 'BLOCKER · Zahlung/Abhebung nicht moeglich' ;;
|
||||
"BLOCKERS (pay/withdraw cannot finish)") printf 'BLOCKER (Zahlung/Abhebung nicht moeglich)' ;;
|
||||
"ERRORS · failed checks") printf 'FEHLER · fehlgeschlagene Pruefungen' ;;
|
||||
"ERRORS (failed checks)") printf 'FEHLER (fehlgeschlagene Pruefungen)' ;;
|
||||
"pay/withdraw cannot finish") printf 'Zahlung/Abhebung kann nicht beendet werden' ;;
|
||||
"SUMMARY") printf 'ZUSAMMENFASSUNG' ;;
|
||||
"totals:") printf 'Summen :' ;;
|
||||
"all clear") printf 'alles in Ordnung' ;;
|
||||
"warnings only (no hard fail)") printf 'nur Warnungen (kein harter Fehler)' ;;
|
||||
"monitoring") printf 'Ueberwachung' ;;
|
||||
"surface") printf 'Surface' ;;
|
||||
"aptdeploy") printf 'aptdeploy' ;;
|
||||
"container") printf 'Container' ;;
|
||||
"disk") printf 'Disk' ;;
|
||||
"package") printf 'Paket' ;;
|
||||
"version") printf 'Version' ;;
|
||||
"reachability") printf 'Erreichbarkeit' ;;
|
||||
"server-header") printf 'Server-Header' ;;
|
||||
"inventory") printf 'Inventar' ;;
|
||||
"flags") printf 'Flags' ;;
|
||||
"target domain") printf 'Zieldomaene' ;;
|
||||
"currency") printf 'Waehrung' ;;
|
||||
"phases") printf 'Phasen' ;;
|
||||
"access") printf 'Zugang' ;;
|
||||
*)
|
||||
local outd="$s"
|
||||
outd=${outd//shared library missing/gemeinsame Bibliothek fehlt}
|
||||
outd=${outd//not active/nicht aktiv}
|
||||
outd=${outd//does not resolve/loest nicht auf}
|
||||
outd=${outd//from Server header/aus Server-Header}
|
||||
outd=${outd//public HTTPS/oeffentliches HTTPS}
|
||||
outd=${outd//no SSH/ohne SSH}
|
||||
outd=${outd//failed — see list above/fehlgeschlagen — siehe Liste oben}
|
||||
printf '%s' "$outd"
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
*)
|
||||
local out="$s"
|
||||
out=${out//shared library missing/bibliotheque partagee manquante}
|
||||
out=${out//shared library error/erreur de bibliotheque partagee}
|
||||
out=${out//not active/non actif}
|
||||
out=${out//after_start/apres_demarrage}
|
||||
out=${out//catalogued but not reachable/catalogue mais inaccessible}
|
||||
out=${out//does not resolve/ne resout pas}
|
||||
out=${out//protocol confirmed/protocole confirme}
|
||||
out=${out//catalogued service OK/service catalogue OK}
|
||||
out=${out//from Server header/depuis en-tete Server}
|
||||
out=${out//OSV clean/OSV sans vulnerabilite}
|
||||
out=${out//actionable vuln/vuln. actionnable}
|
||||
out=${out//header version · not Debian pkg/version en-tete · pas paquet Debian}
|
||||
out=${out//RUN_TIMEOUT exceeded/RUN_TIMEOUT depasse}
|
||||
out=${out//public HTTPS/HTTPS public}
|
||||
out=${out//container status/etat du conteneur}
|
||||
out=${out//remote-only/uniquement distant}
|
||||
out=${out//no SSH/sans SSH}
|
||||
out=${out//failed — see list above/echec — voir la liste ci-dessus}
|
||||
printf '%s' "$out"
|
||||
printf '%s' "$s"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
|
|
|||
32
ladder/README.md
Normal file
32
ladder/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Amount ladder modules
|
||||
|
||||
**Entry:** `../check_amount_ladder.sh` via:
|
||||
|
||||
```bash
|
||||
./taler-monitoring.sh ladder # classic 23-rung withdraw + pay
|
||||
./taler-monitoring.sh max-ladder # max-search withdraw + pay
|
||||
./taler-monitoring.sh goa-ladder # legacy phase name = classic ladder
|
||||
```
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `../check_amount_ladder.sh` | Orchestrator: classic / max **withdraw**, then pay phase |
|
||||
| `lib_pay.sh` | **Pay** module: free-amount template, classic pay list, max **payable** ≤ available |
|
||||
| `extract_rpubs.py` | Force-select: scrape `reservePub` from wallet dumps |
|
||||
|
||||
There is **no** `check_goa_ladder.sh` — removed; use the phases above.
|
||||
|
||||
## Modes
|
||||
|
||||
- **classic** (default): 23-rung withdraw 0→mids→max-1→max, then matching pay plan (never overspend).
|
||||
- **max** (`max-ladder`): adaptive max withdraw, then adaptive max pay via public free template (hacktivism: `goa-free`).
|
||||
|
||||
## Env (pay)
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|----------|---------|---------|
|
||||
| `LADDER_PAY` | `1` | `0` = skip all pays |
|
||||
| `LADDER_USE_FREE_TEMPLATE` | `1` | public POST template with `{"amount":…}` |
|
||||
| `LADDER_FREE_TEMPLATE` | `goa-free` | template id (hacktivism) |
|
||||
| `LADDER_FREE_TEMPLATE_INSTANCE` | merchant instance | e.g. `goa-demo-cp4zqk` |
|
||||
| `LADDER_PAY_WAIT_AVAILABLE_S` | `90` | after withdraw, wait for spendable **available**>0; logs **withdraw board in start order** (OK/OK_BANK) with live `pendingIncoming`→`done` + timer (same wallet DB) |
|
||||
163
ladder/extract_rpubs.py
Executable file
163
ladder/extract_rpubs.py
Executable file
|
|
@ -0,0 +1,163 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Extract reserve_pub candidates from wallet accept + transactions dumps.
|
||||
|
||||
Used by check_amount_ladder.sh force-select. Kept as a file (not a bash
|
||||
heredoc) so shell refactors cannot turn Python ``continue`` into ``return``.
|
||||
|
||||
Usage:
|
||||
ladder_extract_rpubs.py WID AMT accept.out tx.json used-rpubs.txt
|
||||
Prints one reserve pub per line, preferred first.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if len(argv) < 6:
|
||||
print(
|
||||
"usage: ladder_extract_rpubs.py WID AMT accept.out tx.json used-rpubs.txt",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
wid = argv[1]
|
||||
amt = argv[2]
|
||||
paths = argv[3:5]
|
||||
used_path = argv[5]
|
||||
used: set[str] = set()
|
||||
if Path(used_path).is_file():
|
||||
used = {ln.strip() for ln in open(used_path) if ln.strip()}
|
||||
|
||||
amt_num = amt.split(":", 1)[-1] if amt else ""
|
||||
dec = json.JSONDecoder()
|
||||
|
||||
def walk_collect(o, bag, ctx=None):
|
||||
ctx = dict(ctx or {})
|
||||
if isinstance(o, dict):
|
||||
for k, v in o.items():
|
||||
if not isinstance(v, str):
|
||||
continue
|
||||
kl = str(k).lower().replace("-", "_")
|
||||
if kl in (
|
||||
"withdrawal_id",
|
||||
"withdraw_id",
|
||||
"wopid",
|
||||
"id",
|
||||
"transactionid",
|
||||
"transaction_id",
|
||||
):
|
||||
ctx["id"] = v
|
||||
elif kl in (
|
||||
"amount",
|
||||
"rawamount",
|
||||
"instructedamount",
|
||||
"amounteffective",
|
||||
"amountraw",
|
||||
"transferamount",
|
||||
):
|
||||
ctx["amount"] = v
|
||||
elif kl in (
|
||||
"taler_withdraw_uri",
|
||||
"talerwithdrawuri",
|
||||
"uri",
|
||||
"bankconfirmationurl",
|
||||
"confirmtransferurl",
|
||||
):
|
||||
ctx["uri"] = v
|
||||
for k, v in o.items():
|
||||
kl = str(k).lower().replace("-", "_")
|
||||
if kl in ("reserve_pub", "reservepub") and isinstance(v, str) and len(v) >= 40:
|
||||
bag.append((v, dict(ctx)))
|
||||
elif not isinstance(v, (str, int, float, bool)) and v is not None:
|
||||
walk_collect(v, bag, ctx)
|
||||
elif isinstance(o, list):
|
||||
for i in o:
|
||||
walk_collect(i, bag, ctx)
|
||||
|
||||
def score_ctx(pub, ctx, window=""):
|
||||
score = 0
|
||||
cid = str(ctx.get("id") or "")
|
||||
camt = str(ctx.get("amount") or "")
|
||||
curi = str(ctx.get("uri") or "")
|
||||
if wid and wid in curi:
|
||||
score += 120
|
||||
if wid and wid in cid:
|
||||
score += 100
|
||||
if wid and wid in window:
|
||||
score += 80
|
||||
if amt and (camt == amt or camt.endswith(amt_num)):
|
||||
score += 50
|
||||
if amt and amt in window:
|
||||
score += 30
|
||||
if pub in used:
|
||||
score -= 200
|
||||
return score
|
||||
|
||||
def iter_json_objects(text: str):
|
||||
i, n = 0, len(text)
|
||||
while i < n:
|
||||
if text[i] == "{":
|
||||
try:
|
||||
o, end = dec.raw_decode(text, i)
|
||||
yield o
|
||||
i = end
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
i += 1
|
||||
|
||||
raw_pubs: list[str] = []
|
||||
scored: list[tuple[int, str]] = []
|
||||
|
||||
for p in paths:
|
||||
try:
|
||||
t = open(p, errors="replace").read()
|
||||
except Exception:
|
||||
continue
|
||||
for o in iter_json_objects(t):
|
||||
bag = []
|
||||
walk_collect(o, bag)
|
||||
for pub, ctx in bag:
|
||||
raw_pubs.append(pub)
|
||||
scored.append((score_ctx(pub, ctx), pub))
|
||||
|
||||
rpub_pats = (
|
||||
r"\"reservePub\"\s*:\s*\"([A-Z0-9]{40,})\"",
|
||||
r"\"reserve_pub\"\s*:\s*\"([A-Z0-9]{40,})\"",
|
||||
r"reserve[_ ]?pub[\"\s:=]+([A-Z0-9]{40,})",
|
||||
)
|
||||
for p in paths:
|
||||
try:
|
||||
blob = open(p, errors="replace").read()
|
||||
except Exception:
|
||||
continue
|
||||
for pat in rpub_pats:
|
||||
for m in re.finditer(pat, blob, re.I):
|
||||
pub = m.group(1)
|
||||
raw_pubs.append(pub)
|
||||
window = blob[max(0, m.start() - 600) : m.end() + 600]
|
||||
scored.append((score_ctx(pub, {}, window) + 10, pub))
|
||||
|
||||
order: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for score, pub in sorted(scored, key=lambda x: x[0], reverse=True):
|
||||
if pub in seen or pub in used:
|
||||
continue
|
||||
seen.add(pub)
|
||||
order.append(pub)
|
||||
for pub in reversed(raw_pubs):
|
||||
if pub in seen or pub in used:
|
||||
continue
|
||||
seen.add(pub)
|
||||
order.append(pub)
|
||||
for pub in order:
|
||||
print(pub)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
559
ladder/lib_pay.sh
Normal file
559
ladder/lib_pay.sh
Normal file
|
|
@ -0,0 +1,559 @@
|
|||
# ladder/lib_pay.sh — payment side of amount ladder (sourced by check_amount_ladder.sh)
|
||||
# Expects: lib.sh + metrics.sh already sourced; CUR BANK MER INST EX SCRATCH WDB
|
||||
# wcli wallet_avail format_amount_alt ladder_over now_ms elapsed_ms
|
||||
# LADDER_* env, AUTH optional, PAY_TSV PAY_OK_N PAY_FAIL_N
|
||||
#
|
||||
# Public free-amount template (hacktivism): goa-free @ goa-demo-cp4zqk
|
||||
# Never spends more than wallet available.
|
||||
|
||||
: "${LADDER_FREE_TEMPLATE:=goa-free}"
|
||||
: "${LADDER_FREE_TEMPLATE_INSTANCE:=}"
|
||||
: "${LADDER_USE_FREE_TEMPLATE:=1}"
|
||||
: "${LADDER_PAY_WAIT_AVAILABLE_S:=180}"
|
||||
: "${LADDER_PAY_SETTLE_ROUNDS:=6}"
|
||||
|
||||
MAX_PAY_BEST_AMT="${MAX_PAY_BEST_AMT:-}"
|
||||
MAX_PAY_LO="${MAX_PAY_LO:-0}"
|
||||
MAX_PAY_HI="${MAX_PAY_HI:-0}"
|
||||
|
||||
ladder_pay_resolve_merchant() {
|
||||
# Sets AUTH (may be empty if free template only)
|
||||
MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}"
|
||||
if [ -z "$MPW" ]; then
|
||||
MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true)
|
||||
fi
|
||||
if [ -z "$MPW" ]; then
|
||||
MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true)
|
||||
fi
|
||||
if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then
|
||||
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
|
||||
MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt")
|
||||
fi
|
||||
fi
|
||||
AUTH=""
|
||||
if [ -n "$MPW" ]; then
|
||||
case "$MPW" in
|
||||
secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;;
|
||||
*) AUTH="Authorization: Bearer secret-token:${MPW}" ;;
|
||||
esac
|
||||
fi
|
||||
: "${LADDER_FREE_TEMPLATE_INSTANCE:=${INST}}"
|
||||
}
|
||||
|
||||
# Create order for PAMT → OID OTOK PAYURI. Prefers public free template.
|
||||
ladder_create_order() {
|
||||
local pamt="$1" ptag="$2"
|
||||
local inst mh body tpl_https sum_json
|
||||
OID=""; OTOK=""; PAYURI=""
|
||||
inst="${LADDER_FREE_TEMPLATE_INSTANCE:-$INST}"
|
||||
mh=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"${MER}"'").hostname or "taler.hacktivism.ch")' 2>/dev/null || echo "taler.hacktivism.ch")
|
||||
|
||||
if [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; then
|
||||
tpl_https="${MER}/instances/${inst}/templates/${LADDER_FREE_TEMPLATE}"
|
||||
body=$(python3 -c 'import json,sys; print(json.dumps({"amount":sys.argv[1]}))' "$pamt" 2>/dev/null || printf '{"amount":"%s"}' "$pamt")
|
||||
curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$body" \
|
||||
"$tpl_https" 2>"$SCRATCH/ord-$ptag.err" || true
|
||||
OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
|
||||
try: print(json.loads(t).get("order_id") or "")
|
||||
except Exception:
|
||||
m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
|
||||
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
|
||||
OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
|
||||
try: print(json.loads(t).get("token") or "")
|
||||
except Exception:
|
||||
m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
|
||||
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
|
||||
if [ -n "$OID" ] && [ -n "$OTOK" ]; then
|
||||
PAYURI="taler://pay/${mh}/instances/${inst}/${OID}/?c=${OTOK}"
|
||||
PAYURI=$(normalize_taler_uri "$PAYURI")
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
[ -n "${AUTH:-}" ] || return 1
|
||||
sum_json=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "ladder pay ${pamt}" 2>/dev/null || echo '"ladder pay"')
|
||||
curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \
|
||||
-H "$AUTH" -H 'Content-Type: application/json' \
|
||||
-d "{\"order\":{\"summary\":${sum_json},\"amount\":\"${pamt}\",\"fulfillment_message\":\"ok\"},\"create_token\":true}" \
|
||||
"${MER}/instances/${INST}/private/orders" 2>"$SCRATCH/ord-$ptag.err" || true
|
||||
OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
|
||||
try: print(json.loads(t).get("order_id") or "")
|
||||
except Exception:
|
||||
m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
|
||||
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
|
||||
OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
|
||||
try: print(json.loads(t).get("token") or "")
|
||||
except Exception:
|
||||
m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
|
||||
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
|
||||
[ -n "$OID" ] || return 1
|
||||
curl -skS -m 12 -o "$SCRATCH/ord-det-$ptag.json" -H "$AUTH" \
|
||||
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
|
||||
PAYURI=$(python3 -c 'import json,sys
|
||||
try: print(json.load(open(sys.argv[1])).get("taler_pay_uri") or "")
|
||||
except Exception: print("")
|
||||
' "$SCRATCH/ord-det-$ptag.json" 2>/dev/null || true)
|
||||
if [ -z "$PAYURI" ] && [ -n "$OTOK" ]; then
|
||||
PAYURI="taler://pay/${mh}/instances/${INST}/${OID}/?c=${OTOK}"
|
||||
fi
|
||||
PAYURI=$(normalize_taler_uri "$PAYURI")
|
||||
[ -n "$PAYURI" ]
|
||||
}
|
||||
|
||||
# One pay attempt; never overspends. Always soft (no hard ladder stop).
|
||||
ladder_pay_attempt() {
|
||||
local pamt="$1" prange="$2" prung="$3"
|
||||
local ptag pnum bal t_pay t0 ms_order ms_handle ms_psettle after settled r pstatus pnote
|
||||
ptag=$(printf '%s' "$pamt" | tr '.:' '__')
|
||||
pnum=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$pamt")
|
||||
t_pay=$(now_ms)
|
||||
ms_order=0; ms_handle=0; ms_psettle=0
|
||||
pnote=""; pstatus="FAIL"; OID="-"
|
||||
|
||||
bal=$(wallet_avail)
|
||||
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[1])>=Decimal(sys.argv[2]) else 1)' "$bal" "$pnum" 2>/dev/null; then
|
||||
pstatus="SKIP_BALANCE"
|
||||
pnote="skip: would spend more than available avail=${CUR}:${bal} need=${pamt}"
|
||||
warn pay "pay $pamt skipped (balance)" \
|
||||
"problem: never overspend. $(format_amount_alt "${CUR}:${bal}") free vs $(format_amount_alt "$pamt")"
|
||||
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t0\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV"
|
||||
return 0
|
||||
fi
|
||||
|
||||
t0=$(now_ms)
|
||||
if ! ladder_create_order "$pamt" "$ptag"; then
|
||||
ms_order=$(elapsed_ms "$t0")
|
||||
pstatus="FAIL_ORDER"
|
||||
pnote="order create failed $(head -c 100 "$SCRATCH/ord-$ptag.json" 2>/dev/null | tr '\n\"' ' ')"
|
||||
warn pay "order $pamt" "$pnote"
|
||||
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV"
|
||||
return 0
|
||||
fi
|
||||
ms_order=$(elapsed_ms "$t0")
|
||||
ok "order $OID ($pamt) ${ms_order}ms · $(format_amount_alt "$pamt")"
|
||||
|
||||
t0=$(now_ms)
|
||||
if ! wcli handle-uri --yes "$PAYURI" >"$SCRATCH/pay-$ptag.out" 2>&1; then
|
||||
ms_handle=$(elapsed_ms "$t0")
|
||||
pstatus="FAIL_HANDLE"
|
||||
pnote="handle-uri failed $(tail -c 100 "$SCRATCH/pay-$ptag.out" | tr '\n\"' ' ')"
|
||||
warn pay "handle $pamt" "$pnote"
|
||||
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
|
||||
return 0
|
||||
fi
|
||||
ms_handle=$(elapsed_ms "$t0")
|
||||
ok "handle-uri $pamt ${ms_handle}ms"
|
||||
|
||||
t0=$(now_ms)
|
||||
settled=0
|
||||
r=0
|
||||
while [ "$r" -lt "${LADDER_PAY_SETTLE_ROUNDS}" ]; do
|
||||
r=$((r + 1))
|
||||
wcli transactions >"$SCRATCH/tx-pay-$ptag.out" 2>&1 || true
|
||||
if [ -n "${AUTH:-}" ]; then
|
||||
curl -skS -m 8 -o "$SCRATCH/ord-paid-$ptag.json" -H "$AUTH" \
|
||||
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
|
||||
fi
|
||||
if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-pay-$ptag.out" 2>/dev/null \
|
||||
|| python3 -c 'import json,sys
|
||||
try:
|
||||
d=json.load(open(sys.argv[1]))
|
||||
sys.exit(0 if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid" else 1)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
' "$SCRATCH/ord-paid-$ptag.json" 2>/dev/null; then
|
||||
settled=1
|
||||
break
|
||||
fi
|
||||
sleep 0.5
|
||||
done
|
||||
ms_psettle=$(elapsed_ms "$t0")
|
||||
after=$(wallet_avail)
|
||||
if [ "$settled" = "1" ]; then
|
||||
pstatus="OK"
|
||||
pnote="avail=${CUR}:${after}"
|
||||
ok "pay settled $pamt · $(format_amount_alt "$pamt") → bal ${CUR}:${after}"
|
||||
PAY_OK_N=$((PAY_OK_N + 1))
|
||||
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
|
||||
metrics_report_coins "after-pay-${ptag}" || true
|
||||
metrics_record_flow spent "$pamt" || true
|
||||
else
|
||||
pstatus="FAIL_SETTLE"
|
||||
pnote="not settled order=$OID avail=${CUR}:${after}"
|
||||
warn pay "settle $pamt" "$pnote"
|
||||
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# Print withdraw board: OK/OK_BANK rungs in *start order* + wallet tx state.
|
||||
# Uses TSV (ladder order) + transactions dump. Call after refreshing tx dump.
|
||||
# Args: left_secs available_num pendingIncoming_num
|
||||
ladder_print_wd_board() {
|
||||
local left="${1:-?}" bal="${2:-0}" pend="${3:-0}"
|
||||
local tsv="${TSV:-}" txf="${SCRATCH:-/tmp}/tx-wait-pay.out"
|
||||
local board
|
||||
board=$(
|
||||
python3 - "$tsv" "$txf" "${CUR:-GOA}" "$left" "$bal" "$pend" <<'PY' 2>/dev/null || true
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
|
||||
tsv, txf, cur, left, bal, pend = sys.argv[1:7]
|
||||
dec = json.JSONDecoder()
|
||||
|
||||
def num(a: str) -> str:
|
||||
a = (a or "").strip()
|
||||
if ":" in a:
|
||||
return a.split(":", 1)[1]
|
||||
return a
|
||||
|
||||
def iter_json(text: str):
|
||||
i, n = 0, len(text)
|
||||
while i < n:
|
||||
if text[i] == "{":
|
||||
try:
|
||||
o, end = dec.raw_decode(text, i)
|
||||
yield o
|
||||
i = end
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
i += 1
|
||||
|
||||
# TSV rows in start order (OK / OK_BANK only — successful bank path)
|
||||
rows = []
|
||||
if tsv and Path(tsv).is_file():
|
||||
for ln in open(tsv, errors="replace"):
|
||||
if not ln.strip() or ln.startswith("rung"):
|
||||
continue
|
||||
p = ln.rstrip("\n").split("\t")
|
||||
if len(p) < 10:
|
||||
continue
|
||||
status = p[3]
|
||||
if status not in ("OK", "OK_BANK"):
|
||||
continue
|
||||
rows.append({
|
||||
"rung": p[0],
|
||||
"amt": p[2],
|
||||
"bank": status,
|
||||
"wid": (p[9] if len(p) > 9 else "")[:12],
|
||||
})
|
||||
|
||||
# Wallet withdrawal txs (chronological if timestamp present)
|
||||
wds = []
|
||||
if txf and Path(txf).is_file():
|
||||
try:
|
||||
text = open(txf, errors="replace").read()
|
||||
except Exception:
|
||||
text = ""
|
||||
for o in iter_json(text):
|
||||
txs = o.get("transactions") if isinstance(o, dict) else None
|
||||
if not isinstance(txs, list):
|
||||
continue
|
||||
for t in txs:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
typ = str(t.get("type") or t.get("Type") or "").lower()
|
||||
if "withdraw" not in typ:
|
||||
continue
|
||||
amt = t.get("amountRaw") or t.get("amountEffective") or t.get("amount") or ""
|
||||
st = (
|
||||
t.get("txState")
|
||||
or t.get("status")
|
||||
or t.get("pendingMajor")
|
||||
or t.get("txMajorState")
|
||||
or ""
|
||||
)
|
||||
st = str(st).lower()
|
||||
det = t.get("withdrawalDetails") or t.get("withdrawal_details") or {}
|
||||
if not isinstance(det, dict):
|
||||
det = {}
|
||||
conf = det.get("confirmed")
|
||||
if conf is True and not st:
|
||||
st = "done"
|
||||
elif conf is False and not st:
|
||||
st = "pending"
|
||||
# pending* / dialog / kyc → still incoming; done/abort final
|
||||
if any(x in st for x in ("done", "abort", "fail", "delete")):
|
||||
phase = "done" if "done" in st else st
|
||||
elif st in ("", "none", "null"):
|
||||
phase = "unknown"
|
||||
else:
|
||||
phase = "pendingIncoming"
|
||||
ts = 0
|
||||
for k in ("timestamp", "timestamp_ms", "age"):
|
||||
v = t.get(k)
|
||||
if isinstance(v, dict):
|
||||
v = v.get("t_s") or v.get("t_ms") or v.get("t_usec")
|
||||
try:
|
||||
ts = int(v or 0)
|
||||
except Exception:
|
||||
ts = 0
|
||||
if ts:
|
||||
break
|
||||
wds.append({"amt": str(amt), "n": num(str(amt)), "phase": phase, "st": st or "?", "ts": ts})
|
||||
|
||||
wds.sort(key=lambda x: x["ts"])
|
||||
used = [False] * len(wds)
|
||||
|
||||
lines = [
|
||||
f"withdraw board (start order) · timer {left}s left · available={cur}:{bal} · pendingIncoming={cur}:{pend}"
|
||||
]
|
||||
if not rows:
|
||||
lines.append(" (no OK/OK_BANK rows in TSV yet)")
|
||||
else:
|
||||
for i, r in enumerate(rows, 1):
|
||||
n = num(r["amt"])
|
||||
match = None
|
||||
for j, w in enumerate(wds):
|
||||
if used[j]:
|
||||
continue
|
||||
if w["n"] == n or w["amt"] == r["amt"] or w["amt"].endswith(":" + n):
|
||||
used[j] = True
|
||||
match = w
|
||||
break
|
||||
if match is None:
|
||||
wlabel = "wallet=? (no tx match)"
|
||||
elif match["phase"] == "pendingIncoming":
|
||||
wlabel = f"wallet=pendingIncoming ({match['st']})"
|
||||
elif match["phase"] == "done":
|
||||
wlabel = f"wallet=done ({match['st']})"
|
||||
else:
|
||||
wlabel = f"wallet={match['phase']} ({match['st']})"
|
||||
wid = r["wid"] or "-"
|
||||
lines.append(
|
||||
f" #{i:<3} rung={r['rung']:<4} {r['amt']:<22} bank={r['bank']:<8} {wlabel} wid={wid}"
|
||||
)
|
||||
pending_n = sum(1 for w in wds if w["phase"] == "pendingIncoming")
|
||||
done_n = sum(1 for w in wds if w["phase"] == "done")
|
||||
lines.append(
|
||||
f" summary: tsv_ok={len(rows)} · wallet_wd={len(wds)} (pendingIncoming={pending_n} done={done_n})"
|
||||
)
|
||||
|
||||
print("\n".join(lines))
|
||||
PY
|
||||
)
|
||||
if [ -n "$board" ]; then
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
[ -n "$line" ] && info pay "$line"
|
||||
done <<<"$board"
|
||||
else
|
||||
info pay "timer ${left}s left · available=${CUR}:${bal} · pendingIncoming=${CUR}:${pend} · same wallet (board n/a)"
|
||||
fi
|
||||
}
|
||||
|
||||
# Wait for spendable coins on the *same* wallet DB as withdraw (WDB).
|
||||
# After OK_BANK, available is often still 0 while pendingIncoming > 0 — pay needs available.
|
||||
# Shows withdraw board in start order + countdown (never looks hung).
|
||||
ladder_wait_available() {
|
||||
local wait_s="${1:-$LADDER_PAY_WAIT_AVAILABLE_S}"
|
||||
local end now left bal pend tick=0
|
||||
end=$(( $(date +%s) + wait_s ))
|
||||
bal=$(wallet_avail)
|
||||
pend=$(wallet_pending_in 2>/dev/null || echo "0")
|
||||
if python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$bal" 2>/dev/null; then
|
||||
printf '%s' "$bal"
|
||||
return 0
|
||||
fi
|
||||
info pay "same wallet as withdraw (db=$(basename "${WDB:-wallet}")) · available=${CUR}:${bal} · pendingIncoming=${CUR}:${pend}"
|
||||
info pay "waiting up to ${wait_s}s for spendable available>0 — board lists OK/OK_BANK withdraws in start order"
|
||||
# initial board before loop
|
||||
wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true
|
||||
ladder_print_wd_board "$wait_s" "$bal" "$pend"
|
||||
while python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=0 else 1)' "$bal" 2>/dev/null; do
|
||||
now=$(date +%s)
|
||||
left=$(( end - now ))
|
||||
if [ "$left" -le 0 ]; then
|
||||
wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true
|
||||
ladder_print_wd_board "0" "$bal" "$pend"
|
||||
info pay "timer 0s left — stop waiting (available still ${CUR}:0)"
|
||||
break
|
||||
fi
|
||||
# nudge wallet (same DB); do not run-until-done (can hang)
|
||||
wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true
|
||||
if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then
|
||||
local _to
|
||||
_to=$(command -v gtimeout 2>/dev/null || command -v timeout)
|
||||
"$_to" 8 wcli advanced run-pending >"$SCRATCH/run-pending.out" 2>&1 || true
|
||||
fi
|
||||
# progress board every ~5s (tick every 2s → every 3rd)
|
||||
if [ $((tick % 3)) -eq 0 ]; then
|
||||
ladder_print_wd_board "$left" "$bal" "$pend"
|
||||
fi
|
||||
sleep 2
|
||||
tick=$((tick + 1))
|
||||
bal=$(wallet_avail)
|
||||
pend=$(wallet_pending_in 2>/dev/null || echo "0")
|
||||
done
|
||||
if python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$bal" 2>/dev/null; then
|
||||
wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true
|
||||
ladder_print_wd_board "ready" "$bal" "$pend"
|
||||
ok pay "spendable coins ready available=${CUR}:${bal} · $(format_amount_alt "${CUR}:${bal}") (same wallet)"
|
||||
else
|
||||
warn pay "no spendable coins after ${wait_s}s" \
|
||||
"problem: available still ${CUR}:0 (pendingIncoming=${CUR}:${pend}). Board above shows which withdraws still pendingIncoming. Pay will SKIP_BALANCE / skip max-pay until wirewatch+wallet finish. Same wallet as withdraw."
|
||||
fi
|
||||
printf '%s' "$bal"
|
||||
}
|
||||
|
||||
# Classic planned pays (PAY_LIST)
|
||||
ladder_run_pay_classic() {
|
||||
# shellcheck disable=SC2086
|
||||
set -- $PAY_LIST
|
||||
local pay_n=$# prung=0 PAMT prange PNUM
|
||||
if [ "$pay_n" -eq 0 ]; then
|
||||
info pay "empty pay plan — skip"
|
||||
return 0
|
||||
fi
|
||||
info "pay rungs" "$pay_n · $*"
|
||||
for PAMT in "$@"; do
|
||||
prung=$((prung + 1))
|
||||
ladder_over && break
|
||||
if [ "$prung" -eq 1 ]; then
|
||||
prange="pin:0"
|
||||
elif [ "$prung" -eq "$pay_n" ]; then
|
||||
prange="pin:max"
|
||||
elif [ "$prung" -eq $((pay_n - 1)) ] && [ "$pay_n" -ge 3 ]; then
|
||||
prange="pin:max-1"
|
||||
else
|
||||
prange="random"
|
||||
fi
|
||||
section "ladder · pay rung $prung $PAMT · $(format_amount_alt "$PAMT")"
|
||||
PNUM=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$PAMT")
|
||||
if python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])==0 else 1)' "$PNUM" 2>/dev/null; then
|
||||
echo -e "${prung}\t${prange}\t${PAMT}\tZERO_SKIP\t0\t0\t0\t0\t-\tzero" >>"$PAY_TSV"
|
||||
PAY_OK_N=$((PAY_OK_N + 1))
|
||||
continue
|
||||
fi
|
||||
ladder_pay_attempt "$PAMT" "$prange" "$prung"
|
||||
done
|
||||
}
|
||||
|
||||
# Max-search: highest payable ≤ available (uses max_search_next from withdraw module)
|
||||
ladder_run_pay_max_search() {
|
||||
local bal prung _next _anum prange _act _last _slo _shi
|
||||
section "ladder · phase B · max-search pay (highest payable ≤ available)"
|
||||
bal=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S")
|
||||
MAX_PAY_LO="0"
|
||||
MAX_PAY_HI="$bal"
|
||||
MAX_PAY_BEST_AMT=""
|
||||
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$MAX_PAY_HI" 2>/dev/null; then
|
||||
warn pay "max-search pay skipped" \
|
||||
"problem: wallet available is ${CUR}:0 — need spendable coins (pendingIncoming is not enough)"
|
||||
return 0
|
||||
fi
|
||||
info "max-search pay" "hi=available $(format_amount_alt "${CUR}:${MAX_PAY_HI}") · template=${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE}"
|
||||
info "pay budget" "never order more than $(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
|
||||
prung=0
|
||||
_last=""
|
||||
while [ "${LADDER_MAX_PROBES}" = "0" ] || [ "$prung" -lt "${LADDER_MAX_PROBES}" ]; do
|
||||
ladder_over && break
|
||||
# reuse withdraw amount picker with pay bounds
|
||||
_slo=${MAX_LO:-0}; _shi=${MAX_HI:-0}
|
||||
MAX_LO=$MAX_PAY_LO
|
||||
MAX_HI=$MAX_PAY_HI
|
||||
if ! type max_search_next >/dev/null 2>&1; then
|
||||
warn pay "max_search_next missing — cannot run pay max-search"
|
||||
MAX_LO=$_slo; MAX_HI=$_shi
|
||||
return 0
|
||||
fi
|
||||
_next=$(max_search_next)
|
||||
MAX_LO=$_slo; MAX_HI=$_shi
|
||||
_anum=$(printf '%s' "$_next" | cut -d'|' -f1)
|
||||
prange=$(printf '%s' "$_next" | cut -d'|' -f2 | sed 's/^max:/pay:/')
|
||||
_act=$(printf '%s' "$_next" | cut -d'|' -f3)
|
||||
if [ "$_act" = "done" ]; then
|
||||
info "max-search pay" "converged best=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
|
||||
break
|
||||
fi
|
||||
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=Decimal(sys.argv[2]) else 1)' "$_anum" "$MAX_PAY_HI" 2>/dev/null; then
|
||||
_anum=$MAX_PAY_HI
|
||||
fi
|
||||
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$_anum" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
PAMT="${CUR}:${_anum}"
|
||||
if [ -n "$_last" ] && [ "$PAMT" = "$_last" ]; then
|
||||
MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); print(max(Decimal(0), a-(1 if a>=1 else Decimal("0.000001"))))')
|
||||
continue
|
||||
fi
|
||||
_last=$PAMT
|
||||
prung=$((prung + 1))
|
||||
section "ladder · max pay probe $prung $PAMT · $(format_amount_alt "$PAMT") [lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")]"
|
||||
ladder_pay_attempt "$PAMT" "$prange" "$prung"
|
||||
_last_st=$(tail -1 "$PAY_TSV" 2>/dev/null | cut -f4)
|
||||
case "$_last_st" in
|
||||
OK)
|
||||
MAX_PAY_LO=$_anum
|
||||
MAX_PAY_BEST_AMT=$PAMT
|
||||
info "max-search pay" "success bound lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}")"
|
||||
;;
|
||||
SKIP_BALANCE|FAIL_ORDER|FAIL_HANDLE|FAIL_SETTLE)
|
||||
MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); h=Decimal("'"$MAX_PAY_HI"'"); print(min(a,h))')
|
||||
info "max-search pay" "too-high/fail hi:=$(format_amount_alt "${CUR}:${MAX_PAY_HI}") status=${_last_st}"
|
||||
;;
|
||||
esac
|
||||
if python3 -c 'from decimal import Decimal; import sys; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[2])<=Decimal(sys.argv[1]) else 1)' "$MAX_PAY_LO" "$MAX_PAY_HI" 2>/dev/null; then
|
||||
info "max-search pay" "bounds crossed — stop"
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [ -n "${MAX_PAY_BEST_AMT:-}" ]; then
|
||||
ok "max-search best payable $(format_amount_alt "$MAX_PAY_BEST_AMT") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
|
||||
else
|
||||
warn pay "max-search no successful pay" "available was ${CUR}:$(wallet_avail)"
|
||||
fi
|
||||
{
|
||||
echo "best_pay=${MAX_PAY_BEST_AMT:-}"
|
||||
echo "best_pay_alt=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}" 2>/dev/null || true)"
|
||||
echo "pay_lo=${MAX_PAY_LO}"
|
||||
echo "pay_hi=${MAX_PAY_HI}"
|
||||
} >>"${SCRATCH}/ladder-plan.txt" 2>/dev/null || true
|
||||
printf '%s\n' "${MAX_PAY_BEST_AMT:-}" >"${SCRATCH}/ladder-pay-plan.txt" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Entry: glue from check_amount_ladder.sh after withdraw phase
|
||||
ladder_run_pay_phase() {
|
||||
PAY_OK_N=0
|
||||
PAY_FAIL_N=0
|
||||
if [ "${LADDER_PAY}" != "1" ]; then
|
||||
info pay "LADDER_PAY=0 — pay phase skipped"
|
||||
return 0
|
||||
fi
|
||||
if [ "${FAIL_N_L:-0}" -gt 0 ]; then
|
||||
warn pay "skipped pay ladder" "withdraw phase already failed"
|
||||
return 0
|
||||
fi
|
||||
|
||||
set_group pay
|
||||
section "ladder · phase B · pay"
|
||||
metrics_report_coins "before-pay-ladder" || true
|
||||
ladder_pay_resolve_merchant
|
||||
|
||||
if [ -z "${AUTH:-}" ] && ! { [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; }; then
|
||||
warn pay "no merchant token and free template off — skip pay"
|
||||
return 0
|
||||
fi
|
||||
if [ -n "${AUTH:-}" ]; then
|
||||
ok "merchant token" "instance ${INST}"
|
||||
else
|
||||
info pay "public free template ${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE} (no merchant token)"
|
||||
fi
|
||||
|
||||
# Same cumulative wallet as withdraw (WDB=wallet-main.sqlite3)
|
||||
wallet_prepare "main"
|
||||
info pay "using same wallet as withdraw · WDB=${WDB:-?}"
|
||||
local bal0
|
||||
bal0=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S")
|
||||
info "pay budget" "wallet available ${CUR}:${bal0} · $(format_amount_alt "${CUR}:${bal0}") — never order more than this"
|
||||
|
||||
if [ "${LADDER_MODE}" = "max" ]; then
|
||||
ladder_run_pay_max_search
|
||||
else
|
||||
ladder_run_pay_classic
|
||||
fi
|
||||
metrics_report_coins "after-pay-ladder" || true
|
||||
info "pay summary" "ok=$PAY_OK_N fail=$PAY_FAIL_N tsv=$PAY_TSV"
|
||||
}
|
||||
549
lib.sh
549
lib.sh
|
|
@ -1,5 +1,22 @@
|
|||
# shellcheck shell=bash
|
||||
# Shared helpers for taler-monitoring (laptop or koopa).
|
||||
#
|
||||
# Machine paths/SSH hosts: only from env (~/.config/taler-monitoring/env or
|
||||
# TALER_MONITORING_ENV=…), installed via taler-monitoring-env — never hardcode.
|
||||
|
||||
# Load host env once (before any := defaults)
|
||||
if [ "${_TALER_MON_ENV_LOADED:-0}" != "1" ]; then
|
||||
_mon_cfg="${TALER_MONITORING_ENV:-${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env}"
|
||||
if [ -f "$_mon_cfg" ]; then
|
||||
# shellcheck disable=SC1090
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
. "$_mon_cfg"
|
||||
set +a
|
||||
fi
|
||||
unset _mon_cfg
|
||||
export _TALER_MON_ENV_LOADED=1
|
||||
fi
|
||||
|
||||
# Default stack = GOA / hacktivism (overridden by TALER_DOMAIN / --domain)
|
||||
# i18n (en default; fr for FrancPaysan — see i18n.sh / TALER_MON_LANG)
|
||||
|
|
@ -22,16 +39,32 @@ fi
|
|||
: "${EXCHANGE_LOCAL:=http://127.0.0.1:9011}"
|
||||
: "${MERCHANT_LOCAL:=https://127.0.0.1:9010}"
|
||||
: "${LANDING_LOCAL:=http://127.0.0.1:9013}"
|
||||
: "${KOOPA_SSH:=koopa}"
|
||||
# When LAN Host "koopa" is unreachable, try WAN DNAT (see ~/.ssh/config Host koopa-external).
|
||||
: "${KOOPA_SSH_FALLBACKS:=koopa-external}"
|
||||
# Hostnames/paths: set via ~/.config/taler-monitoring/env (no hardcoded machine values).
|
||||
: "${KOOPA_SSH:=}"
|
||||
: "${KOOPA_SSH_FALLBACKS:=}"
|
||||
: "${INSIDE_SSH:=}"
|
||||
: "${FRANCPAYSAN_SSH:=}"
|
||||
: "${DEVTESTING_SSH:=}"
|
||||
: "${DEPLOY_SSH:=}"
|
||||
: "${FIRECUDA_SSH:=}"
|
||||
: "${HTML_OUT:=${HOME}/monitoring-sites-staging}"
|
||||
: "${DEPLOY_STAGING:=${HTML_OUT}}"
|
||||
: "${DEPLOY_WWW_ROOT:=}"
|
||||
: "${APPLY_MONITORING_LIVE:=}"
|
||||
: "${KOOPA_CADDY_DIR:=}"
|
||||
: "${FRANCPAYSAN_SECRETS:=}"
|
||||
: "${FRANCPAYSAN_REMOTE_SECRETS_ROOT:=}"
|
||||
: "${FRANCPAYSAN_REMOTE_BANK_ADMIN_SECRET:=}"
|
||||
: "${FRANCPAYSAN_REMOTE_BANK_EXPLORER_SECRET:=}"
|
||||
: "${FRANCPAYSAN_REMOTE_MERCHANT_TOKEN:=}"
|
||||
: "${WALLET_CLI:=}"
|
||||
: "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}"
|
||||
# Merchant portal /private API: instance ids must be lowercase or you get odd 401s.
|
||||
# (Display names may use capitals; path segment /instances/{id}/ must not.)
|
||||
MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower:]')
|
||||
: "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses ATM notes
|
||||
: "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses bank cash-note amounts
|
||||
: "${PAY_AMT:=GOA:0.01}"
|
||||
: "${CREDIT_AMT:=GOA:4700}" # covers ATM ladder 20+50+100+200+4200 (paivana)
|
||||
: "${CREDIT_AMT:=GOA:4700}" # covers bank withdraw ladder 20+50+100+200+4200 (paivana)
|
||||
: "${TIMEOUT:=12}"
|
||||
: "${E2E_TIMEOUT:=55}" # whole e2e budget; skip rest when exceeded (e2e raises as needed)
|
||||
: "${E2E_PAY_SECS:=22}" # dedicated seconds for pay handle-uri (avoid Alarm clock)
|
||||
|
|
@ -63,7 +96,7 @@ MERCHANT_PUBLIC=${MERCHANT_PUBLIC%/}
|
|||
# ---------------------------------------------------------------------------
|
||||
# Domain profiles → bank / exchange / merchant base URLs
|
||||
#
|
||||
# Single place to declare a stack: scripts/taler-monitoring/domains.conf
|
||||
# Single place to declare a stack: domains.conf (suite root)
|
||||
# (or TALER_DOMAINS_CONF). Each profile names the three public endpoints.
|
||||
#
|
||||
# CLI still wins after profile load:
|
||||
|
|
@ -133,13 +166,53 @@ set_taler_stack() {
|
|||
fi
|
||||
}
|
||||
|
||||
# Normalize domains.conf UI lang token → en|fr|de (empty if not a lang token).
|
||||
# Note: de is valid for --lang / explicit profile, but no stock profile defaults to de.
|
||||
_domain_lang_token() {
|
||||
case "${1:-}" in
|
||||
en|EN|eng|english) printf 'en' ;;
|
||||
fr|FR|fra|french) printf 'fr' ;;
|
||||
de|DE|deu|ger|german|deutsch) printf 'de' ;;
|
||||
*) printf '' ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Normalize regional locale → BCP 47 (fr-CH, de-CH, en-CH, …). Empty if unknown.
|
||||
# Accepts fr-CH, fr_CH, ch-FR / de-CH, ch-DE (user-friendly aliases).
|
||||
_domain_locale_token() {
|
||||
local t="${1:-}"
|
||||
t=$(printf '%s' "$t" | tr '[:upper:]' '[:lower:]' | tr '_' '-')
|
||||
case "$t" in
|
||||
fr-ch|ch-fr|fr_ch) printf 'fr-CH' ;;
|
||||
de-ch|ch-de|de_ch) printf 'de-CH' ;;
|
||||
en-ch|ch-en|en_ch) printf 'en-CH' ;;
|
||||
en-us|en-gb|fr-fr|de-de|it-ch|rm-ch)
|
||||
# pass through other xx-YY
|
||||
printf '%s' "$t" | awk -F- '{printf "%s-%s", tolower($1), toupper($2)}'
|
||||
;;
|
||||
*)
|
||||
# bare xx-YY pattern
|
||||
if printf '%s' "$t" | grep -qE '^[a-z]{2}-[a-z]{2}$'; then
|
||||
printf '%s' "$t" | awk -F- '{printf "%s-%s", tolower($1), toupper($2)}'
|
||||
else
|
||||
printf ''
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Load first matching profile from domains.conf.
|
||||
# Fields (whitespace-separated; # comments; blank lines ignored):
|
||||
# name bank exchange merchant currency local[0|1] landing[0|1] [canonical_domain]
|
||||
# name bank exchange merchant currency local landing lang locale [canonical]
|
||||
# Legacy:
|
||||
# … landing lang [canonical] → locale defaulted
|
||||
# … landing [canonical] → lang=en, locale defaulted
|
||||
# Sets TALER_DOMAIN_LANG + TALER_DOMAIN_LOCALE from the profile.
|
||||
# Returns 0 if found, 1 if not.
|
||||
load_domain_profile() {
|
||||
local want="$1" conf="${TALER_DOMAINS_CONF:-}"
|
||||
local line name bank exchange merchant currency local_stack landing canon
|
||||
local profile_lang profile_locale f8 f9 f10
|
||||
[ -n "$want" ] || return 1
|
||||
[ -n "$conf" ] && [ -f "$conf" ] || return 1
|
||||
|
||||
|
|
@ -156,12 +229,55 @@ load_domain_profile() {
|
|||
currency="$5"
|
||||
local_stack="${6:-0}"
|
||||
landing="${7:-}"
|
||||
canon="${8:-$name}"
|
||||
f8="${8:-}"
|
||||
f9="${9:-}"
|
||||
f10="${10:-}"
|
||||
canon="$name"
|
||||
profile_lang=""
|
||||
profile_locale=""
|
||||
# Backward compat: old 7th field was canonical domain (contains a dot)
|
||||
if [ -n "$landing" ] && [[ "$landing" == *.* && "$landing" != "0" && "$landing" != "1" ]]; then
|
||||
canon="$landing"
|
||||
landing=""
|
||||
f8="${7:-}"
|
||||
f9="${8:-}"
|
||||
f10="${9:-}"
|
||||
fi
|
||||
if [ -n "$f8" ]; then
|
||||
profile_lang=$(_domain_lang_token "$f8")
|
||||
if [ -n "$profile_lang" ]; then
|
||||
# modern: lang [locale] [canonical]
|
||||
profile_locale=$(_domain_locale_token "$f9")
|
||||
if [ -n "$profile_locale" ]; then
|
||||
canon="${f10:-$name}"
|
||||
else
|
||||
# lang + canonical (no locale column)
|
||||
if [ -n "$f9" ] && [ -z "$(_domain_lang_token "$f9")" ]; then
|
||||
canon="$f9"
|
||||
else
|
||||
canon="${f9:-$name}"
|
||||
fi
|
||||
profile_locale=$(_domain_locale_token "$f10")
|
||||
fi
|
||||
else
|
||||
# legacy: field 8 = canonical only
|
||||
canon="$f8"
|
||||
profile_lang=$(_domain_lang_token "$f9")
|
||||
profile_locale=$(_domain_locale_token "${f10:-$f9}")
|
||||
[ -z "$profile_lang" ] && profile_lang=en
|
||||
fi
|
||||
fi
|
||||
[ -z "$profile_lang" ] && profile_lang=en
|
||||
if [ -z "$profile_locale" ]; then
|
||||
# default regional locale: FP → fr-CH, else de-CH
|
||||
case "$name$canon$bank$merchant" in
|
||||
*lefrancpaysan*|*francpaysan*) profile_locale=fr-CH ;;
|
||||
*) profile_locale=de-CH ;;
|
||||
esac
|
||||
fi
|
||||
TALER_DOMAIN_LANG="$profile_lang"
|
||||
TALER_DOMAIN_LOCALE="$profile_locale"
|
||||
export TALER_DOMAIN_LANG TALER_DOMAIN_LOCALE
|
||||
set_taler_stack "$bank" "$exchange" "$merchant" "$currency" "$local_stack" "$landing" "$canon"
|
||||
return 0
|
||||
done <"$conf"
|
||||
|
|
@ -235,13 +351,13 @@ apply_taler_domain() {
|
|||
: "${E2E_TEMPLATE_MAP:=5=oeufs-6 4.5=pain-seigle 6=jus-pomme 8.5=fromage-chevre 12=miel-printemps 25=panier-legumes}"
|
||||
: "${E2E_PAY_VALUES:=5 8.5 6}"
|
||||
: "${E2E_WITHDRAW_VALUES:=20 50}"
|
||||
# Inside checks as low-priv stagepaysan (podman, no sudo) — not koopa
|
||||
# Inside checks as low-priv stagepaysan (podman, no sudo) — not koopa.
|
||||
# INSIDE_SSH / container names: set in host env (no hardcoded SSH host).
|
||||
INSIDE_PROFILE="${INSIDE_PROFILE:-stage-lfp}"
|
||||
INSIDE_SSH="${INSIDE_SSH:-francpaysan-stage-user}"
|
||||
INSIDE_BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}"
|
||||
INSIDE_EXCHANGE_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
|
||||
INSIDE_MERCHANT_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
|
||||
# Host pasta ports (127.0.0.1 on francpaysan-host)
|
||||
# Host pasta ports (127.0.0.1 on the FP host (INSIDE_SSH))
|
||||
INSIDE_BANK_PORT="${INSIDE_BANK_PORT:-9032}"
|
||||
INSIDE_EXCHANGE_PORT="${INSIDE_EXCHANGE_PORT:-9031}"
|
||||
INSIDE_MERCHANT_PORT="${INSIDE_MERCHANT_PORT:-9030}"
|
||||
|
|
@ -323,13 +439,24 @@ SSH_BASE_OPTS=(
|
|||
# Hard wall-clock timeout so ssh/curl never block the monitoring run forever.
|
||||
with_timeout() {
|
||||
local secs="$1"; shift
|
||||
local rc
|
||||
if command -v gtimeout >/dev/null 2>&1; then
|
||||
gtimeout --kill-after=2 "$secs" "$@"
|
||||
return $?
|
||||
fi
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout -k 2 "$secs" "$@" 2>/dev/null || timeout --kill-after=2 "$secs" "$@"
|
||||
return $?
|
||||
# Prefer GNU -k; if that flag form is unsupported, fall back once.
|
||||
# Do NOT use `cmd || cmd` on the child — a phase exit 1 would re-run it.
|
||||
set +e
|
||||
timeout -k 2 "$secs" "$@" 2>/dev/null
|
||||
rc=$?
|
||||
set -e
|
||||
# 125 = timeout(1) itself failed (bad options); 126/127 = not executable
|
||||
if [ "$rc" -eq 125 ] || [ "$rc" -eq 126 ] || [ "$rc" -eq 127 ]; then
|
||||
timeout --kill-after=2 "$secs" "$@"
|
||||
return $?
|
||||
fi
|
||||
return "$rc"
|
||||
fi
|
||||
# Portable: perl alarm + process group kill
|
||||
perl -e '
|
||||
|
|
@ -370,7 +497,7 @@ mon_ssh_ok() {
|
|||
ssh "${SSH_BASE_OPTS[@]}" "$host" true >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# Pick a working SSH host: KOOPA_SSH first, then KOOPA_SSH_FALLBACKS (koopa-external).
|
||||
# Pick a working SSH host: KOOPA_SSH first, then KOOPA_SSH_FALLBACKS (from env).
|
||||
# Sets KOOPA_SSH to the first host that answers. 0 = ok, 1 = none.
|
||||
KOOPA_SSH_RESOLVED=0
|
||||
resolve_koopa_ssh() {
|
||||
|
|
@ -483,7 +610,13 @@ TEST_GROUP=""
|
|||
TEST_N=0
|
||||
LAST_TID=""
|
||||
# Global / progress: may continue across check_*.sh processes via TALER_MON_STATE
|
||||
# GLOBAL_*_N = whole-run status totals (multi-phase summary at parent end)
|
||||
GLOBAL_N="${GLOBAL_N:-0}"
|
||||
GLOBAL_PASS_N="${GLOBAL_PASS_N:-0}"
|
||||
GLOBAL_FAIL_N="${GLOBAL_FAIL_N:-0}"
|
||||
GLOBAL_WARN_N="${GLOBAL_WARN_N:-0}"
|
||||
GLOBAL_INFO_N="${GLOBAL_INFO_N:-0}"
|
||||
GLOBAL_BLOCK_N="${GLOBAL_BLOCK_N:-0}"
|
||||
LAST_GLOBAL=""
|
||||
PROGRESS_DONE="${PROGRESS_DONE:-0}"
|
||||
PROGRESS_TOTAL="${PROGRESS_TOTAL:-0}"
|
||||
|
|
@ -493,14 +626,28 @@ _mon_state_load() {
|
|||
[ -n "${TALER_MON_STATE:-}" ] && [ -f "${TALER_MON_STATE}" ] || return 0
|
||||
# shellcheck disable=SC1090
|
||||
. "${TALER_MON_STATE}"
|
||||
: "${GLOBAL_N:=0}"
|
||||
: "${GLOBAL_PASS_N:=0}"
|
||||
: "${GLOBAL_FAIL_N:=0}"
|
||||
: "${GLOBAL_WARN_N:=0}"
|
||||
: "${GLOBAL_INFO_N:=0}"
|
||||
: "${GLOBAL_BLOCK_N:=0}"
|
||||
: "${PROGRESS_DONE:=0}"
|
||||
: "${PROGRESS_TOTAL:=0}"
|
||||
: "${PROGRESS_LAST_SHOWN:=0}"
|
||||
}
|
||||
_mon_state_save() {
|
||||
[ -n "${TALER_MON_STATE:-}" ] || return 0
|
||||
{
|
||||
printf 'GLOBAL_N=%s\n' "$GLOBAL_N"
|
||||
printf 'PROGRESS_DONE=%s\n' "$PROGRESS_DONE"
|
||||
printf 'PROGRESS_TOTAL=%s\n' "$PROGRESS_TOTAL"
|
||||
printf 'PROGRESS_LAST_SHOWN=%s\n' "$PROGRESS_LAST_SHOWN"
|
||||
printf 'GLOBAL_N=%s\n' "${GLOBAL_N:-0}"
|
||||
printf 'GLOBAL_PASS_N=%s\n' "${GLOBAL_PASS_N:-0}"
|
||||
printf 'GLOBAL_FAIL_N=%s\n' "${GLOBAL_FAIL_N:-0}"
|
||||
printf 'GLOBAL_WARN_N=%s\n' "${GLOBAL_WARN_N:-0}"
|
||||
printf 'GLOBAL_INFO_N=%s\n' "${GLOBAL_INFO_N:-0}"
|
||||
printf 'GLOBAL_BLOCK_N=%s\n' "${GLOBAL_BLOCK_N:-0}"
|
||||
printf 'PROGRESS_DONE=%s\n' "${PROGRESS_DONE:-0}"
|
||||
printf 'PROGRESS_TOTAL=%s\n' "${PROGRESS_TOTAL:-0}"
|
||||
printf 'PROGRESS_LAST_SHOWN=%s\n' "${PROGRESS_LAST_SHOWN:-0}"
|
||||
} >"${TALER_MON_STATE}"
|
||||
}
|
||||
_mon_state_load
|
||||
|
|
@ -531,7 +678,7 @@ set_area() {
|
|||
}
|
||||
# Start a logical sub-group. Short stable names:
|
||||
# exchange bank merchant perf stats landing paivana
|
||||
# prereq wallet atm settle pay shop dig load report
|
||||
# prereq wallet bankwd settle pay shop dig load report
|
||||
# ssh caddy outside inside compare withdraw
|
||||
# Counter continues if the same group is re-entered later in the area.
|
||||
# Prints a compact group chip so log + issue text map cleanly (www.bank-03).
|
||||
|
|
@ -563,8 +710,9 @@ set_progress_total() {
|
|||
# Optional: expected number of numbered checks in this run.
|
||||
# PROGRESS_TOTAL=0 → no percent, only "done=N".
|
||||
# Does not reset GLOBAL_N / PROGRESS_DONE if already mid-run (state file).
|
||||
# May shrink when parent re-fits after a short phase (total = done + remaining est),
|
||||
# but never below PROGRESS_DONE.
|
||||
local n="${1:-0}"
|
||||
# Never shrink below already completed work (re-estimate mid-run safe)
|
||||
if [ "${PROGRESS_DONE:-0}" -gt 0 ] && [ "$n" -gt 0 ] && [ "$n" -lt "$PROGRESS_DONE" ]; then
|
||||
n=$PROGRESS_DONE
|
||||
fi
|
||||
|
|
@ -579,20 +727,38 @@ add_progress_total() {
|
|||
PROGRESS_TOTAL=$((PROGRESS_TOTAL + n))
|
||||
_mon_state_save
|
||||
}
|
||||
# When estimate was short, grow total so we never print done>total (e.g. 224/205).
|
||||
# Keeps ~10% headroom so the bar is not stuck at 100% while checks continue.
|
||||
# When the global estimate was short, grow total just enough so done≤total.
|
||||
# Never shrink. Mid-run headroom is small and monotonic (stable N in done/N).
|
||||
_progress_rebalance() {
|
||||
[ "${PROGRESS_TOTAL:-0}" -gt 0 ] || return 0
|
||||
[ "${PROGRESS_DONE:-0}" -gt 0 ] || return 0
|
||||
if [ "$PROGRESS_DONE" -gt "$PROGRESS_TOTAL" ]; then
|
||||
local head=$((PROGRESS_DONE / 10))
|
||||
[ "$head" -lt 12 ] && head=12
|
||||
# + max(8, 5% of done) so we do not snap to 100% then jump again next check
|
||||
local head=$((PROGRESS_DONE / 20))
|
||||
[ "$head" -lt 8 ] && head=8
|
||||
PROGRESS_TOTAL=$((PROGRESS_DONE + head))
|
||||
_mon_state_save
|
||||
fi
|
||||
}
|
||||
# Snap total to actual done — only at whole-run end (not after each phase).
|
||||
progress_finish() {
|
||||
[ "${PROGRESS_OFF:-0}" = "1" ] && return 0
|
||||
[ "${PROGRESS_DONE:-0}" -gt 0 ] || return 0
|
||||
if [ "${PROGRESS_TOTAL:-0}" -ne "$PROGRESS_DONE" ]; then
|
||||
PROGRESS_TOTAL=$PROGRESS_DONE
|
||||
_mon_state_save
|
||||
fi
|
||||
_progress_bar_line "$PROGRESS_DONE" "$PROGRESS_TOTAL"
|
||||
if [ "${GLOBAL_N:-0}" -gt 0 ]; then
|
||||
printf -- '%s %s %d/%d (#001…#%03d)%s\n' \
|
||||
"$D" "$(i18n_text 'global checks:')" \
|
||||
"$PROGRESS_DONE" "$PROGRESS_TOTAL" "$GLOBAL_N" "$N"
|
||||
fi
|
||||
}
|
||||
_progress_bar_line() {
|
||||
# Format (grows left → right; badge = percent, not "PROG"):
|
||||
# Format (grows left → right; badge = percent):
|
||||
# ┌ 42% ┐ ████████░░░░░░░░░░░░░░░░ 31/74
|
||||
# done/total = global numbered checks so far / global expected total
|
||||
local done="$1" total="$2" width=28 pct=0 filled empty i bar pct_lab
|
||||
# Defensive: never display done > total
|
||||
if [ "$total" -gt 0 ] && [ "$done" -gt "$total" ]; then
|
||||
|
|
@ -625,7 +791,7 @@ _progress_bar_line() {
|
|||
while [ "$i" -lt "$empty" ]; do bar="${bar}░"; i=$((i + 1)); done
|
||||
pct_lab=$(printf '%3d%%' "$pct")
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
# badge = percent (clearer than "PROG"); bar grows ░→█ left to right
|
||||
# badge = percent; bar grows ░→█ left to right; N/M = global check count
|
||||
printf '%s┌%s┐%s %s%s%s %s%d/%d%s\n' \
|
||||
"$BG_PROG" "$pct_lab" "$N" \
|
||||
"$C" "$bar" "$N" \
|
||||
|
|
@ -636,10 +802,10 @@ _progress_bar_line() {
|
|||
fi
|
||||
else
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf '%s┌ … ┐%s %sdone=%d%s (set PROGRESS_TOTAL= for %%)\n' \
|
||||
printf '%s┌ … ┐%s %s%d/?%s (set PROGRESS_TOTAL= for global %%)\n' \
|
||||
"$BG_PROG" "$N" "$D" "$done" "$N"
|
||||
else
|
||||
printf -- '%sdone=%d%s (set PROGRESS_TOTAL= for %%)\n' "$D" "$done" "$N"
|
||||
printf -- '%s%d/?%s (set PROGRESS_TOTAL= for global %%)\n' "$D" "$done" "$N"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
|
@ -720,6 +886,8 @@ ok() {
|
|||
_take_tid
|
||||
_msg_line "$BG_OK" "OK" "$G" "$label" "$detail"
|
||||
PASS_N=$((PASS_N + 1))
|
||||
GLOBAL_PASS_N=$((GLOBAL_PASS_N + 1))
|
||||
_mon_state_save
|
||||
}
|
||||
# component-scoped error: err bank "what failed" "why / HTTP / path"
|
||||
err() {
|
||||
|
|
@ -727,7 +895,9 @@ err() {
|
|||
_take_tid
|
||||
_msg_line "$BG_ERR" "ERROR" "$R" "${comp}: ${msg}" "$detail"
|
||||
FAIL_N=$((FAIL_N + 1))
|
||||
GLOBAL_FAIL_N=$((GLOBAL_FAIL_N + 1))
|
||||
ERRORS+=("${LAST_TID:+$LAST_TID }[$comp] $msg${detail:+ · $detail}")
|
||||
_mon_state_save
|
||||
}
|
||||
# fail "what failed" ["why / HTTP code / path"]
|
||||
fail() {
|
||||
|
|
@ -735,7 +905,9 @@ fail() {
|
|||
_take_tid
|
||||
_msg_line "$BG_ERR" "ERROR" "$R" "$label" "$detail"
|
||||
FAIL_N=$((FAIL_N + 1))
|
||||
GLOBAL_FAIL_N=$((GLOBAL_FAIL_N + 1))
|
||||
ERRORS+=("${LAST_TID:+$LAST_TID }$label${detail:+ · $detail}")
|
||||
_mon_state_save
|
||||
}
|
||||
warn() {
|
||||
# warn "what is soft-bad" ["why still ok to continue"]
|
||||
|
|
@ -756,6 +928,8 @@ warn() {
|
|||
fi
|
||||
_msg_line "$BG_WARN" "WARN" "$Y" "$head" "$detail"
|
||||
WARN_N=$((WARN_N + 1))
|
||||
GLOBAL_WARN_N=$((GLOBAL_WARN_N + 1))
|
||||
_mon_state_save
|
||||
}
|
||||
info() {
|
||||
# info "topic" ["concrete fact / value / next step"]
|
||||
|
|
@ -763,6 +937,8 @@ info() {
|
|||
_take_tid
|
||||
_msg_line "$BG_INFO" "INFO" "$C" "$label" "$detail"
|
||||
INFO_N=$((INFO_N + 1))
|
||||
GLOBAL_INFO_N=$((GLOBAL_INFO_N + 1))
|
||||
_mon_state_save
|
||||
}
|
||||
blocker() {
|
||||
# Hard stop on pay/withdraw path: blocker "step" "why it cannot continue"
|
||||
|
|
@ -771,7 +947,10 @@ blocker() {
|
|||
_msg_line "$BG_BLK" "BLOCKER" "$M" "${step}" "$msg"
|
||||
BLOCKERS+=("${LAST_TID:+$LAST_TID }[$step] $msg")
|
||||
FAIL_N=$((FAIL_N + 1))
|
||||
GLOBAL_FAIL_N=$((GLOBAL_FAIL_N + 1))
|
||||
GLOBAL_BLOCK_N=$((GLOBAL_BLOCK_N + 1))
|
||||
ERRORS+=("BLOCKER ${LAST_TID:+$LAST_TID }[$step] $msg")
|
||||
_mon_state_save
|
||||
}
|
||||
section() {
|
||||
# Boxed section header (3 lines when colour on)
|
||||
|
|
@ -797,20 +976,156 @@ section() {
|
|||
fi
|
||||
}
|
||||
|
||||
# Print SUMMARY totals box for given counts (phase-local or global).
|
||||
# Each severity row is colour-distinct: badge + full-line bg tint + bold count (v1.14.0).
|
||||
# $1=pass $2=fail $3=warn $4=info $5=block
|
||||
_summary_totals_box() {
|
||||
local pass_n="$1" fail_n="$2" warn_n="$3" info_n="$4" blk_n="$5"
|
||||
local hdr_bg hdr_fg _sum_lbl
|
||||
_sum_lbl=$(i18n_tag SUMMARY)
|
||||
# Header colour follows overall verdict
|
||||
if [ "$fail_n" -gt 0 ] || [ "$blk_n" -gt 0 ]; then
|
||||
hdr_bg="${BG_ERR}"
|
||||
hdr_fg="${R}"
|
||||
elif [ "$warn_n" -gt 0 ]; then
|
||||
hdr_bg="${BG_WARN}"
|
||||
hdr_fg="${Y}"
|
||||
else
|
||||
hdr_bg="${BG_OK}"
|
||||
hdr_fg="${G}"
|
||||
fi
|
||||
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
# Dim full-line tints (bold fg on soft bg) — distinct per severity
|
||||
local ROW_OK=$'\e[1;32;48;5;22m' # bold green on dark green
|
||||
local ROW_ERR=$'\e[1;97;48;5;52m' # bold white on dark red
|
||||
local ROW_WARN=$'\e[1;30;48;5;178m' # bold black on gold
|
||||
local ROW_INFO=$'\e[1;97;48;5;24m' # bold white on dark blue
|
||||
local ROW_BLK=$'\e[1;97;48;5;53m' # bold white on dark magenta
|
||||
local ROW_TOT=$'\e[1;37;48;5;236m' # bold white on dark gray
|
||||
|
||||
# Include SUMMARY on all three frame lines so HTML classify maps them to sum-header
|
||||
# (phase section() boxes use the same ╔═║╚ glyphs but must not get sum-* paint).
|
||||
printf -- '\n%s╔════════ %s ════════════════════╗%s\n' "$hdr_bg" "$_sum_lbl" "$N"
|
||||
printf -- '%s║ %-34s ║%s\n' "$hdr_bg" "$_sum_lbl" "$N"
|
||||
printf -- '%s╚════════ %s ════════════════════╝%s\n' "$hdr_bg" "$_sum_lbl" "$N"
|
||||
|
||||
# $1=row_bg $2=badge_bg $3=tag $4=n $5=label_fg — full-width-ish row
|
||||
_sum_row() {
|
||||
local row_bg="$1" badge="$2" tag="$3" n="$4" fg="$5" _tg pad
|
||||
_tg=$(i18n_tag "$tag")
|
||||
pad=$(printf '%-5s' "$_tg")
|
||||
printf -- ' %s┌ %s┐%s %s %s%6d%s %s%-12s%s\n' \
|
||||
"$badge" "$pad" "$N" \
|
||||
"$row_bg" "$B" "$n" "$N" \
|
||||
"$fg$B" "$_tg" "$N"
|
||||
}
|
||||
_sum_row "$ROW_OK" "$BG_OK" "OK" "$pass_n" "$G"
|
||||
[ "$fail_n" -gt 0 ] && _sum_row "$ROW_ERR" "$BG_ERR" "ERROR" "$fail_n" "$R"
|
||||
[ "$warn_n" -gt 0 ] && _sum_row "$ROW_WARN" "$BG_WARN" "WARN" "$warn_n" "$Y"
|
||||
[ "$info_n" -gt 0 ] && _sum_row "$ROW_INFO" "$BG_INFO" "INFO" "$info_n" "$C"
|
||||
[ "$blk_n" -gt 0 ] && _sum_row "$ROW_BLK" "$BG_BLK" "BLOCK" "$blk_n" "$M"
|
||||
|
||||
# totals rollup on tinted bar
|
||||
printf -- ' %s %s %s %s%d OK%s' "$ROW_TOT" "$B" "$(i18n_text 'totals:')" "$G$B" "$pass_n" "$N$ROW_TOT"
|
||||
[ "$fail_n" -gt 0 ] && printf -- ' %s·%s %s%d ERROR%s' "$D" "$N$ROW_TOT" "$R$B" "$fail_n" "$N$ROW_TOT"
|
||||
[ "$warn_n" -gt 0 ] && printf -- ' %s·%s %s%d WARN%s' "$D" "$N$ROW_TOT" "$Y$B" "$warn_n" "$N$ROW_TOT"
|
||||
[ "$info_n" -gt 0 ] && printf -- ' %s·%s %s%d INFO%s' "$D" "$N$ROW_TOT" "$C$B" "$info_n" "$N$ROW_TOT"
|
||||
[ "$blk_n" -gt 0 ] && printf -- ' %s·%s %s%d BLOCKER%s' "$D" "$N$ROW_TOT" "$M$B" "$blk_n" "$N$ROW_TOT"
|
||||
printf -- ' %s\n' "$N"
|
||||
else
|
||||
printf -- '\n[ %s ]\n' "$_sum_lbl"
|
||||
printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag OK)" "$G$B" "$pass_n" "$N"
|
||||
[ "$fail_n" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag ERROR)" "$R$B" "$fail_n" "$N"
|
||||
[ "$warn_n" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag WARN)" "$Y$B" "$warn_n" "$N"
|
||||
[ "$info_n" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag INFO)" "$C$B" "$info_n" "$N"
|
||||
[ "$blk_n" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag BLOCK)" "$M$B" "$blk_n" "$N"
|
||||
printf -- ' %s%s%s %s%d OK%s' "$B" "$(i18n_text 'totals:')" "$N" "$G$B" "$pass_n" "$N"
|
||||
[ "$fail_n" -gt 0 ] && printf -- ' · %s%d ERROR%s' "$R$B" "$fail_n" "$N"
|
||||
[ "$warn_n" -gt 0 ] && printf -- ' · %s%d WARN%s' "$Y$B" "$warn_n" "$N"
|
||||
[ "$info_n" -gt 0 ] && printf -- ' · %s%d INFO%s' "$C$B" "$info_n" "$N"
|
||||
[ "$blk_n" -gt 0 ] && printf -- ' · %s%d BLOCKER%s' "$M$B" "$blk_n" "$N"
|
||||
printf '\n'
|
||||
fi
|
||||
# overall verdict — bold label, filled badge (i18n for FR console/HTML)
|
||||
local _ok_msg _warn_msg _fail_msg
|
||||
_ok_msg=$(i18n_text 'all clear')
|
||||
_warn_msg=$(i18n_text 'warnings only (no hard fail)')
|
||||
if [ "$fail_n" -eq 0 ] && [ "$blk_n" -eq 0 ] && [ "$warn_n" -eq 0 ]; then
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ OK ┐%s %s%s%s%s\n' "$BG_OK" "$N" "$G" "$B" "$_ok_msg" "$N"
|
||||
else
|
||||
printf -- ' %s[ OK ] %s%s%s\n' "$G" "$B" "$_ok_msg" "$N"
|
||||
fi
|
||||
elif [ "$fail_n" -eq 0 ] && [ "$blk_n" -eq 0 ]; then
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ WARN ┐%s %s%s%s%s\n' "$BG_WARN" "$N" "$Y" "$B" "$_warn_msg" "$N"
|
||||
else
|
||||
printf -- ' %s[ WARN ] %s%s%s\n' "$Y" "$B" "$_warn_msg" "$N"
|
||||
fi
|
||||
else
|
||||
_fail_msg=$(i18n_text 'failed — see list above')
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ ERROR ┐%s %s%s%s%s\n' "$BG_ERR" "$N" "$R" "$B" "$_fail_msg" "$N"
|
||||
else
|
||||
printf -- ' %s[ ERROR ] %s%s%s\n' "$R" "$B" "$_fail_msg" "$N"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# Whole-run SUMMARY from TALER_MON_STATE (parent after all phases).
|
||||
# Multi-phase: phases never print SUMMARY totals; only this does.
|
||||
summary_global() {
|
||||
_mon_state_load
|
||||
local pass_n="${GLOBAL_PASS_N:-0}"
|
||||
local fail_n="${GLOBAL_FAIL_N:-0}"
|
||||
local warn_n="${GLOBAL_WARN_N:-0}"
|
||||
local info_n="${GLOBAL_INFO_N:-0}"
|
||||
local blk_n="${GLOBAL_BLOCK_N:-0}"
|
||||
# Progress bar + numbered checks already printed by progress_finish when present.
|
||||
if [ "${GLOBAL_N:-0}" -gt 0 ]; then
|
||||
if [ "${PROGRESS_TOTAL:-0}" -gt 0 ]; then
|
||||
printf -- '%s %s #001…#%03d · %d/%d %s%s\n' \
|
||||
"$D" "$(i18n_text 'numbered checks:')" "$GLOBAL_N" \
|
||||
"${PROGRESS_DONE:-0}" "$PROGRESS_TOTAL" \
|
||||
"$(i18n_text '(global done/total)')" "$N"
|
||||
else
|
||||
printf -- '%s %s #001…#%03d%s\n' \
|
||||
"$D" "$(i18n_text 'numbered checks this run:')" "$GLOBAL_N" "$N"
|
||||
fi
|
||||
fi
|
||||
_summary_totals_box "$pass_n" "$fail_n" "$warn_n" "$info_n" "$blk_n"
|
||||
[ "$fail_n" -eq 0 ]
|
||||
}
|
||||
|
||||
summary() {
|
||||
local blk_n=${#BLOCKERS[@]}
|
||||
echo ""
|
||||
# final progress line (no _take_tid — plain printf)
|
||||
if [ "${PROGRESS_OFF:-0}" != "1" ] && [ "$PROGRESS_DONE" -gt 0 ]; then
|
||||
# Snap total to actual so last line is exact N/N 100%
|
||||
if [ "$PROGRESS_TOTAL" -gt 0 ] && [ "$PROGRESS_DONE" -ne "$PROGRESS_TOTAL" ]; then
|
||||
PROGRESS_TOTAL=$PROGRESS_DONE
|
||||
_mon_state_save
|
||||
fi
|
||||
_progress_bar_line "$PROGRESS_DONE" "$PROGRESS_TOTAL"
|
||||
# Multi-phase runs: never print phase SUMMARY / progress standings (v1.13.10+).
|
||||
# Parent prints one global SUMMARY via summary_global after progress_finish.
|
||||
# Solo check_*.sh (TALER_MON_MULTI_PHASE unset/0) still full summary.
|
||||
# Phase still lists ERRORS/BLOCKERS so mid-run failures stay visible.
|
||||
local _full=1
|
||||
if [ "${TALER_MON_MULTI_PHASE:-0}" = "1" ]; then
|
||||
_full=0
|
||||
fi
|
||||
if [ "$GLOBAL_N" -gt 0 ]; then
|
||||
printf -- '%s %s #001…#%03d%s\n' "$D" "$(i18n_text 'numbered checks this run:')" "$GLOBAL_N" "$N"
|
||||
|
||||
echo ""
|
||||
if [ "$_full" = "1" ]; then
|
||||
# Progress snapshot — whole-run snap also in progress_finish.
|
||||
if [ "${PROGRESS_OFF:-0}" != "1" ] && [ "${PROGRESS_DONE:-0}" -gt 0 ]; then
|
||||
_progress_rebalance
|
||||
_progress_bar_line "$PROGRESS_DONE" "$PROGRESS_TOTAL"
|
||||
fi
|
||||
if [ "$GLOBAL_N" -gt 0 ]; then
|
||||
if [ "${PROGRESS_TOTAL:-0}" -gt 0 ]; then
|
||||
printf -- '%s %s #001…#%03d · %d/%d %s%s\n' \
|
||||
"$D" "$(i18n_text 'numbered checks:')" "$GLOBAL_N" \
|
||||
"$PROGRESS_DONE" "$PROGRESS_TOTAL" \
|
||||
"$(i18n_text '(global done/total)')" "$N"
|
||||
else
|
||||
printf -- '%s %s #001…#%03d%s\n' "$D" "$(i18n_text 'numbered checks this run:')" "$GLOBAL_N" "$N"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
if [ "$blk_n" -gt 0 ]; then
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
|
|
@ -836,54 +1151,13 @@ summary() {
|
|||
done
|
||||
fi
|
||||
|
||||
# Coloured totals — same severity badges as check lines (always; NO_COLOR blanks ANSI)
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- '\n%s┌ %s ┐%s\n' "$BG_SEC" "$(i18n_tag SUMMARY)" "$N"
|
||||
_sum_badge() { # $1=bg $2=tag $3=fg $4=n
|
||||
local _tg
|
||||
_tg=$(i18n_tag "$2")
|
||||
printf -- ' %s┌ %-5s┐%s %s%4d%s\n' "$1" "$_tg" "$N" "$3" "$4" "$N"
|
||||
}
|
||||
_sum_badge "$BG_OK" "OK" "$G" "$PASS_N"
|
||||
[ "$FAIL_N" -gt 0 ] && _sum_badge "$BG_ERR" "ERROR" "$R" "$FAIL_N"
|
||||
[ "$WARN_N" -gt 0 ] && _sum_badge "$BG_WARN" "WARN" "$Y" "$WARN_N"
|
||||
[ "$INFO_N" -gt 0 ] && _sum_badge "$BG_INFO" "INFO" "$C" "$INFO_N"
|
||||
[ "$blk_n" -gt 0 ] && _sum_badge "$BG_BLK" "BLOCK" "$M" "$blk_n"
|
||||
else
|
||||
printf -- '\n[ %s ]\n' "$(i18n_tag SUMMARY)"
|
||||
printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag OK)" "$G" "$PASS_N" "$N"
|
||||
[ "$FAIL_N" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag ERROR)" "$R" "$FAIL_N" "$N"
|
||||
[ "$WARN_N" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag WARN)" "$Y" "$WARN_N" "$N"
|
||||
[ "$INFO_N" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag INFO)" "$C" "$INFO_N" "$N"
|
||||
[ "$blk_n" -gt 0 ] && printf -- ' [ %-5s ] %s%4d%s\n' "$(i18n_tag BLOCK)" "$M" "$blk_n" "$N"
|
||||
fi
|
||||
# one-line rollup (bold label, severity-coloured counts)
|
||||
printf -- ' %stotals:%s %s%d OK%s' "$B" "$N" "$G" "$PASS_N" "$N"
|
||||
[ "$FAIL_N" -gt 0 ] && printf -- ' · %s%d ERROR%s' "$R" "$FAIL_N" "$N"
|
||||
[ "$WARN_N" -gt 0 ] && printf -- ' · %s%d WARN%s' "$Y" "$WARN_N" "$N"
|
||||
[ "$INFO_N" -gt 0 ] && printf -- ' · %s%d INFO%s' "$C" "$INFO_N" "$N"
|
||||
[ "$blk_n" -gt 0 ] && printf -- ' · %s%d BLOCKER%s' "$M" "$blk_n" "$N"
|
||||
printf '\n'
|
||||
# overall verdict
|
||||
if [ "$FAIL_N" -eq 0 ] && [ "$blk_n" -eq 0 ] && [ "$WARN_N" -eq 0 ]; then
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ OK ┐%s %sall clear%s\n' "$BG_OK" "$N" "$G" "$N"
|
||||
else
|
||||
printf -- ' %s[ OK ] all clear%s\n' "$G" "$N"
|
||||
fi
|
||||
elif [ "$FAIL_N" -eq 0 ] && [ "$blk_n" -eq 0 ]; then
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ WARN ┐%s %swarnings only (no hard fail)%s\n' "$BG_WARN" "$N" "$Y" "$N"
|
||||
else
|
||||
printf -- ' %s[ WARN ] warnings only (no hard fail)%s\n' "$Y" "$N"
|
||||
fi
|
||||
else
|
||||
if [ "${BOX:-0}" = "1" ]; then
|
||||
printf -- ' %s┌ ERROR ┐%s %s"$(i18n_text "failed — see list above")"%s\n' "$BG_ERR" "$N" "$R" "$N"
|
||||
else
|
||||
printf -- ' %s[ ERROR ] "$(i18n_text "failed — see list above")"%s\n' "$R" "$N"
|
||||
fi
|
||||
# Coloured totals — only at full phase summary (solo script / single-phase)
|
||||
if [ "$_full" != "1" ]; then
|
||||
# Still fail the phase exit when this phase had errors (set -e + trailing summary)
|
||||
[ "$FAIL_N" -eq 0 ]
|
||||
return
|
||||
fi
|
||||
_summary_totals_box "$PASS_N" "$FAIL_N" "$WARN_N" "$INFO_N" "$blk_n"
|
||||
[ "$FAIL_N" -eq 0 ]
|
||||
}
|
||||
|
||||
|
|
@ -1152,26 +1426,8 @@ if [ -z "$SECRETS_ROOT" ]; then
|
|||
SECRETS_ROOT="${KOOPA_ADMIN_SECRETS}/koopa/host-root"
|
||||
fi
|
||||
fi
|
||||
if [ -z "$SECRETS_ROOT" ]; then
|
||||
_mon_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
_admin_log="$(cd "${_mon_dir}/../.." && pwd)"
|
||||
for d in \
|
||||
"${_admin_log}/../koopa-admin-secrets/koopa/host-root" \
|
||||
"${_admin_log}/../../koopa-admin-secrets/koopa/host-root" \
|
||||
"${HOME}/taler/src/koopa-admin-secrets/koopa/host-root" \
|
||||
"${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root" \
|
||||
"${HOME}/koopa-admin-secrets/koopa/host-root" \
|
||||
"/Users/newkamek/src/koopa/koopa-admin-secrets/koopa/host-root" \
|
||||
"$(cd "${_admin_log}/../.." 2>/dev/null && pwd)/koopa-admin-secrets/koopa/host-root"
|
||||
do
|
||||
# resolve .. without requiring the dir to exist first for cd
|
||||
if [ -d "$d/taler-bank" ]; then
|
||||
SECRETS_ROOT=$(cd "$d" && pwd)
|
||||
break
|
||||
fi
|
||||
done
|
||||
unset _mon_dir _admin_log d
|
||||
fi
|
||||
# No further home-path probes — set SECRETS_ROOT or KOOPA_ADMIN_SECRETS in env.
|
||||
|
||||
|
||||
read_secret() {
|
||||
local rel="$1" base val=""
|
||||
|
|
@ -1214,9 +1470,9 @@ read_secret() {
|
|||
secrets_hint() {
|
||||
cat <<EOF
|
||||
Prefer a local secrets file (not committed):
|
||||
cp scripts/taler-monitoring/secrets.env.example scripts/taler-monitoring/secrets.env
|
||||
\$EDITOR scripts/taler-monitoring/secrets.env
|
||||
Or set SECRETS_ROOT=\$HOME/src/koopa/koopa-admin-secrets/koopa/host-root
|
||||
cp secrets.env.example secrets.env
|
||||
\$EDITOR secrets.env
|
||||
Or set SECRETS_ROOT / KOOPA_ADMIN_SECRETS in ~/.config/taler-monitoring/env
|
||||
Or export E2E_BANK_ADMIN_PASS / E2E_MERCHANT_TOKEN for one shot.
|
||||
EOF
|
||||
}
|
||||
|
|
@ -1248,6 +1504,53 @@ load_monitoring_secrets_env() {
|
|||
export E2E_BANK_ADMIN_PASS E2E_MERCHANT_TOKEN SECRETS_ROOT KOOPA_ADMIN_SECRETS
|
||||
}
|
||||
|
||||
# Strip default HTTPS/HTTP ports from taler:// URIs (libeufin often emits host:443).
|
||||
# Wallets and mon QR rules reject default :443/:80 in the authority.
|
||||
# Usage: normalize_taler_uri "taler://withdraw/host:443/…" → prints cleaned URI
|
||||
normalize_taler_uri() {
|
||||
local u="${1-}"
|
||||
if [ -z "$u" ]; then
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "$u" | python3 -c '
|
||||
import re, sys
|
||||
u = sys.stdin.read().strip()
|
||||
# any taler://SCHEME/HOST:443|80/...
|
||||
u = re.sub(
|
||||
r"(taler://[A-Za-z0-9._-]+/)([^/?#:]+):443(?=/|$|\?|#)",
|
||||
r"\1\2",
|
||||
u,
|
||||
flags=re.I,
|
||||
)
|
||||
u = re.sub(
|
||||
r"(taler://[A-Za-z0-9._-]+/)([^/?#:]+):80(?=/|$|\?|#)",
|
||||
r"\1\2",
|
||||
u,
|
||||
flags=re.I,
|
||||
)
|
||||
# leftover path/query forms
|
||||
for a, b in (
|
||||
(":443/", "/"),
|
||||
(":443?", "?"),
|
||||
(":443#", "#"),
|
||||
(":80/", "/"),
|
||||
(":80?", "?"),
|
||||
(":80#", "#"),
|
||||
):
|
||||
u = u.replace(a, b)
|
||||
# bare trailing :443 / :80 on host (no slash)
|
||||
u = re.sub(r"(taler://[A-Za-z0-9._-]+/[^/?#:]+):443$", r"\1", u, flags=re.I)
|
||||
u = re.sub(r"(taler://[A-Za-z0-9._-]+/[^/?#:]+):80$", r"\1", u, flags=re.I)
|
||||
sys.stdout.write(u)
|
||||
' 2>/dev/null || printf '%s' "$u"
|
||||
}
|
||||
|
||||
# True (exit 0) if URI still has default port in taler:// authority.
|
||||
taler_uri_has_default_port() {
|
||||
# host:443 or host:80 before next path/query/end
|
||||
python3 -c "import re,sys; u=sys.argv[1] if len(sys.argv)>1 else ''; sys.exit(0 if re.search(r'taler://\\S+:(443|80)(/|$|\\?|#)', u, re.I) else 1)" "${1-}" 2>/dev/null
|
||||
}
|
||||
|
||||
find_wallet_cli() {
|
||||
# Must return a path suitable for: node "$WALLET_CLI" …
|
||||
# Prefer *.mjs / bundled entry; plain /usr/bin/taler-wallet-cli is often a
|
||||
|
|
@ -1256,16 +1559,20 @@ find_wallet_cli() {
|
|||
echo "$WALLET_CLI"; return 0
|
||||
fi
|
||||
local c cand
|
||||
for c in \
|
||||
# WALLET_CLI / WALLET_CLI_CANDIDATES from env; then portable package paths.
|
||||
# Prefer local ts-core / mon-env builds (schema-compatible) over stale system
|
||||
# wrappers (e.g. /usr/local 1.3.x vs DB schema 29 from 1.6.x).
|
||||
# shellcheck disable=SC2086
|
||||
for c in ${WALLET_CLI_CANDIDATES:-} \
|
||||
"${HOME}/.local/bin/taler-wallet-cli.mjs" \
|
||||
"${HOME}/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
|
||||
"${HOME}/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
|
||||
"${HOME}/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
|
||||
/home/hernani/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
/home/hernani/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
/Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
"${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
|
||||
"${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli.mjs" \
|
||||
"${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli-local.mjs" \
|
||||
"${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli-bundled.cjs" \
|
||||
/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
/usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs
|
||||
/usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
/usr/local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
|
||||
/usr/local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/dist/taler-wallet-cli-bundled.cjs
|
||||
do
|
||||
[ -n "$c" ] && [ -f "$c" ] && { echo "$c"; return 0; }
|
||||
done
|
||||
|
|
@ -1281,6 +1588,14 @@ find_wallet_cli() {
|
|||
c=$(readlink -f "$cand" 2>/dev/null || true)
|
||||
[ -n "$c" ] && [ -f "$c" ] && { echo "$c"; return 0; }
|
||||
fi
|
||||
# macOS /usr/local wrapper often runs an older bundle; prefer its bundled.cjs only
|
||||
# when no newer candidate above matched (already returned).
|
||||
for c in \
|
||||
/usr/local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/dist/taler-wallet-cli-bundled.cjs \
|
||||
/usr/local/lib/node_modules/taler-wallet-cli/dist/taler-wallet-cli-bundled.cjs
|
||||
do
|
||||
[ -f "$c" ] && { echo "$c"; return 0; }
|
||||
done
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
|
|
|||
20
mail-catalog.conf
Normal file
20
mail-catalog.conf
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# mail-catalog.conf — domains and mail hosts for phase `mail` (v1.8.0+)
|
||||
#
|
||||
# Format (whitespace-separated; # comments):
|
||||
# domain expected_mx (comma) mail_hosts (comma) smtp_ports imap_ports label
|
||||
#
|
||||
# Mail planes (outside-in only — no host login):
|
||||
# • firefly.gnunet.org — taler.net + gnunet.org
|
||||
# • mail.anastasis.lu — Anastasis / TSA (A → pixel.taler-systems.com)
|
||||
# (legacy catalog host anastasis.taler-systems.com does NOT resolve)
|
||||
#
|
||||
# Override path: MAIL_CATALOG=/path/to/file
|
||||
|
||||
# --- firefly.gnunet.org ---
|
||||
taler.net firefly.gnunet.org firefly.gnunet.org 25,465,587 143,993 firefly-taler
|
||||
gnunet.org firefly.gnunet.org firefly.gnunet.org 25,465,587 143,993 firefly-gnunet
|
||||
|
||||
# --- Anastasis / TSA (MX mail.anastasis.lu → pixel) ---
|
||||
# Port 587 often closed/filtered on this plane → WARN only (see check_mail.sh).
|
||||
taler-systems.com mail.anastasis.lu mail.anastasis.lu,pixel.taler-systems.com 25,465,587 143,993 tsa-anastasis
|
||||
anastasis.lu mail.anastasis.lu mail.anastasis.lu 25,465,587 143,993 anastasis-lu
|
||||
21
meta/README.md
Normal file
21
meta/README.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
# meta/ — optional & deprecated tooling
|
||||
|
||||
Not required for the normal **host-agent** monitoring pipeline
|
||||
(`host-agent/run-*.sh` + `site-gen/console_to_html.py`).
|
||||
|
||||
| Subdir | Contents |
|
||||
|--------|----------|
|
||||
| **`deprecated/`** | Pre-v1.8 mail/mattermost host-agent wrappers + user systemd units (folded into **surface**) |
|
||||
| **`firecuda/`** | Outside-runner install (launchd) + pull helpers — optional; prefer koopa host-agent |
|
||||
| **`site-gen-legacy/`** | Multi-host `generate-monitoring-sites` / `deploy-monitoring-sites` (older laptop/outside path) |
|
||||
|
||||
## Production path (keep using)
|
||||
|
||||
```text
|
||||
host-agent/run-host-report.sh
|
||||
host-agent/run-{hacktivism,surface,aptdeploy,fp-*}.sh
|
||||
host-agent/update-suite.sh
|
||||
site-gen/console_to_html.py
|
||||
```
|
||||
|
||||
Machine-local paths/SSH: **`taler-monitoring-env`** → `~/.config/taler-monitoring/env`.
|
||||
20
meta/deprecated/run-mail-monitoring.sh
Executable file
20
meta/deprecated/run-mail-monitoring.sh
Executable file
|
|
@ -0,0 +1,20 @@
|
|||
#!/usr/bin/env bash
|
||||
# DEPRECATED (v1.8.0): mail is part of taler-monitoring-surface.
|
||||
# Kept as a thin alias so old timers/docs do not break.
|
||||
#
|
||||
# Prefer: run-surface-monitoring.sh
|
||||
# Public page: https://taler.hacktivism.ch/taler-monitoring-surface/
|
||||
#
|
||||
set -uo pipefail
|
||||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
echo "NOTE: run-mail-monitoring.sh is deprecated (v1.8.0) — folding into surface" >&2
|
||||
export PHASES="${PHASES:-mail surface monpages}"
|
||||
# force surface HTML paths (no /taler-monitoring-mail*)
|
||||
export HTML_OK_DIR="taler-monitoring-surface"
|
||||
export HTML_ERR_DIR="taler-monitoring-surface_err"
|
||||
export HTML_URL_OK="/taler-monitoring-surface/"
|
||||
export HTML_URL_ERR="/taler-monitoring-surface_err/"
|
||||
export PAGE_LABEL="taler-monitoring-surface"
|
||||
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
|
||||
export MONPAGES_INVENTORY=job
|
||||
exec bash "$AGENT_DIR/run-surface-monitoring.sh" "$@"
|
||||
19
meta/deprecated/run-mattermost-monitoring.sh
Executable file
19
meta/deprecated/run-mattermost-monitoring.sh
Executable file
|
|
@ -0,0 +1,19 @@
|
|||
#!/usr/bin/env bash
|
||||
# DEPRECATED (v1.8.0): mattermost is part of taler-monitoring-surface.
|
||||
# Kept as a thin alias so old timers/docs do not break.
|
||||
#
|
||||
# Prefer: run-surface-monitoring.sh
|
||||
# Public page: https://taler.hacktivism.ch/taler-monitoring-surface/
|
||||
#
|
||||
set -uo pipefail
|
||||
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||||
echo "NOTE: run-mattermost-monitoring.sh is deprecated (v1.8.0) — folding into surface" >&2
|
||||
export PHASES="${PHASES:-mattermost surface monpages}"
|
||||
export HTML_OK_DIR="taler-monitoring-surface"
|
||||
export HTML_ERR_DIR="taler-monitoring-surface_err"
|
||||
export HTML_URL_OK="/taler-monitoring-surface/"
|
||||
export HTML_URL_ERR="/taler-monitoring-surface_err/"
|
||||
export PAGE_LABEL="taler-monitoring-surface"
|
||||
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
|
||||
export MONPAGES_INVENTORY=job
|
||||
exec bash "$AGENT_DIR/run-surface-monitoring.sh" "$@"
|
||||
18
meta/deprecated/taler-monitoring-mail.service
Normal file
18
meta/deprecated/taler-monitoring-mail.service
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
[Unit]
|
||||
Description=Taler monitoring mail (firefly/pixel MX SMTP IMAP)
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
|
||||
Environment=CONTINUE_ON_ERROR=1
|
||||
Environment=RUN_TIMEOUT=600
|
||||
WorkingDirectory=%h/src/taler-monitoring
|
||||
ExecStart=%h/src/taler-monitoring/meta/deprecated/run-mail-monitoring.sh
|
||||
Nice=10
|
||||
TimeoutStartSec=20min
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
14
meta/deprecated/taler-monitoring-mail.timer
Normal file
14
meta/deprecated/taler-monitoring-mail.timer
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
[Unit]
|
||||
Description=Periodic taler-monitoring mail (MX/SMTP/IMAP)
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
|
||||
[Timer]
|
||||
OnBootSec=30min
|
||||
OnUnitActiveSec=4h
|
||||
AccuracySec=5min
|
||||
Persistent=true
|
||||
RandomizedDelaySec=10min
|
||||
Unit=taler-monitoring-mail.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
18
meta/deprecated/taler-monitoring-mattermost.service
Normal file
18
meta/deprecated/taler-monitoring-mattermost.service
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
[Unit]
|
||||
Description=Taler monitoring Mattermost (mattermost.taler.net outside-in)
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
|
||||
Environment=CONTINUE_ON_ERROR=1
|
||||
Environment=RUN_TIMEOUT=300
|
||||
WorkingDirectory=%h/src/taler-monitoring
|
||||
ExecStart=%h/src/taler-monitoring/meta/deprecated/run-mattermost-monitoring.sh
|
||||
Nice=10
|
||||
TimeoutStartSec=15min
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
14
meta/deprecated/taler-monitoring-mattermost.timer
Normal file
14
meta/deprecated/taler-monitoring-mattermost.timer
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
[Unit]
|
||||
Description=Periodic taler-monitoring Mattermost (chat health)
|
||||
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
|
||||
|
||||
[Timer]
|
||||
OnBootSec=25min
|
||||
OnUnitActiveSec=1h
|
||||
AccuracySec=2min
|
||||
Persistent=true
|
||||
RandomizedDelaySec=5min
|
||||
Unit=taler-monitoring-mattermost.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
|
|
@ -11,8 +11,12 @@
|
|||
set -euo pipefail
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
MON_ROOT=$(cd "$ROOT/.." && pwd)
|
||||
FIRECUDA="${FIRECUDA_SSH:-firecuda-external}"
|
||||
MON_ROOT=$(cd "$ROOT/../.." && pwd)
|
||||
FIRECUDA="${FIRECUDA_SSH:-}"
|
||||
if [ -z "$FIRECUDA" ]; then
|
||||
echo "Set FIRECUDA_SSH to your outside-runner SSH host alias" >&2
|
||||
exit 2
|
||||
fi
|
||||
REMOTE_BASE="${FIRECUDA_INSTALL_DIR:-taler-monitoring-site-gen}"
|
||||
RUN_NOW=0
|
||||
[ "${1:-}" = "--run-now" ] && RUN_NOW=1
|
||||
|
|
@ -39,8 +43,8 @@ SKIP_SSH=1
|
|||
CONTINUE_ON_ERROR=1
|
||||
PHASES_ERR=urls versions
|
||||
PHASES_OK=urls versions
|
||||
DEPLOY_SSH=koopa-external
|
||||
DEPLOY_STAGING=/home/hernani/monitoring-sites-staging
|
||||
DEPLOY_SSH=${DEPLOY_SSH:-}
|
||||
DEPLOY_STAGING=${DEPLOY_STAGING:-$HOME/monitoring-sites-staging}
|
||||
WORK_ROOT=
|
||||
MONITORING_ROOT=..
|
||||
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
|
||||
|
|
@ -56,12 +60,14 @@ sed -i.bak \"s|^WORK_ROOT=.*|WORK_ROOT=\$HOME/var/taler-monitoring-sites-work|\"
|
|||
ssh -o BatchMode=yes "$FIRECUDA" bash -s <<REMOTE
|
||||
set -euo pipefail
|
||||
HOME_DIR=\$HOME
|
||||
FIRECUDA_META="\$HOME_DIR/$REMOTE_BASE/meta/firecuda"
|
||||
SITE_GEN="\$HOME_DIR/$REMOTE_BASE/site-gen"
|
||||
PLIST_SRC="\$SITE_GEN/com.hacktivism.taler-monitoring-sites.plist"
|
||||
PLIST_SRC="\$FIRECUDA_META/com.hacktivism.taler-monitoring-sites.plist"
|
||||
PLIST_DST="\$HOME_DIR/Library/LaunchAgents/com.hacktivism.taler-monitoring-sites.plist"
|
||||
sed -e "s|__SITE_GEN__|\${SITE_GEN}|g" -e "s|__HOME__|\${HOME_DIR}|g" "\$PLIST_SRC" > "\$PLIST_DST"
|
||||
chmod +x "\$SITE_GEN"/*.sh "\$SITE_GEN"/console_to_html.py 2>/dev/null || true
|
||||
chmod +x "\$SITE_GEN/run-on-firecuda.sh"
|
||||
sed -e "s|__SITE_GEN__|\${FIRECUDA_META}|g" -e "s|__HOME__|\${HOME_DIR}|g" "\$PLIST_SRC" > "\$PLIST_DST"
|
||||
chmod +x "\$FIRECUDA_META"/*.sh "\$SITE_GEN"/console_to_html.py 2>/dev/null || true
|
||||
chmod +x "\$HOME_DIR/$REMOTE_BASE/meta/site-gen-legacy"/*.sh 2>/dev/null || true
|
||||
chmod +x "\$FIRECUDA_META/run-on-firecuda.sh"
|
||||
launchctl bootout "gui/\$(id -u)/com.hacktivism.taler-monitoring-sites" 2>/dev/null || true
|
||||
launchctl unload "\$PLIST_DST" 2>/dev/null || true
|
||||
launchctl bootstrap "gui/\$(id -u)" "\$PLIST_DST" 2>/dev/null || launchctl load "\$PLIST_DST"
|
||||
|
|
@ -72,8 +78,8 @@ REMOTE
|
|||
|
||||
if [ "$RUN_NOW" = "1" ]; then
|
||||
echo "run once now…"
|
||||
ssh -o BatchMode=yes "$FIRECUDA" "\"\$HOME/$REMOTE_BASE/site-gen/run-on-firecuda.sh\"" || true
|
||||
ssh -o BatchMode=yes "$FIRECUDA" "\"\$HOME/$REMOTE_BASE/meta/firecuda/run-on-firecuda.sh\"" || true
|
||||
fi
|
||||
|
||||
echo "done. logs: $FIRECUDA:~/Library/Logs/taler-monitoring-sites/"
|
||||
echo "manual: ssh $FIRECUDA '~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh'"
|
||||
echo "manual: ssh $FIRECUDA '~/taler-monitoring-site-gen/meta/firecuda/run-on-firecuda.sh'"
|
||||
|
|
@ -2,14 +2,18 @@
|
|||
# pull-firecuda-to-koopa.sh — copy HTML from firecuda work dir → koopa staging
|
||||
# Run on laptop (hernani) where both SSH aliases work.
|
||||
set -euo pipefail
|
||||
FIRECUDA="${FIRECUDA_SSH:-firecuda-external}"
|
||||
KOOPA="${DEPLOY_SSH:-koopa-external}"
|
||||
FIRECUDA="${FIRECUDA_SSH:-}"
|
||||
if [ -z "$FIRECUDA" ]; then
|
||||
echo "Set FIRECUDA_SSH to your outside-runner SSH host alias" >&2
|
||||
exit 2
|
||||
fi
|
||||
KOOPA="${DEPLOY_SSH:?set DEPLOY_SSH in env}"
|
||||
REMOTE_HTML="${FIRECUDA_HTML:-var/taler-monitoring-sites-work/html/}"
|
||||
STAGE="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
|
||||
STAGE="${DEPLOY_STAGING:-${DEPLOY_STAGING:-$HOME/monitoring-sites-staging}}"
|
||||
|
||||
echo "firecuda:$REMOTE_HTML → $KOOPA:$STAGE"
|
||||
ssh -o BatchMode=yes -o ConnectTimeout=20 "$KOOPA" "mkdir -p '$STAGE'"
|
||||
rsync -az --delete \
|
||||
"${FIRECUDA}:${REMOTE_HTML}" \
|
||||
"${KOOPA}:${STAGE}/"
|
||||
echo "OK staged. Root: rsync to /var/www/monitoring-sites (see ROOT-ON-KOOPA.md)"
|
||||
echo "OK staged. Root: rsync to (see ROOT-ON-KOOPA.md)"
|
||||
|
|
@ -1,24 +1,26 @@
|
|||
#!/usr/bin/env bash
|
||||
# run-on-firecuda.sh — entrypoint for launchd/cron on firecuda-external.
|
||||
# run-on-firecuda.sh — entrypoint for launchd/cron on outside-runner.
|
||||
# Outside-only monitoring (SKIP_SSH=1), all 9 sites, then stage HTML to koopa.
|
||||
#
|
||||
# Installed path on firecuda (default):
|
||||
# ~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh
|
||||
# Lives in meta/firecuda/ (optional). Prefer koopa host-agent for production mon.
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
SUITE_ROOT=$(cd "$HERE/../.." && pwd)
|
||||
LEGACY="$SUITE_ROOT/meta/site-gen-legacy"
|
||||
SITE_GEN="$SUITE_ROOT/site-gen"
|
||||
|
||||
LOG_DIR="${FIRECUDA_LOG_DIR:-$HOME/Library/Logs/taler-monitoring-sites}"
|
||||
mkdir -p "$LOG_DIR"
|
||||
STAMP=$(date +%Y%m%d-%H%M%S)
|
||||
LOG="$LOG_DIR/run-${STAMP}.log"
|
||||
exec >>"$LOG" 2>&1
|
||||
|
||||
echo "======== $(date -Iseconds 2>/dev/null || date) firecuda monitoring sites ========"
|
||||
echo "======== $(date -Iseconds 2>/dev/null || date) outside-runner monitoring sites ========"
|
||||
|
||||
# Always: public DNS only, no SSH checks into stacks
|
||||
export SKIP_SSH=1
|
||||
export CONTINUE_ON_ERROR=1
|
||||
export AUTH401_CONTINUE=1
|
||||
|
|
@ -26,33 +28,31 @@ export RUNNER_SSH=
|
|||
export PHASES_ERR="${PHASES_ERR:-urls versions}"
|
||||
export PHASES_OK="${PHASES_OK:-urls versions}"
|
||||
export WORK_ROOT="${WORK_ROOT:-$HOME/var/taler-monitoring-sites-work}"
|
||||
export MONITORING_ROOT="${MONITORING_ROOT:-$ROOT/..}"
|
||||
export DEPLOY_SSH="${DEPLOY_SSH:-koopa-external}"
|
||||
export DEPLOY_STAGING="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
|
||||
export MONITORING_ROOT="${MONITORING_ROOT:-$SUITE_ROOT}"
|
||||
export DEPLOY_SSH="${DEPLOY_SSH:-}"
|
||||
export DEPLOY_STAGING="${DEPLOY_STAGING:-$HOME/monitoring-sites-staging}"
|
||||
export SITE_GEN_DIR="$SITE_GEN"
|
||||
|
||||
# Prefer full 9-site list from settings or defaults inside generate script
|
||||
unset ONLY_HOSTS
|
||||
|
||||
cd "$ROOT" || exit 1
|
||||
chmod +x generate-monitoring-sites.sh deploy-monitoring-sites.sh console_to_html.py 2>/dev/null || true
|
||||
cd "$LEGACY" || exit 1
|
||||
chmod +x generate-monitoring-sites.sh deploy-monitoring-sites.sh 2>/dev/null || true
|
||||
chmod +x "$SITE_GEN/console_to_html.py" 2>/dev/null || true
|
||||
|
||||
./generate-monitoring-sites.sh
|
||||
ec=$?
|
||||
|
||||
# HTML stays on firecuda under WORK_ROOT/html.
|
||||
# Optional push to koopa if this host can SSH there (often only from laptop):
|
||||
if [ -n "${DEPLOY_SSH:-}" ] && [ "${DEPLOY_FROM_FIRECUDA:-0}" = "1" ]; then
|
||||
if [ -x ./deploy-monitoring-sites.sh ]; then
|
||||
./deploy-monitoring-sites.sh || echo "WARN: deploy staging failed"
|
||||
fi
|
||||
else
|
||||
echo "note: HTML at ${WORK_ROOT}/html — pull from laptop:"
|
||||
echo " rsync -az firecuda-external:var/taler-monitoring-sites-work/html/ \\"
|
||||
echo " koopa-external:monitoring-sites-staging/"
|
||||
echo " # or: ./pull-firecuda-to-koopa.sh"
|
||||
echo " rsync -az outside-runner:var/taler-monitoring-sites-work/html/ \\"
|
||||
echo " \${DEPLOY_SSH}:monitoring-sites-staging/"
|
||||
echo " # or: $HERE/pull-firecuda-to-koopa.sh"
|
||||
fi
|
||||
|
||||
# keep last 20 logs
|
||||
ls -1t "$LOG_DIR"/run-*.log 2>/dev/null | tail -n +21 | xargs rm -f 2>/dev/null || true
|
||||
|
||||
echo "======== done ec=$ec ========"
|
||||
|
|
@ -3,9 +3,12 @@
|
|||
# Does NOT reload Caddy (needs root — see ROOT-ON-KOOPA.md).
|
||||
set -uo pipefail
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
SETTINGS="${SITE_GEN_SETTINGS:-$ROOT/settings.conf}"
|
||||
[ -f "$SETTINGS" ] || SETTINGS="$ROOT/settings.conf.example"
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
SUITE_ROOT=$(cd "$HERE/../.." && pwd)
|
||||
SITE_GEN="${SITE_GEN_DIR:-$SUITE_ROOT/site-gen}"
|
||||
ROOT="$HERE"
|
||||
SETTINGS="${SITE_GEN_SETTINGS:-$SITE_GEN/settings.conf}"
|
||||
[ -f "$SETTINGS" ] || SETTINGS="$SITE_GEN/settings.conf.example"
|
||||
# shellcheck disable=SC1090
|
||||
set -a
|
||||
source <(grep -v '^\s*#' "$SETTINGS" | grep -v '^\s*$' | sed 's/\r$//')
|
||||
|
|
@ -13,8 +16,8 @@ set +a
|
|||
|
||||
WORK_ROOT="${WORK_ROOT:-/tmp/taler-monitoring-sites-work}"
|
||||
HTML_DIR="$WORK_ROOT/html"
|
||||
DEPLOY_SSH="${DEPLOY_SSH:-koopa-external}"
|
||||
DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
|
||||
DEPLOY_SSH="${DEPLOY_SSH:-}"
|
||||
DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-}"
|
||||
|
||||
if [ ! -d "$HTML_DIR" ]; then
|
||||
echo "no HTML_DIR $HTML_DIR — run generate-monitoring-sites.sh first" >&2
|
||||
|
|
@ -23,7 +26,7 @@ fi
|
|||
|
||||
echo "deploy $HTML_DIR → ${DEPLOY_SSH}:${DEPLOY_WWW_ROOT}/"
|
||||
# Prefer writing to a hernani-owned staging dir if /var/www not writable
|
||||
STAGE_REMOTE="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
|
||||
STAGE_REMOTE="${DEPLOY_STAGING:-${DEPLOY_STAGING:-$HOME/monitoring-sites-staging}}"
|
||||
|
||||
ssh -o BatchMode=yes -o ConnectTimeout=20 "$DEPLOY_SSH" \
|
||||
"mkdir -p '$STAGE_REMOTE' '$DEPLOY_WWW_ROOT' 2>/dev/null || mkdir -p '$STAGE_REMOTE'"
|
||||
|
|
@ -18,9 +18,13 @@ export PYTHONUNBUFFERED=1
|
|||
: "${RUN_TIMEOUT:=600}"
|
||||
export RUN_TIMEOUT
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
SETTINGS="${SITE_GEN_SETTINGS:-$ROOT/settings.conf}"
|
||||
EXAMPLE="$ROOT/settings.conf.example"
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
SUITE_ROOT=$(cd "$HERE/../.." && pwd)
|
||||
SITE_GEN="${SITE_GEN_DIR:-$SUITE_ROOT/site-gen}"
|
||||
ROOT="$HERE"
|
||||
SETTINGS="${SITE_GEN_SETTINGS:-$SITE_GEN/settings.conf}"
|
||||
EXAMPLE="${SITE_GEN}/settings.conf.example"
|
||||
[ -f "$EXAMPLE" ] || EXAMPLE="$HERE/settings.conf.example"
|
||||
|
||||
if [ ! -f "$SETTINGS" ]; then
|
||||
if [ -f "$EXAMPLE" ]; then
|
||||
|
|
@ -80,7 +84,7 @@ while [ $# -gt 0 ]; do
|
|||
esac
|
||||
done
|
||||
|
||||
MONITORING_ROOT=$(cd "$ROOT/${MONITORING_ROOT:-..}" && pwd)
|
||||
MONITORING_ROOT=$(cd "${MONITORING_ROOT:-$SUITE_ROOT}" && pwd)
|
||||
WORK_ROOT="${WORK_ROOT:-/tmp/taler-monitoring-sites-work}"
|
||||
LOG_DIR="$WORK_ROOT/logs"
|
||||
HTML_DIR="$WORK_ROOT/html"
|
||||
|
|
@ -174,7 +178,7 @@ run_mon() {
|
|||
htmlify() {
|
||||
local host="$1" mode="$2" log="$3" out="$4" other="$5"
|
||||
mkdir -p "$(dirname "$out")"
|
||||
if [ ! -f "$ROOT/console_to_html.py" ]; then
|
||||
if [ ! -f "$SITE_GEN/console_to_html.py" ]; then
|
||||
echo "WARN: console_to_html.py missing — bootstrap $out"
|
||||
{
|
||||
echo "<!DOCTYPE html><html><head><meta charset=utf-8><title>$host $mode</title></head>"
|
||||
|
|
@ -185,7 +189,7 @@ htmlify() {
|
|||
} >"$out"
|
||||
return 0
|
||||
fi
|
||||
python3 "$ROOT/console_to_html.py" \
|
||||
python3 "$SITE_GEN/console_to_html.py" \
|
||||
--log "$log" \
|
||||
--out "$out" \
|
||||
--hostname "$host" \
|
||||
|
|
@ -269,5 +273,5 @@ echo "HTML under $HTML_DIR"
|
|||
echo "commit $COMMIT_SHORT $COMMIT_URL"
|
||||
echo "RUN_TIMEOUT=${RUN_TIMEOUT}s (global; 0=unlimited)"
|
||||
echo "Deploy (as hernani, may need sudo for /var/www):"
|
||||
echo " ./deploy-monitoring-sites.sh"
|
||||
echo " $HERE/deploy-monitoring-sites.sh"
|
||||
echo "Root on koopa: see ROOT-ON-KOOPA.md"
|
||||
12
metrics.sh
12
metrics.sh
|
|
@ -70,11 +70,15 @@ path = sys.argv[2] if len(sys.argv) > 2 else ""
|
|||
alt = {}
|
||||
if path:
|
||||
try:
|
||||
alt = json.load(open(path))
|
||||
_loaded = json.load(open(path))
|
||||
if isinstance(_loaded, dict):
|
||||
alt = {str(k): str(v) for k, v in _loaded.items()}
|
||||
except Exception:
|
||||
alt = {}
|
||||
if not alt:
|
||||
if not isinstance(alt, dict) or not alt:
|
||||
alt = {"0": "GOA"}
|
||||
elif "0" not in alt:
|
||||
alt["0"] = "GOA"
|
||||
|
||||
def parse(s):
|
||||
if ":" in s:
|
||||
|
|
@ -513,7 +517,7 @@ metrics_taler_load() {
|
|||
_m_bank="${INSIDE_BANK_CTR:-stage-lfp-bank}"
|
||||
_m_mer="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
|
||||
_m_ex="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
|
||||
_m_ssh="${INSIDE_SSH:-francpaysan-stage-user}"
|
||||
_m_ssh="${INSIDE_SSH:-}"
|
||||
fi
|
||||
remote_py=$(cat <<PY
|
||||
import base64, json, os, re, subprocess, time
|
||||
|
|
@ -866,7 +870,7 @@ metrics_report_load() {
|
|||
return 0
|
||||
fi
|
||||
if ! metrics_taler_load "$out" "$label"; then
|
||||
warn "load ${label}" "probe failed via ${KOOPA_SSH:-?} — try KOOPA_SSH=koopa-external"
|
||||
warn "load ${label}" "probe failed via ${KOOPA_SSH:-?} — set KOOPA_SSH / KOOPA_SSH_FALLBACKS in env"
|
||||
return 1
|
||||
fi
|
||||
while IFS= read -r line; do
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
# $EDITOR secrets.env
|
||||
#
|
||||
# Never commit secrets.env. Prefer file paths under koopa-admin-secrets
|
||||
# for long-term storage; this file is for laptop one-shots / overrides.
|
||||
# for long-term storage; this file is for workstation one-shots / overrides.
|
||||
#
|
||||
# Load order: env already set > secrets.env > SECRETS_ROOT files > SSH koopa
|
||||
|
||||
|
|
@ -30,8 +30,8 @@
|
|||
# Default phases for -d stage.lefrancpaysan.ch: urls + inside + versions + e2e
|
||||
# inside uses francpaysan-stage-user (stagepaysan); stats from public HTTPS.
|
||||
# Admin credit (required for ATM withdraw ladder on stage):
|
||||
# E2E_BANK_ADMIN_PASS=… # or auto-load via SSH francpaysan-host
|
||||
# FRANCPAYSAN_SSH=francpaysan-host
|
||||
# E2E_BANK_ADMIN_PASS=… # or auto-load via SSH
|
||||
# FRANCPAYSAN_SSH=
|
||||
# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets
|
||||
# Public shop pays use E2E_USE_TEMPLATES=1 (default on TESTPAYSAN) — no merchant token.
|
||||
|
||||
|
|
@ -69,6 +69,15 @@
|
|||
# AUTH401_ADMIN_USER=admin
|
||||
# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets
|
||||
|
||||
# --- Devtesting / fake-franken CHF (rusty.taler-ops.ch) ---
|
||||
# Phase: devtesting | franken | fake-franken
|
||||
# SSH Host taler-rusty-devtesting (user devtesting, forced taler-devtesting CLI).
|
||||
# DEVTESTING_SSH=taler-rusty-devtesting
|
||||
# DEVTESTING_AMOUNT=CHF:1.00
|
||||
# DEVTESTING_CREDIT_PAYTO= # optional; auto from EXCHANGE_PUBLIC/keys or TOPS IBAN
|
||||
# DEVTESTING_EXCHANGE_IBAN=CH6808573105529100001
|
||||
# DEVTESTING_SKIP=1 # skip phase
|
||||
|
||||
# --- Ladder (GOA withdraw ladder · explorer pool) ---
|
||||
# Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt).
|
||||
# Overrides only if needed:
|
||||
|
|
|
|||
|
|
@ -1,81 +1,20 @@
|
|||
# monitoring site-gen
|
||||
# site-gen — HTML converter + Caddy notes
|
||||
|
||||
Build **console-style HTML** from `taler-monitoring` runs and stage them for
|
||||
**host Caddy** (`/monitoring`, `/monitoring_err`).
|
||||
**Required for host-agent:** `console_to_html.py` (log → sticky HTML).
|
||||
|
||||
Same **global reporting defaults** as host-agents (`run-host-report.sh`):
|
||||
|
||||
- **`RUN_TIMEOUT=600`** on each suite run
|
||||
- **line-buffered** logs (continuous write)
|
||||
- **always** write HTML (first run / empty logs too)
|
||||
- Forgejo **commit link** in the page header
|
||||
|
||||
Applies to all nine fronts (GOA + FP stage + FP prod) in the default `SITES` list.
|
||||
|
||||
| Host | Role |
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| **firecuda-external** | optional runner (public DNS) |
|
||||
| **koopa-external** | deploy staging + Caddy (root service) |
|
||||
| **host-agents** | primary: GOA + FP on-box (`../host-agent/`) |
|
||||
| `console_to_html.py` | Convert suite console log → public mon HTML |
|
||||
| `caddy-monitoring-handles.snippet` | Caddy `handle` / bare redir notes |
|
||||
| `ROOT-ON-KOOPA.md` | Publish staging → `/var/www` (paths from env) |
|
||||
| `settings.conf.example` | Optional settings for **legacy** multi-host generator |
|
||||
|
||||
No secrets in `settings.conf` — only hosts, paths, site list.
|
||||
## Optional / moved
|
||||
|
||||
## firecuda-external (outside-only) — **disabled for now**
|
||||
|
||||
**firecuda-external could** run public-only monitoring (`SKIP_SSH=1`, phases
|
||||
`urls versions`) on a schedule (launchd every 4h) and write HTML under
|
||||
`~/var/taler-monitoring-sites-work/html/`. Scripts for that still live here
|
||||
(`install-firecuda-timer.sh`, `run-on-firecuda.sh`, plist template).
|
||||
|
||||
**Current policy: do not run a real launchd job on firecuda.**
|
||||
Primary GOA monitoring is **koopa host-agent** (`host-agent/`, user `hernani`,
|
||||
local `podman exec`). Re-enable firecuda only if you explicitly want a second,
|
||||
outside-only view.
|
||||
Legacy multi-host generate/deploy and firecuda launchd tooling live under
|
||||
**`../meta/`** (see `meta/README.md`).
|
||||
|
||||
```bash
|
||||
# optional one-shot (no timer):
|
||||
# ssh firecuda-external '~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh'
|
||||
|
||||
# if a timer was installed earlier, ensure it is off:
|
||||
# ssh firecuda-external 'launchctl list | grep taler-monitoring || echo off'
|
||||
```
|
||||
|
||||
Disabled plists may remain as `*.plist.disabled-*` under `~/Library/LaunchAgents/`.
|
||||
|
||||
## Quick start (laptop)
|
||||
|
||||
```bash
|
||||
cd site-gen
|
||||
cp settings.conf.example settings.conf # optional
|
||||
chmod +x generate-monitoring-sites.sh deploy-monitoring-sites.sh console_to_html.py
|
||||
|
||||
ONLY_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch" \
|
||||
SKIP_SSH=1 RUNNER_SSH= \
|
||||
./generate-monitoring-sites.sh
|
||||
|
||||
./deploy-monitoring-sites.sh
|
||||
```
|
||||
|
||||
Then on koopa **as root**: see **[ROOT-ON-KOOPA.md](./ROOT-ON-KOOPA.md)**.
|
||||
|
||||
## URL rules
|
||||
|
||||
| Path | When |
|
||||
|------|------|
|
||||
| `/monitoring_err/` | always: full console log + error list on top (CONTINUE_ON_ERROR run) |
|
||||
| `/monitoring/` | **clean** full log if last strict run exit 0; else **stub** linking to `/monitoring_err/` |
|
||||
|
||||
Footer links **Forgejo commit** of the admin-log tree used for that run.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
site-gen/
|
||||
settings.conf.example
|
||||
generate-monitoring-sites.sh
|
||||
deploy-monitoring-sites.sh
|
||||
console_to_html.py
|
||||
caddy-monitoring-handles.snippet
|
||||
ROOT-ON-KOOPA.md
|
||||
README.md
|
||||
# normal publish is host-agent on the mon host, not these scripts
|
||||
../host-agent/run-hacktivism-monitoring.sh # etc.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -3,31 +3,33 @@
|
|||
Host Caddy runs as **root/systemd** (`caddy` user). Static monitoring HTML is
|
||||
**not** put into Taler containers — only host paths + Caddy `handle`.
|
||||
|
||||
After `generate-monitoring-sites.sh` + `deploy-monitoring-sites.sh` (as hernani),
|
||||
files live in:
|
||||
After host-agent (or optional `meta/site-gen-legacy/generate-monitoring-sites.sh`) + deploy, files live in:
|
||||
|
||||
```text
|
||||
/home/hernani/monitoring-sites-staging/<hostname>/monitoring/index.html
|
||||
/home/hernani/monitoring-sites-staging/<hostname>/monitoring_err/index.html
|
||||
$DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/<hostname>/monitoring/index.html
|
||||
$DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/<hostname>/monitoring_err/index.html
|
||||
```
|
||||
|
||||
## One-shot as root (copy + Caddy)
|
||||
## One-shot as root (preferred)
|
||||
|
||||
```bash
|
||||
# on koopa (koopa-external), as root:
|
||||
# absolute path — as root, ~ is /root (script not found under ~/koopa-caddy/…)
|
||||
sudo $APPLY_MONITORING_LIVE
|
||||
# same script is versioned in the suite:
|
||||
# host-agent/apply-monitoring-live.sh → install/sync to $KOOPA_CADDY_DIR/
|
||||
```
|
||||
|
||||
# 1) web root
|
||||
install -d -o caddy -g caddy -m 755 /var/www/monitoring-sites
|
||||
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
|
||||
chown -R caddy:caddy /var/www/monitoring-sites
|
||||
That rsyncs staging → `/var/www/monitoring-sites`, installs the prepared
|
||||
Caddyfile, reloads Caddy, and smokes mon URLs (including bare surface).
|
||||
|
||||
# 2) Caddyfile — merge handles from admin-log mirror, then:
|
||||
# (either edit /etc/caddy/Caddyfile by hand using snippet below,
|
||||
# or copy full mirror after review)
|
||||
#
|
||||
# install -m 644 /home/hernani/koopa-admin-log/configs/caddy/Caddyfile /etc/caddy/Caddyfile
|
||||
# # or: ~/koopa-caddy/apply.sh after syncing Caddyfile into ~/koopa-caddy/
|
||||
## Manual equivalent
|
||||
|
||||
```bash
|
||||
install -d -o hernani -g caddy -m 755 /var/www/monitoring-sites
|
||||
rsync -a --delete $DEPLOY_STAGING (or $HOME/monitoring-sites-staging)/ /var/www/monitoring-sites/
|
||||
chown -R hernani:caddy /var/www/monitoring-sites
|
||||
|
||||
install -m 644 $KOOPA_CADDY_DIR/Caddyfile /etc/caddy/Caddyfile
|
||||
caddy validate --config /etc/caddy/Caddyfile
|
||||
systemctl reload caddy
|
||||
systemctl is-active caddy
|
||||
|
|
@ -35,23 +37,10 @@ systemctl is-active caddy
|
|||
|
||||
## Snippet per hacktivism site (inside each `*.hacktivism.ch { }` block)
|
||||
|
||||
Place **before** the catch-all `reverse_proxy` (same idea as `/intro*`):
|
||||
Place **before** the catch-all `reverse_proxy`. Canonical copy:
|
||||
`site-gen/caddy-monitoring-handles.snippet`.
|
||||
|
||||
```caddy
|
||||
# Public taler-monitoring console HTML (static; host only)
|
||||
handle /monitoring_err* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
rewrite * /monitoring_err{uri}
|
||||
# uri is /monitoring_err or /monitoring_err/ → serve directory index
|
||||
file_server
|
||||
}
|
||||
handle /monitoring* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
file_server
|
||||
}
|
||||
```
|
||||
|
||||
Simpler layout (recommended): path on disk matches URL under host root:
|
||||
Disk layout (path on disk matches URL under host root):
|
||||
|
||||
```text
|
||||
/var/www/monitoring-sites/bank.hacktivism.ch/monitoring/index.html
|
||||
|
|
@ -59,34 +48,50 @@ Simpler layout (recommended): path on disk matches URL under host root:
|
|||
```
|
||||
|
||||
```caddy
|
||||
handle_path /monitoring_err/* {
|
||||
root * /var/www/monitoring-sites/{host}/monitoring_err
|
||||
# bare → slash (MUST use redir * /path/ — see footgun below)
|
||||
handle /monitoring {
|
||||
redir * /monitoring/ 302
|
||||
}
|
||||
handle /monitoring* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
file_server
|
||||
}
|
||||
handle /monitoring_err {
|
||||
redir /monitoring_err/ 302
|
||||
redir * /monitoring_err/ 302
|
||||
}
|
||||
handle_path /monitoring/* {
|
||||
root * /var/www/monitoring-sites/{host}/monitoring
|
||||
handle /monitoring_err* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
file_server
|
||||
}
|
||||
handle /monitoring {
|
||||
redir /monitoring/ 302
|
||||
}
|
||||
```
|
||||
|
||||
Canonical snippet is also in `caddy-monitoring-handles.snippet` and applied
|
||||
in `configs/caddy/Caddyfile` for the three GOA hosts.
|
||||
### Caddy redir footgun (merchant code 21)
|
||||
|
||||
```caddy
|
||||
# WRONG — parsed as matcher=/monitoring/ to="302" → Location: 302
|
||||
handle /monitoring {
|
||||
redir /monitoring/ 302
|
||||
}
|
||||
|
||||
# RIGHT
|
||||
handle /monitoring {
|
||||
redir * /monitoring/ 302
|
||||
}
|
||||
```
|
||||
|
||||
Symptom: `https://…/monitoring` or `…/taler-monitoring-surface` returns merchant
|
||||
JSON `{"code":21,…}` while the slash form serves HTML.
|
||||
|
||||
## Check
|
||||
|
||||
```bash
|
||||
curl -sS -o /dev/null -w '%{http_code}\n' https://bank.hacktivism.ch/monitoring/
|
||||
curl -sS -o /dev/null -w '%{http_code}\n' https://bank.hacktivism.ch/monitoring_err/
|
||||
curl -sS https://bank.hacktivism.ch/monitoring_err/ | head
|
||||
curl -sS -D- -o /dev/null https://taler.hacktivism.ch/taler-monitoring-surface | head -15
|
||||
# bare: expect 302 Location: /taler-monitoring-surface/ — not JSON code 21
|
||||
curl -sS -o /dev/null -w '%{http_code}\n' https://taler.hacktivism.ch/taler-monitoring-surface/
|
||||
```
|
||||
|
||||
## Not needed as root
|
||||
|
||||
- Running `taler-monitoring` (use **firecuda-external** / laptop)
|
||||
- Running `taler-monitoring` (use **optional outside runner** / laptop)
|
||||
- Writing into podman Taler containers
|
||||
|
|
|
|||
|
|
@ -1,57 +1,76 @@
|
|||
# Host Caddy on koopa (root service). Paste BEFORE catch-all reverse_proxy.
|
||||
# Host Caddy (root service). Place BEFORE catch-all reverse_proxy in each site block.
|
||||
#
|
||||
# CRITICAL: bare paths (no trailing slash) must redirect to slash, or the request
|
||||
# falls through to merchant/bank/exchange API → Taler JSON code 21.
|
||||
#
|
||||
# CORRECT (site-level named matcher — preferred):
|
||||
# @mon_bare path /monitoring /monitoring_err \
|
||||
# /monitoring-max-ladder /monitoring-max-ladder_err \
|
||||
# /taler-monitoring-surface /taler-monitoring-surface_err \
|
||||
# /taler-monitoring-aptdeploy /taler-monitoring-aptdeploy_err
|
||||
# redir @mon_bare {path}/ 302
|
||||
#
|
||||
# ALSO OK inside handle (matcher must be `*`):
|
||||
# handle /taler-monitoring-surface {
|
||||
# redir * /taler-monitoring-surface/ 302
|
||||
# }
|
||||
#
|
||||
# WRONG (Caddy footgun — causes merchant code 21 on bare URLs):
|
||||
# redir /taler-monitoring-surface/ 302
|
||||
# → parsed as matcher=/taler-monitoring-surface/ to="302"
|
||||
# → Location: 302, or no-op then fallthrough to reverse_proxy
|
||||
#
|
||||
# root must be the *host directory* (…/monitoring-sites/{host}), and the
|
||||
# file_server matcher a path *prefix* (`/monitoring*`), so
|
||||
# /monitoring/index.html maps to {root}/monitoring/index.html.
|
||||
#
|
||||
# Disk layout:
|
||||
# /var/www/monitoring-sites/{host}/monitoring/index.html
|
||||
# /var/www/monitoring-sites/{host}/monitoring_err/index.html
|
||||
# /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface*/index.html
|
||||
# /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy*/index.html
|
||||
# /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-*/index.html
|
||||
#
|
||||
# /monitoring* → bank + exchange + taler.hacktivism.ch
|
||||
# /taler-monitoring-surface* → taler.hacktivism.ch only
|
||||
# /taler-monitoring-aptdeploy* → taler.hacktivism.ch only
|
||||
# Live apply on koopa (absolute path — ~ as root is /root):
|
||||
# sudo ${APPLY_MONITORING_LIVE}
|
||||
# Runtime-only (no sudo; hernani admin API):
|
||||
# curl -g -X POST http://[::1]:2019/load -H 'Content-Type: text/caddyfile' \
|
||||
# --data-binary @${KOOPA_CADDY_DIR}/Caddyfile
|
||||
|
||||
# --- only inside taler.hacktivism.ch { ... } ---
|
||||
handle /taler-monitoring-surface_err {
|
||||
redir /taler-monitoring-surface_err/ 302
|
||||
}
|
||||
handle /taler-monitoring-surface_err/ {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface_err
|
||||
@mon_bare path /monitoring /monitoring_err \
|
||||
/monitoring-max-ladder /monitoring-max-ladder_err \
|
||||
/taler-monitoring-surface /taler-monitoring-surface_err \
|
||||
/taler-monitoring-aptdeploy /taler-monitoring-aptdeploy_err \
|
||||
/taler-monitoring-mail /taler-monitoring-mail_err \
|
||||
/taler-monitoring-mattermost /taler-monitoring-mattermost_err
|
||||
redir @mon_bare {path}/ 302
|
||||
|
||||
handle /taler-monitoring-surface_err* {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch
|
||||
file_server
|
||||
}
|
||||
handle /taler-monitoring-surface {
|
||||
redir /taler-monitoring-surface/ 302
|
||||
}
|
||||
handle /taler-monitoring-surface/ {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface
|
||||
handle /taler-monitoring-surface* {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch
|
||||
file_server
|
||||
}
|
||||
handle /taler-monitoring-aptdeploy_err {
|
||||
redir /taler-monitoring-aptdeploy_err/ 302
|
||||
}
|
||||
handle /taler-monitoring-aptdeploy_err/ {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy_err
|
||||
handle /taler-monitoring-aptdeploy_err* {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch
|
||||
file_server
|
||||
}
|
||||
handle /taler-monitoring-aptdeploy {
|
||||
redir /taler-monitoring-aptdeploy/ 302
|
||||
}
|
||||
handle /taler-monitoring-aptdeploy/ {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy
|
||||
handle /taler-monitoring-aptdeploy* {
|
||||
root * /var/www/monitoring-sites/taler.hacktivism.ch
|
||||
file_server
|
||||
}
|
||||
|
||||
# --- bank + exchange + taler (each site block) ---
|
||||
handle /monitoring_err {
|
||||
redir /monitoring_err/ 302
|
||||
}
|
||||
handle /monitoring_err/ {
|
||||
root * /var/www/monitoring-sites/{host}/monitoring_err
|
||||
# --- bank + exchange + taler (each site block; set root host dir) ---
|
||||
# /monitoring* already covers /monitoring-max-ladder*
|
||||
@mon_bare path /monitoring /monitoring_err \
|
||||
/monitoring-max-ladder /monitoring-max-ladder_err
|
||||
redir @mon_bare {path}/ 302
|
||||
|
||||
handle /monitoring_err* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
file_server
|
||||
}
|
||||
handle /monitoring {
|
||||
redir /monitoring/ 302
|
||||
}
|
||||
handle /monitoring/ {
|
||||
root * /var/www/monitoring-sites/{host}/monitoring
|
||||
handle /monitoring* {
|
||||
root * /var/www/monitoring-sites/{host}
|
||||
file_server
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -2,8 +2,8 @@
|
|||
# Copy: cp settings.conf.example settings.conf
|
||||
# Never put passwords/tokens here.
|
||||
|
||||
# On firecuda the suite runs *locally* (RUNNER_SSH empty). From a laptop you
|
||||
# can set RUNNER_SSH=firecuda-external to drive a one-off remote run instead.
|
||||
# On an outside runner the suite runs *locally* (RUNNER_SSH empty). From a laptop you
|
||||
# can set RUNNER_SSH=<outside-runner-ssh-alias> to drive a one-off remote run instead.
|
||||
RUNNER_SSH=
|
||||
RUNNER_SSH_FALLBACKS=
|
||||
RUNNER_REMOTE_WORKDIR=/tmp/taler-monitoring-site-gen
|
||||
|
|
@ -13,15 +13,15 @@ MONITORING_ROOT=..
|
|||
SKIP_SSH=1
|
||||
|
||||
# Deploy staging on koopa (host Caddy later — root copies to /var/www)
|
||||
DEPLOY_SSH=koopa-external
|
||||
DEPLOY_SSH= # set SSH host for deploy
|
||||
DEPLOY_SSH_FALLBACKS=koopa
|
||||
# Static files for Caddy file_server (created per hostname)
|
||||
DEPLOY_WWW_ROOT=/var/www/monitoring-sites
|
||||
DEPLOY_WWW_ROOT= # e.g. /var/www/monitoring-sites
|
||||
DEPLOY_SUDO=sudo
|
||||
CADDY_CONFIG=/etc/caddy/Caddyfile
|
||||
CADDY_MIRROR_REL=configs/caddy/Caddyfile
|
||||
# Working tree on deploy host for apply (optional)
|
||||
KOOPA_CADDY_DIR=/home/hernani/koopa-caddy
|
||||
KOOPA_CADDY_DIR= # absolute path to Caddy tree on mon host
|
||||
|
||||
# Source link for HTML footer (Forgejo)
|
||||
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ git.taler.net 443,22 https taler-git
|
|||
bugs.taler.net 443 https taler-bugs
|
||||
lists.taler.net 443 https taler-lists
|
||||
tutorials.taler.net 443 https taler-tutorials
|
||||
mattermost.taler.net 443 https mattermost
|
||||
bank.demo.taler.net 443 https demo-bank
|
||||
exchange.demo.taler.net 443 https demo-exchange
|
||||
backend.demo.taler.net 443 https demo-merchant
|
||||
|
|
@ -35,7 +36,6 @@ stage.my.taler-ops.ch 443 https tops-stage-merchant
|
|||
exchange.stage.taler-ops.ch 443 https tops-stage-exchange
|
||||
nexus.stage.taler-ops.ch 443 https tops-stage-nexus
|
||||
shops.taler-ops.ch 443 https tops-shops
|
||||
mattermost.taler.net 443 https mattermost
|
||||
|
||||
# --- gnunet.org ---
|
||||
www.gnunet.org 443,80 https gnunet-www
|
||||
|
|
@ -46,6 +46,13 @@ bugs.gnunet.org 443 https gnunet-bugs
|
|||
# --- taler-systems.com ---
|
||||
www.taler-systems.com 443 https tsa-www
|
||||
taler-systems.com 443 https tsa-apex
|
||||
# Anastasis / TSA mail plane (MX mail.anastasis.lu → pixel). Do NOT list
|
||||
# anastasis.taler-systems.com — it does not resolve (was a catalog typo).
|
||||
mail.anastasis.lu 443,25,465,587,143,993 https anastasis-mail
|
||||
pixel.taler-systems.com 443,25,465,587,143,993 https pixel-mail
|
||||
|
||||
# --- mail (also covered by phase mail; listed for surface nmap/ports) ---
|
||||
firefly.gnunet.org 443,25,465,587,143,993 https firefly-mail
|
||||
|
||||
# --- common infra hostnames (remote only; may 404/redirect) ---
|
||||
deb.taler.net 443 https deb-taler
|
||||
|
|
|
|||
|
|
@ -12,6 +12,66 @@
|
|||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||
|
||||
# --- suite identity (also: --ver / --version / -V) ---
|
||||
SOURCE_REPO_WEB="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-monitoring}"
|
||||
SOURCE_REPO_GIT="${SOURCE_REPO_GIT:-https://git.hacktivism.ch/hernani/taler-monitoring.git}"
|
||||
|
||||
print_version() {
|
||||
local ver tag commit commit_full dirty branch
|
||||
ver="unknown"
|
||||
if [ -f "$ROOT/VERSION" ]; then
|
||||
ver=$(tr -d "[:space:]" <"$ROOT/VERSION" | sed "s/^v//")
|
||||
fi
|
||||
tag=""
|
||||
commit="unknown"
|
||||
commit_full=""
|
||||
branch=""
|
||||
dirty=""
|
||||
if git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then
|
||||
tag=$(git -C "$ROOT" describe --tags --exact-match 2>/dev/null || true)
|
||||
if [ -z "$tag" ]; then
|
||||
tag=$(git -C "$ROOT" describe --tags --abbrev=0 2>/dev/null || true)
|
||||
fi
|
||||
commit=$(git -C "$ROOT" rev-parse --short=12 HEAD 2>/dev/null || echo unknown)
|
||||
commit_full=$(git -C "$ROOT" rev-parse HEAD 2>/dev/null || true)
|
||||
branch=$(git -C "$ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || true)
|
||||
if [ -n "$(git -C "$ROOT" status --porcelain 2>/dev/null)" ]; then
|
||||
dirty=" (dirty)"
|
||||
fi
|
||||
fi
|
||||
if [ -n "$tag" ] && [ "$tag" = "v$ver" ]; then
|
||||
printf "taler-monitoring %s\n" "$tag"
|
||||
else
|
||||
printf "taler-monitoring v%s\n" "$ver"
|
||||
fi
|
||||
printf "version: %s\n" "$ver"
|
||||
if [ -n "$tag" ]; then
|
||||
if [ "$tag" = "v$ver" ]; then
|
||||
printf "tag: %s\n" "$tag"
|
||||
else
|
||||
printf "git-tag: %s (tree; VERSION file is %s)\n" "$tag" "$ver"
|
||||
fi
|
||||
fi
|
||||
printf "commit: %s%s\n" "$commit" "$dirty"
|
||||
[ -n "$branch" ] && [ "$branch" != "HEAD" ] && printf "branch: %s\n" "$branch"
|
||||
printf "repo: %s\n" "$SOURCE_REPO_WEB"
|
||||
printf "clone: %s\n" "$SOURCE_REPO_GIT"
|
||||
if [ -n "$commit_full" ]; then
|
||||
printf "commit-url: %s/src/commit/%s\n" "$SOURCE_REPO_WEB" "$commit_full"
|
||||
fi
|
||||
if [ -n "$tag" ]; then
|
||||
printf "tag-url: %s/src/tag/%s\n" "$SOURCE_REPO_WEB" "$tag"
|
||||
fi
|
||||
}
|
||||
|
||||
# Fast path: version before loading lib.sh / phases
|
||||
case "${1:-}" in
|
||||
--ver|--version|-V)
|
||||
print_version
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
|
||||
# Line-buffered stdout/stderr so redirected logs (tee/host-agent) flush each line.
|
||||
if [ "${_TALER_MON_STDBUF:-0}" != "1" ] && command -v stdbuf >/dev/null 2>&1; then
|
||||
export _TALER_MON_STDBUF=1
|
||||
|
|
@ -33,13 +93,22 @@ Phases:
|
|||
sanity public + optional server
|
||||
server server-side only (SSH)
|
||||
e2e withdraw + pay (small amounts; remote aborts on login/KYC)
|
||||
ladder withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire)
|
||||
ladder amount ladder classic (default 23-rung withdraw + pay; any currency)
|
||||
max-ladder max-search: highest withdrawable + highest payable (≤ available)
|
||||
goa-ladder alias of ladder (legacy phase name; same check_amount_ladder.sh)
|
||||
auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
|
||||
aptdeploy koopa podman apt-src smoke: taler-merchant in
|
||||
koopa-taler-deploy-test-apt-src-trixie{,-testing}
|
||||
surface REMOTE-ONLY public inventory (NOT in default/all/full):
|
||||
ecosystem hosts (taler.net, gnunet.org, taler-systems.com, mattermost, …)
|
||||
or -d DOMAIN → that domain’s surface; port/protocol/TLS/CVE (OSV)
|
||||
mattermost Mattermost chat health (SPA + ping + client floors: mobile/desktop/support ≥10.11)
|
||||
mail MX/SMTP/IMAP for Taler mail (firefly, pixel/TSA, catalogued domains)
|
||||
monpages public monitoring HTML via FQDN (obligatory ERROR; GOA full inventory, FP only FP)
|
||||
uses MON_HOSTS + HTML_URL_OK (same as host-agent); v1.3.1+
|
||||
devtesting fake-franken CHF via rusty.taler-ops.ch (taler-devtesting)
|
||||
geniban + fake-incoming; needs SSH Host DEVTESTING_SSH (key to devtesting@rusty)
|
||||
franken|fake-franken aliases for devtesting
|
||||
all urls + inside + versions + sanity + e2e (SSH phases only on koopa)
|
||||
full all + server + ladder + auth401 (maximum; long-running, needs secrets)
|
||||
NOTE: surface is never included in all/full — pass it explicitly
|
||||
|
|
@ -55,8 +124,9 @@ Options:
|
|||
--merchant URL merchant-backend base — overrides profile
|
||||
--currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only
|
||||
--no-probe do not probe alternate hosts (unknown domains only)
|
||||
--lang en|fr UI/console language (sticky bar + badges)
|
||||
--lang en|fr|de UI/console language (sticky bar + badges; de available, not a profile default)
|
||||
overrides auto (FrancPaysan→fr, else en)
|
||||
--ver, --version, -V print suite version, git commit, and repo URLs
|
||||
-h, --help
|
||||
|
||||
Add a stack: edit domains.conf (name + bank + exchange + merchant + currency).
|
||||
|
|
@ -82,7 +152,7 @@ Env (same meaning):
|
|||
PERF_WARN_MS PERF_FAIL_MS (urls latency; default 8000 / 20000)
|
||||
QR_CHECK=0 skip QR form + qrencode/zbarimg (urls phase)
|
||||
QR_ECC=M qrencode ECC level (default M)
|
||||
KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external)
|
||||
KOOPA_SSH KOOPA_SSH_FALLBACKS (set KOOPA_SSH / KOOPA_SSH_FALLBACKS in env)
|
||||
METRICS_LOAD=0 skip host/container RAM/CPU probes (e2e/ladder/inside)
|
||||
EXPECT_WEBUI_VERSION=1.6.11 pin merchant /webui/version.txt (urls)
|
||||
EXPECT_WEBUI_OVERLAY=selfbuild require substr in version-overlay.txt
|
||||
|
|
@ -97,7 +167,8 @@ Env (same meaning):
|
|||
SURFACE_CVE_LEVEL=warn|error bare Server-header versions default warn
|
||||
(Debian package versions default error)
|
||||
SURFACE_CATALOG=path override surface-catalog.conf
|
||||
TALER_MON_LANG=en|fr same as --lang (default en; auto fr for *lefrancpaysan*)
|
||||
TALER_MON_LANG=en|fr|de same as --lang (profile lang from domains.conf; de never stock default)
|
||||
TALER_MON_LOCALE=fr-CH|de-CH regional l10n from domains.conf locale column
|
||||
|
||||
Full load (GOA / hacktivism):
|
||||
./taler-monitoring.sh -d hacktivism.ch full
|
||||
|
|
@ -109,6 +180,8 @@ Remote surface / ecosystem (explicit only — never default):
|
|||
./taler-monitoring.sh surface
|
||||
./taler-monitoring.sh -d hacktivism.ch surface
|
||||
./taler-monitoring.sh -d lefrancpaysan.ch surface
|
||||
./taler-monitoring.sh mattermost # mattermost.taler.net chat health
|
||||
./taler-monitoring.sh mail # firefly + pixel MX/SMTP/IMAP
|
||||
|
||||
SPA pin after selfbuild:
|
||||
EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \\
|
||||
|
|
@ -117,6 +190,21 @@ SPA pin after selfbuild:
|
|||
EOF
|
||||
}
|
||||
|
||||
# Machine-local paths/hosts from taler-monitoring-env install (no hardcodes in suite).
|
||||
# Install: ~/src/taler-monitoring-env/bin/install-env.sh <machine>
|
||||
if [ "${_TALER_MON_ENV_LOADED:-0}" != "1" ]; then
|
||||
_mon_cfg="${TALER_MONITORING_ENV:-${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env}"
|
||||
if [ -f "$_mon_cfg" ]; then
|
||||
# shellcheck disable=SC1090
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
. "$_mon_cfg"
|
||||
set +a
|
||||
fi
|
||||
unset _mon_cfg
|
||||
export _TALER_MON_ENV_LOADED=1
|
||||
fi
|
||||
|
||||
# shellcheck source=lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
# ensure i18n after domain may change later
|
||||
|
|
@ -140,12 +228,13 @@ NO_PROBE=0
|
|||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--lang)
|
||||
[ $# -ge 2 ] || { echo "error: --lang needs en|fr" >&2; exit 2; }
|
||||
[ $# -ge 2 ] || { echo "error: --lang needs en|fr|de" >&2; exit 2; }
|
||||
_lang="$2"
|
||||
case "$_lang" in
|
||||
fr|FR|fra|french) TALER_MON_LANG=fr ;;
|
||||
de|DE|deu|ger|german|deutsch) TALER_MON_LANG=de ;;
|
||||
en|EN|eng|english) TALER_MON_LANG=en ;;
|
||||
*) echo "error: unknown language '$_lang' (use en or fr)" >&2; exit 2 ;;
|
||||
*) echo "error: unknown language '$_lang' (use en, fr, or de)" >&2; exit 2 ;;
|
||||
esac
|
||||
TALER_MON_LANG_SET=1
|
||||
export TALER_MON_LANG TALER_MON_LANG_SET
|
||||
|
|
@ -153,6 +242,7 @@ while [ $# -gt 0 ]; do
|
|||
i18n_init 2>/dev/null || true
|
||||
;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
--ver|--version|-V) print_version; exit 0 ;;
|
||||
-d|--domain)
|
||||
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
|
||||
apply_taler_domain "$2"
|
||||
|
|
@ -177,7 +267,7 @@ while [ $# -gt 0 ]; do
|
|||
CURRENCY_OVERRIDE="$2"; shift 2
|
||||
;;
|
||||
--no-probe) NO_PROBE=1; shift ;;
|
||||
urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|all|full) PHASES+=("$1"); shift ;;
|
||||
urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;;
|
||||
*)
|
||||
# bare domain shorthand: ./taler-monitoring.sh taler.net
|
||||
if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then
|
||||
|
|
@ -207,7 +297,7 @@ if [ "${LOCAL_STACK:-1}" != "1" ]; then
|
|||
SKIP_SSH=1
|
||||
fi
|
||||
|
||||
# Remote: no SSH; ATM withdraw ladder set in check_e2e (smaller notes)
|
||||
# Remote: no SSH; bank withdraw ladder set in check_e2e (smaller notes)
|
||||
if [ "${LOCAL_STACK}" != "1" ]; then
|
||||
E2E_FAKE_INCOMING=0
|
||||
E2E_REMOTE=1
|
||||
|
|
@ -221,14 +311,15 @@ export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE CHECK_LANDING
|
|||
export TALER_DOMAIN_FROM_CLI="${TALER_DOMAIN_FROM_CLI:-0}"
|
||||
export DOMAIN_SET="${DOMAIN_SET:-0}"
|
||||
export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE
|
||||
export E2E_FAKE_INCOMING E2E_REMOTE E2E_VARIABLE E2E_ATM_MAX
|
||||
export E2E_FAKE_INCOMING E2E_REMOTE E2E_VARIABLE E2E_ATM_MAX E2E_BANK_WITHDRAW_MAX
|
||||
export E2E_WITHDRAW_VALUES E2E_PAY_VALUES E2E_USE_TEMPLATES E2E_TEMPLATE_MAP
|
||||
export PAIVANA_PUBLIC E2E_PAIVANA E2E_PAIVANA_TEMPLATE E2E_PAIVANA_AMOUNT E2E_PAIVANA_INSTANCE
|
||||
export INSIDE_PROFILE INSIDE_SSH
|
||||
export INSIDE_BANK_CTR INSIDE_EXCHANGE_CTR INSIDE_MERCHANT_CTR
|
||||
export INSIDE_BANK_PORT INSIDE_EXCHANGE_PORT INSIDE_MERCHANT_PORT
|
||||
export INSIDE_DNS_BANK INSIDE_DNS_EXCHANGE INSIDE_DNS_MERCHANT
|
||||
# Ladder: withdraw then pay — 0 + random mids + max-1 + max (see check_goa_ladder.sh).
|
||||
# Amount ladder — check_amount_ladder.sh + ladder/lib_pay.sh + ladder/extract_rpubs.py
|
||||
# Phases: ladder|goa-ladder (classic), max-ladder|maxladder (max-search).
|
||||
# Defaults so set -u export is safe when vars were never set by caller.
|
||||
: "${LADDER_STEPS:=23}"
|
||||
: "${LADDER_MIN_AMOUNT:=0.000001}"
|
||||
|
|
@ -242,16 +333,23 @@ export INSIDE_DNS_BANK INSIDE_DNS_EXCHANGE INSIDE_DNS_MERCHANT
|
|||
: "${LADDER_INCLUDE_ZERO:=1}"
|
||||
: "${LADDER_INCLUDE_MAX:=1}"
|
||||
: "${LADDER_HIGH_FROM:=1000000}"
|
||||
: "${LADDER_HIGH_RUNGS:=12}"
|
||||
: "${LADDER_HIGH_RUNGS:=4}"
|
||||
: "${LADDER_LOAD:=1}"
|
||||
: "${LADDER_PAY:=1}"
|
||||
: "${LADDER_WITHDRAW_SCALE:=1.5}"
|
||||
: "${LADDER_PAY_SETTLE_ROUNDS:=6}"
|
||||
: "${LADDER_MODE:=classic}"
|
||||
: "${LADDER_MAX_PROBES:=32}"
|
||||
: "${LADDER_MAX_TOL_REL:=0.001}"
|
||||
: "${LADDER_MAX_SEED:=}"
|
||||
: "${LADDER_PAY_WITH_MAX:=0}"
|
||||
: "${LADDER_HIGH_RUNGS:=6}"
|
||||
export LADDER_STEPS LADDER_MIN_AMOUNT LADDER_CONFIRM_POLLS LADDER_MAX_RUNGS LADDER_TIMEOUT_S LADDER_REPORT_DIR
|
||||
export LADDER_SETTLE_ROUNDS LADDER_SETTLE_SLEEP EXP_PW_FILE EXP_USER
|
||||
export LADDER_MAX_AMOUNT LADDER_INCLUDE_ZERO LADDER_INCLUDE_MAX
|
||||
export LADDER_HIGH_FROM LADDER_HIGH_RUNGS LADDER_LOAD
|
||||
export LADDER_PAY LADDER_WITHDRAW_SCALE LADDER_PAY_SETTLE_ROUNDS
|
||||
export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_MAX_SEED LADDER_PAY_WITH_MAX
|
||||
export TALER_DOMAIN_APPLIED=1
|
||||
i18n_init 2>/dev/null || true
|
||||
# Optional: auth401 and other phases may honor CONTINUE_ON_ERROR
|
||||
|
|
@ -324,28 +422,81 @@ if [ "${#PHASES[@]}" -eq 0 ]; then
|
|||
PHASES=(urls)
|
||||
fi
|
||||
|
||||
# Rough expected check counts for progress % (override: PROGRESS_TOTAL=N).
|
||||
# Prefer slightly high — lib.sh rebalances if still short (never shows done>total).
|
||||
# Global expected check count for progress bar done/total (override: PROGRESS_TOTAL=N).
|
||||
# Auto estimate is env-aware (CHECK_BANK=0 etc.) and re-fit after each phase so stage
|
||||
# jobs do not sit at 42/139 then snap to 46/46. Phase summary() must NOT snap mid-run.
|
||||
# Host-agent monpages post-check is outside this process — never counted here.
|
||||
_progress_estimate_phase() {
|
||||
local p="$1" n=0 hosts=1
|
||||
hosts=$(printf '%s' "${MON_HOSTS:-x}" | wc -w)
|
||||
[ "$hosts" -lt 1 ] && hosts=1
|
||||
case "$p" in
|
||||
urls)
|
||||
# Core: exchange + merchant + webui + perf + terms (~28–40). Bank/landing optional.
|
||||
# Stage (CHECK_BANK=0 CHECK_LANDING=0) lands near ~34 numbered lines — keep close.
|
||||
n=28
|
||||
[ "${CHECK_BANK:-1}" != "0" ] && n=$((n + 24))
|
||||
[ "${CHECK_LANDING:-1}" != "0" ] && n=$((n + 18)) # intro + stats.json + qr
|
||||
n=$((n + 6)) # headroom (SPA / terms / privacy / alt paths)
|
||||
;;
|
||||
inside) n=30 ;;
|
||||
versions) n=35 ;;
|
||||
sanity) n=35 ;;
|
||||
server) n=20 ;;
|
||||
# e2e: bank withdraw ladder emits many INFO lines (coins before/after each note)
|
||||
e2e) n=240 ;;
|
||||
ladder|goa-ladder|max-ladder|maxladder) n=120 ;;
|
||||
auth401) n=70 ;;
|
||||
aptdeploy) n=40 ;;
|
||||
surface|ecosystem) n=120 ;;
|
||||
monpages|pages)
|
||||
# slash + bare (+ optional _err) per inventory host; GOA full suite ≈20 URLs
|
||||
case "${TALER_DOMAIN:-}" in
|
||||
*hacktivism*) n=24 ;;
|
||||
*lefrancpaysan*|*francpaysan*) n=$((hosts * 2 + 6)); [ "$n" -lt 12 ] && n=12 ;;
|
||||
*) n=$((hosts * 2 + 4)); [ "$n" -lt 8 ] && n=8 ;;
|
||||
esac
|
||||
[ "$n" -gt 48 ] && n=48
|
||||
;;
|
||||
mattermost) n=25 ;;
|
||||
mail) n=40 ;;
|
||||
# baseline ~8 + ladder rungs (default 23 × ~2 lines) + summary
|
||||
devtesting|franken|fake-franken|fake_franken)
|
||||
n=8
|
||||
if [ "${DEVTESTING_LADDER:-1}" != "0" ]; then
|
||||
_ls="${DEVTESTING_LADDER_STEPS:-23}"
|
||||
n=$((8 + _ls * 2 + 6))
|
||||
fi
|
||||
;;
|
||||
*) n=15 ;;
|
||||
esac
|
||||
printf '%s' "$n"
|
||||
}
|
||||
|
||||
# Recompute PROGRESS_TOTAL = done + estimate(remaining phases). Allows shrink after short phases.
|
||||
_progress_refit_remaining() {
|
||||
[ "${PROGRESS_TOTAL_AUTO:-0}" = "1" ] || return 0
|
||||
local rem=0 rp
|
||||
for rp in "$@"; do
|
||||
rem=$((rem + $(_progress_estimate_phase "$rp")))
|
||||
done
|
||||
if type _mon_state_load >/dev/null 2>&1; then
|
||||
_mon_state_load
|
||||
fi
|
||||
set_progress_total "$(( ${PROGRESS_DONE:-0} + rem ))"
|
||||
}
|
||||
|
||||
PROGRESS_TOTAL_AUTO=0
|
||||
if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then
|
||||
PROGRESS_TOTAL_AUTO=1
|
||||
_pt=0
|
||||
for p in "${PHASES[@]}"; do
|
||||
case "$p" in
|
||||
urls) _pt=$((_pt + 95)) ;; # +qr + webui SPA fingerprints
|
||||
inside) _pt=$((_pt + 30)) ;;
|
||||
versions) _pt=$((_pt + 25)) ;;
|
||||
sanity) _pt=$((_pt + 35)) ;;
|
||||
server) _pt=$((_pt + 20)) ;;
|
||||
# e2e: ATM ladder emits many INFO lines (coins before/after each note)
|
||||
e2e) _pt=$((_pt + 240)) ;;
|
||||
ladder|goa-ladder) _pt=$((_pt + 120)) ;;
|
||||
auth401) _pt=$((_pt + 70)) ;;
|
||||
aptdeploy) _pt=$((_pt + 20)) ;;
|
||||
surface|ecosystem) _pt=$((_pt + 80)) ;;
|
||||
esac
|
||||
_pt=$((_pt + $(_progress_estimate_phase "$p")))
|
||||
done
|
||||
set_progress_total "$_pt"
|
||||
unset _pt
|
||||
fi
|
||||
export PROGRESS_TOTAL_AUTO
|
||||
|
||||
# Whole-run wall clock (default 10 min). Override: RUN_TIMEOUT=0 (unlimited).
|
||||
: "${RUN_TIMEOUT:=600}"
|
||||
|
|
@ -408,6 +559,8 @@ elif [ "${LOCAL_STACK:-0}" = "1" ] && command -v podman >/dev/null 2>&1 \
|
|||
fi
|
||||
|
||||
printf 'target domain=%s\n' "${TALER_DOMAIN}"
|
||||
printf ' ui_lang %s\n' "${TALER_MON_LANG:-en}"
|
||||
printf ' locale %s\n' "${TALER_MON_LOCALE:-${TALER_DOMAIN_LOCALE:-?}}"
|
||||
printf ' bank %s\n' "$BANK_PUBLIC"
|
||||
printf ' exchange %s\n' "$EXCHANGE_PUBLIC"
|
||||
printf ' merchant %s\n' "$MERCHANT_PUBLIC"
|
||||
|
|
@ -429,15 +582,28 @@ if [ "${RUN_TIMEOUT:-0}" -eq 0 ]; then
|
|||
else
|
||||
printf ' run_timeout %ss wall clock (RUN_TIMEOUT=; 0=unlimited)\n' "$RUN_TIMEOUT"
|
||||
fi
|
||||
printf ' progress total≈%s (set PROGRESS_TOTAL= to override; PROGRESS_OFF=1 to hide)\n' "${PROGRESS_TOTAL:-0}"
|
||||
printf ' progress %s/%s (global done/total · set PROGRESS_TOTAL= to override; PROGRESS_OFF=1 to hide)\n' \
|
||||
"${PROGRESS_DONE:-0}" "${PROGRESS_TOTAL:-0}"
|
||||
unset _INSIDE_ACCESS_HINT
|
||||
|
||||
chmod +x "$ROOT"/check_*.sh 2>/dev/null || true
|
||||
|
||||
ec=0
|
||||
for p in "${PHASES[@]}"; do
|
||||
_phase_idx=0
|
||||
_n_phases=${#PHASES[@]}
|
||||
# Suppress all phase SUMMARY / progress standings when more than one phase;
|
||||
# parent prints one global SUMMARY after progress_finish (v1.13.10+).
|
||||
if [ "$_n_phases" -gt 1 ]; then
|
||||
export TALER_MON_MULTI_PHASE=1
|
||||
else
|
||||
export TALER_MON_MULTI_PHASE=0
|
||||
fi
|
||||
_MON_WAS_MULTI="${TALER_MON_MULTI_PHASE}"
|
||||
while [ "$_phase_idx" -lt "$_n_phases" ]; do
|
||||
p="${PHASES[$_phase_idx]}"
|
||||
if [ "$RUN_TIMED_OUT" = "1" ]; then
|
||||
RUN_SKIPPED_PHASES+=("$p")
|
||||
_phase_idx=$((_phase_idx + 1))
|
||||
continue
|
||||
fi
|
||||
left=$(mon_seconds_left)
|
||||
|
|
@ -445,6 +611,7 @@ for p in "${PHASES[@]}"; do
|
|||
RUN_TIMED_OUT=1
|
||||
RUN_TIMEOUT_AT_PHASE="${RUN_TIMEOUT_AT_PHASE:-$p}"
|
||||
RUN_SKIPPED_PHASES+=("$p")
|
||||
_phase_idx=$((_phase_idx + 1))
|
||||
continue
|
||||
fi
|
||||
case "$p" in
|
||||
|
|
@ -454,12 +621,41 @@ for p in "${PHASES[@]}"; do
|
|||
sanity) run_phase sanity "$ROOT/check_sanity.sh" || ec=1 ;;
|
||||
server) run_phase server "$ROOT/check_server.sh" || ec=1 ;;
|
||||
e2e) run_phase e2e "$ROOT/check_e2e.sh" || ec=1 ;;
|
||||
ladder|goa-ladder) run_phase ladder "$ROOT/check_goa_ladder.sh" || ec=1 ;;
|
||||
ladder|goa-ladder)
|
||||
run_phase ladder "$ROOT/check_amount_ladder.sh" || ec=1
|
||||
;;
|
||||
max-ladder|maxladder)
|
||||
# Feature: hunt highest mint+confirm withdrawable amount (any currency)
|
||||
LADDER_MODE=max
|
||||
export LADDER_MODE
|
||||
run_phase ladder "$ROOT/check_amount_ladder.sh" || ec=1
|
||||
;;
|
||||
auth401) run_phase auth401 "$ROOT/check_auth401.sh" || ec=1 ;;
|
||||
aptdeploy|apt-deploy|apt_src) run_phase aptdeploy "$ROOT/check_apt_deploy.sh" || ec=1 ;;
|
||||
surface|ecosystem) run_phase surface "$ROOT/check_surface.sh" || ec=1 ;;
|
||||
mattermost) run_phase mattermost "$ROOT/check_mattermost.sh" || ec=1 ;;
|
||||
mail) run_phase mail "$ROOT/check_mail.sh" || ec=1 ;;
|
||||
monpages|pages) run_phase monpages "$ROOT/check_monitoring_pages.sh" || ec=1 ;;
|
||||
devtesting|franken|fake-franken|fake_franken)
|
||||
run_phase devtesting "$ROOT/check_devtesting.sh" || ec=1
|
||||
;;
|
||||
esac
|
||||
# After each phase: refit total = actual done + estimate(remaining). Fixes
|
||||
# stage over-estimate (e.g. 42/139 while only ~4 checks left).
|
||||
if [ "${PROGRESS_TOTAL_AUTO:-0}" = "1" ]; then
|
||||
_rem_args=()
|
||||
_j=$((_phase_idx + 1))
|
||||
while [ "$_j" -lt "$_n_phases" ]; do
|
||||
_rem_args+=("${PHASES[$_j]}")
|
||||
_j=$((_j + 1))
|
||||
done
|
||||
_progress_refit_remaining "${_rem_args[@]+"${_rem_args[@]}"}"
|
||||
unset _rem_args _j
|
||||
fi
|
||||
_phase_idx=$((_phase_idx + 1))
|
||||
done
|
||||
unset _phase_idx _n_phases
|
||||
unset PROGRESS_FINAL TALER_MON_MULTI_PHASE 2>/dev/null || true
|
||||
|
||||
# Extraordinary run-budget failure: always report at end; HTML links top → here.
|
||||
if [ "$RUN_TIMED_OUT" = "1" ]; then
|
||||
|
|
@ -493,4 +689,20 @@ if [ "$RUN_TIMED_OUT" = "1" ]; then
|
|||
"$RUN_TIMEOUT" "$elapsed"
|
||||
fi
|
||||
|
||||
# Snap progress bar to real global count (done/total) once for the whole run.
|
||||
# Reload TALER_MON_STATE — check_*.sh update done/total + GLOBAL_*_N in a separate process.
|
||||
if type _mon_state_load >/dev/null 2>&1; then
|
||||
_mon_state_load
|
||||
fi
|
||||
if type progress_finish >/dev/null 2>&1; then
|
||||
printf '\n'
|
||||
progress_finish
|
||||
fi
|
||||
# Multi-phase: one global SUMMARY (OK/ERROR/WARN/INFO/BLOCK across all phases).
|
||||
# Single-phase already printed phase-local SUMMARY via check_*.sh.
|
||||
if [ "${_MON_WAS_MULTI:-0}" = "1" ] && type summary_global >/dev/null 2>&1; then
|
||||
summary_global || true
|
||||
fi
|
||||
unset _MON_WAS_MULTI 2>/dev/null || true
|
||||
|
||||
exit "$ec"
|
||||
|
|
|
|||
73
tests/test_normalize_taler_uri.sh
Executable file
73
tests/test_normalize_taler_uri.sh
Executable file
|
|
@ -0,0 +1,73 @@
|
|||
#!/usr/bin/env bash
|
||||
# Unit tests for normalize_taler_uri / taler_uri_has_default_port (lib.sh).
|
||||
# Run: ./tests/test_normalize_taler_uri.sh
|
||||
set -euo pipefail
|
||||
ROOT=$(cd "$(dirname "$0")/.." && pwd)
|
||||
# shellcheck source=../lib.sh
|
||||
source "$ROOT/lib.sh"
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
assert_eq() {
|
||||
local name="$1" got="$2" want="$3"
|
||||
if [ "$got" = "$want" ]; then
|
||||
pass=$((pass + 1))
|
||||
else
|
||||
echo "FAIL $name" >&2
|
||||
echo " got: $got" >&2
|
||||
echo " want: $want" >&2
|
||||
fail=$((fail + 1))
|
||||
fi
|
||||
}
|
||||
assert_true() {
|
||||
local name="$1"
|
||||
if "$@"; then
|
||||
# shift name
|
||||
:
|
||||
fi
|
||||
}
|
||||
|
||||
# --- strip :443 / :80 ---
|
||||
assert_eq "withdraw :443/" \
|
||||
"$(normalize_taler_uri 'taler://withdraw/bank.hacktivism.ch:443/taler-integration/abc-123')" \
|
||||
"taler://withdraw/bank.hacktivism.ch/taler-integration/abc-123"
|
||||
|
||||
assert_eq "withdraw :443 end" \
|
||||
"$(normalize_taler_uri 'taler://withdraw/bank.example:443')" \
|
||||
"taler://withdraw/bank.example"
|
||||
|
||||
assert_eq "pay :443" \
|
||||
"$(normalize_taler_uri 'taler://pay/taler.hacktivism.ch:443/instances/x/orders/y?c=z')" \
|
||||
"taler://pay/taler.hacktivism.ch/instances/x/orders/y?c=z"
|
||||
|
||||
assert_eq "pay-template :80" \
|
||||
"$(normalize_taler_uri 'taler://pay-template/shop.example:80/instances/i/t')" \
|
||||
"taler://pay-template/shop.example/instances/i/t"
|
||||
|
||||
assert_eq "already clean" \
|
||||
"$(normalize_taler_uri 'taler://withdraw/bank.hacktivism.ch/taler-integration/uuid')" \
|
||||
"taler://withdraw/bank.hacktivism.ch/taler-integration/uuid"
|
||||
|
||||
assert_eq "empty" "$(normalize_taler_uri '')" ""
|
||||
|
||||
# non-default port kept
|
||||
assert_eq "keep :8443" \
|
||||
"$(normalize_taler_uri 'taler://withdraw/bank.example:8443/taler-integration/x')" \
|
||||
"taler://withdraw/bank.example:8443/taler-integration/x"
|
||||
|
||||
# has_default_port detector
|
||||
if taler_uri_has_default_port 'taler://withdraw/h:443/p'; then
|
||||
pass=$((pass + 1))
|
||||
else
|
||||
echo "FAIL detector should match :443" >&2
|
||||
fail=$((fail + 1))
|
||||
fi
|
||||
if taler_uri_has_default_port 'taler://withdraw/h/p'; then
|
||||
echo "FAIL detector should not match clean URI" >&2
|
||||
fail=$((fail + 1))
|
||||
else
|
||||
pass=$((pass + 1))
|
||||
fi
|
||||
|
||||
echo "normalize_taler_uri tests: pass=$pass fail=$fail"
|
||||
[ "$fail" -eq 0 ]
|
||||
Loading…
Add table
Add a link
Reference in a new issue