monitoring/merge: origin ladder+metrics + stage domains/profiles

scripts/taler-monitoring/README.md

@@ -135,6 +135,7 @@ E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e   # s
 | **versions** | `deb.taler.net` reachable (InRelease/Packages/pool `.deb`); containers can reach it + have apt source; installed Taler packages vs **trixie** |
 | **sanity** | bank · exchange · merchant sections (public + server) |
 | **e2e** | account → credit → withdraw → wallet → confirm → order → pay |
+| **ladder** | GOA withdraw: fixed **0** + low bands + **random high ranges** + fixed **max** |
 
 ```bash
 # package suite (default trixie on deb.taler.net)
@@ -150,8 +151,86 @@ E2E maps failures to blockers, e.g.:
 - `Alarm clock` during pay → usually missing `handle-uri --yes` or wrong pay URI (must be `…/instances/{inst}/{oid}/?c={token}` from merchant `taler_pay_uri`)
 - `insufficient balance` → withdraw incomplete  
 
+## Performance (urls phase)
+
+Outside-in HTTPS RTT for bank / exchange / merchant critical paths. Each probe prints
+`HTTP … · N ms` on the `[OK]`/`[WARN]`/`[ERROR]` line; end of the block prints
+`perf summary` (n / min / p50 / avg / max).
+
+| Env | Default | Meaning |
+|-----|---------|---------|
+| `PERF_WARN_MS` | `8000` | WARN if latency ≥ this |
+| `PERF_FAIL_MS` | `20000` | ERROR if latency ≥ this |
+| `PERF_CURL_TIMEOUT` | `25` | curl max seconds per probe |
+
+```bash
+PERF_WARN_MS=3000 PERF_FAIL_MS=10000 ./taler-monitoring.sh urls
+```
+
+## Wallet coins (e2e · ladder)
+
+At every relevant withdraw/pay step the wallet is snapped via `advanced dump-coins`.
+Amounts use exchange **`alt_unit_names`** (Kilo-GOA, Mega-GOA, …) with the base
+**GOA value in parentheses**:
+
+```text
+[INFO]  coins after-ATM — amount_circ=2 Kilo-GOA (GOA:2000)
+[INFO]  coins after-ATM — denoms_in_circ 1 Kilo-GOA (GOA:1000)×2 (=2 Kilo-GOA (GOA:2000))
+== ladder · rung 12  GOA:1000000 · 1 Mega-GOA (GOA:1000000) ==
+```
+
+Also: coin counts (in circulation / spent), status histogram, Δ vs previous snap.
+History TSV: `$METRICS_DIR/coins-history.tsv`. Map loaded from
+`${EXCHANGE_PUBLIC}/config` → `currency_specification.alt_unit_names`.
+
+### Final statistics dashboard (e2e · ladder)
+
+At the end of **e2e** and **ladder**, a boxed **FINAL STATISTICS** block prints:
+
+| Section | Content |
+|---------|---------|
+| **Coins** | in circulation / spent counts + amounts + denoms (alt names) |
+| **Money flow** | withdrawn events/amount, spent (paid), net (wd − spent) |
+| **Tendency** | coin/amount trend over snaps (↑↓→), slope, last history rows |
+| **Performance** | min / p50 / avg / max per phase; first-half vs second-half latency |
+| **Load** | host loadavg + mem before/after; per-container RSS/CPU; tendencies |
+
+Successful withdraw/pay steps append to `flow-withdrawn.txt` / `flow-spent.txt` (ladder also uses its TSVs).
+
+## Load / memory (inside · e2e · ladder)
+
+Host **loadavg**, **RAM used/avail**, and per-container **RSS / CPU / block I/O**
+(plus postgres DB sizes and role RSS: libeufin, exchange, merchant, …) are
+snapshotted via SSH whenever the stack is under load:
+
+| Phase | When |
+|-------|------|
+| **inside** | end of container status |
+| **e2e** | before work · after ATM withdraws · after payments · after shop · e2e-end (+ delta) |
+| **ladder** | before first rung · after last rung (+ delta + timing overall) |
+
+```bash
+# LAN down? force WAN DNAT host
+KOOPA_SSH=koopa-external ./taler-monitoring.sh inside
+# or rely on automatic fallback (default KOOPA_SSH_FALLBACKS=koopa-external)
+
+METRICS_LOAD=0 ./taler-monitoring.sh e2e   # skip host probes
+LADDER_LOAD=0 ./taler-monitoring.sh ladder
+```
+
+| Env | Default | Meaning |
+|-----|---------|---------|
+| `KOOPA_SSH` | `koopa` | primary SSH host |
+| `KOOPA_SSH_FALLBACKS` | `koopa-external` | tried if primary fails |
+| `METRICS_LOAD` | `1` | `0` = skip all host/container load probes |
+| `LADDER_LOAD` | `1` | `0` = skip load in ladder only |
+| `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws |
+| `LADDER_WITHDRAW_SCALE` | `1.5` | withdraw mids / pay mids ratio (funding headroom) |
+| `LADDER_STEPS` | `23` | rungs for both withdraw and pay (0 + mids + max−1 + max) |
+| `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python |
+
 ## Needs
 
-- SSH `koopa` (inside/sanity server bits)
+- SSH `koopa` or `koopa-external` (inside / load / sanity server bits)
 - secrets under `koopa-admin-secrets/...` for e2e
 - `taler-wallet-cli` for e2e