# taler-monitoring — grouped test IDs

Every check line is numbered **`area.group-NN`** (two-digit group counter):

```text
┌ OK    ┐ www.exchange-01  exchange /config · HTTP 200
┌ ERROR ┐ e2e.pay-03       order create failed · HTTP 502
┌ BLOCKER┐ e2e.prereq-02   merchant secret missing
```

| Area | Phase script | Groups (examples) |
|------|--------------|-------------------|
| **www** | `check_urls.sh` | `exchange` `perf` `stats` `bank` `merchant` `paivana` `landing` |
| **inside** | `check_inside.sh` | `ssh` `bank` `exchange` `merchant` `caddy` `load` |
| **versions** | `check_versions.sh` | `outside` `inside` `compare` |
| **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
| **server** | `check_server.sh` | (flat `server-NN` or host groups) |
| **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `atm` `settle` `pay` `shop` `paivana` `dig` `report` |
| **ladder** | `check_goa_ladder.sh` | `plan` `load` `withdraw` `pay` `report` |

**Why groups:** flat `www-001`…`www-080` was hard to talk about.  
`www.bank-04` / `e2e.atm-02` pin the failure to a logical block.

Usage in scripts:

```bash
set_area www
set_group exchange   # → www.exchange-01, www.exchange-02, …
set_group bank       # counter resets → www.bank-01
```

`set_group` also prints a small group chip (`┌ www.exchange ┐`) so logs show section boundaries.  
Re-entering the same group later (e.g. exchange legal after perf) **continues** NN — `www.exchange-03`, not a second `-01`.  
Without `set_group`, IDs stay flat: `area-01`, `area-02`, …

Optional soft checks still consume a number when they WARN.  
Numbering follows **executed** checks (early skip may shift later NN inside the same group).

---

## www — public URLs (`./taler-monitoring.sh urls`)

| Group | Checks (typical) |
|-------|------------------|
| **www.exchange-** | `/config`, currency, **alt_unit_names**; `/keys` (+ alt soft); `/intro/`, `/`; **`/terms`**, **`/privacy`**, `/terms/` |
| **www.perf-** | outside-in RTT: bank/exchange/merchant `/config`, keys, webui, intro — ms; WARN ≥ `PERF_WARN_MS` (8000); ERROR ≥ `PERF_FAIL_MS` (20000) |
| **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three |
| **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json**; `/terms`, `/privacy` |
| **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** |
| **www.paivana-** | local GOA paywall front (redirect to template) |
| **www.landing-** | every own-stack link on bank/merchant/exchange intros; static assets; demo-withdraw / cross-links |

**Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. Local stack may require content needle (terms/privacy/FADP/GOA…).

**Performance rule:** Measured from the **monitoring runner** (public via Caddy), not container loopback. Latency (ms) on every perf line + **perf summary** (n / min / p50 / avg / max).

**Landing links rule:** Own-stack must be HTTP 200 (or redirect→200). External stores/docs soft WARN. Auto-account wallet link must be `taler://withdraw/HOST/taler-integration/…` (default ports stripped), never payto.

**alt_unit_names rule:** non-empty map including scale key `"0"`. Multi-currency merchant: follow each `exchanges[]` public `/config`.

---

## inside — koopa SSH (`./taler-monitoring.sh inside`)

| Group | Checks |
|-------|--------|
| **inside.ssh-** | ssh reachability / remote collect |
| **inside.bank-** | container, libeufin, postgres, local `/config`, nginx, DNS pins |
| **inside.exchange-** | container, httpd, wirewatch, aggregator, transfer, local keys, DNS |
| **inside.merchant-** | container, httpd, wirewatch, depositcheck, DNS |
| **inside.caddy-** | host reverse-proxy process |
| **inside.load-** | host loadavg + RAM; per-container RSS/CPU |

Remote lines `E|comp|LEVEL|key|detail` each become one numbered result under that component group.

SSH: `KOOPA_SSH` (default `koopa`), then `KOOPA_SSH_FALLBACKS` when LAN is unreachable.

---

## e2e — withdraw + pay (`./taler-monitoring.sh e2e`)

| Group | Checks |
|-------|--------|
| **e2e.prereq-** | wallet-cli, currency, budgets, secrets, public reachability |
| **e2e.load-** | host/container load snapshots (before / after ATM / after pay) |
| **e2e.bank-** | account, credit, withdraw ops |
| **e2e.wallet-** | exchange + ToS, wallet setup |
| **e2e.atm-** | ATM withdraw ladder rungs |
| **e2e.settle-** | wallet settlement wait / spendable balance |
| **e2e.pay-** | variable payments |
| **e2e.shop-** | GOA shop product catalog pays |
| **e2e.paivana-** | template paywall pay |
| **e2e.dig-** | coins-missing dig (inside + withdrawal-operation) |
| **e2e.report-** | final tallies |

When filing an issue, quote the full id + label, e.g.  
`e2e.atm-03  ATM withdraw 5 GOA · bank confirmed, wallet empty`.

---

## versions / sanity / ladder / server

| Area.group | Meaning |
|------------|---------|
| **versions.outside-** | deb.taler.net suite index |
| **versions.inside-** | packages in containers |
| **versions.compare-** | installed vs suite |
| **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component |
| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | GOA amount ladder |
| **server-** | SSH host ports / processes (flat unless grouped later) |

---

## Visual format (colour on)

```text
╔══════════════════════════════╗
║ www · public URLs · …        ║
╚══════════════════════════════╝
┌ www.exchange ┐
┌ OK    ┐ www.exchange-01  exchange /config · HTTP 200
┌ www.bank ┐
┌ OK    ┐ www.bank-01  bank /config · HTTP 200
┌ ERROR ┐ www.bank-04  bank /intro/auto-account.json · invalid withdraw

┌ ERRORS · failed checks ┐
  • www.bank-04 bank /intro/auto-account.json · invalid withdraw
totals: 40 OK, 1 ERROR, 2 WARN, 5 INFO
```

`NO_COLOR=1` or `CLICOLOR=0` disables boxes/colours (ASCII `[ OK ]` + `-- group --` headers).  
Always use `printf --` friendly plain headers when colour is off (leading `---` is not a printf option).