hernani/taler-monitoring
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

taler-monitoring v1.0: standalone suite

Browse source
This commit is contained in:
Hernâni Marques 2026-07-18 13:55:49 +02:00
commit a2afe690b7
No known key found for this signature in database
GPG key ID: CB5738652768F7E9
73 changed files with 18268 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

9
.gitignore vendored Normal file
View file

@ -0,0 +1,9 @@
secrets.env
__pycache__/
*.pyc
android-test/apks/
android-test/out*/
android-test/artifacts/
site-gen/__pycache__/
*.log
.DS_Store

228
CLI-AUTOMATION-NOTES.md Normal file
View file

@ -0,0 +1,228 @@
# CLI automation notes (`taler-wallet-cli` + monitoring)
Living list of **recurring issues** seen while driving GOA (hacktivism) and
stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring
e2e/ladder). Ordered roughly by **how often they hurt automation** and how
useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be.
Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md).
Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`.
Legend:
| Tag | Meaning |
|-----|---------|
| **cli** | wallet-cli UX / flags / hang behaviour |
| **core** | wallet-core protocol / state machine |
| **bank** | libeufin-bank / integration API |
| **ops** | secrets, paths, multi-host |
| **doc** | missing or wrong documentation |
---
## 1. `run-until-done` is unusable for automation
| | |
|--|--|
| **Seen** | e2e, ladder, Android notes, macOS comments: *hangs / banned in monitoring* |
| **Area** | **cli** / **core** |
| **Problem** | `taler-wallet-cli run-until-done` (or long shepherd runs) often **never returns** or blocks CI-length timeouts. Monitoring **forbids** it and polls balance + bank `transfer_done` instead. |
| **Workaround** | `advanced serve` + socket (`WSOCK`) or pure poll loops (`LADDER_SETTLE_*`, e2e settle wait). |
| **Wanted for wallet-cli** | Bounded wait: e.g. `run-until-done --timeout=Ns --exit-on=withdrawn|paid|idle`; stable non-zero exit on timeout; progress on stdout/JSON. |
---
## 2. Withdraw completion is not observable from CLI alone
| | |
|--|--|
| **Seen** | e2e “coins missing after withdraw”; ladder `OK_BANK` vs wallet avail |
| **Area** | **cli** / **core** / **bank** |
| **Problem** | After `accept-uri` + bank confirm, coins may lag (wirewatch). CLI has no single “withdraw settled” command with clear success/failure; automation reimplements status from bank integration JSON + balance polls. |
| **Workaround** | Poll `balance` + `…/withdrawal-operation/{id}` (`status`, `transfer_done`, `selected_reserve_pub`). |
| **Wanted** | `wallet-cli withdrawals wait --id=… --timeout=` or JSON event stream: `selected → confirmed → coins-available`. |
---
## 3. `reserve_pub` / force-select is fragile (5114)
| | |
|--|--|
| **Seen** | ladder force-select, e2e confirm path, Android spinner issues |
| **Area** | **cli** / **core** / **bank** |
| **Problem** | Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). Automation scrapes accept/tx dumps for candidates. |
| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. |
| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. |
---
## 4. Exchange not auto-known for bank deep links / withdraw URIs
| | |
|--|--|
| **Seen** | Android endless spinner; GOA `bank.hacktivism.ch` withdraw; fixed in app branch `fix/bank-withdraw-auto-exchange` |
| **Area** | **core** / **cli** |
| **Problem** | Withdraw URI names an exchange the wallet has never added → stuck loading / empty exchange list. CLI requires explicit `exchanges add` + `update` + `accept-tos` every fresh DB. |
| **Workaround** | e2e always adds exchange before ATM ladder; Android needs ensureExchange-style fix. |
| **Wanted** | CLI/core: on `accept-uri` withdraw, **auto-add** exchange from URI/bank details (`allowCompletion`), then surface ToS if needed in one command. |
---
## 5. Default ports in URIs (`:443` / `:80`) break round-trips
| | |
|--|--|
| **Seen** | ladder/e2e strip ports; QR checks; mint URIs with `:443` |
| **Area** | **cli** / **bank** / **doc** |
| **Problem** | Some bank/wallet outputs include `taler://…host:443/…`. QR validators and some clients reject or double-normalize inconsistently. |
| **Workaround** | `sed 's/:443\//\//g'` everywhere in automation. |
| **Wanted** | Canonical form without default ports in **all** wallet-cli and bank integration outputs; document as invariant. |
---
## 6. No first-class “demo withdraw” / explorer mint in wallet-cli
| | |
|--|--|
| **Seen** | e2e uses bank admin + ATM create; ladder/gui-chain reimplement explorer mint in Python |
| **Area** | **cli** / **ops** |
| **Problem** | Automation always hand-rolls Basic auth → token → `POST …/withdrawals` → `taler_withdraw_uri`. Easy to get wrong (auth header, amount currency). |
| **Workaround** | Shared Python in `goa-chain` / ladder / e2e; secrets files. |
| **Wanted** | Optional helper: `wallet-cli testing mint-withdraw --bank=URL --user=explorer --password-file=… --amount=GOA:10` (or bank-side only tool in libeufin) emitting a clean URI. |
---
## 7. Long-lived wallet process required for pay/withdraw progress
| | |
|--|--|
| **Seen** | e2e `advanced serve` / shepherd socket |
| **Area** | **cli** |
| **Problem** | One-shot CLI invocations do not keep background work alive; without serve/shepherd, withdraw/pay may stall mid-flight. |
| **Workaround** | Start serve once per e2e run; route `wcli` through the socket. |
| **Wanted** | Documented, supported “session mode”: start/stop serve; or each mutating command optionally `--background-until=…` with timeout. |
---
## 8. ToS accept is a separate step that fails silently or blocks
| | |
|--|--|
| **Seen** | e2e `exchanges accept-tos`; ladder after add; Android GUI ToS screens |
| **Area** | **cli** / **core** |
| **Problem** | Fresh exchange → operations need ToS; CLI must call `accept-tos` explicitly. Failures are easy to miss in multi-step scripts. |
| **Workaround** | Always `accept-tos` after `update` in e2e/ladder. |
| **Wanted** | `accept-uri --accept-tos` or auto-prompt with non-interactive `--yes` that covers exchange ToS for that URI’s exchange. |
---
## 9. Pay template / public order path is merchant-shaped, not CLI-shaped
| | |
|--|--|
| **Seen** | e2e shop templates, stage farmer shops, ladder private orders |
| **Area** | **cli** / **doc** |
| **Problem** | Wallet-cli pays via `handle-uri` on `taler://pay/…`; creating the order is always custom curl (template POST or private order + token). No unified “pay this amount to instance” for public templates. |
| **Workaround** | Monitoring builds URI externally then `handle-uri --yes`. |
| **Wanted** | Documented recipe only, or `wallet-cli testing pay-template --base=… --instance=… --id=…` for demos. |
---
## 10. Amount / currency parsing edge cases
| | |
|--|--|
| **Seen** | ladder min denom 0.01 vs GOA 1e-6; zero withdraw rejected (HTTP 409 / amount too low); max wire ceilings |
| **Area** | **cli** / **bank** |
| **Problem** | `CURRENCY:0` and sub-min amounts fail differently per stack; CLI error strings are not machine-stable. |
| **Workaround** | Soft-skip zero rung; clamp ladder min to exchange denoms / bank max_wire. |
| **Wanted** | Structured errors (`AMOUNT_too_small`, `currency_unknown`) in JSON mode; `wallet-cli amount validate --exchange=`. |
---
## 11. Finding the right `taler-wallet-cli` binary
| | |
|--|--|
| **Seen** | `find_wallet_cli` in `lib.sh`; hardcoded laptop paths; wrapper vs `.mjs` |
| **Area** | **ops** / **cli** |
| **Problem** | Debian package is a shell wrapper; some tools need `node …/taler-wallet-cli.mjs`. Hardcoded `/Users/…` paths break other hosts. |
| **Workaround** | `find_wallet_cli` search list; `WALLET_CLI=` override. |
| **Wanted** | Single install story: `wallet-cli --version` JSON with path + libversion; no need to pass `.mjs` to node by hand. |
---
## 12. Secrets and multi-stack confusion
| | |
|--|--|
| **Seen** | explorer vs admin password; GOA secrets used on stage; ladder EXP_PW |
| **Area** | **ops** |
| **Problem** | CLI does not know “which stack”; wrong password → 401 mid-ladder. Not a wallet-cli bug, but every CLI automation hits it. |
| **Workaround** | `SECRETS_ROOT`, stage SSH, `STACK=` profiles. |
| **Wanted** | Optional `~/.config/taler/stacks.d/goa.env` convention documented next to wallet-cli; still no secrets in repo. |
---
## 13. No stable machine-readable “step result” for scripts
| | |
|--|--|
| **Seen** | e2e greps accept output; ladder scrapes JSON from mixed stdout |
| **Area** | **cli** |
| **Problem** | Human logs + occasional JSON blobs; hard to parse reliably. |
| **Workaround** | Python scrapers, temp files, `tee`. |
| **Wanted** | Global `--json` / `--ndjson` for all commands; one object per completed operation with `ok`, `op`, `ids`, `amounts`. |
---
## 14. Pay settlement wait is symmetric to withdraw pain
| | |
|--|--|
| **Seen** | e2e pay settle loops; ladder pay settle rounds |
| **Area** | **cli** / **core** |
| **Problem** | After `handle-uri` pay, success is “order paid” / balance drop / merchant order status — not one CLI wait. |
| **Workaround** | Short poll loops; never run-until-done. |
| **Wanted** | Same as §1–2: bounded wait on transaction id / order id. |
---
## Priority shortlist for `taler-wallet-cli` / core
If only a few changes land, these unlock the most automation:
1. **Bounded `run-until-done` / wait-for-state** (§1, §2, §14)
2. **Structured JSON on every command** (§13)
3. **Auto-add exchange + ToS on accept-uri** (§4, §8)
4. **Canonical URIs without :443** (§5)
5. **Emit reserve_pub for last withdraw** (§3)
---
## How we work around today (monitoring)
| Issue | Monitoring behaviour |
|-------|----------------------|
| run-until-done | Disabled; balance + bank poll |
| serve | Optional long-lived socket in e2e |
| exchange | Explicit add/update/accept-tos |
| ladder explorer | `read_secret` / EXP_PW_FILE |
| stage maxima | bank `max_wire` + keys min denom |
| force-select | multi-rpub try; soft-skip confirm |
---
## How to add an issue
```markdown
### N. short title
| | |
|--|--|
| **Seen** | where / which stack |
| **Area** | cli / core / bank / ops / doc |
| **Problem** | … |
| **Workaround** | … |
| **Wanted** | concrete CLI/core behaviour |
```

70
DEPENDENCIES.md Normal file
View file

@ -0,0 +1,70 @@
# taler-monitoring — host package dependencies
## koopa (openSUSE Tumbleweed, user `hernani`)
### Required (host-agent / GOA primary: `urls inside versions`)
| Binary | Package (zypper) | Role |
|--------|------------------|------|
| `bash` | (base) | runner |
| `curl` | `curl` | HTTPS/API probes |
| `python3` | `python313-base` (provides `/usr/bin/python3`) | JSON, HTML site-gen, metrics |
| `podman` | `podman` | **inside**: `podman exec` into taler-hacktivism* |
| `git` | `git` | pin commit SHA in monitoring HTML footer |
| `rsync` | `rsync` | optional: deploy/sync suite |
| `timeout` / `mktemp` / `getent` | coreutils / glibc | helpers |
**Status on koopa (2026-07-18):** all of the above present; rootless podman owns GOA containers.
### Recommended (full `urls` QR group)
| Binary | Package | Role |
|--------|---------|------|
| `qrencode` | **`qrencode`** | encode taler:// / payto:// → PNG |
| `zbarimg` | **`zbar`** | decode PNG; exact payload match |
Without these, QR checks **WARN** and skip encode/decode (`apt install` messages in code are Debian-oriented; on Tumbleweed use zypper).
```bash
# as root on koopa
zypper refresh
zypper in qrencode zbar
# verify as hernani:
command -v qrencode zbarimg
```
### Optional (not needed for host-agent default)
| Binary / tool | When |
|---------------|------|
| `jq` | nice-to-have; suite uses python3 for JSON |
| `dig` / `bind-utils` | debug DNS; not required for default phases |
| `ssh` | only for access mode **ssh** (laptop → `KOOPA_SSH` / stage `INSIDE_SSH`); not needed for **host-podman** |
| `node` / `pnpm` + wallet-cli | **e2e** / ladder only (+ secrets) |
| `secrets.env` / admin passwords | e2e only — never system packages |
### systemd user (no extra packages)
- `systemd` user session + **`loginctl enable-linger hernani`** (already **Linger=yes** on koopa)
- Units: `host-agent/*.service|path|timer`
### Caddy HTML publish (separate from monitoring run)
- Host Caddy already installed (root service).
- Writing `/var/www/monitoring-sites`: either root `rsync` or directory owned/writable by `hernani` (you set this up).
---
## firecuda-external (macOS, outside-only timer)
| Need | Notes |
|------|--------|
| `bash`, `curl`, `python3`, `rsync`, `git` (optional) | usually Homebrew / Xcode CLT |
| **No** `podman` | phases are `urls versions` only (`SKIP_SSH=1`) |
| `qrencode` / `zbarimg` | optional for QR; `brew install qrencode zbar` if desired |
---
## Franc Paysan hosts (francpaysan / stagepaysan)
Document package lists under **`francpaysan-admin-log`** when those host-agents are installed (same idea: curl, python3, podman if inside, qrencode/zbar for full urls).

362
README.md Normal file
View file

@ -0,0 +1,362 @@
# taler-monitoring
Standalone repository: https://git.hacktivism.ch/hernani/taler-monitoring (tag **v1.0**).
Formerly: `koopa-admin-log/scripts/taler-monitoring/`.
# taler-monitoring
Host packages (koopa / runners): **[DEPENDENCIES.md](./DEPENDENCIES.md)**.
Host-agents (GOA + FP; shared report pipeline): **[host-agent/README.md](./host-agent/README.md)**.
Public HTML site-gen (same timeout/log/HTML defaults): **[site-gen/README.md](./site-gen/README.md)**.
FP install notes: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
Report for the **GOA** stack with boxed severity badges and **grouped test IDs**
`area.group-NN` (e.g. `www.exchange-01`, `e2e.pay-03`):
| Tag | Meaning |
|-----|---------|
| `┌ OK ┐` | check passed |
| `┌ INFO ┐` | status / note |
| `┌ WARN ┐` | degraded but maybe not fatal |
| `┌ ERROR ┐` | component problem |
| `┌ BLOCKER ┐` | **withdraw/pay path cannot complete** because of this |
IDs reset per **group** inside an area — so “too many www tests” become
`www.exchange-*`, `www.bank-*`, `www.landing-*`, etc.
Catalog: **[TESTS.md](./TESTS.md)**.
Android / tooling: **[VERSIONS.md](./VERSIONS.md)**.
GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**.
CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**.
```text
┌ OK ┐ www.exchange-01 exchange /config · HTTP 200
┌ BLOCKER┐ e2e.prereq-02 merchant HTTP 502
```
End of each phase: totals + list of **BLOCKERS** and **ERRORS** (ids included).
## Secrets (local GOA e2e)
**Do not commit passwords.** Canonical trees: `~/src/koopa/koopa-admin-log` +
sibling `~/src/koopa/koopa-admin-secrets`.
### Local file (recommended on laptop)
```bash
cd ~/src/koopa/src/taler-monitoring
cp secrets.env.example secrets.env # gitignored
$EDITOR secrets.env
./taler-monitoring.sh e2e
```
See **[secrets.env.example](./secrets.env.example)** for every variable that
may appear (admin pass, merchant token, SECRETS_ROOT, progress knobs).
### Search order
1. Already-exported env (`E2E_BANK_ADMIN_PASS`, …)
2. `secrets.env` next to this suite (`MONITORING_SECRETS_ENV=` override)
3. `SECRETS_ROOT` / `KOOPA_ADMIN_SECRETS` file tree
4. Sibling `../koopa-admin-secrets/koopa/host-root`
5. `$HOME/src/koopa/koopa-admin-secrets/koopa/host-root`
6. `$HOME/.config/taler-landing/bank-admin-password.txt`
7. SSH koopa `/root/…` or bank container
### Line IDs + progress
Each check prints **global** and **grouped** ids:
```text
┌ OK ┐ #014 www.exchange-03 exchange /config · HTTP 200
┌ WARN ┐ #015 www.stats-02 stats age soft · yellow badge (not red)
┌ 42% ┐ ████████░░░░░░░░░░░░░░░░░░░░ 14/55
```
- `#014` — run-wide counter (whole `./taler-monitoring.sh` invocation)
- `www.exchange-03` — area.group-NN
- **Colours (left badge + label):**
- **OK** green · **INFO** blue/cyan · **WARN** yellow (never red)
- **ERROR** red · **BLOCKER** magenta
- Progress: badge shows **percent** (not “PROG”); bar fills **left → right**;
`done/total` on the right. Estimate auto-grows if short. Override:
`PROGRESS_TOTAL=N` / `PROGRESS_OFF=1`
- **QR (urls):** harvest `taler://` / `payto://` (+ app store `data-qr-url`) from landings and
`demo-withdraw.json` / `auto-account.json` → validate form → `qrencode` PNG → `zbarimg`
decode must match. Requires packages **`qrencode`** + **`zbar-tools`**.
## Commands
```bash
# Local GOA stack (may use SSH for inside/e2e)
./taler-monitoring.sh # urls + inside + versions + e2e
./taler-monitoring.sh inside # containers on koopa
./taler-monitoring.sh versions # deb.taler.net + package versions vs trixie
./taler-monitoring.sh sanity
./taler-monitoring.sh e2e
# FrancPaysan STAGE (TESTPAYSAN) — like hacktivism breadth:
# default phases: urls + inside + versions + e2e
./taler-monitoring.sh -d stage.lefrancpaysan.ch
./taler-monitoring.sh -d stage.lefrancpaysan.ch inside # stagepaysan only
# inside: low-priv SSH francpaysan-stage-user (User stagepaysan) → podman
# containers stage-lfp-{bank,exchange-ansible,merchant} · ports 9032/9031/9030
# stats: outside-in public …/intro/stats.json (age via generated_at_unix)
# Admin credit for e2e: E2E_BANK_ADMIN_PASS=… or francpaysan-host secrets
# (bank-admin-password.txt under /mnt/data/stagepaysan/bank/secrets/)
# Override: INSIDE_SSH=… INSIDE_PROFILE=stage-lfp
# Other domains — public HTTPS only (pass e2e/inside explicitly)
./taler-monitoring.sh -d taler.net
./taler-monitoring.sh --domain taler-ops.ch
./taler-monitoring.sh -d my.taler-ops.ch urls
./taler-monitoring.sh -d demo.taler.net urls
./taler-monitoring.sh taler.net # bare domain = same as -d
# Explicit endpoints (any mix; override profile or skip -d)
./taler-monitoring.sh --bank https://bank.demo.taler.net \
--exchange https://exchange.demo.taler.net \
--merchant https://backend.demo.taler.net --currency KUDOS urls
# Landing stats.json freshness (outside-in; age via generated_at_unix)
STATS_STALE_SECS=900 STATS_FAIL_SECS=3600 ./taler-monitoring.sh -d stage.lefrancpaysan.ch urls
# --- Full load (GOA / hacktivism) ---
# all = urls + inside + versions + sanity + e2e
# full = all + server + ladder + auth401 (long; needs secrets.env)
./taler-monitoring.sh -d hacktivism.ch full
./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401
# SPA pin after selfbuild-webui.sh (container):
EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \
WEBUI_OVERLAY_DENY='c778af04|master-11340' \
./taler-monitoring.sh -d hacktivism.ch urls
# Only 401 matrix (may create throwaway instance):
./taler-monitoring.sh -d hacktivism.ch auth401
# Stage FrancPaysan — durable mon401 account (password in francpaysan-secrets):
./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401
# secrets: ~/francpaysan-secrets/stage/instances/mon401.password
# ~/francpaysan-secrets/stage/default-instance-token.txt
# force throwaway create instead: AUTH401_PREFER_DURABLE=0 AUTH401_CREATE=1 …
```
### Landing `stats.json` (www.stats)
Public `GET …/intro/stats.json` must expose timestamps so monitoring can tell
whether the collector is still updating:
| Field | Role |
|-------|------|
| `generated_at_unix` | epoch seconds — **preferred** for age = now − unix |
| `generated_at` | ISO-8601 (fallback if unix missing) |
| `generated_at_human` | UI “updated” / footer string |
Also checks display payloads (bank-shape `withdraws` / accounts, or exchange-db /
merchant-db counters). Soft WARN if age > `STATS_STALE_SECS` (default 900);
ERROR if age > `STATS_FAIL_SECS` (default 3600; set `0` to disable).
Stage shared feed: same `gen_unix` on bank + exchange + monnaies.
## Domain profiles (`domains.conf`)
Each monitored stack must say **what is bank, exchange, and merchant-backend**.
Edit **[domains.conf](./domains.conf)** (or set `TALER_DOMAINS_CONF=`):
```text
# name bank exchange merchant-backend currency local [canonical]
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0
```
| Field | Meaning |
|-------|---------|
| **name** | `-d NAME` / `TALER_DOMAIN` match (add alias lines as needed) |
| **bank** | libeufin/bank public host or `https://…` |
| **exchange** | exchange public host or URL |
| **merchant** | merchant **backend** (SPA/API), e.g. `my.taler-ops.ch` |
| **currency** | `GOA` / `KUDOS` / `CHF` / … |
| **local** | `1` = koopa SSH stack; `0` = public only |
| **landing** | `1` = check `/intro` landings + assets; `0` = skip (TOPS / mytops) |
| **canonical** | optional label after alias (e.g. `my.taler-ops.ch` → `taler-ops.ch`) |
Then:
```bash
./taler-monitoring.sh -d taler-ops.ch urls
```
Overrides (always win after profile load):
```bash
./taler-monitoring.sh -d taler-ops.ch \
--merchant https://my.taler-ops.ch \
--exchange https://exchange.taler-ops.ch \
--bank https://bank.taler-ops.ch urls
```
Helpers in `lib.sh`: `set_taler_stack`, `load_domain_profile`, `apply_taler_domain`.
### Built-in profiles (see `domains.conf`)
| Domain / alias | Bank | Exchange | Merchant | Currency |
|----------------|------|----------|----------|----------|
| `hacktivism.ch` / `koopa` | bank.hacktivism.ch | exchange.hacktivism.ch | taler.hacktivism.ch | GOA |
| `taler.net` / `demo.taler.net` | bank.demo.taler.net | exchange.demo.taler.net | backend.demo.taler.net | KUDOS |
| `test.taler.net` | bank.test… | exchange.test… | backend.test… | **TESTKUDOS** · no landings |
| `lefrancpaysan.ch` | bank.… | exchange.… | **monnaie.…** (merchant) | report-only · no landings |
| **`stage.lefrancpaysan.ch`** | stage.bank.… | stage.exchange.… | **stage.monnaie.…** | **TESTPAYSAN** · landings · **inside via stagepaysan** |
| `taler-ops.ch` / `my.taler-ops.ch` | bank.taler-ops.ch | exchange.taler-ops.ch | **my.taler-ops.ch** | CHF · **no landings** |
| `stage.taler-ops.ch` | bank.stage… | exchange.stage… | **my.stage…** | CHF · **no landings** |
| unknown | bank.DOMAIN | exchange.DOMAIN | my.DOMAIN then probe | any · no landings |
**Landings:** GOA, stage LFP (`TESTPAYSAN`), and optionally demo use public `/intro` pages. **taler-ops.ch does not** — `CHECK_LANDING=0` skips intro crawl.
**SSH:**
| Stack | Host alias | User | Scope |
|-------|------------|------|--------|
| koopa / hacktivism | `koopa` | hernani | full host + containers |
| stage LFP | `francpaysan-stage-user` | **stagepaysan** | own podman only (no sudo / no prod data) |
Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC.
### E2E amounts (variable)
| | Local (koopa) | Remote |
|--|---------------|--------|
| **ATM withdraw** | 20 · 50 · 100 · 200 · **4200** (paivana) | 10 · 20 · 50 |
| **Pay ladder** | 0.01 … 10 | 0.01 … 1 |
| **Paivana** | HTTP paywall + template pay **GOA:4200** (`goa-shop` / `paivana`) | skipped |
```bash
./taler-monitoring.sh e2e
./taler-monitoring.sh -d taler.net urls e2e
# Amount ladder (withdraw 0 → … → max, then pay): same script for GOA + stage
./taler-monitoring.sh ladder # GOA / current domain
./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima
# Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01).
# Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=).
# Override: LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 …
# customize e2e:
E2E_WITHDRAW_VALUES="20 50 100" E2E_PAY_VALUES="0.05 1 5" ./taler-monitoring.sh e2e
E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # single fixed
# GOA shop catalog (local hacktivism): full list in E2E_SHOP_PRODUCTS; each run
# shuffles and pays E2E_SHOP_PICK_N products (default 2).
# E2E_SHOP_PRODUCTS lines: id|Product name|GOA:amount
# E2E_SHOP_PICK_N=2
# (landing QR = taler://pay-template/…/{id}; popup = live taler://pay after POST templates/{id})
# Paivana paywall (local GOA only):
# E2E_PAIVANA=1 (default) PAIVANA_PUBLIC=https://paivana.hacktivism.ch
# E2E_PAIVANA_TEMPLATE=paivana E2E_PAIVANA_AMOUNT=GOA:4200 E2E_PAIVANA_INSTANCE=goa-shop
# E2E_PAIVANA=0 # skip
# remote secrets:
# E2E_BANK_ADMIN_PASS=… E2E_MERCHANT_TOKEN=…
```
## Phases
| Phase | What |
|-------|------|
| **inside** | SSH koopa: processes, postgres, local /config,/keys, wirewatch, recent log ERRORs |
| **versions** | `deb.taler.net` reachable (InRelease/Packages/pool `.deb`); containers can reach it + have apt source; installed Taler packages vs **trixie** |
| **sanity** | bank · exchange · merchant sections (public + server) |
| **e2e** | account → credit → withdraw → wallet → confirm → order → pay |
| **ladder** | GOA withdraw: fixed **0** + low bands + **random high ranges** + fixed **max** |
```bash
# package suite (default trixie on deb.taler.net)
TALER_APT_SUITE=trixie ./taler-monitoring.sh versions
TALER_PKG_BEHIND=error ./taler-monitoring.sh versions # any behind = ERROR
```
E2E maps failures to blockers, e.g.:
- `bank-confirm HTTP 409` → wallet did not select exchange
- `no GOA balance` → wirewatch / transfer
- `create order failed` → merchant auth/instance
- `Alarm clock` during pay → usually missing `handle-uri --yes` or wrong pay URI (must be `…/instances/{inst}/{oid}/?c={token}` from merchant `taler_pay_uri`)
- `insufficient balance` → withdraw incomplete
## Performance (urls phase)
Outside-in HTTPS RTT for bank / exchange / merchant critical paths. Each probe prints
`HTTP … · N ms` on the `[OK]`/`[WARN]`/`[ERROR]` line; end of the block prints
`perf summary` (n / min / p50 / avg / max).
| Env | Default | Meaning |
|-----|---------|---------|
| `PERF_WARN_MS` | `8000` | WARN if latency ≥ this |
| `PERF_FAIL_MS` | `20000` | ERROR if latency ≥ this |
| `PERF_CURL_TIMEOUT` | `25` | curl max seconds per probe |
```bash
PERF_WARN_MS=3000 PERF_FAIL_MS=10000 ./taler-monitoring.sh urls
```
## Wallet coins (e2e · ladder)
At every relevant withdraw/pay step the wallet is snapped via `advanced dump-coins`.
Amounts use exchange **`alt_unit_names`** (Kilo-GOA, Mega-GOA, …) with the base
**GOA value in parentheses**:
```text
[INFO] coins after-ATM — amount_circ=2 Kilo-GOA (GOA:2000)
[INFO] coins after-ATM — denoms_in_circ 1 Kilo-GOA (GOA:1000)×2 (=2 Kilo-GOA (GOA:2000))
== ladder · rung 12 GOA:1000000 · 1 Mega-GOA (GOA:1000000) ==
```
Also: coin counts (in circulation / spent), status histogram, Δ vs previous snap.
History TSV: `$METRICS_DIR/coins-history.tsv`. Map loaded from
`${EXCHANGE_PUBLIC}/config` → `currency_specification.alt_unit_names`.
### Final statistics dashboard (e2e · ladder)
At the end of **e2e** and **ladder**, a boxed **FINAL STATISTICS** block prints:
| Section | Content |
|---------|---------|
| **Coins** | in circulation / spent counts + amounts + denoms (alt names) |
| **Money flow** | withdrawn events/amount, spent (paid), net (wd − spent) |
| **Tendency** | coin/amount trend over snaps (↑↓→), slope, last history rows |
| **Performance** | min / p50 / avg / max per phase; first-half vs second-half latency |
| **Load** | host loadavg + mem before/after; per-container RSS/CPU; tendencies |
Successful withdraw/pay steps append to `flow-withdrawn.txt` / `flow-spent.txt` (ladder also uses its TSVs).
## Load / memory (inside · e2e · ladder)
Host **loadavg**, **RAM used/avail**, and per-container **RSS / CPU / block I/O**
(plus postgres DB sizes and role RSS: libeufin, exchange, merchant, …) are
snapshotted via SSH whenever the stack is under load:
| Phase | When |
|-------|------|
| **inside** | end of container status |
| **e2e** | before work · after ATM withdraws · after payments · after shop · e2e-end (+ delta) |
| **ladder** | before first rung · after last rung (+ delta + timing overall) |
```bash
# LAN down? force WAN DNAT host
KOOPA_SSH=koopa-external ./taler-monitoring.sh inside
# or rely on automatic fallback (default KOOPA_SSH_FALLBACKS=koopa-external)
METRICS_LOAD=0 ./taler-monitoring.sh e2e # skip host probes
LADDER_LOAD=0 ./taler-monitoring.sh ladder
```
| Env | Default | Meaning |
|-----|---------|---------|
| `KOOPA_SSH` | `koopa` | primary SSH host |
| `KOOPA_SSH_FALLBACKS` | `koopa-external` | tried if primary fails |
| `METRICS_LOAD` | `1` | `0` = skip all host/container load probes |
| `LADDER_LOAD` | `1` | `0` = skip load in ladder only |
| `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws |
| `LADDER_WITHDRAW_SCALE` | `1.5` | withdraw mids / pay mids ratio (funding headroom) |
| `LADDER_STEPS` | `23` | rungs for both withdraw and pay (0 + mids + max−1 + max) |
| `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python |
## Needs
- SSH `koopa` or `koopa-external` (inside / load / sanity server bits)
- secrets under `koopa-admin-secrets/...` for e2e
- `taler-wallet-cli` for e2e

193
TESTS.md Normal file
View file

@ -0,0 +1,193 @@
# taler-monitoring — grouped test IDs
Every check line has a **global** run number and a **grouped** id:
```text
┌ OK ┐ #001 www.exchange-01 exchange /config · HTTP 200
┌ WARN ┐ #002 www.stats-02 soft age · yellow (never red)
┌ ERROR ┐ #003 e2e.pay-03 order create failed · HTTP 502
┌ BLOCKER┐ #004 e2e.prereq-02 merchant secret missing
┌ PROG ┐ ████░░░░ 25% 4/16
```
- `#NNN` — monotonic for the whole monitoring run
- `area.group-NN` — logical block (resets per group, continues if re-entered)
| Area | Phase script | Groups (examples) |
|------|--------------|-------------------|
| **www** | `check_urls.sh` | `exchange` `perf` `stats` `bank` `merchant` `paivana` `landing` `qr` |
| **inside** | `check_inside.sh` | `ssh` `bank` `exchange` `merchant` `caddy` `load` |
| **versions** | `check_versions.sh` | `outside` `inside` `compare` |
| **aptdeploy** | `check_apt_deploy.sh` | `trixie` `trixie-testing` (koopa podman apt-src merchant smoke) |
| **surface** | `check_surface.sh` | **remote-only** ecosystem / `-d` domain inventory (not in default/all/full) |
| **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
| **server** | `check_server.sh` | (flat `server-NN` or host groups) |
| **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `atm` `settle` `pay` `shop` `paivana` `dig` `report` |
| **ladder** | `check_goa_ladder.sh` | `plan` `load` `withdraw` `pay` `report` |
**Why groups:** flat `www-001`…`www-080` was hard to talk about.
`www.bank-04` / `e2e.atm-02` pin the failure to a logical block.
Usage in scripts:
```bash
set_area www
set_group exchange # → www.exchange-01, www.exchange-02, …
set_group bank # counter resets → www.bank-01
```
`set_group` also prints a small group chip (`┌ www.exchange ┐`) so logs show section boundaries.
Re-entering the same group later (e.g. exchange legal after perf) **continues** NN — `www.exchange-03`, not a second `-01`.
Without `set_group`, IDs stay flat: `area-01`, `area-02`, …
Optional soft checks still consume a number when they WARN.
Numbering follows **executed** checks (early skip may shift later NN inside the same group).
---
## www — public URLs (`./taler-monitoring.sh urls`)
| Group | Checks (typical) |
|-------|------------------|
| **www.exchange-** | `/config`, currency, **alt_unit_names**; `/keys` (+ alt soft); `/intro/`, `/`; **`/terms`**, **`/privacy`**, `/terms/` |
| **www.perf-** | outside-in RTT: bank/exchange/merchant `/config`, keys, webui, intro — ms; WARN ≥ `PERF_WARN_MS` (8000); ERROR ≥ `PERF_FAIL_MS` (20000) |
| **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three |
| **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` |
| **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** |
| **www.webui-** | SPA fingerprints: **`/webui/version.txt`**, **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` |
| **www.paivana-** | local GOA paywall front (redirect to template) |
| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local |
| **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` |
**QR rule:** `taler://withdraw/HOST/taler-integration/UUID` (no default `:443`/`:80`); `taler://pay/` / `pay-template/`; `payto://…` shape OK; decoded PNG must equal payload. Form errors on withdraw/pay/payto are **ERROR** on local/GOA.
**Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. Local stack may require content needle (terms/privacy/FADP/GOA…).
**Performance rule:** Measured from the **monitoring runner** (public via Caddy), not container loopback. Latency (ms) on every perf line + **perf summary** (n / min / p50 / avg / max).
**Landing links rule:** Own-stack must be HTTP 200 (or redirect→200). External stores/docs soft WARN. Auto-account wallet link must be `taler://withdraw/HOST/taler-integration/…` (default ports stripped), never payto.
**alt_unit_names rule:** non-empty map including scale key `"0"`. Multi-currency merchant: follow each `exchanges[]` public `/config`.
---
## inside — koopa SSH (`./taler-monitoring.sh inside`)
| Group | Checks |
|-------|--------|
| **inside.host-** | **host-podman**: reachability via `podman exec` on this machine (GOA host / `INSIDE_PODMAN=1`) — not SSH |
| **inside.ssh-** | **ssh**: from laptop/remote — SSH to `KOOPA_SSH` / `INSIDE_SSH`, then podman there |
| **inside.bank-** | container, libeufin, postgres, local `/config`, nginx, DNS pins |
| **inside.exchange-** | container, httpd, wirewatch, aggregator, transfer, local keys, DNS |
| **inside.merchant-** | container, httpd, wirewatch, depositcheck, DNS |
| **inside.caddy-** | host reverse-proxy process |
| **inside.load-** | host loadavg + RAM; per-container RSS/CPU |
| **inside.disk-** / **aptdeploy.disk-** / **versions.disk-** / **server.disk-** | free space via `df`: **WARN** if use ≥ `DISK_WARN_USED_PCT` (85), **ERROR** if full or use ≥ `DISK_ERR_USED_PCT` (95); host + containers |
Remote lines `E|comp|LEVEL|key|detail` each become one numbered result under that component group.
SSH: `KOOPA_SSH` (default `koopa`), then `KOOPA_SSH_FALLBACKS` when LAN is unreachable.
---
## e2e — withdraw + pay (`./taler-monitoring.sh e2e`)
| Group | Checks |
|-------|--------|
| **e2e.prereq-** | wallet-cli, currency, budgets, secrets, public reachability |
| **e2e.load-** | host/container load snapshots (before / after ATM / after pay) |
| **e2e.bank-** | account, credit, withdraw ops |
| **e2e.wallet-** | exchange + ToS, wallet setup |
| **e2e.atm-** | ATM withdraw ladder rungs |
| **e2e.settle-** | wallet settlement wait / spendable balance |
| **e2e.pay-** | variable payments |
| **e2e.shop-** | GOA shop product catalog pays |
| **e2e.paivana-** | template paywall pay |
| **e2e.dig-** | coins-missing dig (inside + withdrawal-operation) |
| **e2e.report-** | final tallies |
When filing an issue, quote the full id + label, e.g.
`e2e.atm-03 ATM withdraw 5 GOA · bank confirmed, wallet empty`.
---
## versions / sanity / ladder / server / auth401
| Area.group | Meaning |
|------------|---------|
| **versions.outside-** | deb.taler.net suite index |
| **versions.host-** | host-podman: packages via local `podman exec` (`INSIDE_PODMAN=1`) |
| **versions.ssh-** | ssh: packages via `KOOPA_SSH` / `INSIDE_SSH` from laptop/remote |
| **versions.compare-** | installed vs suite |
| **aptdeploy.trixie-fresh-** | fresh rebuild container `…-apt-src-trixie` |
| **aptdeploy.trixie-testing-fresh-** | fresh rebuild `…-trixie-testing` |
| **aptdeploy.trixie-upgrade-** | upgrade-track `…-trixie-upgrade` (install once, then mytops-style upgrade) |
| **aptdeploy.trixie-testing-upgrade-** | upgrade-track `…-trixie-testing-upgrade` |
Standalone: `host-agent/run-aptdeploy.sh` (on koopa local; else `ssh koopa-external`).
### surface (remote-only public inventory)
```bash
./taler-monitoring.sh surface # taler.net / gnunet / taler-systems / mattermost / …
./taler-monitoring.sh -d hacktivism.ch surface # all things under that domain
```
- Catalog: `surface-catalog.conf` (extend as hosts are found)
- DNS, optional ICMP, TCP ports, then **protocol** checks (HTTPS/HTTP/SSH) before ERROR
- TLS cert expiry; Server-header / `/config` version hints; else **WARN: version unknown**
- OSV CVE query when package+version known → **ERROR** on hits
- Never SSH / never podman on the *targets*
| **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component |
| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder (GOA ceiling or stage TESTPAYSAN `max_wire`) |
| **server-** | SSH host ports / processes (flat unless grouped later) |
| **auth401.*** | merchant 401 / case matrix (WebUI APIs): **signup** `POST /instances` MIX id, **login** Basic user low vs MIX/UPPER, full **case** matrix, **password** bad/empty, **pwchange** `POST private/auth` + re-login, **admin-create** `POST /management/instances` (if admin token), **bearer**, **durable mon401**, **webui** SPA `toLowerCase` probe (`check_auth401.sh`) |
### SPA pin (after selfbuild-webui)
```bash
EXPECT_WEBUI_VERSION=1.6.11 \
EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \
WEBUI_OVERLAY_DENY='c778af04|master-11340' \
./taler-monitoring.sh -d hacktivism.ch urls
```
Skip SPA block: `CHECK_WEBUI_SPA=0`.
### Full load (maximum phases)
| Alias | Expands to (local GOA) |
|-------|-------------------------|
| **`all`** | `urls inside versions sanity e2e` |
| **`full`** | `urls inside versions sanity server e2e ladder auth401` |
```bash
cd ~/src/koopa/src/taler-monitoring
# secrets.env with bank admin + merchant tokens (see secrets.env.example)
./taler-monitoring.sh -d hacktivism.ch full
# same, explicit:
./taler-monitoring.sh -d hacktivism.ch \
urls inside versions sanity server e2e ladder auth401
```
---
## Visual format (colour on)
```text
╔══════════════════════════════╗
║ www · public URLs · … ║
╚══════════════════════════════╝
┌ www.exchange ┐
┌ OK ┐ www.exchange-01 exchange /config · HTTP 200
┌ www.bank ┐
┌ OK ┐ www.bank-01 bank /config · HTTP 200
┌ ERROR ┐ www.bank-04 bank /intro/auto-account.json · invalid withdraw
┌ ERRORS · failed checks ┐
• www.bank-04 bank /intro/auto-account.json · invalid withdraw
totals: 40 OK, 1 ERROR, 2 WARN, 5 INFO
```
`NO_COLOR=1` or `CLICOLOR=0` disables boxes/colours (ASCII `[ OK ]` + `-- group --` headers).
Always use `printf --` friendly plain headers when colour is off (leading `---` is not a printf option).

1
VERSION Normal file
View file

@ -0,0 +1 @@
1.0.0

14
VERSIONS.md Normal file
View file

@ -0,0 +1,14 @@
# VERSIONS — taler-monitoring
| Tag | Date (UTC) | Notes |
|-----|------------|--------|
| **v1.0** | 2026-07-18 | Initial standalone release. Split from `koopa-admin-log` at `scripts/taler-monitoring/` (tip after monitoring host-agent/site-gen split commits). |
Source: https://git.hacktivism.ch/hernani/taler-monitoring
```bash
git clone https://git.hacktivism.ch/hernani/taler-monitoring.git ~/src/taler-monitoring
cd ~/src/taler-monitoring && git checkout v1.0 # or stay on main
```
Default host-agent tracks `origin/main` (`SUITE_UPDATE_MODE=reset`). Pin with `SUITE_GIT_REF=v1.0` in `~/.config/taler-monitoring/env`.

11
android-test/.gitignore vendored Normal file
View file

@ -0,0 +1,11 @@
apks/*.apk
out/
*.png
out-source/
out-source-*/
out-gui/
out-source-gui/
out-gui-chain/
out-matrix-*/
out-stack-run-*/
out-survey-*/

362
android-test/GUI-AUTOMATION-NOTES.md Normal file
View file

@ -0,0 +1,362 @@
# GUI automation notes (Android wallet)
Canonical notes for **graphical** Android wallet tests against GOA / stage.
---
## Minimal variant (implemented first) — status 2026-07
This is the **minimal vanilla** path we actually shipped; full dual-platform
GUI is planned via flags (below), not required to use these scripts today.
| Piece | Status | Notes |
|-------|--------|--------|
| F-Droid published APK install | **done** | `net.taler.wallet.fdroid` 1.6.1 / 854 |
| Source build + same smoke | **done** | `:wallet:assembleFdroidDebug` → `net.taler.wallet.fdroid.debug` |
| Default source branch | **done** | `dev/hernani-inference/fix-bank-withdraw-auto-exchange` (minimal GOA exchange auto-add) |
| Deep-link entry (≡ QR) | **done** | `adb` `VIEW` **withdraw then pay** (`taler://withdraw/…` → `taler://pay…`) |
| Explorer API mint (chain) | **done** | `run-goa-gui-chain.sh` (GOA + stage) |
| UI taps via uiautomator | **done** | best-effort Confirm/ToS/Pay; ANR → prefer **Wait** |
| Screenshots / logcat artifacts | **done** | under `out*`, `out-gui-chain/` (gitignored) |
| Host CLI e2e settlement | **separate** | `taler-monitoring.sh e2e` / `ladder` (not Android UI) |
| Reliable unattended Confirm on 4 GiB Linux host | **limited** | System UI ANR; use phone or more RAM |
| iOS GUI | **not in this tree** | see taler-ios `dev/hernani-inference/gui-workflows` |
| macOS full dual AVD helpers | **upstream** | taler-android `gui-workflows` (Homebrew); ported ideas here for Linux |
**Ground rule:** bring it to run; app diffs only if needed and **minimal**; prefer
extending existing `dev/hernani-inference/*` branches over new trees.
### Minimalvariante: was geht / was nicht
Einschränkungen der **aktuell implementierten** Minimalvariante (deep-link +
uiautomator best-effort). Kein voller Dual-Platform-GUI-Stack, kein Ersatz für
Host-CLI-Settlement.
#### Geht (grün)
| Fähigkeit | Wie / Hinweis |
|-----------|----------------|
| APK installieren | F-Droid `net.taler.wallet.fdroid` **oder** Source `assembleFdroidDebug` (`.debug`) |
| GOA-Exchange ohne manuelles Add | Source-Build von `fix-bank-withdraw-auto-exchange` (Default in `run-android-build-and-smoke.sh`) |
| **Abheben + Bezahlen** | Beide Beine: erst Withdraw-URI + Taps, dann Pay-URI + Taps (`DO_WITHDRAW`/`DO_PAY`) |
| Light UI-Drive | Confirm / Accept / Pay-Taps über Label-Suche (`lib_ui.py`, gui-smoke, chain) |
| Multi-Runden-Taps | `run-android-gui-smoke.sh` / `run-goa-gui-chain.sh` (best-effort) |
| Explorer-Mint (Kette) | `run-goa-gui-chain.sh` + `EXP_PW_FILE` (GOA + stage) |
| Stack-Wahl | `STACK=goa` / `stage` (Bank/Merchant-URLs) |
| Artefakte | Screenshots, UI-XML, logcat unter `out*` / `out-gui-chain/` (gitignored) |
| Hybrid-Smoke | Install + Intent + leichte Taps: `run-android-pay-smoke.sh` |
| Settlement **nachweisen** | **Host-seitig**: `taler-monitoring.sh e2e` / `ladder` (nicht Android-UI) |
| Flags abschalten | `AUTO_ANDROID=0` skip; `AUTO_GUI=0` → nur deep-link/pay-smoke |
#### Geht **nicht** oder nur eingeschränkt (rot / gelb)
| Beschränkung | Folge / Workaround |
|--------------|--------------------|
| **Unattended Confirm → settled im Emulator (~4 GiB Host)** | Häufig **System UI ANR**; Taps bleiben hängen. Prefer **Wait** auf ANR-Dialog, echtes Gerät, oder Host mit mehr RAM. Kein zuverlässiges grünes E2E nur über GUI auf dem schwachen Emulator. |
| **Vollständige Wallet-Settlement-Assertion in der GUI** | Minimalvariante prüft **nicht** „Balance final / transfer_done“ in der App-Oberfläche. Proof bleibt CLI/e2e. |
| **Kamera / physischer QR-Scan** | Absichtlich nicht automatisiert; deep-link ist der Shortcut. Landing-QR-Pfade sind **HTTP**-Thema, nicht Android-GUI. |
| **Browser-Shop-Checkout-UI** | Kein Chromium/WebView-Drive; Pay-URI kommt per Template-`POST` + Intent. |
| **Volles ToS-Scroll / alle Dialoge** | Best-effort Accept-Tap; lange ToS oder unerwartete Sheets können hängen bleiben. |
| **Published F-Droid allein auf GOA** | Kann Exchange-Add / Spinner-Probleme zeigen → **fix-branch-APK** bauen; bei Spinner-Hang optional `fix-withdraw-spinner-fallback`. |
| **iOS-GUI** | Nicht in diesem Tree; siehe taler-ios `gui-workflows`. |
| **macOS dual AVD / Homebrew-Helpers** | Upstream `gui-workflows`; hier nur Linux-taugliche Port-Ideen. |
| **`AUTO_*` in jedem Script erzwungen** | Design + teilweise verdrahtet; Wrapper-Checkliste noch offen. |
| **`run-until-done` / CLI-Wallet in GUI-Skripten** | Gehört zu CLI-Automation, nicht GUI; siehe `CLI-AUTOMATION-NOTES.md`. |
| **CI grün = „User hat bezahlt“** | Smoke = Intent + best-effort UI. Wirtschaftlicher Erfolg = Host-e2e / ladder / Explorer. |
#### Erwartetes Ergebnis pro Schicht
| Schicht | Erfolgskriterium Minimalvariante |
|---------|----------------------------------|
| A Install | APK installed, App startet |
| B Deep-link | Intent delivered, Wallet öffnet Withdraw/Pay-Flow |
| C GUI taps | Dump+Tap ohne Crash; ANR ggf. mit Wait; **kein** Garant für „Paid“ |
| D Evidence | Artefakte geschrieben |
| Settlement | **Außerhalb** GUI: Host e2e/ladder |
#### Wann Minimalvariante reicht
- Smoke nach APK-/Branch-Build (Intent kommt an, App crasht nicht).
- Manuelle Nacharbeit am Gerät nach vorbereiteten URIs.
- GOA/stage **Erreichbarkeit** von Landing/Template + Wallet-Einstieg.
#### Wann nicht reicht → nächster Schritt
- Unattended grünes Pay-E2E auf dem Emulator → mehr RAM / physisches Gerät, ggf. `fix-withdraw-spinner-fallback`.
- Beweis „Münzen da / Merchant paid“ → CLI e2e/ladder, nicht GUI-smoke.
- iOS / Dual-Platform → andere Repos/Flags, nicht diese Skripte.
```bash
cd scripts/taler-monitoring/android-test
# Minimal hybrid (install + deep-link + light taps)
STACK=goa ./run-android-pay-smoke.sh
# GUI multi-round taps
STACK=goa ./run-android-gui-smoke.sh
STACK=stage ./run-android-gui-smoke.sh
# Explorer mint + multi withdraw/pay chain (from gui-workflows)
EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \
STACK=goa PKG=net.taler.wallet.fdroid.debug \
./run-goa-gui-chain.sh
# Build fix-branch APK then smoke (GUI=1 for gui smoke)
./run-android-build-and-smoke.sh
GUI=1 ./run-android-build-and-smoke.sh
```
---
## Platform capability flags (planned + defaults)
Automation can run on hosts that support **Android only**, **GUI tooling only**
(conceptual), or **both**. Flags keep that explicit for CI and laptops.
### Proposed env flags
| Variable | Values | Meaning |
|----------|--------|---------|
| `AUTO_ANDROID` | `0` / `1` | Run Android wallet automation (adb, APK, emulator/device) |
| `AUTO_GUI` | `0` / `1` | Drive **graphical** UI (uiautomator taps, multi-round). If `0`, deep-link/CLI-only install smoke. |
| `AUTO_PLATFORM` | `auto` / `linux` / `macos` / `ios` | Host family (optional override) |
| `EMULATOR_HEADLESS` | **`1` (default)** / `0` | No host window (`-no-window`). Server/CI-safe. |
| `WINDOWED` | `0` / `1` | Shortcut: `WINDOWED=1` ⇒ show emulator window (`EMULATOR_HEADLESS=0`) |
| `EMULATOR_GPU` | default **`swiftshader_indirect`** | Guest GLES still runs headless (layout/taps/screenshots). Override `host` only if windowed + real GPU. |
| `AUTO_START_EMULATOR` | **`1` (default)** / `0` | If no adb device, start AVD via `start-android-emulator.sh --wait` |
---
## Headless mode (default)
**Default for all GUI / pay smokes:** no window on the host. Opening an emulator
GUI on servers (or headless CI) is wrong; these scripts therefore start the AVD
with **`-no-window`** and still keep **graphics logic** in the guest:
| Piece | Headless default |
|-------|------------------|
| Host window | **off** (`-no-window`, `QT_QPA_PLATFORM=offscreen`, no `DISPLAY`) |
| Guest GLES | **on** via **SwiftShader** (`-gpu swiftshader_indirect`) |
| UI automation | **on** — `uiautomator` dump/tap + `screencap` (no X11 needed) |
| Audio / boot anim | off (`-no-audio -no-boot-anim`) |
```bash
# explicit (same as default)
./start-android-emulator.sh --wait
# or let smoke auto-start headless AVD when no phone is plugged in
STACK=goa ./run-android-gui-smoke.sh
# laptop: show the emulator window
WINDOWED=1 ./start-android-emulator.sh --wait
# or: EMULATOR_HEADLESS=0 EMULATOR_GPU=host ./start-android-emulator.sh --wait
```
Shared code: `lib_android_env.sh` + `start-android-emulator.sh`.
**Semantics:**
| `AUTO_ANDROID` | `AUTO_GUI` | Behaviour |
|----------------|------------|-----------|
| `1` | `0` | Android install + deep-link smoke only (no multi-round taps) |
| `1` | `1` | Android + graphical drive (vanilla GUI / chain) |
| `0` | `1` | Reserved (e.g. future desktop/web GUI); currently **no-op** with a clear message |
| `0` | `0` | Skip mobile automation |
### Defaults by host (when flags unset)
| Host (`uname -s`) | Default `AUTO_ANDROID` | Default `AUTO_GUI` | Rationale |
|-------------------|------------------------|--------------------|-----------|
| **Linux** | **`1`** | **`1`** if device/emulator present, else scripts exit 3 | This repo’s day-to-day path; **Android only** (no iOS here) |
| **Darwin (macOS)** | `1` | `1` | Can run Android emulators **and** (separately) taler-ios GUI helpers; both flags on for Android scripts; iOS is out of tree |
| **Other** | `0` | `0` | Fail closed |
**Linux default = Android** (no second mobile platform in this suite).
**macOS** may enable both ecosystems in the wider monorepo sense; for *these*
scripts only Android is implemented — set `AUTO_ANDROID=1` (default) and keep
iOS under `taler-ios`.
### Resolution helper (convention for future wrappers)
```bash
# Example for a future run-all-mobile.sh
os=$(uname -s)
: "${AUTO_PLATFORM:=auto}"
case "$AUTO_PLATFORM" in
auto) case "$os" in Linux) AUTO_PLATFORM=linux ;; Darwin) AUTO_PLATFORM=macos ;; *) AUTO_PLATFORM=other ;; esac ;;
esac
case "$AUTO_PLATFORM" in
linux)
: "${AUTO_ANDROID:=1}"
: "${AUTO_GUI:=1}"
# no iOS
;;
macos)
: "${AUTO_ANDROID:=1}"
: "${AUTO_GUI:=1}"
# optional later: AUTO_IOS=1 for taler-ios scripts
;;
*)
: "${AUTO_ANDROID:=0}"
: "${AUTO_GUI:=0}"
;;
esac
```
Scripts **today** implement Android only; they should honour:
- `AUTO_ANDROID=0` → exit 0 with “skipped (AUTO_ANDROID=0)”
- `AUTO_GUI=0` → call deep-link smoke without multi-round GUI (or set `GUI_ROUNDS=0`)
---
## Vanilla layers (detail)
| Layer | What | Tooling |
|-------|------|---------|
| **A — Install** | F-Droid or from-source APK | `adb install` |
| **B — Entry (shortcut)** | `taler://withdraw/…` / `taler://pay…` | `adb am start -a VIEW` |
| **C — Graphical UI** | Confirm / ToS / Pay taps; ANR Wait | `uiautomator` + `lib_ui.py` / chain |
| **D — Evidence** | Screenshots, XML, logcat, URIs | `out*/` |
### Legitimate shortcuts
| Shortcut | Replaces |
|----------|----------|
| Deep-link withdraw/pay | Camera QR / opening paywall |
| Host `POST` template → pay URI | Browser shop checkout UI |
| Label-list taps | Human reading button text |
| Skip full ToS scroll | Tap Accept if shown |
### Scripts
| Script | Role |
|--------|------|
| `run-android-pay-smoke.sh` | Hybrid: install + deep-link + light taps |
| `run-android-gui-smoke.sh` | Multi-round GUI drive |
| `run-goa-gui-chain.sh` | Multi mint/withdraw/pay (gui-workflows port) |
| `run-android-build-and-smoke.sh` | Build git ref (branch/tag) + smoke (`GUI=1` optional) |
| `run-android-variant-matrix.sh` | **published + stable-self + master** (compare builds) |
| `start-android-emulator.sh` | Start AVD (**headless default**, SwiftShader) |
| `lib_android_env.sh` | Flags, PATH, `android_ensure_device` |
| `lib_ui.py` | Dump/tap/ANR helpers |
### Build variants (anti–“weird local build”)
| Variant | Source | Severity | Purpose |
|---------|--------|----------|---------|
| **stable-self** | **only** rebuild of current stable tag | WARN | Self-build vs published for **one** release |
| **published** | F-Droid current APK | **≤14d → BLOCKER**, else WARN | What users install |
| **older / milestones** | F-Droid APK ~**3 / 6 / 9 / 12 mo** | **always WARN** | No self-build of old tags |
| **master** | `origin/master` rebuild | WARN | Tip — never hard-blocks |
| **fix** (optional) | inference branch | soft | GOA automation fixes |
**Self-build:** current stable only (plus master/fix if requested) — not older releases.
**Blocker window:** releases ≤ 14 days.
**Milestones:** nearest tags to 90/180/270/365d with a still-hosted F-Droid APK.
---
## Stacks under test
| Stack | Withdraw | Pay |
|-------|----------|-----|
| **GOA** | explorer mint / `demo-withdraw.json` @ bank.hacktivism.ch | goa-shop templates / Paivana pay-template |
| **stage** | explorer / demo-withdraw @ stage.bank… | fermes / jardin public templates |
### Lageübersicht GOA + stage (as-is survey · 2026-07-17 ~22:05–22:18)
**Device:** headless `emulator-5554` · host ~3.7 GiB RAM (often <300 MiB free) · timeout per leg ≤420 s.
**Legs:** Abheben + Bezahlen (`DO_WITHDRAW=1 DO_PAY=1`). Artefakte: `out-survey-20260717-220527/` (gitignored).
#### Varianten
| Variante | Quelle | Bemerkung |
|----------|--------|-----------|
| **stable (published)** | F-Droid `net.taler.wallet.fdroid` **1.6.1 / 854** | as-is |
| **master as-is** | rebuild `origin/master` @ `f65a976c` → `.fdroid.debug` | as-is, no patches |
| **ältere Releases** (3/6/9/12 mon) | F-Droid APKs für `1.5.0`/`1.3.0`/`1.1.0`/`1.0.8` | **nicht auffindbar** (HTTP miss) → **SKIP** (kein Selbstbau älterer Tags) |
| **master + inference** (extra) | local branch `local/master-with-inference` @ `3ebe8b09` | merges `fix-bank-withdraw-auto-exchange` + `fix-withdraw-spinner-fallback` · **not pushed** |
#### Ergebnis-Matrix
| Variant | Stack | Install / deep-link | Withdraw URI | Pay URI | wallet-core Abheben | wallet-core Pay | ANR | Exit |
|---------|-------|---------------------|--------------|---------|---------------------|-----------------|-----|------|
| stable published | **goa** (hacktivism) | OK | OK bank.hacktivism.ch | OK paivana template | **NO** | soft/YES once* | **YES** | **10** |
| stable published | **stage** (*.lefrancpaysan) | OK | OK stage.bank… | OK fermes order | **NO** | **NO** | **YES** | **10** |
| master as-is | **goa** | OK | OK | OK paivana | **NO** | **NO** | **YES** | **10** |
| master as-is | **stage** | OK | OK | OK fermes | **NO** | **NO** | **YES** | **10** |
| master+inference | **goa** | OK | OK | OK | **NO** | **NO** | **YES** | **10** |
| master+inference | **stage** | OK | OK | OK | **NO** | **NO** | **YES** | **10** |
\*stable-goa console once reported pay wallet-core YES mid-run; final score still ANR-dominated / not reliable settlement.
#### Was pro Stack **steht** (Infra)
| Check | **goa / hacktivism** | **stage / \*.lefrancpaysan** |
|-------|----------------------|------------------------------|
| Bank `…/intro/demo-withdraw.json` → `taler://withdraw/…` | **OK** | **OK** |
| Pay entry | paivana `pay-template` **OK** | fermes `panier-legumes` POST → `taler://pay/…` **OK** |
| adb `VIEW` Intent delivery | **OK** | **OK** |
| Unattended Confirm/Abheben/Pay GUI | **FAIL (ANR)** | **FAIL (ANR)** |
| wallet-core prepare/accept withdraw in logcat | **not seen** | **not seen** |
**Fazit as-is:** Beide Stacks liefern die **Eingänge** (Withdraw- + Pay-URIs). Die **GUI/wallet-core-Strecke** ist auf diesem Emulator für **alle** getesteten App-Varianten (stable, master, inference) **gleich tot** — System UI / Android System ANR, keine brauchbaren Taps. Das ist **kein Stack-Unterschied** GOA vs stage und **kein** belastbarer App-Versions-Vergleich.
#### Inference-Patch-Versuch (lokal, **kein Push**)
Weil **nichts** der as-is-Varianten wallet-core-grün war, wurde zusätzlich gebaut:
```text
taler-android branch: local/master-with-inference # NOT pushed
base: origin/master
merged: origin/dev/hernani-inference/fix-bank-withdraw-auto-exchange
origin/dev/hernani-inference/fix-withdraw-spinner-fallback
commit: 3ebe8b091379
APK: apks/wallet-master-inference-3ebe8b091379.apk (local)
```
| Stack | Inference smoke | Besser als master/stable? |
|-------|-----------------|---------------------------|
| goa | exit 10, ANR, no withdraw core | **Nein** |
| stage | exit 10, ANR, no withdraw core | **Nein** |
**Kein grünes Inference-Signal** auf diesem Host → wir können **nicht** belegen, dass die Patches Withdraw/Pay „retten“.
Ebenso können wir **nicht** belegen, dass master ohne Patches „schuld“ ist: die Umgebung erreicht wallet-core gar nicht.
> **Wenn später auf Gerät/mehr RAM die Inference-APK plötzlich grün und master/stable rot ist:**
> das ist der **klare Fix-Hinweis** — Patches aus `dev/hernani-inference/*` upstreamen / in master holen.
> **Heute:** noch **nicht** der Fall; primärer Blocker = **Emulator/RAM/ANR**.
#### Nächste sinnvolle Schritte (später)
1. Physisches Gerät oder Host ≥6–8 GiB free RAM
2. Nur **eine** Wallet-Package-ID installiert (kein Open-with)
3. Survey wiederholen: stable · master · `local/master-with-inference` auf **goa + stage**
4. Ältere F-Droid-APKs nur wenn wieder gehostet; kein Selbstbau alter Tags
---
## Upstream branches (`taler-android` `dev/hernani-inference/*`)
| Branch | Role |
|--------|------|
| `gui-workflows` | macOS emulator helpers + original `goa-chain-emu.sh` |
| `fix/bank-withdraw-auto-exchange` | **Minimal** GOA exchange auto-add (default build branch) |
| `fix/withdraw-spinner-fallback` | Spinner + OIM; use if spinner still hangs |
| `experimental-oim*` | Optional cash UI |
---
## Checklist
- [x] Minimal variant documented
- [x] Minimalvariante: was geht / was nicht (Beschränkungen)
- [x] Two stacks (goa / stage)
- [x] Dual-stack run documented (2026-07-17: both FAIL ANR, landings OK)
- [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults)
- [x] **Headless emulator default** (`EMULATOR_HEADLESS=1`, SwiftShader)
- [x] Linux default = Android suite only
- [x] Inference-branch policy
- [ ] Wrapper enforces flags in every entry script (incremental)
- [ ] Reliable unattended Confirm on low-RAM Linux emulator

111
android-test/README.md Normal file
View file

@ -0,0 +1,111 @@
# Android wallet smoke (Abheben + Bezahlen)
Smokes always cover **withdraw first**, then **pay** (not pay-only).
| Leg | Default | What |
|-----|---------|------|
| **1 WITHDRAW** | on | `taler://withdraw/…` + Confirm/Abheben taps |
| **2 PAY** | on | `taler://pay…` / pay-template + Pay taps |
```bash
# needs adb device/emulator
./run-android-pay-smoke.sh
STACK=goa ./run-android-pay-smoke.sh
DO_PAY=0 ./run-android-pay-smoke.sh # withdraw only
DO_WITHDRAW=0 DO_PAY=1 … # pay only (unusual)
```
APK: F-Droid **1.6.1** (`net.taler.wallet.fdroid`).
## 2) Source build + same tests
Builds `taler-android` at a git **ref** (`:wallet:assembleFdroidDebug`), then
runs the **same** smoke (install + withdraw/pay deep-links).
```bash
# default: inference fix branch (GOA withdraw reliability)
./run-android-build-and-smoke.sh
# tip of master
BRANCH=master ./run-android-build-and-smoke.sh
# self-build of published stable **tag** (same release as F-Droid 1.6.1)
BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh
SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh
TALER_ANDROID_SRC=/path/to/taler-android ./run-android-build-and-smoke.sh
```
Built package id: `net.taler.wallet.fdroid.debug`.
Needs `ANDROID_HOME` (e.g. `$HOME/Android/Sdk`), Java 17+, network for Maven.
## 2b) Variant matrix (published + stable self-build + master)
Runs several smokes so a single odd developer build cannot be the only sample:
| Variant | What |
|---------|------|
| Order | Variant | Severity | What |
|------|---------|----------|------|
| **1** | **stable-self** | WARN | **Only** self-build: current stable tag |
| **2** | **published** | age ≤14d → **BLOCKER**, else WARN | Current F-Droid APK |
| **3** | **older / milestones** | **always WARN** | F-Droid APKs nearest **3 / 6 / 9 / 12 months** (if still hosted) — **no self-build** |
| **4** | **master** | WARN | Rebuild `origin/master` |
**Self-build policy:** only **current published stable** (`stable-self`) + optional **master**.
Historical versions: published F-Droid APK only; skip if APK gone.
**Milestones:** `MILESTONE_DAYS="90 180 270 365"` (±50d), warn only.
**Young releases** (≤ 14 days): hard blocker on fail.
```bash
./run-android-variant-matrix.sh
STACK=goa ./run-android-variant-matrix.sh
INCLUDE_OLDER=0 VARIANTS=stable-self,published,master ./run-android-variant-matrix.sh
# custom milestones (days):
MILESTONE_DAYS="90 180 270 365" ./run-android-variant-matrix.sh
```
Artifacts + `SUMMARY.txt` under `out-matrix-<timestamp>/`.
## 3) Graphical UI (vanilla)
Deep-link entry (≡ QR) **plus** multi-round uiautomator taps:
```bash
STACK=goa ./run-android-gui-smoke.sh
STACK=stage ./run-android-gui-smoke.sh
# after source build:
GUI=1 ./run-android-build-and-smoke.sh
```
## 4) Full GOA/stage chain (from `dev/hernani-inference/gui-workflows`)
Bank API mint → withdraw deep-link → taps → template pay → taps:
```bash
# needs explorer password + device
EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \
STACK=goa PKG=net.taler.wallet.fdroid.debug \
./run-goa-gui-chain.sh
STACK=stage ./run-goa-gui-chain.sh
```
Canonical notes (minimal variant, **flags**, Linux defaults):
**[GUI-AUTOMATION-NOTES.md](./GUI-AUTOMATION-NOTES.md)**.
| Flag | Linux default | Meaning |
|------|---------------|---------|
| `AUTO_ANDROID` | `1` | Run Android adb/APK automation |
| `AUTO_GUI` | `1` | Graphical uiautomator drive (else deep-link only) |
| `AUTO_PLATFORM` | `auto` → `linux` | Host family; macOS can later pair with iOS out of tree |
| **`EMULATOR_HEADLESS`** | **`1`** | **No host window** (`-no-window`). Server-safe. |
| `EMULATOR_GPU` | `swiftshader_indirect` | Guest GLES still active for UI/taps/screenshots |
| `AUTO_START_EMULATOR` | `1` | Auto-start headless AVD if no adb device |
| `WINDOWED` | `0` | `WINDOWED=1` → show emulator window |
```bash
# headless AVD only (default path for servers)
./start-android-emulator.sh --wait
# windowed (laptops)
WINDOWED=1 ./start-android-emulator.sh --wait
```
See also parent [VERSIONS.md](../VERSIONS.md).

158
android-test/lib_android_env.sh Normal file
View file

@ -0,0 +1,158 @@
# shellcheck shell=bash
# Shared Android env for GUI / adb smokes.
# Source: . "$ROOT/lib_android_env.sh"
#
# Defaults favour **servers / CI**: no host window (headless emulator), while the
# guest still runs GLES via SwiftShader so UI layout / uiautomator work.
#
# See GUI-AUTOMATION-NOTES.md → Headless mode
# --- SDK / PATH ---------------------------------------------------------------
export ANDROID_HOME="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-$HOME/Android/Sdk}}"
export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}"
for d in \
"$ANDROID_HOME/platform-tools" \
"$ANDROID_HOME/emulator" \
"$ANDROID_HOME/cmdline-tools/latest/bin"
do
[ -d "$d" ] && case ":$PATH:" in *":$d:"*) ;; *) PATH="$d:$PATH" ;; esac
done
export PATH
# AVD path (some hosts keep AVDs under ~/.config/.android)
if [ -z "${ANDROID_AVD_HOME:-}" ]; then
if [ -d "$HOME/.android/avd" ]; then
export ANDROID_AVD_HOME="$HOME/.android/avd"
elif [ -d "$HOME/.config/.android/avd" ]; then
export ANDROID_AVD_HOME="$HOME/.config/.android/avd"
# emulator also looks at ~/.android/avd — soft-link if missing
if [ ! -e "$HOME/.android/avd" ]; then
mkdir -p "$HOME/.android"
ln -sfn "$HOME/.config/.android/avd" "$HOME/.android/avd" 2>/dev/null || true
fi
fi
fi
# --- Platform / feature flags -------------------------------------------------
os=$(uname -s)
case "${AUTO_PLATFORM:-auto}" in
auto)
case "$os" in
Linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;;
Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;;
*) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;;
esac
;;
linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;;
*) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;;
esac
# Headless = default (no X11/Wayland window on host). GUI *logic* still runs in
# the guest via software GPU. Set EMULATOR_HEADLESS=0 or WINDOWED=1 for a window.
: "${EMULATOR_HEADLESS:=1}"
if [ "${WINDOWED:-0}" = "1" ]; then
EMULATOR_HEADLESS=0
fi
export EMULATOR_HEADLESS
# Soft GLES for headless; windowed may override to host/auto if set by user.
if [ "${EMULATOR_HEADLESS}" = "1" ]; then
: "${EMULATOR_GPU:=swiftshader_indirect}"
else
: "${EMULATOR_GPU:=swiftshader_indirect}" # still safe default; host GPU: EMULATOR_GPU=host
fi
export EMULATOR_GPU
: "${EMULATOR_AVD:=TalerWallet34}"
: "${EMULATOR_MEMORY:=1536}"
: "${EMULATOR_CORES:=2}"
# If no adb device: start headless AVD (0 = require phone/manual emulator)
: "${AUTO_START_EMULATOR:=1}"
: "${EMULATOR_BOOT_TIMEOUT:=360}"
: "${EMULATOR_LOG:=/tmp/taler-android-emulator.log}"
: "${EMULATOR_NO_SNAPSHOT:=1}"
android_adb_serial() {
adb devices 2>/dev/null | awk '/\tdevice$/{print $1; exit}'
}
android_has_device() {
[ -n "$(android_adb_serial)" ]
}
# Build emulator argv for current headless/windowed mode.
# Prints args; caller exec/nohup.
android_emulator_args() {
local avd="${1:-$EMULATOR_AVD}"
local args=( -avd "$avd" -gpu "$EMULATOR_GPU" -no-audio -no-boot-anim )
if [ "${EMULATOR_HEADLESS}" = "1" ]; then
args+=( -no-window )
# Avoid Qt trying to open a display on servers
export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}"
fi
# Broken default_boot snapshots leave adb offline forever — cold boot by default.
if [ "${EMULATOR_NO_SNAPSHOT:-1}" = "1" ]; then
args+=( -no-snapshot -no-snapshot-load -no-snapshot-save )
fi
if [ -n "${EMULATOR_MEMORY:-}" ]; then
args+=( -memory "$EMULATOR_MEMORY" )
fi
if [ -n "${EMULATOR_CORES:-}" ]; then
args+=( -cores "$EMULATOR_CORES" )
fi
# extra user flags
# shellcheck disable=SC2206
if [ -n "${EMULATOR_EXTRA_ARGS:-}" ]; then
# intentional word-split for extra flags
args+=( ${EMULATOR_EXTRA_ARGS} )
fi
printf '%s\n' "${args[@]}"
}
# Ensure an adb "device" is online. Optionally start headless emulator.
# Sets SERIAL / ANDROID_SERIAL on success. Returns 0 ok, 3 no device.
android_ensure_device() {
local serial
serial="${SERIAL:-${ANDROID_SERIAL:-}}"
if [ -z "$serial" ]; then
serial=$(android_adb_serial || true)
fi
if [ -n "$serial" ]; then
SERIAL="$serial"
export ANDROID_SERIAL="$serial"
return 0
fi
if [ "${AUTO_START_EMULATOR}" != "1" ]; then
echo "no adb device (AUTO_START_EMULATOR=0 — plug phone or start AVD manually)" >&2
return 3
fi
if ! command -v emulator >/dev/null 2>&1; then
echo "no adb device and emulator binary missing under $ANDROID_HOME" >&2
return 3
fi
echo "no adb device → starting AVD ${EMULATOR_AVD} (headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU})"
local starter
starter="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/start-android-emulator.sh"
if [ ! -x "$starter" ]; then
echo "missing $starter" >&2
return 3
fi
EMULATOR_HEADLESS="$EMULATOR_HEADLESS" EMULATOR_GPU="$EMULATOR_GPU" \
EMULATOR_AVD="$EMULATOR_AVD" EMULATOR_MEMORY="$EMULATOR_MEMORY" \
EMULATOR_CORES="$EMULATOR_CORES" EMULATOR_LOG="$EMULATOR_LOG" \
EMULATOR_BOOT_TIMEOUT="$EMULATOR_BOOT_TIMEOUT" \
"$starter" --wait || return 3
serial=$(android_adb_serial || true)
if [ -z "$serial" ]; then
echo "emulator started but no adb device yet — see $EMULATOR_LOG" >&2
return 3
fi
SERIAL="$serial"
export ANDROID_SERIAL="$serial"
echo "device: $SERIAL (headless=${EMULATOR_HEADLESS})"
return 0
}

133
android-test/lib_release_age.sh Normal file
View file

@ -0,0 +1,133 @@
# shellcheck shell=bash
# Release age helpers for Android wallet matrix.
# Source after ROOT/lib_android_env.sh (optional).
#
# BLOCKER only when a *released* app is within BLOCKER_MAX_AGE_DAYS (default 14).
# Older releases: try them, but failures are WARN only.
: "${BLOCKER_MAX_AGE_DAYS:=14}"
: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}"
# Days since git tag (tagger/committer date). Empty if unknown.
# Usage: release_age_days wallet-1.6.1 → e.g. 28
release_age_days() {
local tag="$1"
local src="${2:-$TALER_ANDROID_SRC}"
[ -d "$src/.git" ] || return 1
local epoch
epoch=$(git -C "$src" log -1 --format=%ct "refs/tags/${tag}" 2>/dev/null || true)
[ -n "$epoch" ] || epoch=$(git -C "$src" log -1 --format=%ct "$tag" 2>/dev/null || true)
[ -n "$epoch" ] || return 1
local now
now=$(date +%s)
echo $(( (now - epoch) / 86400 ))
}
# severity from age: blocker if age_days <= BLOCKER_MAX_AGE_DAYS, else warn
# Unknown age → warn (never accidental hard fail on mystery builds)
severity_from_age_days() {
local days="${1:-}"
if [ -z "$days" ] || ! [[ "$days" =~ ^[0-9]+$ ]]; then
echo warn
return
fi
if [ "$days" -le "${BLOCKER_MAX_AGE_DAYS}" ]; then
echo blocker
else
echo warn
fi
}
severity_for_tag() {
local tag="$1"
local days
days=$(release_age_days "$tag" 2>/dev/null || true)
severity_from_age_days "$days"
}
# List wallet-* release tags newest first (exclude rc/dev).
# Args: max count (default 5). Pass 0 or "all" for no limit.
list_wallet_release_tags() {
local n="${1:-5}"
local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}"
[ -d "$src/.git" ] || return 1
git -C "$src" fetch --tags origin 2>/dev/null || true
local cmd=(git -C "$src" for-each-ref --sort=-creatordate --format='%(refname:short)' refs/tags)
if [ "$n" = "all" ] || [ "$n" = "0" ]; then
"${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$'
else
"${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' | head -n "$n"
fi
}
# Pick one tag closest to each age milestone (days). Always intended as WARN samples.
# Default milestones: 3,6,9,12 months ≈ 90,180,270,365 days.
# Env: MILESTONE_DAYS="90 180 270 365" MILESTONE_TOLERANCE_DAYS=50
# Prints lines: TAG AGE_DAYS TARGET_DAYS (only when a tag within tolerance exists)
select_milestone_tags() {
local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}"
local milestones="${MILESTONE_DAYS:-90 180 270 365}"
local tol="${MILESTONE_TOLERANCE_DAYS:-50}"
local skip="${1:-}" # optional tag to exclude (e.g. current STABLE_TAG)
local -a tags=()
mapfile -t tags < <(list_wallet_release_tags all 2>/dev/null || true)
[ "${#tags[@]}" -gt 0 ] || return 0
local target best_tag best_age best_delta age t delta
local -A used=()
for target in $milestones; do
best_tag=""
best_age=""
best_delta=99999
for t in "${tags[@]}"; do
[ -n "$t" ] || continue
[ "$t" = "$skip" ] && continue
[ -n "${used[$t]:-}" ] && continue
age=$(release_age_days "$t" "$src" 2>/dev/null || true)
[[ "$age" =~ ^[0-9]+$ ]] || continue
delta=$(( age > target ? age - target : target - age ))
if [ "$delta" -le "$tol" ] && [ "$delta" -lt "$best_delta" ]; then
best_delta=$delta
best_tag=$t
best_age=$age
fi
done
if [ -n "$best_tag" ]; then
used[$best_tag]=1
echo "$best_tag $best_age $target"
fi
done
}
# Guess F-Droid APK URL for a wallet version name (e.g. 1.6.1).
# Uses versionCode from tag's build.gradle if possible, else STABLE mapping.
# Prints: versionCode apk_url or returns 1
fdroid_apk_for_version_name() {
local vn="$1" # 1.6.1
local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}"
local tag="wallet-${vn}"
local vc=""
if git -C "$src" cat-file -e "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null; then
vc=$(git -C "$src" show "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null \
| sed -n 's/^[[:space:]]*versionCode[[:space:]]\+\([0-9]\+\).*/\1/p' | head -1)
fi
# ABI x86_64 → +4 in override scheme 10*vc+abi (common F-Droid multi-apk)
# Prefer universal-style code used by F-Droid: for 1.6.1 → 854 = 10*85+4
local apk_vc=""
if [ -n "$vc" ]; then
apk_vc=$((10 * vc + 4))
fi
# Known overrides / suggested
case "$vn" in
1.6.1) apk_vc=854 ;;
esac
[ -n "$apk_vc" ] || return 1
echo "${apk_vc}|https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk"
}
# Check HTTP HEAD/GET that APK exists (0 = ok)
fdroid_apk_exists() {
local url="$1"
curl -sfS -m 15 -o /dev/null -r 0-0 "$url" 2>/dev/null \
|| curl -sfS -m 15 -o /dev/null -I "$url" 2>/dev/null
}

192
android-test/lib_ui.py Executable file
View file

@ -0,0 +1,192 @@
#!/usr/bin/env python3
"""Minimal adb + uiautomator helpers for Taler wallet GUI smoke (no root).
Does not replace Maestro/Appium; uses only adb shell + uiautomator dump.
"""
from __future__ import annotations
import re
import subprocess
import time
from pathlib import Path
from typing import Iterable, List, Optional, Sequence, Tuple
Bounds = Tuple[int, int, int, int]
def adb(serial: str, *args: str, check: bool = False, timeout: Optional[float] = 60) -> subprocess.CompletedProcess:
cmd = ["adb", "-s", serial, *args]
return subprocess.run(cmd, check=check, capture_output=True, text=True, timeout=timeout)
def dump_ui(serial: str, dest: Path) -> str:
adb(serial, "shell", "uiautomator", "dump", "/sdcard/ui.xml", timeout=90)
adb(serial, "pull", "/sdcard/ui.xml", str(dest), timeout=30)
return dest.read_text(errors="replace") if dest.is_file() else ""
def screencap(serial: str, dest: Path) -> None:
p = subprocess.run(
["adb", "-s", serial, "exec-out", "screencap", "-p"],
capture_output=True,
timeout=60,
)
if p.returncode == 0 and p.stdout:
dest.write_bytes(p.stdout)
def texts(xml: str) -> List[str]:
return [t for t in re.findall(r'text="([^"]*)"', xml) if t.strip()]
def find_bounds(xml: str, label: str) -> List[Bounds]:
"""Return list of (x1,y1,x2,y2) for nodes whose text or content-desc matches label (exact)."""
out: List[Bounds] = []
esc = re.escape(label)
patterns = [
rf'(?:text|content-desc)="{esc}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"',
rf'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*(?:text|content-desc)="{esc}"',
]
for pat in patterns:
for m in re.finditer(pat, xml, flags=re.I):
out.append(tuple(map(int, m.groups()))) # type: ignore[arg-type]
return out
def find_bounds_re(xml: str, pattern: str) -> List[Tuple[str, Bounds]]:
out: List[Tuple[str, Bounds]] = []
for m in re.finditer(
rf'(?:text|content-desc)="([^"]*{pattern}[^"]*)"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"',
xml,
flags=re.I,
):
label = m.group(1)
b = tuple(map(int, m.group(2, 3, 4, 5)))
out.append((label, b)) # type: ignore[arg-type]
return out
def center(b: Bounds) -> Tuple[int, int]:
x1, y1, x2, y2 = b
return (x1 + x2) // 2, (y1 + y2) // 2
def tap(serial: str, x: int, y: int) -> None:
adb(serial, "shell", "input", "tap", str(x), str(y))
def tap_bounds(serial: str, b: Bounds) -> None:
x, y = center(b)
tap(serial, x, y)
def dismiss_anr(serial: str, xml: str) -> bool:
"""If System UI isn't responding, prefer Wait over Close."""
if "isn't responding" not in xml and "reagiert nicht" not in xml.lower():
return False
for label in ("Wait", "Warten", "OK"):
bs = find_bounds(xml, label)
if bs:
tap_bounds(serial, bs[0])
return True
# fallback: right-ish button often Wait
adb(serial, "shell", "input", "keyevent", "22")
adb(serial, "shell", "input", "keyevent", "66")
return True
# Labels seen on wallet UI (EN/DE/FR) — expand as needed
CONFIRM_LABELS = [
"Confirm",
"Confirm withdrawal",
"Withdraw",
"Accept",
"I accept",
"Agree",
"Continue",
"Next",
"OK",
"Pay",
"Pay now",
"Bestätigen",
"Abheben",
"Akzeptieren",
"Zustimmen",
"Weiter",
"Bezahlen",
"Confirmer",
"Retirer",
"Accepter",
"Payer",
]
TOS_LABELS = [
"I accept",
"Accept",
"Agree",
"Akzeptieren",
"Zustimmen",
"Accepter",
]
def try_tap_any(serial: str, xml: str, labels: Sequence[str]) -> Optional[str]:
for label in labels:
bs = find_bounds(xml, label)
if bs:
tap_bounds(serial, bs[0])
return label
return None
def gui_drive(
serial: str,
out_dir: Path,
*,
rounds: int = 8,
sleep_s: float = 3.0,
phase: str = "gui",
) -> dict:
"""Poll UI, dismiss ANR, tap confirm/ToS-like buttons. Returns status dict."""
out_dir.mkdir(parents=True, exist_ok=True)
seen_texts: List[str] = []
taps: List[str] = []
anr = 0
for i in range(1, rounds + 1):
xml_path = out_dir / f"{phase}-{i:02d}-ui.xml"
png_path = out_dir / f"{phase}-{i:02d}.png"
try:
xml = dump_ui(serial, xml_path)
except Exception as e:
taps.append(f"dump-fail:{e}")
time.sleep(sleep_s)
continue
try:
screencap(serial, png_path)
except Exception:
pass
ts = texts(xml)
seen_texts.extend(ts[:20])
if dismiss_anr(serial, xml):
anr += 1
time.sleep(sleep_s)
continue
hit = try_tap_any(serial, xml, CONFIRM_LABELS)
if hit:
taps.append(hit)
time.sleep(sleep_s)
continue
# partial regex for amount confirm rows
for label, b in find_bounds_re(xml, r"Confirm|Withdraw|Pay|Accept|Bestätig|Abheben|Payer"):
tap_bounds(serial, b)
taps.append(f"re:{label}")
time.sleep(sleep_s)
break
else:
time.sleep(sleep_s)
return {
"taps": taps,
"anr_dismissals": anr,
"sample_texts": seen_texts[-40:],
"rounds": rounds,
}

167
android-test/run-android-build-and-smoke.sh Executable file
View file

@ -0,0 +1,167 @@
#!/usr/bin/env bash
# Build GNU Taler Android wallet from source at a git **ref**, then run the same
# adb deep-link (or GUI) smoke as run-android-pay-smoke.sh.
#
# Source (default): $HOME/taler/taler-android
# Build: ./gradlew :wallet:assembleFdroidDebug
# Package id: net.taler.wallet.fdroid.debug
#
# Typical refs:
# BRANCH=master # tip of master
# BRANCH=wallet-1.6.1 # published stable **tag** (self-build)
# BRANCH=dev/hernani-inference/fix-… # GOA automation fix branch
#
# For a multi-variant matrix (master + stable self-build + published APK):
# ./run-android-variant-matrix.sh
#
# Usage:
# ./run-android-build-and-smoke.sh
# BRANCH=master STACK=goa ./run-android-build-and-smoke.sh
# BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh
# SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
SMOKE="$ROOT/run-android-pay-smoke.sh"
SRC="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}"
SKIP_PULL="${SKIP_PULL:-0}"
# Prefer existing inference fix branch for GOA withdraw (spinner/exchange resolve).
# Matrix uses BRANCH=master and BRANCH=$STABLE_TAG explicitly.
BRANCH="${BRANCH:-dev/hernani-inference/fix-bank-withdraw-auto-exchange}"
# Optional human label for logs / artifact names (default = sanitized BRANCH)
LABEL="${LABEL:-}"
VARIANT="${VARIANT:-fdroidDebug}"
export ANDROID_HOME="${ANDROID_HOME:-$HOME/Android/Sdk}"
export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}"
if [ ! -x "$SRC/gradlew" ]; then
echo "taler-android not found at $SRC (set TALER_ANDROID_SRC=…)" >&2
exit 2
fi
if [ ! -d "$ANDROID_HOME" ]; then
echo "ANDROID_HOME missing: $ANDROID_HOME" >&2
exit 2
fi
printf 'sdk.dir=%s\n' "$ANDROID_HOME" >"$SRC/local.properties"
# --- checkout REF (branch, origin/branch, or tag) --------------------------------
cd "$SRC"
if [ -d .git ]; then
git fetch origin --tags 2>/dev/null || git fetch origin 2>/dev/null || true
REF="$BRANCH"
checked=0
if [ -n "$REF" ]; then
if git rev-parse --verify "refs/tags/${REF}" >/dev/null 2>&1; then
echo "checkout tag ${REF} (stable / release self-build)"
git checkout -f "refs/tags/${REF}"
checked=1
SKIP_PULL=1
elif git rev-parse --verify "origin/${REF}" >/dev/null 2>&1; then
echo "checkout origin/${REF}"
git checkout -B "$REF" "origin/${REF}" 2>/dev/null || git checkout "$REF" 2>/dev/null || true
checked=1
elif git rev-parse --verify "${REF}" >/dev/null 2>&1; then
echo "checkout ${REF}"
git checkout -f "$REF"
checked=1
else
echo "WARN: ref '${REF}' not found locally after fetch — staying on current HEAD" >&2
fi
fi
if [ "$SKIP_PULL" != "1" ] && [ "$checked" = "1" ]; then
# only pull real branches, never tags
if git symbolic-ref -q HEAD >/dev/null 2>&1; then
br=$(git rev-parse --abbrev-ref HEAD)
git pull --ff-only origin "$br" 2>/dev/null || true
fi
fi
fi
COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown)
HEAD_DESC=$(git describe --tags --always 2>/dev/null || echo "$COMMIT")
BR_NOW=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo detached)
if [ -z "$LABEL" ]; then
LABEL=$(printf '%s' "$BRANCH" | tr '/:' '--' | tr -c 'A-Za-z0-9._-' '-' | cut -c1-48)
fi
echo "source: $SRC @ $COMMIT ($HEAD_DESC) ref=$BRANCH label=$LABEL head=$BR_NOW"
echo "variant: $VARIANT"
# Map VARIANT → gradle task
case "$VARIANT" in
fdroidDebug|FdroidDebug) task=assembleFdroidDebug ;;
fdroidRelease|FdroidRelease) task=assembleFdroidRelease ;;
nightlyDebug|NightlyDebug) task=assembleNightlyDebug ;;
googleDebug|GoogleDebug) task=assembleGoogleDebug ;;
*)
task="assemble$(printf '%s' "$VARIANT" | python3 -c 'import sys; s=sys.stdin.read().strip(); print(s[:1].upper()+s[1:] if s else "FdroidDebug")')"
;;
esac
# Low-RAM hosts: cap JVM
export GRADLE_OPTS="${GRADLE_OPTS:--Xmx1280m -Dorg.gradle.daemon=false -Dorg.gradle.workers.max=1}"
echo "gradle :wallet:$task …"
./gradlew ":wallet:$task" --no-daemon --max-workers="${GRADLE_MAX_WORKERS:-1}" \
-Dorg.gradle.jvmargs="${GRADLE_JVMARGS:--Xmx1280m}"
# Prefer fdroid debug APK if multiple exist
APK=$(find "$SRC/wallet/build/outputs/apk" -path '*fdroid*debug*.apk' -type f 2>/dev/null | sort | tail -1)
if [ -z "$APK" ] || [ ! -f "$APK" ]; then
APK=$(find "$SRC/wallet/build/outputs/apk" -name '*.apk' -type f 2>/dev/null | sort | tail -1)
fi
if [ -z "$APK" ] || [ ! -f "$APK" ]; then
echo "no APK under wallet/build/outputs/apk" >&2
exit 4
fi
echo "built apk: $APK ($(wc -c <"$APK") bytes)"
PKG="${PKG:-}"
if [ -z "$PKG" ] && command -v aapt >/dev/null 2>&1; then
PKG=$(aapt dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1)
fi
if [ -z "$PKG" ] && command -v aapt2 >/dev/null 2>&1; then
PKG=$(aapt2 dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1)
fi
: "${PKG:=net.taler.wallet.fdroid.debug}"
mkdir -p "$ROOT/apks" "$ROOT/out"
STAMP_APK="$ROOT/apks/wallet-${LABEL}-${COMMIT}.apk"
cp -f "$APK" "$STAMP_APK"
echo "copied → $STAMP_APK"
{
echo "label=$LABEL"
echo "ref=$BRANCH"
echo "commit=$COMMIT"
echo "describe=$HEAD_DESC"
echo "apk=$STAMP_APK"
echo "pkg=$PKG"
echo "task=$task"
} | tee "$ROOT/out/build-${LABEL}.txt"
echo "$COMMIT" >"$ROOT/out/build-commit.txt"
echo "$APK" >"$ROOT/out/build-apk-path.txt"
export APK_PATH="$STAMP_APK"
export APK_NAME
APK_NAME="$(basename "$STAMP_APK")"
export PKG
export OUT_DIR="${OUT_DIR:-$ROOT/out-source-$LABEL}"
mkdir -p "$OUT_DIR"
cp -f "$ROOT/out/build-${LABEL}.txt" "$OUT_DIR/build-meta.txt" 2>/dev/null || true
SMOKE_CMD="$SMOKE"
if [ "${GUI:-0}" = "1" ]; then
SMOKE_CMD="$ROOT/run-android-gui-smoke.sh"
case "$OUT_DIR" in
*-gui) ;;
*) OUT_DIR="${OUT_DIR}-gui"; export OUT_DIR; mkdir -p "$OUT_DIR" ;;
esac
fi
echo "=== smoke label=$LABEL PKG=$PKG GUI=${GUI:-0} OUT_DIR=$OUT_DIR ==="
# Do not exec: allow matrix wrapper to run multiple variants
set +e
env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" APK_PATH="$APK_PATH" PKG="$PKG" OUT_DIR="$OUT_DIR" \
"$SMOKE_CMD"
rc=$?
set -e
echo "=== smoke label=$LABEL exit=$rc ==="
exit "$rc"

201
android-test/run-android-gui-smoke.sh Executable file
View file

@ -0,0 +1,201 @@
#!/usr/bin/env bash
# GUI-oriented Android smoke: **withdraw (Abheben)** then **pay**, each with
# deep-link entry + multi-round uiautomator taps. Not pay-only.
#
# 1) taler://withdraw/… + Confirm/Abheben/ToS taps
# 2) taler://pay… + Pay/Confirm taps
#
# See GUI-AUTOMATION-NOTES.md
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib_android_env.sh
. "$ROOT/lib_android_env.sh"
[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; }
if [ "${AUTO_GUI}" != "1" ]; then
echo "AUTO_GUI=0 → deep-link smoke only"
exec env AUTO_ANDROID=1 "$ROOT/run-android-pay-smoke.sh"
fi
export OUT_DIR="${OUT_DIR:-$ROOT/out-gui}"
export GUI_ROUNDS="${GUI_ROUNDS:-10}"
export GUI_SLEEP="${GUI_SLEEP:-3}"
# Install + deliver intents via existing smoke, but skip its weak single-tap
# by setting GUI_MODE=1 which we implement here more fully.
# Simpler: do install/URI here by sourcing patterns from pay-smoke via env + python.
# Delegate install + URI start to pay-smoke with GUI_AFTER=1 if we patch it;
# instead run a self-contained flow calling pay-smoke pieces.
APK_DIR="${APK_DIR:-$ROOT/apks}"
APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}"
APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}"
APK_PATH="${APK_PATH:-}"
PKG="${PKG:-net.taler.wallet.fdroid}"
STACK="${STACK:-auto}"
SERIAL="${SERIAL:-}"
mkdir -p "$OUT_DIR"
command -v adb >/dev/null || { echo "adb missing" >&2; exit 2; }
# Headless default: no host window; guest GLES via SwiftShader (uiautomator).
android_ensure_device || { echo "no adb device (try ./start-android-emulator.sh --wait)" >&2; exit 3; }
ADB=(adb -s "$SERIAL")
echo "device: $SERIAL mode: GUI (uiautomator) headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU}"
case "$STACK" in
auto)
if curl -sfS -m 5 -o /dev/null https://bank.hacktivism.ch/config; then STACK=goa; else STACK=stage; fi
;;
esac
case "$STACK" in
goa)
BANK=https://bank.hacktivism.ch
MERCHANT=https://taler.hacktivism.ch
WITHDRAW_JSON="$BANK/intro/demo-withdraw.json"
;;
stage)
BANK=https://stage.bank.lefrancpaysan.ch
MERCHANT=https://stage.monnaie.lefrancpaysan.ch
WITHDRAW_JSON="$BANK/intro/demo-withdraw.json"
;;
*) echo "unknown STACK" >&2; exit 2 ;;
esac
echo "stack: $STACK"
if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then
:
elif [ -f "$APK_DIR/$APK_NAME" ]; then
APK_PATH="$APK_DIR/$APK_NAME"
else
APK_PATH="$APK_DIR/$APK_NAME"
mkdir -p "$APK_DIR"
curl -fL --retry 3 -o "$APK_PATH" "$APK_URL"
fi
echo "apk: $APK_PATH pkg: $PKG"
"${ADB[@]}" wait-for-device
"${ADB[@]}" install -r "$APK_PATH"
"${ADB[@]}" shell settings put global window_animation_scale 0 || true
"${ADB[@]}" shell settings put global transition_animation_scale 0 || true
"${ADB[@]}" shell settings put global animator_duration_scale 0 || true
: "${DO_WITHDRAW:=1}"
: "${DO_PAY:=1}"
: "${SMOKE_STRICT:=0}"
WURI=""
PAY_URI=""
WITHDRAW_OK=0
PAY_OK=0
"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true
"${ADB[@]}" logcat -c 2>/dev/null || true
if [ "$DO_WITHDRAW" = "1" ]; then
echo
echo "======== 1 WITHDRAW (Abheben) ========"
WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")')
[ -n "$WURI" ] || { echo "no withdraw uri" >&2; exit 4; }
echo "withdraw: $WURI"
echo "$WURI" >"$OUT_DIR/last-withdraw.uri"
"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" | tee "$OUT_DIR/start-withdraw.txt"
sleep 5
python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY'
import json, sys, importlib.util
from pathlib import Path
lib_path, serial, out, rounds, sleep_s = (
Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5])
)
spec = importlib.util.spec_from_file_location("lib_ui", lib_path)
lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(lib)
st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd")
(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2))
print("gui-withdraw taps:", st.get("taps"))
print("gui-withdraw anr:", st.get("anr_dismissals"))
PY
else
echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========"
fi
if [ "$DO_PAY" = "1" ]; then
echo
echo "======== 2 PAY (Bezahlen) ========"
if [ "$STACK" = "stage" ]; then
if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \
"$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then
PAY_URI=$(printf '%s' "$RESP" | python3 -c 'import json,sys;d=json.load(sys.stdin);oid=d.get("order_id")or"";tok=d.get("token")or"";
print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok) if oid and tok else "")')
fi
else
PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana"
fi
if [ -n "$PAY_URI" ]; then
echo "pay: $PAY_URI"
echo "$PAY_URI" >"$OUT_DIR/last-pay.uri"
"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" | tee "$OUT_DIR/start-pay.txt"
sleep 5
python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY'
import json, sys, importlib.util
from pathlib import Path
lib_path, serial, out, rounds, sleep_s = (
Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5])
)
spec = importlib.util.spec_from_file_location("lib_ui", lib_path)
lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(lib)
st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay")
(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2))
print("gui-pay taps:", st.get("taps"))
print("gui-pay anr:", st.get("anr_dismissals"))
PY
else
echo "pay: (no URI)"
fi
else
echo "======== 2 PAY skipped (DO_PAY=0) ========"
fi
"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true
grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|Error|success|withdraw|pay' \
"$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true
if [ "$DO_WITHDRAW" = "1" ] && grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \
"$OUT_DIR/logcat.txt" 2>/dev/null; then
WITHDRAW_OK=1
fi
if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ] && \
grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \
"$OUT_DIR/logcat.txt" 2>/dev/null; then
PAY_OK=1
fi
rc=0
echo
echo "=== GUI smoke summary ($STACK) — withdraw + pay ==="
echo "package: $PKG DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY STRICT=$SMOKE_STRICT"
if [ "$DO_WITHDRAW" = "1" ]; then
echo "withdraw URI: delivered → $OUT_DIR/gui-withdraw-status.json"
echo "withdraw wallet-core: $([ "$WITHDRAW_OK" = 1 ] && echo YES || echo NO)"
if [ "$SMOKE_STRICT" = "1" ] && [ "$WITHDRAW_OK" != "1" ]; then
echo "FAIL: strict withdraw missing"; rc=11
fi
fi
if [ "$DO_PAY" = "1" ]; then
echo "pay URI: ${PAY_URI:-none} → $OUT_DIR/gui-pay-status.json"
echo "pay wallet-core: $([ "$PAY_OK" = 1 ] && echo YES || echo NO)"
if [ "$SMOKE_STRICT" = "1" ] && [ "$PAY_OK" != "1" ]; then
echo "FAIL: strict pay missing"; [ "$rc" -eq 0 ] && rc=13
fi
fi
if grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \
|| grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" "$OUT_DIR/gui-pay" 2>/dev/null; then
echo "FAIL: ANR"; rc=10
fi
echo "artifacts: $OUT_DIR/"
ls -la "$OUT_DIR" | sed 's/^/ /'
echo "Documented shortcuts → GUI-AUTOMATION-NOTES.md"
exit "$rc"

308
android-test/run-android-pay-smoke.sh Executable file
View file

@ -0,0 +1,308 @@
#!/usr/bin/env bash
# Android wallet smoke: install APK, then **withdraw (Abheben)** and **pay** via
# deep-link + best-effort UI taps. Both legs are first-class (not pay-only).
#
# Flow (default both on):
# 1) WITHDRAW taler://withdraw/… from bank demo-withdraw.json + confirm taps
# 2) PAY taler://pay… / pay-template + optional taps
#
# Env:
# DO_WITHDRAW=1 (default) DO_PAY=1 (default)
# SMOKE_STRICT=1 → require wallet-core withdraw (+ pay if DO_PAY) + no ANR
# WITHDRAW_GUI_ROUNDS multi-round Abheben taps (default 6)
#
# Usage:
# ./android-test/run-android-pay-smoke.sh
# STACK=goa ./android-test/run-android-pay-smoke.sh
# DO_PAY=0 ./android-test/run-android-pay-smoke.sh # withdraw-only
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib_android_env.sh
. "$ROOT/lib_android_env.sh"
if [ "${AUTO_ANDROID}" != "1" ]; then
echo "skipped: AUTO_ANDROID=${AUTO_ANDROID} (platform flags — see GUI-AUTOMATION-NOTES.md)"
exit 0
fi
APK_DIR="${APK_DIR:-$ROOT/apks}"
APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}"
APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}"
# Optional override: full path to an already-built APK (from-source)
APK_PATH="${APK_PATH:-}"
PKG="${PKG:-net.taler.wallet.fdroid}"
STACK="${STACK:-auto}" # auto | goa | stage
SERIAL="${SERIAL:-}"
OUT_DIR="${OUT_DIR:-$ROOT/out}"
mkdir -p "$APK_DIR" "$OUT_DIR"
if ! command -v adb >/dev/null 2>&1; then
echo "adb missing — install android-tools-adb or user SDK platform-tools" >&2
exit 2
fi
# Default: headless AVD (no host window). WINDOWED=1 / EMULATOR_HEADLESS=0 for UI.
if ! android_ensure_device; then
cat >&2 <<EOF
No adb device. Options:
1) Plug phone with USB debugging
2) Headless AVD (default): $ROOT/start-android-emulator.sh --wait
3) Windowed AVD: WINDOWED=1 $ROOT/start-android-emulator.sh --wait
4) AUTO_START_EMULATOR=0 disables auto-start from this smoke
EOF
exit 3
fi
ADB=(adb -s "$SERIAL")
echo "device: $SERIAL emulator_headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU}"
# Resolve stack endpoints
case "$STACK" in
auto)
if curl -sfS -m 5 -o /dev/null https://bank.hacktivism.ch/config 2>/dev/null; then
STACK=goa
else
STACK=stage
fi
;;
esac
case "$STACK" in
goa)
BANK=https://bank.hacktivism.ch
MERCHANT=https://taler.hacktivism.ch
WITHDRAW_JSON="$BANK/intro/demo-withdraw.json"
PAY_HINT="paivana template / goa-shop"
;;
stage)
BANK=https://stage.bank.lefrancpaysan.ch
MERCHANT=https://stage.monnaie.lefrancpaysan.ch
WITHDRAW_JSON="$BANK/intro/demo-withdraw.json"
PAY_HINT="farmer shop templates"
;;
*) echo "unknown STACK=$STACK" >&2; exit 2 ;;
esac
echo "stack: $STACK bank: $BANK"
# APK: explicit path, or download published F-Droid into APK_DIR
if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then
:
elif [ -f "$APK_DIR/$APK_NAME" ]; then
APK_PATH="$APK_DIR/$APK_NAME"
else
APK_PATH="$APK_DIR/$APK_NAME"
echo "downloading published wallet APK…"
curl -fL --retry 3 -o "$APK_PATH" "$APK_URL"
fi
echo "apk: $APK_PATH ($(wc -c <"$APK_PATH") bytes)"
echo "pkg: $PKG"
: "${DO_WITHDRAW:=1}"
: "${DO_PAY:=1}"
: "${WITHDRAW_GUI_ROUNDS:=6}"
: "${PAY_GUI_ROUNDS:=4}"
: "${SMOKE_STRICT:=0}"
"${ADB[@]}" wait-for-device
"${ADB[@]}" install -r "$APK_PATH"
"${ADB[@]}" shell settings put global window_animation_scale 0 || true
"${ADB[@]}" shell settings put global transition_animation_scale 0 || true
"${ADB[@]}" shell settings put global animator_duration_scale 0 || true
"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true
"${ADB[@]}" logcat -c 2>/dev/null || true
WURI=""
PAY_URI=""
WITHDRAW_OK=0
PAY_OK=0
# ═══════════════════════════════════════════════════════════════
# 1) WITHDRAW / Abheben (required by default — not pay-only)
# ═══════════════════════════════════════════════════════════════
if [ "$DO_WITHDRAW" = "1" ]; then
echo
echo "======== 1 WITHDRAW (Abheben) ========"
WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")')
if [ -z "$WURI" ]; then
echo "FAIL: no taler_withdraw_uri from $WITHDRAW_JSON" >&2
exit 4
fi
echo "withdraw URI: $WURI"
echo "$WURI" >"$OUT_DIR/last-withdraw.uri"
"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" 2>&1 | tee "$OUT_DIR/start-withdraw.txt"
sleep 6
"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/01-after-withdraw-uri.png" || true
"${ADB[@]}" shell uiautomator dump /sdcard/ui.xml 2>/dev/null || true
"${ADB[@]}" pull /sdcard/ui.xml "$OUT_DIR/01-ui.xml" 2>/dev/null || true
# Multi-round confirm / Abheben / ToS (same idea as GUI smoke)
if [ -f "$ROOT/lib_ui.py" ]; then
python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$WITHDRAW_GUI_ROUNDS" 2 <<'PY' || true
import json, sys, importlib.util
from pathlib import Path
lib_path, serial, out, rounds, sleep_s = (
Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5])
)
spec = importlib.util.spec_from_file_location("lib_ui", lib_path)
lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(lib)
st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd")
(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2))
print("withdraw GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals"))
PY
else
python3 - "$SERIAL" "$OUT_DIR/01-ui.xml" <<'PY' || true
import re, subprocess, sys
serial, path = sys.argv[1], sys.argv[2]
try:
xml = open(path, errors="replace").read()
except FileNotFoundError:
raise SystemExit(0)
labels = [
"Confirm", "Confirm withdrawal", "Withdraw", "Accept", "Continue", "OK",
"I accept", "Agree", "Bestätigen", "Abheben", "Akzeptieren", "Retirer",
]
for label in labels:
for pat in (
r'text="%s"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"' % re.escape(label),
r'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*text="%s"' % re.escape(label),
):
for m in re.finditer(pat, xml, flags=re.I):
x1, y1, x2, y2 = map(int, m.groups())
x, y = (x1 + x2) // 2, (y1 + y2) // 2
print(f"tap {label} @ {x},{y}")
subprocess.run(["adb", "-s", serial, "shell", "input", "tap", str(x), str(y)], check=False)
PY
sleep 8
fi
"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/02-after-withdraw-taps.png" || true
# Mid-flow logcat: withdraw must register before pay
"${ADB[@]}" logcat -d -t 400 >"$OUT_DIR/logcat-after-withdraw.txt" || true
if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \
"$OUT_DIR/logcat-after-withdraw.txt" 2>/dev/null; then
WITHDRAW_OK=1
echo "withdraw: wallet-core activity YES"
else
echo "withdraw: wallet-core activity NOT seen yet (onboarding/ANR/slow?)"
fi
else
echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========"
fi
# ═══════════════════════════════════════════════════════════════
# 2) PAY (after withdraw; still first-class, but second leg)
# ═══════════════════════════════════════════════════════════════
if [ "$DO_PAY" = "1" ]; then
echo
echo "======== 2 PAY (Bezahlen) ========"
if [ "$STACK" = "stage" ]; then
if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \
"$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then
PAY_URI=$(printf '%s' "$RESP" | python3 -c '
import json,sys
d=json.load(sys.stdin)
oid=d.get("order_id") or ""
tok=d.get("token") or ""
if oid and tok:
print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok))
' 2>/dev/null || true)
fi
elif [ "$STACK" = "goa" ]; then
PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana"
fi
if [ -n "$PAY_URI" ]; then
echo "pay URI: $PAY_URI ($PAY_HINT)"
echo "$PAY_URI" >"$OUT_DIR/last-pay.uri"
"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" 2>&1 | tee "$OUT_DIR/start-pay.txt"
sleep 6
if [ -f "$ROOT/lib_ui.py" ]; then
python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$PAY_GUI_ROUNDS" 2 <<'PY' || true
import json, sys, importlib.util
from pathlib import Path
lib_path, serial, out, rounds, sleep_s = (
Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5])
)
spec = importlib.util.spec_from_file_location("lib_ui", lib_path)
lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(lib)
st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay")
(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2))
print("pay GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals"))
PY
fi
"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/03-after-pay-uri.png" || true
else
echo "pay: (no public template URI for stack=$STACK)"
fi
else
echo "======== 2 PAY skipped (DO_PAY=0) ========"
fi
# Final evidence
"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true
grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|error|Error|success|withdraw|pay' \
"$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true
# Re-evaluate withdraw/pay from full logcat
if [ "$DO_WITHDRAW" = "1" ]; then
if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \
"$OUT_DIR/logcat.txt" 2>/dev/null; then
WITHDRAW_OK=1
fi
fi
if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ]; then
if grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \
"$OUT_DIR/logcat.txt" 2>/dev/null; then
PAY_OK=1
fi
fi
rc=0
echo
echo "=== summary (withdraw + pay) ==="
echo "package: $PKG stack: $STACK"
echo "DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY SMOKE_STRICT=$SMOKE_STRICT"
if [ "$DO_WITHDRAW" = "1" ]; then
echo "withdraw URI: ${WURI:-(none)}"
if [ "$WITHDRAW_OK" = "1" ]; then
echo "withdraw wallet-core: YES"
else
echo "withdraw wallet-core: NO"
if [ "$SMOKE_STRICT" = "1" ]; then
echo "FAIL: SMOKE_STRICT requires withdraw (prepareBank/accept/confirm) in logcat"
rc=11
fi
fi
fi
if [ "$DO_PAY" = "1" ]; then
echo "pay URI: ${PAY_URI:-(none)}"
if [ -z "$PAY_URI" ]; then
echo "pay: no URI (template/API)"
if [ "$SMOKE_STRICT" = "1" ]; then
echo "FAIL: SMOKE_STRICT requires a pay URI when DO_PAY=1"
rc=12
fi
elif [ "$PAY_OK" = "1" ]; then
echo "pay wallet-core: YES"
else
echo "pay wallet-core: NO (intent may still have been delivered)"
if [ "$SMOKE_STRICT" = "1" ]; then
echo "FAIL: SMOKE_STRICT requires pay path activity in logcat"
[ "$rc" -eq 0 ] && rc=13
fi
fi
fi
if grep -qiE "isn't responding|isn.t responding|System UI isn" "$OUT_DIR/01-ui.xml" 2>/dev/null \
|| grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \
|| grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" 2>/dev/null; then
echo "FAIL: System UI / app ANR"
rc=10
fi
echo "artifacts: $OUT_DIR/"
ls -la "$OUT_DIR" | sed 's/^/ /'
echo
echo "NOTE: Full unattended Abheben+Bezahlen needs responsive device (≥6–8 GiB host RAM)."
echo "Smoke legs: withdraw=$( [ "$DO_WITHDRAW" = 1 ] && echo on || echo off) pay=$( [ "$DO_PAY" = 1 ] && echo on || echo off) rc=$rc"
exit "$rc"

322
android-test/run-android-variant-matrix.sh Executable file
View file

@ -0,0 +1,322 @@
#!/usr/bin/env bash
# Run Android wallet smoke against several build / published variants.
#
# Default order:
# 1) stable-self — **only** self-build of current published stable (WARN)
# 2) published — current F-Droid APK
# 3) older — F-Droid APKs ~3/6/9/12 mo if still hosted (WARN, no self-build)
# 4) master — tip self-build (WARN)
#
# Self-build policy: current stable + master/fix only — never rebuild old releases.
#
# Severity: age ≤14d → BLOCKER; milestones/older/self-builds → WARN
#
# Env:
# STABLE_TAG, STABLE_APK_URL, STABLE_APK_NAME
# MASTER_BRANCH, FIX_BRANCH
# VARIANTS default: stable-self,published,older,master
# INCLUDE_OLDER=1 expand token "older" / "milestones" (default 1)
# MILESTONE_DAYS default "90 180 270 365" (≈ 3,6,9,12 months)
# MILESTONE_TOLERANCE_DAYS default 50 — max distance to accept a tag
# BLOCKER_MAX_AGE_DAYS default 14
# PUBLISHED_FORCE_BLOCKER=1 always treat current published as blocker
# FAIL_FAST=1, GUI=0, STACK=…
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib_android_env.sh
. "$ROOT/lib_android_env.sh"
# shellcheck source=lib_release_age.sh
. "$ROOT/lib_release_age.sh"
: "${STABLE_TAG:=wallet-1.6.1}"
: "${STABLE_APK_URL:=https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}"
: "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}"
: "${MASTER_BRANCH:=master}"
: "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}"
: "${VARIANTS:=stable-self,published,older,master}"
: "${INCLUDE_OLDER:=1}"
: "${MILESTONE_DAYS:=90 180 270 365}"
: "${MILESTONE_TOLERANCE_DAYS:=50}"
: "${BLOCKER_MAX_AGE_DAYS:=14}"
: "${PUBLISHED_FORCE_BLOCKER:=0}"
: "${FAIL_FAST:=1}"
: "${GUI:=0}"
: "${STACK:=auto}"
: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}"
export TALER_ANDROID_SRC BLOCKER_MAX_AGE_DAYS MILESTONE_DAYS MILESTONE_TOLERANCE_DAYS
TS=$(date +%Y%m%d-%H%M%S)
MATRIX_OUT="${MATRIX_OUT:-$ROOT/out-matrix-$TS}"
mkdir -p "$MATRIX_OUT"
SUMMARY="$MATRIX_OUT/SUMMARY.txt"
: >"$SUMMARY"
echo "=== Android variant matrix ===" | tee -a "$SUMMARY"
echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY"
echo "STABLE_TAG=$STABLE_TAG BLOCKER_MAX_AGE_DAYS=$BLOCKER_MAX_AGE_DAYS" | tee -a "$SUMMARY"
echo "milestones(days)=$MILESTONE_DAYS tolerance=$MILESTONE_TOLERANCE_DAYS (always WARN)" | tee -a "$SUMMARY"
echo "out: $MATRIX_OUT" | tee -a "$SUMMARY"
android_ensure_device || {
echo "no adb device — start ./start-android-emulator.sh --wait" >&2
exit 3
}
export SERIAL ANDROID_SERIAL
echo "device: $SERIAL" | tee -a "$SUMMARY"
# --- expand VARIANTS (token "older" → concrete older-pub / older-self entries) ---
# Append one older release as WARN-only — **F-Droid APK only** (no self-build).
# Returns 0 if queued, 1 if skipped (no published APK).
append_older_warn() {
local t="$1" age="$2" target="${3:-}"
local vn meta apk_vc apk_url
vn="${t#wallet-}"
if [ -n "$target" ]; then
echo " milestone ~${target}d → $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY"
else
echo " older $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY"
fi
meta=$(fdroid_apk_for_version_name "$vn" 2>/dev/null || true)
if [ -n "$meta" ]; then
apk_vc="${meta%%|*}"
apk_url="${meta#*|}"
if fdroid_apk_exists "$apk_url"; then
out+=("older-pub:${vn}:${apk_vc}:warn")
return 0
fi
echo " SKIP $t — no F-Droid APK at ${apk_url:-?} (self-build only for current stable)" | tee -a "$SUMMARY"
else
echo " SKIP $t — cannot map F-Droid versionCode (self-build only for current stable)" | tee -a "$SUMMARY"
fi
return 1
}
expand_variants() {
local raw="$1"
local -a out=()
local v tag vn age sev line t target
IFS=',' read -r -a toks <<<"$raw"
for v in "${toks[@]}"; do
v=$(echo "$v" | tr -d '[:space:]')
[ -n "$v" ] || continue
if [ "$v" = "older" ] || [ "$v" = "older-releases" ] || [ "$v" = "milestones" ]; then
[ "$INCLUDE_OLDER" = "1" ] || continue
# 3/6/9/12-month samples: published APK only, always WARN.
local found=0 queued=0
while read -r t age target; do
[ -n "$t" ] || continue
[ "$t" = "$STABLE_TAG" ] && continue
found=1
if append_older_warn "$t" "$age" "$target"; then
queued=$((queued + 1))
fi
done < <(select_milestone_tags "$STABLE_TAG" 2>/dev/null || true)
if [ "$found" = "0" ]; then
echo " (no milestone tags within ±${MILESTONE_TOLERANCE_DAYS}d of $MILESTONE_DAYS)" | tee -a "$SUMMARY"
elif [ "$queued" = "0" ]; then
echo " (milestone tags found but none still on F-Droid — no older APK smokes)" | tee -a "$SUMMARY"
fi
else
out+=("$v")
fi
done
printf '%s\n' "${out[@]}"
}
mapfile -t VLIST < <(expand_variants "$VARIANTS")
echo "expanded variants: ${VLIST[*]}" | tee -a "$SUMMARY"
declare -a RESULTS=()
declare -a WARNINGS=()
overall=0
severity_for_published_current() {
if [ "${PUBLISHED_FORCE_BLOCKER}" = "1" ]; then
echo blocker
return
fi
local days
days=$(release_age_days "$STABLE_TAG" 2>/dev/null || true)
severity_from_age_days "$days"
}
run_published_apk() {
local label="$1"
local apk_url="$2"
local apk_name="$3"
local sev="$4"
local out="$MATRIX_OUT/$label"
mkdir -p "$out"
echo "" | tee -a "$SUMMARY"
echo "-------- variant: $label (F-Droid APK) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY"
echo " url=$apk_url" | tee -a "$SUMMARY"
local smoke="$ROOT/run-android-pay-smoke.sh"
[ "$GUI" = "1" ] && smoke="$ROOT/run-android-gui-smoke.sh"
local strict=1
[ "$sev" = "warn" ] && strict="${SMOKE_STRICT_WARN:-0}"
[ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}"
set +e
env STACK="$STACK" \
APK_URL="$apk_url" \
APK_NAME="$apk_name" \
APK_PATH="" \
PKG=net.taler.wallet.fdroid \
OUT_DIR="$out" \
SERIAL="$SERIAL" \
AUTO_START_EMULATOR=0 \
SMOKE_STRICT="$strict" \
"$smoke"
local rc=$?
set -e
echo "$label exit=$rc severity=$sev" | tee -a "$SUMMARY"
echo "$label $rc $sev $apk_url" >>"$MATRIX_OUT/results.tsv"
return "$rc"
}
run_selfbuild() {
local label="$1"
local ref="$2"
local sev="${3:-warn}"
local out="$MATRIX_OUT/$label"
mkdir -p "$out"
echo "" | tee -a "$SUMMARY"
echo "-------- variant: $label (self-build ref=$ref) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY"
local strict="${SMOKE_STRICT:-1}"
[ "$sev" = "warn" ] && strict="${SMOKE_STRICT_SELF:-0}"
[ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}"
set +e
env STACK="$STACK" \
BRANCH="$ref" \
LABEL="$label" \
GUI="$GUI" \
OUT_DIR="$out" \
SERIAL="$SERIAL" \
ANDROID_SERIAL="$SERIAL" \
AUTO_START_EMULATOR=0 \
PKG=net.taler.wallet.fdroid.debug \
SMOKE_STRICT="$strict" \
TALER_ANDROID_SRC="$TALER_ANDROID_SRC" \
"$ROOT/run-android-build-and-smoke.sh"
local rc=$?
set -e
echo "$label (ref=$ref) exit=$rc severity=$sev" | tee -a "$SUMMARY"
echo "$label $rc $ref $sev" >>"$MATRIX_OUT/results.tsv"
if [ -f "$out/build-meta.txt" ]; then
echo "build-meta:" | tee -a "$SUMMARY"
cat "$out/build-meta.txt" | tee -a "$SUMMARY"
fi
return "$rc"
}
record_result() {
local v="$1" rc="$2" sev="$3"
if [ "$rc" -eq 0 ]; then
RESULTS+=("$v:OK($sev)")
return
fi
case "$sev" in
warn)
echo "WARN: $v failed rc=$rc — not a blocker; continuing" | tee -a "$SUMMARY"
WARNINGS+=("$v:$rc")
RESULTS+=("$v:WARN($rc)")
;;
blocker)
echo "BLOCKER: $v failed rc=$rc (release within ${BLOCKER_MAX_AGE_DAYS}d window)" | tee -a "$SUMMARY"
RESULTS+=("$v:BLOCKER($rc)")
overall=1
if [ "$FAIL_FAST" = "1" ]; then
echo "FAIL_FAST: stopping after blocker $v" | tee -a "$SUMMARY"
return 2
fi
;;
soft)
echo "FAIL: $v rc=$rc" | tee -a "$SUMMARY"
RESULTS+=("$v:FAIL($rc)")
overall=1
if [ "$FAIL_FAST" = "1" ]; then
echo "FAIL_FAST: stopping after $v" | tee -a "$SUMMARY"
return 2
fi
;;
esac
return 0
}
for raw in "${VLIST[@]}"; do
v=$(echo "$raw" | tr -d '[:space:]')
[ -n "$v" ] || continue
rc=0
sev=warn
stop=0
case "$v" in
published|fdroid|stable-published)
sev=$(severity_for_published_current)
age=$(release_age_days "$STABLE_TAG" 2>/dev/null || echo "?")
echo "published current: tag=$STABLE_TAG age_days=$age → severity=$sev" | tee -a "$SUMMARY"
run_published_apk "published" "$STABLE_APK_URL" "$STABLE_APK_NAME" "$sev" || rc=$?
;;
stable-self|stable|self-stable|release-self)
# rebuild of stable: always warn (toolchain), age does not make self-build a blocker
sev=warn
run_selfbuild "stable-self" "$STABLE_TAG" warn || rc=$?
;;
master|main)
sev=warn
run_selfbuild "master" "$MASTER_BRANCH" warn || rc=$?
;;
fix|fix-branch|inference)
sev=soft
run_selfbuild "fix" "$FIX_BRANCH" soft || rc=$?
;;
older-pub:*)
# older-pub:VERSION:VC:SEV
IFS=':' read -r _ vn apk_vc sev <<<"$v"
: "${sev:=warn}"
label="older-pub-${vn}"
url="https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk"
name="net.taler.wallet.fdroid_${apk_vc}.apk"
age=$(release_age_days "wallet-${vn}" 2>/dev/null || echo "?")
echo "older published $vn age_days=$age severity=$sev" | tee -a "$SUMMARY"
run_published_apk "$label" "$url" "$name" "$sev" || rc=$?
v="$label"
;;
older-self:*)
# Explicit older-self is disabled by policy (self-build = current stable only).
echo "SKIP $v — self-build only for current stable ($STABLE_TAG); use older-pub or published" | tee -a "$SUMMARY"
RESULTS+=("$v:SKIP(no-selfbuild-older)")
continue
;;
*)
# raw git ref: only master-like / fix via named tokens; bare wallet-* tags → no self-build
if [[ "$v" =~ ^wallet-[0-9] ]]; then
echo "SKIP self-build $v — only current stable self-build ($STABLE_TAG) is enabled" | tee -a "$SUMMARY"
RESULTS+=("$v:SKIP(no-selfbuild-older)")
continue
fi
sev=warn
run_selfbuild "ref-${v//\//-}" "$v" warn || rc=$?
;;
esac
record_result "$v" "$rc" "$sev" || stop=$?
[ "$stop" = "2" ] && break
done
echo "" | tee -a "$SUMMARY"
echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY"
echo "rule: age ≤ ${BLOCKER_MAX_AGE_DAYS}d → BLOCKER; 3/6/9/12mo milestones + older → WARN only" | tee -a "$SUMMARY"
for r in "${RESULTS[@]}"; do
echo " $r" | tee -a "$SUMMARY"
done
if [ "${#WARNINGS[@]}" -gt 0 ]; then
echo "warnings (non-blocking):" | tee -a "$SUMMARY"
for w in "${WARNINGS[@]}"; do
echo " WARN $w" | tee -a "$SUMMARY"
done
fi
echo ""
echo "Artifacts: $MATRIX_OUT"
exit "$overall"

366
android-test/run-goa-gui-chain.sh Executable file
View file

@ -0,0 +1,366 @@
#!/usr/bin/env bash
# GOA (and optionally stage) GUI chain on Android — adapted from
# taler-android branch **dev/hernani-inference/gui-workflows**
# (`scripts/goa-chain-emu.sh`, docs/gui-workflows.md).
#
# Linux + macOS: uses adb + uiautomator only (no Homebrew hard dependency).
# Upstream scripts are documented as macOS-only for SDK install paths.
#
# Flow:
# 1) API mint withdraw (explorer) → VIEW taler://withdraw → GUI taps
# 2) Merchant template POST → VIEW taler://pay → GUI taps
#
# Usage:
# ./run-goa-gui-chain.sh
# STACK=stage ./run-goa-gui-chain.sh
# WITHDRAW_AMOUNTS='GOA:10' PAY_LIMIT=2 ./run-goa-gui-chain.sh
# PKG=net.taler.wallet.fdroid.debug ./run-goa-gui-chain.sh --no-clear
#
# Env: ANDROID_SERIAL, PKG, EXP_PW_FILE, BANK, MERCHANT, INSTANCE,
# WITHDRAW_AMOUNTS, PRODUCTS, PAY_LIMIT, SHOTDIR, PAUSE
#
# See GUI-AUTOMATION-NOTES.md
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
CLEAR=1
for a in "$@"; do
case "$a" in
--no-clear) CLEAR=0 ;;
-h|--help) sed -n '2,28p' "$0"; exit 0 ;;
esac
done
die() { echo "ERROR: $*" >&2; exit 1; }
step() { echo; echo "======== $* ========"; }
info() { echo " $*"; }
pause() { sleep "${1:-${PAUSE:-0.8}}"; }
# Platform flags + headless emulator defaults (see GUI-AUTOMATION-NOTES.md)
# shellcheck source=lib_android_env.sh
. "$ROOT/lib_android_env.sh"
[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; }
[ "${AUTO_GUI}" = "1" ] || { echo "skipped: AUTO_GUI=${AUTO_GUI} (chain is GUI)"; exit 0; }
command -v adb >/dev/null || die "adb not found"
command -v python3 >/dev/null || die "python3 required"
STACK="${STACK:-goa}"
case "$STACK" in
goa|hacktivism)
PKG="${PKG:-net.taler.wallet.fdroid.debug}"
BANK="${BANK:-https://bank.hacktivism.ch}"
MERCHANT="${MERCHANT:-https://taler.hacktivism.ch}"
INSTANCE=$(printf '%s' "${INSTANCE:-goa-shop}" | tr '[:upper:]' '[:lower:]')
DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-goa-demo-cp4zqk}" | tr '[:upper:]' '[:lower:]')
DONATE_TEMPLATE="${DONATE_TEMPLATE:-goa-free}"
DEFAULT_AMOUNTS=("GOA:10" "GOA:20")
DEFAULT_PAYS=("product:orbit-sticker" "product:nebula-coffee" "product:voidwave-playlist" "donate:GOA:12")
CUR_HINT=GOA
;;
stage|testpaysan)
PKG="${PKG:-net.taler.wallet.fdroid.debug}"
BANK="${BANK:-https://stage.bank.lefrancpaysan.ch}"
MERCHANT="${MERCHANT:-https://stage.monnaie.lefrancpaysan.ch}"
INSTANCE=$(printf '%s' "${INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]')
DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]')
DONATE_TEMPLATE="${DONATE_TEMPLATE:-don-panier-libre}"
DEFAULT_AMOUNTS=("TESTPAYSAN:10" "TESTPAYSAN:20")
DEFAULT_PAYS=("product:panier-legumes" "product:fromage-chevre" "product:oeufs-6")
CUR_HINT=TESTPAYSAN
;;
*) die "STACK must be goa|stage (got $STACK)" ;;
esac
ACT="${ACT:-$PKG/net.taler.wallet.main.MainActivity}"
EXP_USER="${EXP_USER:-explorer}"
PAY_LIMIT="${PAY_LIMIT:-4}"
PAUSE="${PAUSE:-0.8}"
SHOTDIR="${SHOTDIR:-$ROOT/out-gui-chain/$(date +%Y%m%d-%H%M%S)-$STACK}"
mkdir -p "$SHOTDIR"
# Explorer password (same search as taler-monitoring ladder)
if [[ -z "${EXP_PW_FILE:-}" ]]; then
for cand in \
"${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \
"${HOME}/.config/taler-landing/bank-explorer-password.txt"
do
[[ -f "$cand" ]] && EXP_PW_FILE="$cand" && break
done
fi
# stage: try stagepaysan secret via ssh if missing
if [[ ! -f "${EXP_PW_FILE:-}" && "$STACK" = "stage" ]]; then
for host in francpaysan-stage-user francpaysan-host; do
tmp=$(mktemp)
if ssh -o BatchMode=yes -o ConnectTimeout=10 "$host" \
'tr -d "\n\r" </mnt/data/stagepaysan/bank/secrets/bank-explorer-password.txt' \
>"$tmp" 2>/dev/null && [[ -s "$tmp" ]]; then
EXP_PW_FILE="$tmp"
info "explorer password via $host"
break
fi
rm -f "$tmp"
done
fi
[[ -n "${EXP_PW_FILE:-}" && -f "$EXP_PW_FILE" ]] || die "set EXP_PW_FILE (explorer password)"
# Headless AVD by default (no host window). WINDOWED=1 for a visible emulator.
SERIAL="${SERIAL:-${ANDROID_SERIAL:-}}"
android_ensure_device || die "no adb device (./start-android-emulator.sh --wait)"
export ANDROID_SERIAL="$SERIAL"
info "ANDROID_SERIAL=$ANDROID_SERIAL STACK=$STACK PKG=$PKG SHOTDIR=$SHOTDIR headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU"
ADB=(adb -s "$ANDROID_SERIAL")
adb_sh() { "${ADB[@]}" shell "$@"; }
shot() {
local n="$1"
adb_sh screencap -p "/sdcard/demo-$n.png" 2>/dev/null || true
"${ADB[@]}" pull "/sdcard/demo-$n.png" "$SHOTDIR/$n.png" >/dev/null 2>&1 || true
info "[shot] $SHOTDIR/$n.png"
}
tap_text() {
local label="$1"
adb_sh uiautomator dump /sdcard/ui.xml >/dev/null 2>&1 || return 1
"${ADB[@]}" pull /sdcard/ui.xml "$SHOTDIR/ui.xml" >/dev/null 2>&1 || return 1
SERIAL="$ANDROID_SERIAL" python3 - "$label" "$SHOTDIR/ui.xml" <<'PY'
import os, re, subprocess, sys
from pathlib import Path
label, path = sys.argv[1], sys.argv[2]
serial = os.environ.get("SERIAL") or os.environ.get("ANDROID_SERIAL") or ""
xml = Path(path).read_text(errors="replace")
# ANR dismiss
if "isn't responding" in xml or "reagiert nicht" in xml.lower():
for lab in ("Wait", "Warten"):
for m in re.finditer(rf'text="{lab}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', xml):
x = (int(m.group(1))+int(m.group(3)))//2
y = (int(m.group(2))+int(m.group(4)))//2
subprocess.run(["adb","-s",serial,"shell","input","tap",str(x),str(y)], check=False)
print(f"anr-wait @{x},{y}")
sys.exit(0)
for pat in [
rf'text="{re.escape(label)}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"',
rf'text="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"',
rf'content-desc="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"',
]:
for m in re.finditer(pat, xml, re.I):
chunk = xml[max(0, m.start()-120):m.end()]
if 'enabled="false"' in chunk:
continue
x = (int(m.group(1))+int(m.group(3)))//2
y = (int(m.group(2))+int(m.group(4)))//2
print(f"tap {label!r} @{x},{y}")
cmd = ["adb","shell","input","tap",str(x),str(y)]
if serial:
cmd = ["adb","-s",serial,"shell","input","tap",str(x),str(y)]
subprocess.run(cmd, check=False)
sys.exit(0)
print("miss", label)
sys.exit(1)
PY
}
mint_withdraw() {
local amount="$1"
EXP_USER="$EXP_USER" EXP_PW_FILE="$EXP_PW_FILE" BANK="$BANK" \
python3 - "$amount" <<'PY'
import base64, json, os, ssl, sys, urllib.request
from pathlib import Path
amount = sys.argv[1]
bank = os.environ["BANK"].rstrip("/")
user = os.environ["EXP_USER"]
pw = Path(os.environ["EXP_PW_FILE"]).read_text().strip()
auth = "Basic " + base64.b64encode(f"{user}:{pw}".encode()).decode()
ctx = ssl.create_default_context()
def req(method, url, data=None, headers=None):
h = dict(headers or {})
body = None
if data is not None:
body = json.dumps(data).encode()
h["Content-Type"] = "application/json"
r = urllib.request.Request(url, data=body, headers=h, method=method)
with urllib.request.urlopen(r, context=ctx, timeout=25) as resp:
return json.load(resp)
tok = req(
"POST",
f"{bank}/accounts/{user}/token",
{"scope": "readwrite", "duration": {"d_us": 3600_000_000}},
{"Authorization": auth},
)["access_token"]
wd = req(
"POST",
f"{bank}/accounts/{user}/withdrawals",
{"amount": amount},
{"Authorization": f"Bearer {tok}"},
)
print(wd.get("taler_withdraw_uri") or wd.get("talerWithdrawUri") or "")
PY
}
mint_pay() {
local mode="$1"
MERCHANT="$MERCHANT" INSTANCE="$INSTANCE" \
DONATE_INSTANCE="$DONATE_INSTANCE" DONATE_TEMPLATE="$DONATE_TEMPLATE" \
python3 - "$mode" <<'PY'
import json, os, ssl, sys, urllib.error, urllib.request
mode = sys.argv[1]
merchant = os.environ["MERCHANT"].rstrip("/")
inst = os.environ["INSTANCE"]
d_inst = os.environ["DONATE_INSTANCE"]
d_tmpl = os.environ["DONATE_TEMPLATE"]
ctx = ssl.create_default_context()
def post(url, data):
body = json.dumps(data).encode()
r = urllib.request.Request(url, data=body, method="POST",
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(r, context=ctx, timeout=25) as resp:
return resp.status, json.load(resp)
except urllib.error.HTTPError as e:
raw = e.read() if hasattr(e, "read") else b""
try:
return getattr(e, "code", None), json.loads(raw)
except Exception as err:
raise SystemExit(f"POST {url} failed: {e} body={raw[:200]!r}") from err
if mode.startswith("product:"):
pid = mode.split(":", 1)[1]
st, d = post(f"{merchant}/instances/{inst}/templates/{pid}", {})
use_inst = inst
elif mode.startswith("donate:"):
amt = mode.split(":", 1)[1]
st, d = post(f"{merchant}/instances/{d_inst}/templates/{d_tmpl}", {"amount": amt})
use_inst = d_inst
else:
raise SystemExit(f"bad mode {mode!r}")
if "taler_pay_uri" in d:
print(d["taler_pay_uri"]); raise SystemExit(0)
oid, tok = d.get("order_id"), d.get("token")
if not oid or not tok:
raise SystemExit(f"no order: {d!r}"[:200])
host = merchant.replace("https://","").replace("http://","").split("/")[0]
print(f"taler://pay/{host}/instances/{use_inst}/{oid}/?c={tok}")
PY
}
click_through() {
local labels=("$@")
local i lab misses=0
# Prefer dismissing ANR before hunting Confirm (low-RAM hosts thrash System UI)
for i in 1 2 3 4 5 6; do
if tap_text "Wait" 2>/dev/null || tap_text "Warten" 2>/dev/null; then
info "dismissed ANR (Wait)"
pause 2.0
misses=0
continue
fi
local hit=0
for lab in "${labels[@]}"; do
if tap_text "$lab" 2>/dev/null; then
hit=1
misses=0
pause 0.9
break
fi
done
if [[ "$hit" -eq 0 ]]; then
misses=$((misses + 1))
# stop early if hierarchy has nothing useful (avoid burning order deadlines)
[[ "$misses" -ge 3 ]] && break
fi
pause 0.5
done
}
if [[ -n "${WITHDRAW_AMOUNTS:-}" ]]; then
# shellcheck disable=SC2206
AMOUNTS=($WITHDRAW_AMOUNTS)
else
AMOUNTS=("${DEFAULT_AMOUNTS[@]}")
fi
if [[ -n "${PRODUCTS:-}" ]]; then
PAYS=()
for p in $PRODUCTS; do PAYS+=("product:$p"); done
else
PAYS=("${DEFAULT_PAYS[@]}")
fi
############################
step "0 device + wallet ($STACK / $CUR_HINT)"
adb_sh input keyevent KEYCODE_WAKEUP 2>/dev/null || true
adb_sh wm dismiss-keyguard 2>/dev/null || true
if [[ "$CLEAR" -eq 1 ]]; then
info "pm clear $PKG"
adb_sh pm clear "$PKG" >/dev/null 2>&1 || true
fi
adb_sh am force-stop "$PKG" 2>/dev/null || true
pause 0.4
# Prefer explicit MainActivity (from gui-workflows); fall back to VIEW launcher
if ! adb_sh am start -n "$ACT" >/dev/null 2>&1; then
info "MainActivity path failed — monkey launch"
adb_sh monkey -p "$PKG" -c android.intent.category.LAUNCHER 1 >/dev/null 2>&1 || true
fi
pause 2.0
shot "00-start"
############################
step "2 withdraw chain"
idx=0
for AMT in "${AMOUNTS[@]}"; do
idx=$((idx + 1))
step "2.$idx withdraw $AMT"
URI="$(mint_withdraw "$AMT" | head -1)"
[[ -n "$URI" ]] || die "mint failed for $AMT"
# wallet / monitoring QR rules: strip default ports
URI=$(printf '%s' "$URI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g')
info "URI=$URI"
echo "$URI" >>"$SHOTDIR/withdraw-uris.txt"
adb_sh am start -a android.intent.action.VIEW -d "$URI" "$PKG" >/dev/null
pause 1.5
shot "w${idx}-open"
click_through "Accept" "I accept" "Accept terms" "Confirm" "Continue" "Withdraw" "Next" "OK" "Agree" "Bestätigen" "Abheben" "Akzeptieren"
pause 3.0
shot "w${idx}-after"
adb_sh am start -n "$ACT" >/dev/null 2>&1 || true
pause 0.6
tap_text "Assets" 2>/dev/null || true
pause 0.4
shot "w${idx}-assets"
done
############################
step "3 pay chain (limit=$PAY_LIMIT)"
pidx=0
for PAY in "${PAYS[@]}"; do
pidx=$((pidx + 1))
[[ "$pidx" -gt "$PAY_LIMIT" ]] && break
step "3.$pidx pay $PAY"
if ! PAYURI="$(mint_pay "$PAY" | head -1)"; then
info "mint_pay failed for $PAY — skip"
continue
fi
[[ -n "$PAYURI" ]] || { info "empty pay uri — skip"; continue; }
PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g')
info "PAYURI=$PAYURI"
echo "$PAYURI" >>"$SHOTDIR/pay-uris.txt"
adb_sh am start -a android.intent.action.VIEW -d "$PAYURI" "$PKG" >/dev/null
pause 1.5
shot "p${pidx}-open"
click_through "Pay" "Confirm" "Accept" "Continue" "OK" "Next" "Bezahlen" "Bestätigen" "Payer"
pause 1.5
shot "p${pidx}-after"
adb_sh am start -n "$ACT" >/dev/null 2>&1 || true
pause 0.5
done
step "4 done → $SHOTDIR"
ls -la "$SHOTDIR" | tail -40
adb_sh dumpsys window 2>/dev/null | grep mCurrentFocus | head -1 || true
info "PKG=$PKG amounts=${AMOUNTS[*]} pays(limit)=$PAY_LIMIT"
echo "See GUI-AUTOMATION-NOTES.md (ported from taler-android dev/hernani-inference/gui-workflows)"

90
android-test/start-android-emulator.sh Executable file
View file

@ -0,0 +1,90 @@
#!/usr/bin/env bash
# Start the Taler Android AVD. **Default = headless** (no host window).
# Guest GLES still runs via SwiftShader so uiautomator / screenshots work.
#
# Usage:
# ./start-android-emulator.sh # headless, background
# ./start-android-emulator.sh --wait # headless + wait for boot
# ./start-android-emulator.sh --windowed # show emulator window
# EMULATOR_HEADLESS=0 ./start-android-emulator.sh
# EMULATOR_GPU=host WINDOWED=1 ./start-android-emulator.sh
#
# Env: EMULATOR_AVD EMULATOR_MEMORY EMULATOR_CORES EMULATOR_GPU
# EMULATOR_HEADLESS EMULATOR_LOG EMULATOR_BOOT_TIMEOUT EMULATOR_EXTRA_ARGS
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib_android_env.sh
. "$ROOT/lib_android_env.sh"
WAIT=0
for a in "$@"; do
case "$a" in
--wait) WAIT=1 ;;
--windowed| --gui) EMULATOR_HEADLESS=0; export EMULATOR_HEADLESS ;;
--headless) EMULATOR_HEADLESS=1; export EMULATOR_HEADLESS ;;
-h|--help)
sed -n '2,18p' "$0"
exit 0
;;
esac
done
command -v emulator >/dev/null || {
echo "emulator not found (ANDROID_HOME=$ANDROID_HOME)" >&2
exit 2
}
command -v adb >/dev/null || {
echo "adb not found" >&2
exit 2
}
# Already booted?
if serial=$(android_adb_serial); [ -n "$serial" ]; then
boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true)
if [ "$boot" = "1" ]; then
echo "already running: $serial (boot_completed=1) — headless default still applies to *new* starts"
exit 0
fi
fi
# Avoid pkill patterns that match this wrapper's argv (self-kill).
mapfile -t EMU_ARGS < <(android_emulator_args "$EMULATOR_AVD")
echo "starting: emulator ${EMU_ARGS[*]}"
echo " headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU log=$EMULATOR_LOG"
if [ "${EMULATOR_HEADLESS}" = "1" ]; then
export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}"
# Do not require DISPLAY
unset DISPLAY || true
fi
mkdir -p "$(dirname "$EMULATOR_LOG")"
# Background; do not use pkill -f emulator from callers matching full cmdline
nohup emulator "${EMU_ARGS[@]}" >"$EMULATOR_LOG" 2>&1 &
echo "emulator pid=$! log=$EMULATOR_LOG"
if [ "$WAIT" != "1" ]; then
echo "not waiting (pass --wait for boot). adb devices when ready."
exit 0
fi
echo "waiting for boot (timeout ${EMULATOR_BOOT_TIMEOUT}s)…"
deadline=$((SECONDS + EMULATOR_BOOT_TIMEOUT))
serial=""
while (( SECONDS < deadline )); do
adb wait-for-device 2>/dev/null || true
serial=$(android_adb_serial || true)
if [ -n "$serial" ]; then
boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true)
if [ "$boot" = "1" ]; then
echo "BOOT_OK serial=$serial headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU"
exit 0
fi
fi
sleep 2
done
echo "boot timeout — last log:" >&2
tail -30 "$EMULATOR_LOG" >&2 || true
exit 3

279
check_apt_deploy.sh Executable file
View file

@ -0,0 +1,279 @@
#!/usr/bin/env bash
# check_apt_deploy.sh — podman apt-src merchant deploy tests on koopa.
#
# Containers (default):
# Fresh (rebuild when repo versions change):
# koopa-taler-deploy-test-apt-src-trixie
# koopa-taler-deploy-test-apt-src-trixie-testing
# Upgrade track (initial install, then mytops-style apt upgrade):
# koopa-taler-deploy-test-apt-src-trixie-upgrade
# koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
#
# Env:
# APT_DEPLOY_CONTAINERS "name:suite:mode …" mode=fresh|upgrade (optional, default fresh)
# APT_DEPLOY_SKIP=1
# APT_DEPLOY_PODMAN=podman
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
if [ "${APT_DEPLOY_SKIP:-0}" = "1" ]; then
echo "[INFO] skip phase aptdeploy (APT_DEPLOY_SKIP=1)"
exit 0
fi
set_area aptdeploy
section "aptdeploy · podman apt-src merchant (fresh + upgrade tracks)"
PODMAN_BIN="${APT_DEPLOY_PODMAN:-podman}"
DEFAULT_LIST="\
koopa-taler-deploy-test-apt-src-trixie:trixie:fresh \
koopa-taler-deploy-test-apt-src-trixie-testing:trixie-testing:fresh \
koopa-taler-deploy-test-apt-src-trixie-upgrade:trixie:upgrade \
koopa-taler-deploy-test-apt-src-trixie-testing-upgrade:trixie-testing:upgrade"
LIST="${APT_DEPLOY_CONTAINERS:-$DEFAULT_LIST}"
if ! command -v "$PODMAN_BIN" >/dev/null 2>&1; then
err "podman" "podman binary missing ($PODMAN_BIN)"
summary
exit 1
fi
ok "podman" "$("$PODMAN_BIN" --version 2>/dev/null | head -1)"
info "host" "$(hostname 2>/dev/null || echo unknown) · whoami=$(whoami)"
fail_any=0
print_pkg_table() {
local name=$1
local rows
rows=$("$PODMAN_BIN" exec "$name" bash -lc '
echo "| package | version |"
echo "|---------|---------|"
for p in taler-merchant libtalermerchant libtalerexchange libdonau libgnunet \
taler-merchant-webui taler-merchant-typst taler-terms-generator; do
v=$(dpkg-query -W -f="\${Version}" "$p" 2>/dev/null || echo missing)
echo "| $p | $v |"
done
' 2>/dev/null || echo "| (query failed) | |")
info "pkg-table" "$name"
# emit as plain lines so HTML log keeps the table
while IFS= read -r line; do
[ -n "$line" ] || continue
info "pkg" "$line"
done <<<"$rows"
}
check_ldd_deep() {
local name=$1
local out
out=$("$PODMAN_BIN" exec "$name" bash -lc '
set +e
bin=$(command -v taler-merchant-httpd)
echo "=== ldd taler-merchant-httpd ==="
ldd "$bin" 2>&1 | grep -E "libtalerutil|libdonau|libgnunet|not found" || true
echo "=== ldd libdonau (if present) ==="
for f in /usr/lib/*/libdonau.so* /usr/lib/*/libdonauutil.so*; do
[ -e "$f" ] || continue
echo "-- $f --"
ldd "$f" 2>&1 | grep -E "libtalerutil|not found" || true
done
echo "=== libtalerutil files ==="
ls -la /usr/lib/*/libtalerutil* 2>/dev/null || echo "(none)"
' 2>/dev/null || echo "ldd probe failed")
while IFS= read -r line; do
[ -n "$line" ] || continue
info "ldd" "$line"
done <<<"$out"
if echo "$out" | grep -q 'not found'; then
err "ldd" "shared library missing in $name" "see ldd lines above"
return 1
fi
return 0
}
check_systemd() {
local name=$1
local out active_target active_httpd enabled
out=$("$PODMAN_BIN" exec "$name" bash -lc '
set +e
echo "system: $(systemctl is-system-running 2>&1)"
echo "target_enabled: $(systemctl is-enabled taler-merchant.target 2>&1)"
echo "target_active: $(systemctl is-active taler-merchant.target 2>&1)"
echo "httpd_active: $(systemctl is-active taler-merchant-httpd.service 2>&1)"
echo "httpd_enabled: $(systemctl is-enabled taler-merchant-httpd.service 2>&1)"
systemctl start taler-merchant.target 2>&1 | tail -5
sleep 2
echo "after_start_target: $(systemctl is-active taler-merchant.target 2>&1)"
echo "after_start_httpd: $(systemctl is-active taler-merchant-httpd.service 2>&1)"
systemctl --failed --no-legend 2>/dev/null | grep -i taler || echo "failed_taler: none"
' 2>/dev/null || echo "systemd probe failed")
while IFS= read -r line; do
[ -n "$line" ] || continue
info "systemd" "$line"
done <<<"$out"
active_httpd=$(echo "$out" | sed -n 's/^after_start_httpd: //p' | tail -1)
if [ "$active_httpd" = "active" ]; then
ok "systemd httpd" "taler-merchant-httpd active after start taler-merchant.target"
return 0
fi
# soft if target not shipped as enabled in container — still ERROR for deploy smoke
err "systemd httpd" "taler-merchant-httpd not active" "after_start_httpd=${active_httpd:-?} · see systemd lines"
return 1
}
check_merchant_basics() {
local name=$1
local out code
# unix socket or curl localhost if httpd listens
out=$("$PODMAN_BIN" exec "$name" bash -lc '
set +e
echo "=== binaries ==="
command -v taler-merchant-httpd taler-merchant-dbinit taler-config 2>/dev/null
echo "=== --version ==="
taler-merchant-httpd --version 2>&1
echo "=== config probe ==="
# try common paths without TLS
for u in \
http://127.0.0.1:9966/config \
http://127.0.0.1:8081/config \
http://127.0.0.1/config
do
code=$(curl -sS -m 2 -o /tmp/mcfg -w "%{http_code}" "$u" 2>/dev/null || echo 000)
echo "curl $u -> $code"
[ "$code" = "200" ] && head -c 120 /tmp/mcfg && echo
done
sock=$(ls /run/taler-merchant/httpd/*.sock 2>/dev/null | head -1)
if [ -n "$sock" ]; then
echo "socket: $sock"
code=$(curl -sS -m 2 --unix-socket "$sock" -o /tmp/mcfg -w "%{http_code}" http://localhost/config 2>/dev/null || echo 000)
echo "curl --unix-socket -> $code"
[ "$code" = "200" ] && head -c 160 /tmp/mcfg && echo
else
echo "socket: (none under /run/taler-merchant/httpd/)"
fi
' 2>/dev/null || echo "basics probe failed")
while IFS= read -r line; do
[ -n "$line" ] || continue
info "basics" "$line"
done <<<"$out"
if echo "$out" | grep -qi 'error while loading shared libraries'; then
err "httpd" "shared library error on --version" "see basics/ldd"
return 1
fi
if ! echo "$out" | grep -q 'taler-merchant-httpd --version\|v\.\|taler-merchant-httpd'; then
# version line varies; success if exit was ok — check via separate exec
:
fi
set +e
"$PODMAN_BIN" exec "$name" taler-merchant-httpd --version >/dev/null 2>&1
local ec=$?
set -e
if [ "$ec" -eq 0 ]; then
ok "httpd --version" "exit 0"
else
err "httpd --version" "exit $ec"
return 1
fi
# /config is nice-to-have if stack is fully configured
if echo "$out" | grep -qE 'curl .* -> 200'; then
ok "merchant /config" "HTTP 200 (local)"
else
warn "merchant /config" "no local HTTP 200 (unit may need BASE_URL/db — still report version/ldd)"
fi
return 0
}
check_one() {
local name="$1" expect_suite="$2" mode="${3:-fresh}"
local st suite_line pkgs
# group ids: aptdeploy.trixie-01 / aptdeploy.trixie-testing-upgrade-01
set_group "${expect_suite}${mode:+-$mode}"
section "aptdeploy · $name (suite=$expect_suite mode=$mode)"
if ! "$PODMAN_BIN" container exists "$name" 2>/dev/null; then
err "container" "$name missing" "run ensure-apt-deploy-test-containers.sh"
fail_any=1
return
fi
st=$("$PODMAN_BIN" inspect -f '{{.State.Status}}' "$name" 2>/dev/null || echo unknown)
if [ "$st" != "running" ]; then
err "container" "$name not running" "status=$st"
fail_any=1
return
fi
ok "container" "$name running (mode=$mode)"
suite_line=$("$PODMAN_BIN" exec "$name" bash -lc \
'grep -h "^Suites:" /etc/apt/sources.list.d/*taler* 2>/dev/null | head -1' 2>/dev/null || true)
if echo "$suite_line" | grep -qE "Suites:[[:space:]]*${expect_suite}([[:space:]]|$)"; then
ok "apt suite" "$expect_suite"
else
err "apt suite" "expected $expect_suite" "got: ${suite_line:-empty}"
fail_any=1
fi
print_pkg_table "$name"
if ! check_merchant_basics "$name"; then
fail_any=1
check_ldd_deep "$name" || fail_any=1
else
# still show ldd summary as info when ok
ldd_out=$("$PODMAN_BIN" exec "$name" bash -lc \
'ldd "$(command -v taler-merchant-httpd)" 2>&1 | grep -E "libtalerutil|not found" || true' 2>/dev/null || true)
if echo "$ldd_out" | grep -q 'not found'; then
err "libtalerutil" "not found" "$ldd_out"
check_ldd_deep "$name" || true
fail_any=1
else
info "libtalerutil" "$(echo "$ldd_out" | tr '\n' ' ')"
fi
fi
if ! check_systemd "$name"; then
fail_any=1
check_ldd_deep "$name" || true
fi
# Disk free inside this deploy-test container (+ host once, first container only)
if [ "${_APT_DEPLOY_HOST_DISK_DONE:-0}" != "1" ]; then
set_group disk
mon_disk_check_host "host" || fail_any=1
_APT_DEPLOY_HOST_DISK_DONE=1
fi
set_group disk
if ! mon_disk_check_podman "$name" "$PODMAN_BIN"; then
fail_any=1
fi
}
for entry in $LIST; do
# name:suite or name:suite:mode
cname=${entry%%:*}
rest=${entry#*:}
case "$rest" in
*:*)
suite=${rest%%:*}
mode=${rest#*:}
;;
*)
suite=$rest
mode=fresh
;;
esac
check_one "$cname" "$suite" "$mode"
done
summary
if [ "$fail_any" -ne 0 ]; then
exit 1
fi
exit 0

704
check_auth401.sh Executable file
View file

@ -0,0 +1,704 @@
#!/usr/bin/env bash
# auth401 — merchant HTTP 401 / case-sensitivity matrix (WebUI-shaped flows)
#
# Exercises the same APIs the merchant SPA uses for:
# • self-provision signup (POST /instances) — mixed-case id
# • login / createAccessToken (POST …/private/token) — Basic user case
# • password change (POST …/private/auth)
# • admin-create (POST /management/instances) when admin bearer available
# • access-token + private/ with Bearer
#
# Backend fact (#11340): Basic-auth *username* is case-sensitive; URL path is
# usually not. SPA must always lowercase the username (createAccessToken, signup
# id, admin-create, pw change re-login). These tests assert the *raw API*
# behaviour so a missing SPA lowercasing shows up as 401 where 200 is wanted
# only after lowercasing.
#
# ./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401
# ./taler-monitoring.sh -d hacktivism.ch auth401
#
# Continue through all groups / collect every ERROR (optional):
# AUTH401_CONTINUE=1 ./taler-monitoring.sh -d stage… auth401
# CONTINUE_ON_ERROR=1 ./taler-monitoring.sh -d stage… auth401 # same
#
# Env: see secrets.env.example (AUTH401_*).
#
# NOTE: we intentionally do NOT use `set -e`. A single failed curl/grep under
# set -e would abort with no ERROR summary — the opposite of what monitoring
# needs. Optional CONTINUE_ON_ERROR still controls hard prereq early-exit.
set -uo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
load_monitoring_secrets_env 2>/dev/null || true
tmp=$(mktemp -d)
AUTH401_FINISHED=0
_auth401_finish() {
[ "${AUTH401_FINISHED}" = "1" ] && return 0
AUTH401_FINISHED=1
# summary() returns non-zero when FAIL_N>0 — never let that skip the print
summary || true
rm -rf "${tmp:-}"
if [ "${FAIL_N:-0}" -gt 0 ]; then
exit 1
fi
exit 0
}
trap '_auth401_finish' EXIT
# ---------------------------------------------------------------------------
# Parameters
# ---------------------------------------------------------------------------
: "${AUTH401_TIMEOUT:=${TIMEOUT:-20}}"
: "${AUTH401_CREATE:=1}"
: "${AUTH401_INSTANCE:=}"
: "${AUTH401_PASSWORD:=${MERCHANT_INSTANCE_PASSWORD:-${AUTH401_PASS:-}}}"
: "${AUTH401_ID_PREFIX:=mon401}"
: "${AUTH401_SKIP_IF_MFA:=1}"
: "${AUTH401_BEARER:=${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}}"
: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}"
: "${AUTH401_DURABLE_ID:=mon401}"
# Prefer durable alone (skip throwaway) — default 0 so full matrix always runs
: "${AUTH401_PREFER_DURABLE:=0}"
# Run expanded groups (signup/login/pwchange/admin/webui). Default 1.
: "${AUTH401_FULL:=1}"
# Admin bearer for POST /management/instances (optional)
: "${AUTH401_ADMIN_TOKEN:=${MERCHANT_ADMIN_TOKEN:-}}"
: "${AUTH401_ADMIN_PASSWORD:=}"
: "${AUTH401_ADMIN_USER:=admin}"
# 1 = do not abort on hard prereq fail; run remaining groups (default 0)
# Alias: CONTINUE_ON_ERROR=1
: "${AUTH401_CONTINUE:=${CONTINUE_ON_ERROR:-0}}"
# Never rotate password on durable account (default 1 — keep mon401.password valid)
: "${AUTH401_PWCHANGE_THROWAY_ONLY:=1}"
TIMEOUT="$AUTH401_TIMEOUT"
export TIMEOUT
_auth401_abort_or_continue() {
local why="${1:-prereq failed}"
if [ "${AUTH401_CONTINUE}" = "1" ]; then
warn "continue" "${why} (AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1)"
return 0
fi
# hard stop — EXIT trap prints summary + all ERRORS collected so far
exit 1
}
BASE="${MERCHANT_PUBLIC%/}"
if [ -z "$BASE" ]; then
echo "MERCHANT_PUBLIC empty — set -d domain or --merchant" >&2
exit 2
fi
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
expect_http() {
local label="$1" want="$2" got="$3" detail="${4:-}"
case ",$want," in
*",$got,"*) ok "$label" "HTTP $got${detail:+ · $detail}" ;;
*) fail "$label" "HTTP $got (want $want)${detail:+ · $detail}" ;;
esac
}
to_lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; }
to_upper() { printf '%s' "$1" | tr '[:lower:]' '[:upper:]'; }
# First char upper, rest lower → synthetic MIX for Basic user (always ≠ low if len>0)
synth_mix() {
python3 -c "s='''$1'''; print((s[:1].upper()+s[1:].lower()) if s else s)"
}
# First 3 upper (partial mix)
synth_partmix() {
python3 -c "s='''$1'''; print(s[:3].upper()+s[3:] if len(s)>=3 else s.upper())"
}
# Normalize curl status: "200", "000", never "200000" / "000000"
_http_norm() {
local raw="$1" dig
dig=$(printf '%s' "$raw" | tr -cd '0-9')
if [ -z "$dig" ]; then
printf '000'
else
# first 3 digits = real code (curl 200 then || echo 000 → 200000)
printf '%s' "$dig" | head -c 3
fi
}
token_http() {
local path_id="$1" user="$2" password="$3"
local b64 raw
b64=$(printf '%s' "${user}:${password}" | base64 -w0 2>/dev/null \
|| printf '%s' "${user}:${password}" | base64)
raw=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/last_token.json" -w '%{http_code}' \
-X POST "${BASE}/instances/${path_id}/private/token" \
-H "Authorization: Basic ${b64}" \
-H 'Content-Type: application/json' \
-d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \
2>/dev/null || true)
_http_norm "${raw:-}"
}
json_code_hint() {
python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true
import json, sys
try:
d = json.load(open(sys.argv[1]))
except Exception:
print("")
raise SystemExit
c = d.get("code", "")
h = (d.get("hint") or d.get("detail") or "")[:70]
print(f"code={c} {h}".strip())
PY
}
extract_token() {
python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true
import json, sys
try:
d = json.load(open(sys.argv[1]))
except Exception:
print("")
raise SystemExit(0)
print(d.get("access_token") or d.get("token") or "")
PY
}
instance_body() {
local id="$1" pass="$2" name="${3:-auth401 probe}"
python3 - "$id" "$pass" "$name" <<'PY'
import json, sys
print(json.dumps({
"address": {},
"auth": {"method": "token", "password": sys.argv[2]},
"default_pay_delay": {"d_us": 60000000},
"default_wire_transfer_delay": {"d_us": 30000000},
"default_refund_delay": {"d_us": 30000000},
"id": sys.argv[1],
"jurisdiction": {},
"name": sys.argv[3],
"use_stefan": True,
}))
PY
}
auth_change_http() {
local path_id="$1" bearer="$2" new_pass="$3"
local tok="$bearer" body
case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac
body=$(NEW_PASS="$new_pass" python3 -c 'import json,os; print(json.dumps({"method":"token","password":os.environ["NEW_PASS"]}))')
curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/auth_change.json" -w '%{http_code}' \
-X POST "${BASE}/instances/${path_id}/private/auth" \
-H "Authorization: Bearer ${tok}" \
-H 'Content-Type: application/json' \
-d "$body" \
2>/dev/null || echo 000
}
# ---------------------------------------------------------------------------
# Stage secrets (quiet)
# ---------------------------------------------------------------------------
_auth401_stage_secret_roots() {
printf '%s\n' \
"${FRANCPAYSAN_SECRETS:-}" \
"${HOME}/francpaysan-secrets" \
"${HOME}/src/francpaysan-secrets" \
"${HOME}/taler/src/francpaysan-secrets"
}
_auth401_resolve_secrets_quiet() {
local root inst_id pass
for root in $(_auth401_stage_secret_roots); do
[ -n "$root" ] && [ -d "$root" ] || continue
# durable ids: prefer mon401fix when mon401 was rotated by old tests
if [ -z "${DURABLE_PASS:-}" ]; then
for inst_id in "${AUTH401_DURABLE_ID}" mon401fix mon401 "${MERCHANT_INSTANCE:-}"; do
[ -n "$inst_id" ] || continue
pass=""
if [ -f "$root/stage/instances/${inst_id}.password" ]; then
pass=$(tr -d '\n\r' <"$root/stage/instances/${inst_id}.password")
fi
# skip empty password files
if [ -n "$pass" ]; then
# verify file is for this id (mon401.password may be stale copy of mon401fix)
DURABLE_PASS="$pass"
DURABLE_ID=$(to_lower "$inst_id")
AUTH401_SECRET_NOTE="durable ${inst_id} from ${root}/stage/instances/"
# only set AUTH401_PASSWORD if caller did not pin an instance
if [ -z "${AUTH401_PASSWORD}" ] && [ -z "${AUTH401_INSTANCE}" ]; then
AUTH401_PASSWORD="$pass"
fi
break
fi
done
fi
if [ -z "${AUTH401_BEARER}" ] && [ -f "$root/stage/default-instance-token.txt" ]; then
AUTH401_BEARER=$(tr -d '\n\r' <"$root/stage/default-instance-token.txt")
AUTH401_BEARER_NOTE="default-instance-token.txt"
fi
if [ -z "${AUTH401_ADMIN_TOKEN}" ] && [ -f "$root/stage/admin-instance-token.txt" ]; then
AUTH401_ADMIN_TOKEN=$(tr -d '\n\r' <"$root/stage/admin-instance-token.txt")
AUTH401_ADMIN_NOTE="admin-instance-token.txt"
fi
[ -n "${DURABLE_PASS:-}" ] && break
done
# also keep AUTH401_INSTANCE for durable-only mode
if [ "${AUTH401_PREFER_DURABLE}" = "1" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then
AUTH401_INSTANCE="${AUTH401_INSTANCE:-$DURABLE_ID}"
AUTH401_PASSWORD="${AUTH401_PASSWORD:-$DURABLE_PASS}"
AUTH401_CREATE=0
AUTH401_MODE_NOTE="PREFER_DURABLE=1 → only durable ${AUTH401_INSTANCE}"
fi
}
_auth401_resolve_secrets_quiet
: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}"
export AUTH401_INSTANCE AUTH401_PASSWORD AUTH401_BEARER AUTH401_CREATE AUTH401_ADMIN_TOKEN
# ---------------------------------------------------------------------------
set_area auth401
section "auth401 · merchant 401 / case matrix (WebUI APIs) · ${BASE}"
if [ -n "${AUTH401_SECRET_NOTE:-}" ]; then info "stage secret" "$AUTH401_SECRET_NOTE"; fi
if [ -n "${AUTH401_BEARER_NOTE:-}" ]; then info "stage secret" "AUTH401_BEARER from ${AUTH401_BEARER_NOTE}"; fi
if [ -n "${AUTH401_ADMIN_NOTE:-}" ]; then info "stage secret" "AUTH401_ADMIN_TOKEN from ${AUTH401_ADMIN_NOTE}"; fi
if [ -n "${AUTH401_MODE_NOTE:-}" ]; then info "auth401 mode" "$AUTH401_MODE_NOTE"; fi
info "auth401 scope" "FULL=${AUTH401_FULL} CREATE=${AUTH401_CREATE} PREFER_DURABLE=${AUTH401_PREFER_DURABLE} CONTINUE=${AUTH401_CONTINUE}"
if [ "${AUTH401_CONTINUE}" = "1" ]; then
info "continue mode" "CONTINUE_ON_ERROR/AUTH401_CONTINUE=1 — hard prereqs soft; always full ERROR list via EXIT trap"
fi
# ===========================================================================
set_group config
# ===========================================================================
cfg_code=$(http_body "$BASE/config" "$tmp/mer-config.json")
expect_http "merchant /config" 200 "$cfg_code" "$BASE/config"
HAVE_SP=0
MANDATORY_TAN=0
if [ "$cfg_code" = "200" ]; then
eval "$(python3 - "$tmp/mer-config.json" <<'PY'
import json, sys
d = json.load(open(sys.argv[1]))
sp = 1 if d.get("have_self_provisioning") else 0
tans = d.get("mandatory_tan_channels") or []
print(f"HAVE_SP={sp}")
print(f"MANDATORY_TAN={1 if tans else 0}")
print(f"TAN_CH={','.join(tans) if tans else ''}")
print(f"API_VER={d.get('version','')}")
PY
)"
info "self-provision" "have_self_provisioning=${HAVE_SP} mandatory_tan=${MANDATORY_TAN}${TAN_CH:+ ($TAN_CH)} api=${API_VER:-?}"
else
fail "merchant config required for auth401" "HTTP $cfg_code"
_auth401_abort_or_continue "no /config"
fi
# ===========================================================================
set_group webui
# SPA surface: /webui/ loads + version stamp; optional toLowerCase in bundle
# ===========================================================================
if [ "${AUTH401_FULL}" = "1" ]; then
wcode=$(http_code "$BASE/webui/")
case "$wcode" in
200|301|302) ok "webui /" "HTTP $wcode" ;;
*) warn "webui /" "HTTP $wcode" ;;
esac
vcode=$(http_body "$BASE/webui/version.txt" "$tmp/spa-ver.txt" 2>/dev/null || echo 000)
if [ "$vcode" = "200" ] && [ -s "$tmp/spa-ver.txt" ]; then
ok "webui version.txt" "$(tr -d '\n\r' <"$tmp/spa-ver.txt")"
else
warn "webui version.txt" "HTTP ${vcode:-000}"
fi
# Download a slice of index.js — look for lowercasing mitigation (#11340)
jcode=$(curl -skS -m 60 -o "$tmp/spa-index.js" -w '%{http_code}' "$BASE/webui/index.js" 2>/dev/null || echo 000)
jcode=$(printf '%s' "$jcode" | tr -cd '0-9' | head -c 3)
if [ "$jcode" = "200" ] && [ -s "$tmp/spa-index.js" ]; then
ok "webui index.js" "HTTP 200 · $(wc -c <"$tmp/spa-index.js" | tr -d ' ') bytes"
if grep -q 'toLowerCase' "$tmp/spa-index.js" 2>/dev/null; then
ok "webui bundle toLowerCase" "present (SPA may mitigate #11340)"
else
warn "webui bundle toLowerCase" "not found in index.js — login MIX may 401 without lowercasing"
fi
else
warn "webui index.js" "HTTP ${jcode:-000}"
fi
fi
# ===========================================================================
set_group signup
# WebUI self-provision: POST /instances with mixed-case id (as typed)
# ===========================================================================
CREATED=0
USE_ID=""
USE_PASS=""
ID_MIX=""
ID_LOW=""
CREATE_PASS="$AUTH401_CREATE_PASSWORD"
stamp=$(date +%H%M%S)
# Mixed-case id like the SPA would send before lowercasing (or if it forgot)
ID_MIX="$(printf '%s' "${AUTH401_ID_PREFIX}" | awk '{print toupper(substr($0,1,1)) substr($0,2)}')X${stamp}"
ID_LOW=$(to_lower "$ID_MIX")
if [ "${AUTH401_FULL}" = "1" ] && [ "$AUTH401_CREATE" = "1" ] && [ "$HAVE_SP" = "1" ]; then
if [ "$MANDATORY_TAN" = "1" ] && [ "$AUTH401_SKIP_IF_MFA" = "1" ]; then
warn "signup skipped" "mandatory_tan_channels set — set AUTH401_INSTANCE+PASSWORD"
else
body=$(instance_body "$ID_MIX" "$CREATE_PASS" "auth401 signup MIX")
ccode=$(_http_norm "$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/create.json" -w '%{http_code}' \
-X POST "${BASE}/instances" \
-H 'Content-Type: application/json' \
-d "$body" 2>/dev/null || true)")
case "$ccode" in
200|204)
ok "signup POST /instances id=MIX" "HTTP $ccode id=$ID_MIX (SPA self-provision)"
CREATED=1
USE_ID="$ID_MIX"
USE_PASS="$CREATE_PASS"
# Immediate login as SPA after signup: must use *lowercase* Basic user
code=$(token_http "$ID_LOW" "$ID_LOW" "$CREATE_PASS")
expect_http "signup→login SPA-sim path=low user=low → 200" 200 "$code" "$(json_code_hint)"
# Bug path: SPA forgot toLowerCase on login after signup with MIX id typed
code=$(token_http "$ID_LOW" "$ID_MIX" "$CREATE_PASS")
expect_http "signup→login RAW user=MIX (no lower) → 401" 401 "$code" "$(json_code_hint)"
;;
202)
warn "signup needs MFA" "HTTP 202 — use AUTH401_INSTANCE"
;;
*)
fail "signup POST /instances" "HTTP $ccode $(head -c 120 "$tmp/create.json" 2>/dev/null | tr '\n' ' ')"
;;
esac
fi
elif [ -n "${AUTH401_INSTANCE:-}" ] && [ -n "${AUTH401_PASSWORD:-}" ]; then
USE_ID=$(to_lower "$AUTH401_INSTANCE")
USE_PASS="$AUTH401_PASSWORD"
ID_LOW=$(to_lower "$USE_ID")
ID_MIX=$(synth_mix "$ID_LOW")
info "signup" "using existing AUTH401_INSTANCE=$USE_ID (no POST /instances)"
ok "using existing instance" "$USE_ID"
else
warn "signup skipped" "CREATE=${AUTH401_CREATE} HAVE_SP=${HAVE_SP} (need instance for matrix)"
fi
# Fallback durable
if [ -z "$USE_ID" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then
USE_ID="$DURABLE_ID"
USE_PASS="$DURABLE_PASS"
ID_LOW=$(to_lower "$USE_ID")
ID_MIX=$(synth_mix "$ID_LOW")
info "fallback durable" "$USE_ID"
ok "using durable instance" "$USE_ID"
fi
if [ -z "$USE_ID" ]; then
fail "no instance under test" "need signup OK or AUTH401_INSTANCE+PASSWORD or mon401.password"
_auth401_abort_or_continue "no instance under test"
# CONTINUE: synthesize dummy ids so later groups still run (expect many 404/401)
ID_LOW_USE="auth401-missing-instance"
ID_USER_MIX=$(synth_mix "$ID_LOW_USE")
ID_PATH_MIX="$ID_USER_MIX"
ID_UPPER=$(to_upper "$ID_LOW_USE")
ID_PARTMIX=$(synth_partmix "$ID_LOW_USE")
PASS_OK="invalid"
PASS_BAD='auth401-wrong-password-NOT-VALID'
USE_ID="$ID_LOW_USE"
USE_PASS="$PASS_OK"
fi
ID_LOW_USE=$(to_lower "$USE_ID")
# Always synthetic case variants (even for mon401) so MIX ≠ low
ID_USER_MIX=$(synth_mix "$ID_LOW_USE")
ID_PATH_MIX="$ID_USER_MIX"
ID_UPPER=$(to_upper "$ID_LOW_USE")
ID_PARTMIX=$(synth_partmix "$ID_LOW_USE")
PASS_OK="$USE_PASS"
PASS_BAD='auth401-wrong-password-NOT-VALID'
info "probe ids" "low=${ID_LOW_USE} MIX_user=${ID_USER_MIX} UPPER=${ID_UPPER} created=${CREATED} pass_len=${#PASS_OK}"
# ===========================================================================
set_group login
# WebUI login = POST …/private/token with Basic (username, password)
# ===========================================================================
if [ "${AUTH401_FULL}" = "1" ]; then
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK")
expect_http "login SPA-sim user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK")
expect_http "login RAW user=MIX pwd=OK → 401 (#11340)" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK")
expect_http "login RAW user=UPPER pwd=OK → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK")
expect_http "login path=MIX user=low pwd=OK → 200 (path tolerant)" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK")
expect_http "login path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)"
fi
# ===========================================================================
set_group case
# Full Basic case matrix (always hard expects — synthetic MIX)
# ===========================================================================
code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_OK")
expect_http "path=MIX user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK")
expect_http "path=MIX user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK")
expect_http "path=low user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK")
expect_http "path=low user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK")
expect_http "path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK")
expect_http "path=low user=UPPER pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_PARTMIX" "$PASS_OK")
expect_http "path=low user=MiX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)"
# ===========================================================================
set_group password
# ===========================================================================
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_BAD")
expect_http "pwd=BAD → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "")
expect_http "pwd=EMPTY → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_BAD")
expect_http "path=MIX user=MIX pwd=BAD → 401" 401 "$code" "$(json_code_hint)"
# ===========================================================================
set_group pwchange
# WebUI password change: POST private/auth then re-login
# Only on throwaway signup instances — never rotate durable mon401.password
# ===========================================================================
if [ "${AUTH401_FULL}" = "1" ]; then
if [ "${AUTH401_PWCHANGE_THROWAY_ONLY}" = "1" ] && [ "${CREATED:-0}" != "1" ]; then
info "pwchange skipped" "not a throwaway instance (CREATED=0) — refuse to rotate durable password"
else
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK")
if [ "$code" != "200" ]; then
fail "pwchange: login for token" "HTTP $code — skip pwchange"
else
TOK=$(extract_token)
if [ -z "$TOK" ]; then
fail "pwchange: parse token" "empty access_token"
else
ok "pwchange: got bearer" "len=${#TOK}"
NEW_PASS="Auth401-New-${stamp}!"
ac=$(_http_norm "$(auth_change_http "$ID_LOW_USE" "$TOK" "$NEW_PASS")")
case "$ac" in
200|204)
ok "pwchange POST private/auth" "HTTP $ac (SPA password form)"
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK")
expect_http "pwchange: old pwd → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$NEW_PASS")
expect_http "pwchange: new pwd user=low → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$NEW_PASS")
expect_http "pwchange: new pwd user=MIX → 401" 401 "$code" "$(json_code_hint)"
PASS_OK="$NEW_PASS"
USE_PASS="$NEW_PASS"
;;
202)
warn "pwchange needs MFA" "HTTP 202"
;;
*)
fail "pwchange POST private/auth" "HTTP $ac $(head -c 100 "$tmp/auth_change.json" 2>/dev/null | tr '\n' ' ')"
;;
esac
fi
fi
fi
fi
# ===========================================================================
set_group admin
# Admin UI: POST /management/instances (needs admin bearer)
# ===========================================================================
if [ "${AUTH401_FULL}" = "1" ]; then
ADMIN_TOK="${AUTH401_ADMIN_TOKEN:-}"
# Optional: login as admin user with password
if [ -z "$ADMIN_TOK" ] && [ -n "${AUTH401_ADMIN_PASSWORD}" ]; then
code=$(token_http "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_PASSWORD")
if [ "$code" = "200" ]; then
ADMIN_TOK=$(extract_token)
ok "admin login user=${AUTH401_ADMIN_USER}" "got token"
else
# try lower only
au=$(to_lower "$AUTH401_ADMIN_USER")
code=$(token_http "$au" "$au" "$AUTH401_ADMIN_PASSWORD")
if [ "$code" = "200" ]; then
ADMIN_TOK=$(extract_token)
ok "admin login user=${au}" "got token"
else
warn "admin login" "HTTP $code — set AUTH401_ADMIN_TOKEN"
fi
fi
fi
if [ -z "$ADMIN_TOK" ]; then
info "admin-create skipped" "no AUTH401_ADMIN_TOKEN / AUTH401_ADMIN_PASSWORD (stage default is not admin)"
else
case "$ADMIN_TOK" in secret-token:*) ;; *) ADMIN_TOK="secret-token:${ADMIN_TOK}" ;; esac
AID_MIX="Adm401X${stamp}"
AID_LOW=$(to_lower "$AID_MIX")
APASS="Adm401-Pass-${stamp}!"
body=$(instance_body "$AID_MIX" "$APASS" "auth401 admin-create MIX")
acode=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/admin_create.json" -w '%{http_code}' \
-X POST "${BASE}/management/instances" \
-H "Authorization: Bearer ${ADMIN_TOK}" \
-H 'Content-Type: application/json' \
-d "$body" 2>/dev/null || echo 000)
case "$acode" in
200|204)
ok "admin-create POST /management/instances id=MIX" "HTTP $acode id=$AID_MIX"
code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS")
expect_http "admin-create→login user=low → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$AID_LOW" "$AID_MIX" "$APASS")
expect_http "admin-create→login user=MIX → 401" 401 "$code" "$(json_code_hint)"
# admin-set auth on new instance
code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS")
if [ "$code" = "200" ]; then
AT=$(extract_token)
NEW_A="Adm401-New-${stamp}!"
ch=$(auth_change_http "$AID_LOW" "$AT" "$NEW_A")
case "$ch" in
200|204)
ok "admin-created instance pwchange" "HTTP $ch"
code=$(token_http "$AID_LOW" "$AID_LOW" "$NEW_A")
expect_http "admin-created re-login new pwd → 200" 200 "$code" "$(json_code_hint)"
;;
*) warn "admin-created pwchange" "HTTP $ch" ;;
esac
fi
;;
401|403)
warn "admin-create unauthorized" "HTTP $acode — token not admin (expected on stage default)"
;;
202)
warn "admin-create MFA" "HTTP 202"
;;
*)
fail "admin-create" "HTTP $acode $(head -c 120 "$tmp/admin_create.json" 2>/dev/null | tr '\n' ' ')"
;;
esac
fi
fi
# ===========================================================================
set_group missing
# ===========================================================================
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/last_token.json" -w '%{http_code}' \
-X POST "${BASE}/instances/${ID_LOW_USE}/private/token" \
-H 'Content-Type: application/json' \
-d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \
2>/dev/null || echo 000)
expect_http "private/token NO_AUTH → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "$ID_LOW_USE" "auth401-not-a-real-user" "$PASS_OK")
expect_http "user=OTHER pwd=OK → 401" 401 "$code" "$(json_code_hint)"
code=$(token_http "auth401-does-not-exist-${stamp}" "$ID_LOW_USE" "$PASS_OK")
# Must not look like auth failure (401). Prefer 404; 502 from proxy is soft.
case "$code" in
404) ok "path=missing → 404 (not 401)" "HTTP 404 · $(json_code_hint)" ;;
401) fail "path=missing must not be 401" "HTTP 401 · $(json_code_hint)" ;;
502|503) warn "path=missing" "HTTP $code (want 404; proxy flake, not auth)" ;;
*) fail "path=missing → 404 (not 401)" "HTTP $code (want 404) · $(json_code_hint)" ;;
esac
# ===========================================================================
set_group bearer
# ===========================================================================
code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK")
if [ "$code" = "200" ]; then
TOKEN=$(extract_token)
if [ -n "$TOKEN" ]; then
ok "got access_token" "len=${#TOKEN}"
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/priv.json" -w '%{http_code}' \
"${BASE}/instances/${ID_LOW_USE}/private/" \
-H "Authorization: Bearer ${TOKEN}" 2>/dev/null || echo 000)
expect_http "GET private/ Bearer OK → 200" 200 "$code"
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/priv.json" -w '%{http_code}' \
"${BASE}/instances/${ID_LOW_USE}/private/" \
-H "Authorization: Bearer secret-token:AUTH401INVALID000" 2>/dev/null || echo 000)
expect_http "GET private/ Bearer BAD → 401" 401 "$code"
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/priv.json" -w '%{http_code}' \
"${BASE}/instances/${ID_LOW_USE}/private/" 2>/dev/null || echo 000)
expect_http "GET private/ NO_AUTH → 401" 401 "$code"
else
fail "parse access_token" "HTTP $code but no token field"
fi
else
fail "token for bearer tests" "HTTP $code"
fi
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/mgmt.json" -w '%{http_code}' \
"${BASE}/management/instances" 2>/dev/null || echo 000)
expect_http "GET /management/instances NO_AUTH → 401" 401 "$code"
if [ -n "${AUTH401_BEARER:-}" ]; then
if [ -z "${AUTH401_BEARER_INSTANCE:-}" ]; then
case "${AUTH401_BEARER_NOTE:-}" in
*default-instance-token*) AUTH401_BEARER_INSTANCE=default ;;
*) AUTH401_BEARER_INSTANCE="${DURABLE_ID:-default}" ;;
esac
fi
inst=$(to_lower "$AUTH401_BEARER_INSTANCE")
tok="$AUTH401_BEARER"
case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac
code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \
-o "$tmp/priv2.json" -w '%{http_code}' \
"${BASE}/instances/${inst}/private/" \
-H "Authorization: Bearer ${tok}" 2>/dev/null || echo 000)
case "$code" in
200) ok "GET private/ secrets Bearer" "HTTP 200 instance=$inst" ;;
401) warn "secrets Bearer rejected" "HTTP 401 on $inst" ;;
*) warn "secrets Bearer" "HTTP $code on $inst" ;;
esac
fi
# ===========================================================================
set_group durable
# Extra: durable mon401 still works after throwaway tests (if present)
# ===========================================================================
if [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then
dlow=$(to_lower "$DURABLE_ID")
dmix=$(synth_mix "$dlow")
code=$(token_http "$dlow" "$dlow" "$DURABLE_PASS")
expect_http "durable ${dlow} login user=low → 200" 200 "$code" "$(json_code_hint)"
code=$(token_http "$dlow" "$dmix" "$DURABLE_PASS")
expect_http "durable ${dlow} login user=MIX → 401" 401 "$code" "$(json_code_hint)"
else
info "durable" "no mon401.password — skipped"
fi
# ===========================================================================
set_group report
# ===========================================================================
info "summary matrix" "401: Basic user case≠low | bad/empty pwd | no auth | bad bearer | wrong user · 200: user=low + path any case · 404: missing instance · SPA must toLowerCase username on login/signup/admin/pw-relogin"
if [ "$CREATED" = "1" ]; then
info "throwaway left" "id=${ID_MIX:-?} low=${ID_LOW_USE} (password rotated if pwchange ran)"
fi
# fall through → EXIT trap runs _auth401_finish (summary + full ERROR list)
:

1711
check_e2e.sh Executable file
View file

File diff suppressed because it is too large Load diff

1364
check_goa_ladder.sh Executable file
View file

File diff suppressed because it is too large Load diff

546
check_inside.sh Executable file
View file

@ -0,0 +1,546 @@
#!/usr/bin/env bash
# Inside status for bank / exchange / merchant containers.
#
# Access modes (shown in the log as flags):
# host-podman — this host runs podman exec into containers (no SSH).
# INSIDE_PODMAN=1 or INSIDE_MODE=local-podman, or auto on GOA host.
# ssh — from a laptop/remote: SSH to KOOPA_SSH / INSIDE_SSH, then podman there.
#
# Profiles:
# koopa (default LOCAL_STACK=1) — taler-hacktivism* containers
# stage-lfp (TESTPAYSAN) — SSH INSIDE_SSH (stagepaysan), low-priv podman
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
# shellcheck source=metrics.sh
source "$ROOT/metrics.sh"
set_area inside
PROFILE="${INSIDE_PROFILE:-}"
if [ -z "$PROFILE" ]; then
if [ "${LOCAL_STACK:-0}" = "1" ]; then
PROFILE=koopa
elif [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ] || [ -n "${INSIDE_SSH:-}" ]; then
PROFILE=stage-lfp
else
PROFILE=koopa
fi
fi
# Resolve host-podman vs ssh before first check ID
_use_local_podman=0
if [ "$PROFILE" != "stage-lfp" ]; then
if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; then
_use_local_podman=1
elif command -v podman >/dev/null 2>&1 \
&& podman ps --format '{{.Names}}' 2>/dev/null | grep -qE 'taler-hacktivism'; then
_use_local_podman=1
fi
fi
if [ "$_use_local_podman" = "1" ]; then
INSIDE_ACCESS=host-podman
set_group host
else
INSIDE_ACCESS=ssh
set_group ssh
fi
export INSIDE_ACCESS
section "inside · collect (${PROFILE} · access=${INSIDE_ACCESS})"
info "flags" "INSIDE_ACCESS=${INSIDE_ACCESS} INSIDE_PODMAN=${INSIDE_PODMAN:-0} INSIDE_MODE=${INSIDE_MODE:-} LOCAL_STACK=${LOCAL_STACK:-0} SKIP_SSH=${SKIP_SSH:-0} KOOPA_SSH=${KOOPA_SSH:-} INSIDE_SSH=${INSIDE_SSH:-}"
# ---------------------------------------------------------------------------
# stage-lfp: low-priv stagepaysan on francpaysan-host
# ---------------------------------------------------------------------------
if [ "$PROFILE" = "stage-lfp" ]; then
SSH_HOST="${INSIDE_SSH:-francpaysan-stage-user}"
BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}"
EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
MER_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
BANK_PORT="${INSIDE_BANK_PORT:-9032}"
EX_PORT="${INSIDE_EXCHANGE_PORT:-9031}"
MER_PORT="${INSIDE_MERCHANT_PORT:-9030}"
DNS_BANK="${INSIDE_DNS_BANK:-stage.bank.lefrancpaysan.ch}"
DNS_EX="${INSIDE_DNS_EXCHANGE:-stage.exchange.lefrancpaysan.ch}"
DNS_MER="${INSIDE_DNS_MERCHANT:-stage.monnaie.lefrancpaysan.ch}"
# Stage remote has more podman execs than koopa; allow a bit more wall time
STAGE_SSH_T="${INSIDE_SSH_TIMEOUT:-${SSH_CMD_TIMEOUT:-24}}"
if [ "${STAGE_SSH_T}" -lt 24 ] 2>/dev/null; then STAGE_SSH_T=24; fi
if ! mon_ssh_ok "$SSH_HOST"; then
err "ssh" "cannot reach ${SSH_HOST} (stagepaysan low-priv) — set INSIDE_SSH= or SKIP"
summary
exit 1
fi
ok "ssh ${SSH_HOST}" "stagepaysan (podman, no sudo)"
# Inject names/ports into remote (ssh bash -s does not inherit local env).
RAW=$(
{
printf 'BANK_CTR=%q; EX_CTR=%q; MER_CTR=%q\n' "$BANK_CTR" "$EX_CTR" "$MER_CTR"
printf 'BANK_PORT=%q; EX_PORT=%q; MER_PORT=%q\n' "$BANK_PORT" "$EX_PORT" "$MER_PORT"
printf 'DNS_BANK=%q; DNS_EX=%q; DNS_MER=%q\n' "$DNS_BANK" "$DNS_EX" "$DNS_MER"
cat <<'REMOTE'
set +e
emit() { printf 'E|%s|%s|%s|%s\n' "$1" "$2" "$3" "$(printf '%s' "${4:-}" | tr '\n\r' ' ' | head -c 200)"; }
hc() { curl -skS -m 3 -o /tmp/mb -w '%{http_code}' "$1" 2>/dev/null || echo 000; }
hasp() { podman exec "$1" pgrep -f "$2" >/dev/null 2>&1; }
BANK="${BANK_CTR}"
EX="${EX_CTR}"
MER="${MER_CTR}"
podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$BANK" || BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*bank|lfp-bank' | head -1)
podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$EX" || EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*exchange|lfp-exchange' | head -1)
podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$MER" || MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*merchant|lfp-merchant' | head -1)
check_dns() {
local comp="$1" ctr="$2" host="$3"
local line ip code
[ -n "$host" ] || { emit "$comp" WARN "dns" "empty host"; return 1; }
line=$(podman exec "$ctr" getent ahostsv4 "$host" 2>/dev/null | head -1)
[ -z "$line" ] && line=$(podman exec "$ctr" getent hosts "$host" 2>/dev/null | head -1)
ip=$(echo "$line" | awk '{print $1}')
if [ -z "$ip" ]; then
emit "$comp" WARN "dns $host" "no resolve inside container"
return 1
fi
if [ "$ip" = "127.0.0.1" ] || [ "$ip" = "::1" ]; then
emit "$comp" WARN "dns $host" "$ip (loopback)"
return 1
fi
# 169.254.x = pasta host-gateway pin is OK for stage
code=$(podman exec "$ctr" curl -skS -m 3 -o /dev/null -w '%{http_code}' "https://${host}/config" 2>/dev/null || echo 000)
if [ "$code" = "200" ]; then
emit "$comp" OK "dns $host" "→ $ip /config=$code"
else
emit "$comp" WARN "dns $host" "→ $ip /config=$code"
fi
}
if [ -z "$BANK" ]; then emit bank ERROR container "not running"
else
emit bank INFO container "$(podman ps --filter name=^${BANK}$ --format '{{.Names}} {{.Status}}' | head -1)"
emit bank INFO ports "$(podman ps --filter name=^${BANK}$ --format '{{.Ports}}' | head -1)"
hasp "$BANK" 'MainKt serve|libeufin-bank serve' && emit bank OK libeufin "running" || emit bank ERROR libeufin "not running"
podman exec "$BANK" pg_isready -q 2>/dev/null && emit bank OK postgres "ready" || emit bank WARN postgres "pg_isready failed"
c=$(hc "http://127.0.0.1:${BANK_PORT}/config")
[ "$c" = "200" ] && emit bank OK "host :${BANK_PORT}/config" "HTTP $c" || emit bank ERROR "host :${BANK_PORT}/config" "HTTP $c"
c=$(hc "http://127.0.0.1:${BANK_PORT}/taler-integration/config")
[ "$c" = "200" ] && emit bank OK "host integration" "HTTP $c" || emit bank ERROR "host integration" "HTTP $c"
check_dns bank "$BANK" "$DNS_BANK" || true
check_dns bank "$BANK" "$DNS_EX" || true
fi
if [ -z "$EX" ]; then emit exchange ERROR container "not running"
else
emit exchange INFO container "$(podman ps --filter name=^${EX}$ --format '{{.Names}} {{.Status}}' | head -1)"
c=$(hc "http://127.0.0.1:${EX_PORT}/config")
[ "$c" = "200" ] && emit exchange OK "host :${EX_PORT}/config" "HTTP $c" || emit exchange ERROR "host :${EX_PORT}/config" "HTTP $c"
c=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' "http://127.0.0.1:${EX_PORT}/keys" 2>/dev/null || echo 000)
[ "$c" = "200" ] && emit exchange OK "host :${EX_PORT}/keys" "HTTP $c" || emit exchange ERROR "host :${EX_PORT}/keys" "HTTP $c"
hasp "$EX" 'taler-exchange-httpd' && emit exchange OK httpd "running" || emit exchange ERROR httpd "not running"
hasp "$EX" 'taler-exchange-wirewatch' && emit exchange OK wirewatch "running" || emit exchange ERROR wirewatch "not running"
hasp "$EX" 'taler-exchange-aggregator' && emit exchange OK aggregator "running" || emit exchange WARN aggregator "not running"
hasp "$EX" 'taler-exchange-transfer' && emit exchange OK transfer "running" || emit exchange WARN transfer "not running"
check_dns exchange "$EX" "$DNS_BANK" || true
check_dns exchange "$EX" "$DNS_EX" || true
check_dns exchange "$EX" "$DNS_MER" || true
fi
if [ -z "$MER" ]; then emit merchant ERROR container "not running"
else
emit merchant INFO container "$(podman ps --filter name=^${MER}$ --format '{{.Names}} {{.Status}}' | head -1)"
c=$(hc "http://127.0.0.1:${MER_PORT}/config")
[ "$c" = "200" ] && emit merchant OK "host :${MER_PORT}/config" "HTTP $c" || emit merchant ERROR "host :${MER_PORT}/config" "HTTP $c"
hasp "$MER" 'taler-merchant-httpd' && emit merchant OK httpd "running" || emit merchant ERROR httpd "not running"
# Stage merchant often has no wirewatch unit — INFO not WARN
hasp "$MER" 'taler-merchant-wirewatch' && emit merchant OK wirewatch "running" || emit merchant INFO wirewatch "not running (optional on stage)"
hasp "$MER" 'taler-merchant-depositcheck' && emit merchant OK depositcheck "running" || emit merchant INFO depositcheck "not running (optional on stage)"
hasp "$MER" 'taler-merchant-exchangekeyupdate' && emit merchant OK exchangekeyupdate "running" || emit merchant INFO exchangekeyupdate "not running"
check_dns merchant "$MER" "$DNS_BANK" || true
check_dns merchant "$MER" "$DNS_EX" || true
check_dns merchant "$MER" "$DNS_MER" || true
fi
# Caddy is host-wide (often root); stagepaysan can only see the process
if pgrep -x caddy >/dev/null 2>&1; then
emit caddy OK process "running (host)"
else
emit caddy WARN process "not seen as stagepaysan (may still run as root)"
fi
echo DONE
REMOTE
} | mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T}" || true
)
if [ -z "$RAW" ] || ! echo "$RAW" | grep -q '^E|'; then
err "ssh" "stage remote timed out or empty (cap ${STAGE_SSH_T}s · host=${SSH_HOST})"
summary
exit 1
fi
_last_inside_grp=""
while IFS= read -r line; do
case "$line" in
E\|*)
IFS='|' read -r _ comp level key detail <<<"$line"
case "$comp" in
bank|exchange|merchant|caddy) _g="$comp" ;;
*) _g="ssh" ;;
esac
if [ "$_g" != "$_last_inside_grp" ]; then
set_group "$_g"
_last_inside_grp="$_g"
fi
case "$level" in
OK) ok "[$comp] $key${detail:+ ($detail)}" ;;
ERROR) err "$comp" "$key" "$detail" ;;
WARN) warn "[$comp] $key" "$detail" ;;
INFO) info "[$comp] $key" "$detail" ;;
esac
;;
esac
done <<<"$RAW"
# Stats from outside (laptop) — public HTTPS, no stagepaysan write needed
set_group stats
section "inside · public stats (outside-in)"
_probe_public_stats() {
local comp="$1" url="$2"
local body code age unix now
body=$(mktemp)
code=$(curl -skS -m 8 -o "$body" -w '%{http_code}' "$url" 2>/dev/null || echo 000)
if [ "$code" != "200" ]; then
warn "[$comp] public stats.json" "HTTP $code · $url"
rm -f "$body"
return
fi
unix=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d.get("generated_at_unix") or 0)' "$body" 2>/dev/null || echo 0)
now=$(date +%s)
if [ "${unix:-0}" -gt 0 ] 2>/dev/null; then
age=$((now - unix))
if [ "$age" -lt 0 ]; then age=0; fi
if [ "$age" -le "${STATS_STALE_SECS:-900}" ]; then
ok "[$comp] public stats.json" "HTTP 200 age=${age}s · $url"
elif [ "$age" -le "${STATS_FAIL_SECS:-3600}" ]; then
warn "[$comp] public stats.json" "stale age=${age}s · $url"
else
err "$comp" "public stats.json" "too old age=${age}s · $url"
fi
else
ok "[$comp] public stats.json" "HTTP 200 · $url"
fi
rm -f "$body"
}
_probe_public_stats bank "https://${DNS_BANK}/intro/stats.json"
_probe_public_stats exchange "https://${DNS_EX}/intro/stats.json"
_probe_public_stats merchant "https://${DNS_MER}/intro/stats.json"
# Host load as stagepaysan (no container RSS from koopa metrics)
set_group load
section "inside · load (stagepaysan host)"
LOAD_LINE=$(mon_ssh_bash "$SSH_HOST" 8 <<'EOF' || true
python3 - <<'PY'
import os
la=os.getloadavg()
print("loadavg=%.2f,%.2f,%.2f" % la)
try:
with open("/proc/meminfo") as f:
d={}
for line in f:
k,v=line.split(":")[0], line.split(":")[1].strip().split()[0]
d[k]=int(v)
total=d.get("MemTotal",0)/1024/1024
avail=d.get("MemAvailable",0)/1024/1024
used=total-avail
print("mem_used=%.2fGiB avail=%.2fGiB total=%.2fGiB" % (used, avail, total))
except Exception:
print("mem=?")
PY
EOF
)
if [ -n "$LOAD_LINE" ]; then
info "stage host" "$(echo "$LOAD_LINE" | tr '\n' ' ')"
else
info "stage host" "load probe empty"
fi
set_group disk
section "inside · disk free space (stage host + containers)"
_disk_raw=$(mon_ssh_bash "$SSH_HOST" "${STAGE_SSH_T:-24}" <<'DISK' || true
set +e
echo "###HOST###"
df -Pk / /var /home /tmp /mnt/data 2>/dev/null || df -Pk
echo "###CTRS###"
for c in $(podman ps --format '{{.Names}}' 2>/dev/null); do
echo "###CTR $c###"
podman exec "$c" df -Pk / /var /tmp 2>/dev/null || podman exec "$c" df -Pk 2>/dev/null
done
DISK
)
_host_df=$(printf '%s\n' "$_disk_raw" | sed -n '/^###HOST###$/,/^###CTRS###$/p' | sed '1d;$d')
mon_disk_check_remote_text "ssh:${SSH_HOST}" "$_host_df" || true
_ctr=""; _buf=""
while IFS= read -r _line || [ -n "$_line" ]; do
case "$_line" in
'###CTR '*)
if [ -n "$_ctr" ]; then mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true; fi
_ctr=${_line####CTR }; _ctr=${_ctr%###}; _buf=""
;;
'###CTRS###'|'###HOST###') ;;
*) [ -n "$_ctr" ] && _buf="${_buf}${_line}"$'\n' ;;
esac
done <<<"$_disk_raw"
[ -n "$_ctr" ] && mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true
summary
exit 0
fi
# ---------------------------------------------------------------------------
# koopa (default) — host-podman exec (on GOA host) or SSH to KOOPA_SSH (laptop)
# ---------------------------------------------------------------------------
section "inside · collect from koopa (access=${INSIDE_ACCESS})"
if [ "$_use_local_podman" = "1" ]; then
set_group host
if ! command -v podman >/dev/null 2>&1; then
err "host" "INSIDE_PODMAN/host-podman but podman missing"
summary
exit 1
fi
ok "host→container" "podman exec on this host (INSIDE_ACCESS=host-podman · no SSH)"
elif [ "${SKIP_SSH}" = "1" ] && [ "${LOCAL_STACK:-0}" != "1" ]; then
set_group ssh
warn "ssh" "SKIP_SSH=1 and not host-podman — skipped"
summary
exit 0
elif ! koopa_ssh_ok; then
set_group ssh
err "ssh" "cannot reach ${KOOPA_SSH} — from laptop use KOOPA_SSH=; on host set INSIDE_PODMAN=1"
summary
exit 1
else
set_group ssh
ok "ssh ${KOOPA_SSH}" "remote host then podman (INSIDE_ACCESS=ssh · laptop/remote)"
fi
# Collect script (file → bash local or ssh bash -s)
_INSIDE_SCRIPT=$(mktemp)
trap 'rm -f "${_INSIDE_SCRIPT:-}"' RETURN
cat >"$_INSIDE_SCRIPT" <<'REMOTE'
set +e
emit() { printf 'E|%s|%s|%s|%s\n' "$1" "$2" "$3" "$(printf '%s' "${4:-}" | tr '\n\r' ' ' | head -c 200)"; }
# quick curl
hc() { curl -skS -m 3 -o /tmp/mb -w '%{http_code}' "$1" 2>/dev/null || echo 000; }
# quick process check inside container (pgrep only)
hasp() { podman exec "$1" pgrep -f "$2" >/dev/null 2>&1; }
BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'hacktivism-bank|taler-bank' | head -1)
[ -z "$BANK" ] && BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -i bank | head -1)
EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -i exchange | head -1)
# Exact merchant container name first — never fall through to *-bank
MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -E '^taler-hacktivism$' | head -1)
[ -z "$MER" ] && MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -i merchant | grep -viE 'bank|exchange' | head -1)
[ -z "$MER" ] && MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -E 'hacktivism' | grep -viE 'bank|exchange' | head -1)
# Domain resolve inside container (wirewatch needs bank.hacktivism.ch → real IP)
# emit: comp LEVEL dns "host → ip" or fail
check_dns() {
local comp="$1" ctr="$2" host="$3"
local line ip code
# Prefer IPv4 (wirewatch/libcurl often happier; avoid dead AAAA)
line=$(podman exec "$ctr" getent ahostsv4 "$host" 2>/dev/null | head -1)
[ -z "$line" ] && line=$(podman exec "$ctr" getent hosts "$host" 2>/dev/null | head -1)
ip=$(echo "$line" | awk '{print $1}')
if [ -z "$ip" ]; then
emit "$comp" ERROR "dns $host" "no resolve — run pin-container-hosts.sh"
return 1
fi
# 127.0.0.1 is almost always wrong for public bank/exchange from inside pasta
if [ "$ip" = "127.0.0.1" ] || [ "$ip" = "::1" ]; then
emit "$comp" ERROR "dns $host" "$ip (loopback — wirewatch will fail)"
return 1
fi
code=$(podman exec "$ctr" curl -skS -m 3 -o /dev/null -w '%{http_code}' "https://${host}/config" 2>/dev/null || echo 000)
if [ "$code" = "200" ]; then
emit "$comp" OK "dns $host" "→ $ip /config=$code"
else
emit "$comp" WARN "dns $host" "→ $ip /config=$code"
fi
}
if [ -z "$BANK" ]; then emit bank ERROR container "not running"
else
emit bank INFO container "$(podman ps --filter name=$BANK --format '{{.Names}} {{.Status}}' | head -1)"
emit bank INFO ports "$(podman ps --filter name=$BANK --format '{{.Ports}}' | head -1)"
hasp "$BANK" 'MainKt serve|libeufin-bank serve' && emit bank OK libeufin "running" || emit bank ERROR libeufin "not running — API/withdraw dead"
podman exec "$BANK" pg_isready -q 2>/dev/null && emit bank OK postgres "ready" || emit bank ERROR postgres "not ready"
c=$(hc http://127.0.0.1:9012/config)
[ "$c" = "200" ] && emit bank OK "local /config" "HTTP $c" || emit bank ERROR "local /config" "HTTP $c"
c=$(hc http://127.0.0.1:9012/taler-integration/config)
[ "$c" = "200" ] && emit bank OK "local integration" "HTTP $c" || emit bank ERROR "local integration" "HTTP $c"
hasp "$BANK" 'nginx' && emit bank OK nginx ":9013" || emit bank WARN nginx "not running"
check_dns bank "$BANK" bank.hacktivism.ch || true
check_dns bank "$BANK" exchange.hacktivism.ch || true
fi
if [ -z "$EX" ]; then emit exchange ERROR container "not running"
else
emit exchange INFO container "$(podman ps --filter name=$EX --format '{{.Names}} {{.Status}}' | head -1)"
c=$(hc http://127.0.0.1:9011/config)
[ "$c" = "200" ] && emit exchange OK "local /config" "HTTP $c" || emit exchange ERROR "local /config" "HTTP $c"
c=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:9011/keys 2>/dev/null || echo 000)
[ "$c" = "200" ] && emit exchange OK "local /keys" "HTTP $c" || emit exchange ERROR "local /keys" "HTTP $c"
hasp "$EX" 'taler-exchange-httpd' && emit exchange OK httpd "running" || emit exchange ERROR httpd "not running"
hasp "$EX" 'taler-exchange-wirewatch' && emit exchange OK wirewatch "running" || emit exchange ERROR wirewatch "not running — withdraw stuck after bank confirm"
hasp "$EX" 'taler-exchange-aggregator' && emit exchange OK aggregator "running" || emit exchange WARN aggregator "not running"
hasp "$EX" 'taler-exchange-transfer' && emit exchange OK transfer "running" || emit exchange WARN transfer "not running"
# critical for wire gateway
check_dns exchange "$EX" bank.hacktivism.ch || true
check_dns exchange "$EX" exchange.hacktivism.ch || true
check_dns exchange "$EX" taler.hacktivism.ch || true
fi
if [ -z "$MER" ]; then emit merchant ERROR container "not running"
else
emit merchant INFO container "$(podman ps --filter name=$MER --format '{{.Names}} {{.Status}}' | head -1)"
c=$(hc https://127.0.0.1:9010/config)
[ "$c" = "200" ] && emit merchant OK "local /config" "HTTP $c" || emit merchant ERROR "local /config" "HTTP $c"
hasp "$MER" 'taler-merchant-httpd' && emit merchant OK httpd "running" || emit merchant ERROR httpd "not running"
hasp "$MER" 'taler-merchant-wirewatch' && emit merchant OK wirewatch "running" || emit merchant WARN wirewatch "not running"
# depositcheck: may abort on exchange track 404 (merchant assertion) — try ensure once
if hasp "$MER" 'taler-merchant-depositcheck'; then
emit merchant OK depositcheck "running"
else
podman exec "$MER" /usr/local/bin/ensure_merchant_helpers.sh >/dev/null 2>&1 || true
sleep 1
if hasp "$MER" 'taler-merchant-depositcheck'; then
emit merchant OK depositcheck "running (after ensure_merchant_helpers)"
elif hasp "$MER" 'taler-merchant-wirewatch'; then
# Known: depositcheck can SIGABRT on exchange 404 for stale coins — not startable cleanly
emit merchant INFO depositcheck "not running (often aborts on track 404; wirewatch OK · settlement partial)"
else
emit merchant WARN depositcheck "not running"
fi
fi
check_dns merchant "$MER" bank.hacktivism.ch || true
check_dns merchant "$MER" exchange.hacktivism.ch || true
check_dns merchant "$MER" taler.hacktivism.ch || true
fi
if systemctl is-active caddy >/dev/null 2>&1 || pgrep -x caddy >/dev/null 2>&1; then
emit caddy OK process "active"
else
emit caddy ERROR process "not active"
fi
echo DONE
REMOTE
if [ "$_use_local_podman" = "1" ]; then
RAW=$(bash "$_INSIDE_SCRIPT" || true)
else
RAW=$(koopa_ssh_bash "${SSH_CMD_TIMEOUT}" <"$_INSIDE_SCRIPT" || true)
fi
rm -f "$_INSIDE_SCRIPT"
if [ -z "$RAW" ] || ! echo "$RAW" | grep -q '^E|'; then
if [ "$_use_local_podman" = "1" ]; then
err "host" "collect empty (INSIDE_ACCESS=host-podman · podman exec failed)"
else
err "ssh" "collect empty (INSIDE_ACCESS=ssh · cap ${SSH_CMD_TIMEOUT}s · host=${KOOPA_SSH})"
fi
summary
exit 1
fi
# Fallback group for non-component rows: host (podman) vs ssh (remote)
_access_grp=ssh
[ "$_use_local_podman" = "1" ] && _access_grp=host
_last_inside_grp=""
while IFS= read -r line; do
case "$line" in
E\|*)
IFS='|' read -r _ comp level key detail <<<"$line"
# Group IDs by component: inside.bank-02, inside.exchange-04; access = host|ssh
case "$comp" in
bank|exchange|merchant|caddy) _g="$comp" ;;
*) _g="$_access_grp" ;;
esac
if [ "$_g" != "$_last_inside_grp" ]; then
set_group "$_g"
_last_inside_grp="$_g"
fi
case "$level" in
OK) ok "[$comp] $key${detail:+ ($detail)}" ;;
ERROR) err "$comp" "$key" "$detail" ;;
WARN) warn "[$comp] $key" "$detail" ;;
INFO) info "[$comp] $key" "$detail" ;;
esac
;;
esac
done <<<"$RAW"
# Host loadavg + RAM + per-container RSS/CPU (same probe as e2e/ladder)
set_group load
section "inside · load / memory"
METRICS_DIR="${METRICS_DIR:-$(mktemp -d)}"
export METRICS_DIR
metrics_report_load "${METRICS_DIR}/load-inside.json" "inside" || true
# Disk free space: host + taler containers (WARN tight, ERROR full)
set_group disk
section "inside · disk free space"
_disk_ec=0
if [ "$_use_local_podman" = "1" ]; then
mon_disk_check_host "host" || _disk_ec=1
while read -r _cname; do
[ -n "$_cname" ] || continue
mon_disk_check_podman "$_cname" || _disk_ec=1
done < <(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'hacktivism|taler-|stage-lfp|lfp-' || true)
else
# remote host via SSH: df on host + each container
_disk_raw=$(koopa_ssh_bash "${SSH_CMD_TIMEOUT:-20}" <<'DISK' || true
set +e
echo "###HOST###"
df -Pk / /var /home /tmp 2>/dev/null || df -Pk
echo "###CTRS###"
for c in $(podman ps --format '{{.Names}}' 2>/dev/null); do
echo "###CTR $c###"
podman exec "$c" df -Pk / /var /tmp 2>/dev/null || podman exec "$c" df -Pk 2>/dev/null
done
DISK
)
_host_df=$(printf '%s\n' "$_disk_raw" | sed -n '/^###HOST###$/,/^###CTRS###$/p' | sed '1d;$d')
mon_disk_check_remote_text "ssh:${KOOPA_SSH:-remote}" "$_host_df" || _disk_ec=1
_ctr=""
_buf=""
while IFS= read -r _line || [ -n "$_line" ]; do
case "$_line" in
'###CTR '*)
if [ -n "$_ctr" ]; then
mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || _disk_ec=1
fi
_ctr=${_line####CTR }
_ctr=${_ctr%###}
_buf=""
;;
'###CTRS###'|'###HOST###') ;;
*)
if [ -n "$_ctr" ]; then
_buf="${_buf}${_line}"$'\n'
fi
;;
esac
done <<<"$_disk_raw"
if [ -n "$_ctr" ]; then
mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || _disk_ec=1
fi
fi
unset _disk_raw _host_df _ctr _buf _line _cname _disk_ec
summary

294
check_qr_payloads.py Executable file
View file

@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""Collect + validate QR-related payloads for taler-monitoring (www.qr-*).
Sources (CLI paths / URLs handled by caller writing local files):
- landing HTML (href, data-qr-url, data-qr-taler, raw text)
- demo-withdraw.json / auto-account.json (taler_withdraw_uri / qr_payload)
- optional: already-decoded lines from zbarimg (kind=decoded)
Stdout TSV: source\\tkind\\turi\\tstatus\\tdetail
kind: withdraw | pay | pay-template | refund | payto | https | other | empty
status: ok | bad
Exit 0 always (caller tallies ok/bad lines). stderr for hard parse errors only.
"""
from __future__ import annotations
import json
import re
import sys
from html.parser import HTMLParser
from pathlib import Path
from urllib.parse import unquote, urlparse
# Schemes we care about for wallet / settlement QRs
TALER_RE = re.compile(r"taler://[^\s\"'<>]+", re.I)
PAYTO_RE = re.compile(r"payto://[^\s\"'<>]+", re.I)
HTTPS_RE = re.compile(r"https://[^\s\"'<>]+", re.I)
WITHDRAW_RE = re.compile(
r"^taler://withdraw/([^/]+)/taler-integration/([0-9a-fA-F-]+)/?$"
)
# order pay / template — host may include path segments after
PAY_RE = re.compile(r"^taler://pay/([^/?#]+)/", re.I)
PAY_TEMPLATE_RE = re.compile(r"^taler://pay-template/([^/?#]+)/", re.I)
REFUND_RE = re.compile(r"^taler://refund/", re.I)
PAYTO_OK_RE = re.compile(r"^payto://[a-z0-9][a-z0-9+.-]*/", re.I)
def looks_like_real_uri(uri: str) -> bool:
"""Drop JS regex fragments, docs ellipses, incomplete placeholders."""
if not uri or len(uri) < 12:
return False
if any(c in uri for c in "[]{}<>\n\r\t"):
return False
if "…" in uri or "..." in uri:
return False
# incomplete withdraw stubs in docs
if re.match(r"^taler://withdraw/?$", uri, re.I):
return False
if re.match(r"^taler://pay/?$", uri, re.I):
return False
# trailing junk / JS regex fragments (allow ? & = in query strings)
if uri.endswith((")", "(", "\\", "^", "*")):
return False
if re.search(r"[\\^$*]", uri):
return False
# lone incomplete `taler://…?` without path
if re.match(r"^taler://[^/]+$", uri, re.I):
return False
return True
def strip_default_port_host(host: str) -> str | None:
if not host or host.startswith(":"):
return None
if ":" in host:
h, _, p = host.rpartition(":")
if not h or not p.isdigit():
return None
if p in ("443", "80"):
return None # must be stripped for wallets
return host
def classify_and_validate(uri: str) -> tuple[str, str, str]:
"""Return (kind, status, detail)."""
uri = (uri or "").strip()
if not uri:
return "empty", "bad", "empty"
# strip accidental trailing punctuation from HTML scrape
uri = uri.rstrip(").,;]")
low = uri.lower()
if low.startswith("taler://withdraw/"):
m = WITHDRAW_RE.match(uri)
if not m:
return "withdraw", "bad", "shape need taler://withdraw/HOST/taler-integration/UUID"
host = m.group(1)
if strip_default_port_host(host) is None:
return "withdraw", "bad", "host/port invalid or default :443/:80 not stripped"
return "withdraw", "ok", f"host={host} id={m.group(2)[:12]}…"
if low.startswith("taler://pay-template/"):
m = PAY_TEMPLATE_RE.match(uri)
if not m:
return "pay-template", "bad", "need taler://pay-template/HOST/…"
host = m.group(1)
if strip_default_port_host(host) is None:
return "pay-template", "bad", "host/port invalid or default port not stripped"
return "pay-template", "ok", f"host={host}"
if low.startswith("taler://pay/"):
m = PAY_RE.match(uri)
if not m:
return "pay", "bad", "need taler://pay/HOST/…"
host = m.group(1)
if strip_default_port_host(host) is None:
return "pay", "bad", "host/port invalid or default port not stripped"
if "instances" not in uri and "/orders/" not in uri:
# still often valid private order URIs
return "pay", "ok", f"host={host}"
return "pay", "ok", f"host={host}"
if low.startswith("taler://refund/"):
if not REFUND_RE.match(uri):
return "refund", "bad", "bad refund URI"
return "refund", "ok", "refund"
if low.startswith("taler://"):
return "other", "ok", "other taler:// (accepted)"
if low.startswith("payto://"):
if not PAYTO_OK_RE.match(uri):
return "payto", "bad", "need payto://method/…"
# wallet pay QR must not be payto for GOA demo-withdraw (checked elsewhere)
return "payto", "ok", uri.split("?", 1)[0][:72]
if low.startswith("https://") or low.startswith("http://"):
p = urlparse(uri)
if p.scheme not in ("http", "https") or not p.netloc:
return "https", "bad", "bad URL"
return "https", "ok", p.netloc + (p.path or "/")[:40]
return "other", "bad", "unsupported scheme"
class LinkHarvester(HTMLParser):
def __init__(self) -> None:
super().__init__()
self.uris: list[tuple[str, str]] = [] # (source_hint, uri)
def handle_starttag(self, tag, attrs):
ad = {k.lower(): (v or "") for k, v in attrs}
for key in ("href", "src", "data-qr-url", "data-qr-taler", "data-uri", "data-payto"):
if key in ad and ad[key]:
self.uris.append((f"attr:{key}", ad[key].strip()))
for k, v in ad.items():
if k.startswith("data-") and v and (
"taler://" in v.lower() or "payto://" in v.lower() or k.endswith("qr-url")
):
self.uris.append((f"attr:{k}", v.strip()))
def handle_data(self, data):
if not data or "://" not in data:
return
for m in TALER_RE.finditer(data):
self.uris.append(("text", m.group(0)))
for m in PAYTO_RE.finditer(data):
self.uris.append(("text", m.group(0)))
def harvest_html(path: Path, source: str) -> list[tuple[str, str, str]]:
raw = path.read_text(encoding="utf-8", errors="replace")
h = LinkHarvester()
try:
h.feed(raw)
except Exception:
pass
out: list[tuple[str, str, str]] = []
for hint, u in h.uris:
lu = u.lower()
if lu.startswith(("taler://", "payto://", "https://", "http://")):
if looks_like_real_uri(u):
out.append((f"{source}:{hint}", u, "html"))
# raster image paths that look like QR assets (not .js)
elif re.search(r"\.(png|jpe?g|gif)(\?|$)", lu) and "qr" in lu:
out.append((f"{source}:imgpath", u, "imgpath"))
# Prefer attribute/text harvest; light regex only for complete-looking URIs in pre/code
for m in TALER_RE.finditer(raw):
u = m.group(0).rstrip(").,;]'\"")
if looks_like_real_uri(u) and (
WITHDRAW_RE.match(u)
or PAY_RE.match(u)
or PAY_TEMPLATE_RE.match(u)
or u.lower().startswith("taler://refund/")
):
out.append((f"{source}:re", u, "html"))
for m in PAYTO_RE.finditer(raw):
u = m.group(0).rstrip(").,;]'\"")
if looks_like_real_uri(u) and PAYTO_OK_RE.match(u):
out.append((f"{source}:re", u, "html"))
return out
def harvest_json(path: Path, source: str) -> list[tuple[str, str, str]]:
try:
d = json.loads(path.read_text(encoding="utf-8", errors="replace"))
except Exception as e:
return [(source, "", f"json-error:{e}")]
out: list[tuple[str, str, str]] = []
if not isinstance(d, dict):
return out
for key in (
"taler_withdraw_uri",
"qr_payload",
"taler_pay_uri",
"payto_uri",
"payto",
"uri",
):
v = d.get(key)
if isinstance(v, str) and v.strip():
out.append((f"{source}:{key}", v.strip(), "json"))
# nested
for k, v in d.items():
if isinstance(v, str) and (
v.startswith("taler://") or v.startswith("payto://")
):
out.append((f"{source}:{k}", v.strip(), "json"))
return out
def main(argv: list[str]) -> int:
# args: pairs of source_label path
if len(argv) < 3 or len(argv) % 2 == 0:
print(
"usage: check_qr_payloads.py LABEL path [LABEL path ...]",
file=sys.stderr,
)
return 2
seen: set[str] = set()
rows: list[tuple[str, str, str, str, str]] = []
i = 1
while i < len(argv):
label, path_s = argv[i], argv[i + 1]
i += 2
path = Path(path_s)
if not path.is_file():
rows.append((label, "empty", "", "bad", f"missing file {path_s}"))
continue
if path.suffix.lower() in (".html", ".htm") or "html" in path.name:
items = harvest_html(path, label)
elif path.suffix.lower() == ".json" or path.name.endswith(".json"):
items = harvest_json(path, label)
else:
# plain text: one URI per line or raw
text = path.read_text(encoding="utf-8", errors="replace").strip()
items = []
for line in text.splitlines():
line = line.strip()
if line:
items.append((label, line, "text"))
if not items and text:
items.append((label, text, "text"))
for src, uri, _origin in items:
if not uri:
continue
# skip pure fragment / relative without scheme for validate
if uri.startswith("#") or uri == "/":
continue
lu = uri.lower()
if lu.startswith("/") or "imgpath" in src:
continue
if not looks_like_real_uri(uri):
continue
# https: app-store / wallet / explicit data-qr-* only
if lu.startswith("http"):
if "data-qr" not in src and not any(
x in lu
for x in (
"play.google",
"apps.apple",
"f-droid",
"wallet.taler",
"taler.net/wallet",
)
):
continue
if uri in seen:
continue
seen.add(uri)
kind, status, detail = classify_and_validate(uri)
rows.append((src, kind, uri, status, detail))
for src, kind, uri, status, detail in rows:
# TSV — uri may contain tabs rarely; replace
safe = uri.replace("\t", " ").replace("\n", " ")
print(f"{src}\t{kind}\t{safe}\t{status}\t{detail}")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))

285
check_sanity.sh Executable file
View file

@ -0,0 +1,285 @@
#!/usr/bin/env bash
# Sanity checks for bank · exchange · merchant (public + server-side).
# Sections are independent; continues after failures.
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
expect_code() {
local label="$1" want="$2" url="$3"
local code
code=$(http_code "$url")
case ",$want," in
*",$code,"*) ok "$label" ;;
*) fail "$label" "HTTP $code (want $want) $url" ;;
esac
}
json_currency() {
python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d.get("currency") or "")' "$1" 2>/dev/null || true
}
# Area sanity.* — public + optional server-side per component
# Groups: sanity.bank / sanity.exchange / sanity.merchant
set_area sanity
# ---------------------------------------------------------------------------
set_group bank
section "sanity · bank"
# ---------------------------------------------------------------------------
expect_code "bank public /config" 200 "$BANK_PUBLIC/config"
expect_code "bank public /taler-integration/config" 200 "$BANK_PUBLIC/taler-integration/config"
expect_code "bank public /webui/" 200 "$BANK_PUBLIC/webui/"
code=$(http_body "$BANK_PUBLIC/config" "$tmp/bank-config.json")
if [ "$code" = "200" ]; then
cur=$(json_currency "$tmp/bank-config.json")
[ "$cur" = "GOA" ] && ok "bank currency GOA" || fail "bank currency" "got ${cur:-?}"
# wire type / name if present
python3 - "$tmp/bank-config.json" <<'PY' 2>/dev/null && ok "bank config JSON object" || fail "bank config JSON"
import json,sys
d=json.load(open(sys.argv[1]))
sys.exit(0 if isinstance(d, dict) and d.get("currency") else 1)
PY
if json_has_alt_unit_names "$tmp/bank-config.json" >/tmp/alt-bank-s.$$ 2>&1; then
ok "bank /config alt_unit_names" "$(tr '\n' '; ' </tmp/alt-bank-s.$$ | sed 's/; $//')"
else
fail "bank /config alt_unit_names" "$(tr '\n' '; ' </tmp/alt-bank-s.$$ | sed 's/; $//')"
fi
rm -f /tmp/alt-bank-s.$$
fi
# server-side bank
if koopa_ssh_ok; then
BOUT=$(koopa_ssh_bash 40 <<'REMOTE' || true
set +e
BANK=$(podman ps --format '{{.Names}}' | grep -iE 'hacktivism-bank|taler-bank' | head -1)
[ -z "$BANK" ] && BANK=$(podman ps --format '{{.Names}}' | grep -i bank | head -1)
echo "CTR=$BANK"
if [ -z "$BANK" ]; then echo "NOCTR"; exit 0; fi
code=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:9012/config 2>/dev/null || echo 000)
echo "LOCAL_CONFIG=$code"
code2=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:9012/taler-integration/config 2>/dev/null || echo 000)
echo "LOCAL_INT=$code2"
if podman exec "$BANK" bash -c 'pgrep -f "MainKt serve|libeufin-bank serve" >/dev/null' 2>/dev/null; then
echo "LIBEUFIN=1"
else
echo "LIBEUFIN=0"
fi
if podman exec "$BANK" bash -c 'pg_isready -q' 2>/dev/null; then
echo "PG=1"
else
echo "PG=0"
fi
# in-container health if present
if podman exec "$BANK" test -x /usr/local/bin/check_bank-health.sh 2>/dev/null; then
podman exec "$BANK" /usr/local/bin/check_bank-health.sh 2>&1 | sed 's/^/HEALTH /' | tail -20
echo "HEALTH_EC=${PIPESTATUS[0]}"
fi
REMOTE
)
echo "$BOUT" | grep -q '^CTR=.\+' && ok "bank container $(echo "$BOUT" | sed -n 's/^CTR=//p' | head -1)" || fail "bank container" "not found"
echo "$BOUT" | grep -q 'LOCAL_CONFIG=200' && ok "bank local :9012/config" || fail "bank local :9012/config"
echo "$BOUT" | grep -q 'LOCAL_INT=200' && ok "bank local :9012/taler-integration/config" || fail "bank local integration"
echo "$BOUT" | grep -q 'LIBEUFIN=1' && ok "bank libeufin-bank process" || fail "bank libeufin-bank process"
echo "$BOUT" | grep -q 'PG=1' && ok "bank postgres ready" || warn "bank postgres" "pg_isready failed"
if echo "$BOUT" | grep -q 'HEALTH '; then
if echo "$BOUT" | grep -qE 'HEALTH_EC=0|ALL CRITICAL CHECKS PASSED'; then
ok "bank check_bank-health.sh"
else
# health script may false-fail process grep; warn not fail if local config ok
warn "bank check_bank-health.sh" "non-zero or incomplete"
fi
fi
else
warn "bank server-side" "ssh ${KOOPA_SSH} unavailable"
fi
# ---------------------------------------------------------------------------
set_group exchange
section "sanity · exchange"
# ---------------------------------------------------------------------------
expect_code "exchange public /config" 200 "$EXCHANGE_PUBLIC/config"
expect_code "exchange public /keys" 200 "$EXCHANGE_PUBLIC/keys"
expect_code "exchange public /terms" 200 "$EXCHANGE_PUBLIC/terms"
code=$(http_body "$EXCHANGE_PUBLIC/config" "$tmp/ex-config.json")
if [ "$code" = "200" ]; then
cur=$(json_currency "$tmp/ex-config.json")
[ "$cur" = "GOA" ] && ok "exchange currency GOA" || fail "exchange currency" "got ${cur:-?}"
if json_has_alt_unit_names "$tmp/ex-config.json" "GOA" >/tmp/alt-ex-s.$$ 2>&1; then
ok "exchange /config alt_unit_names" "$(tr '\n' '; ' </tmp/alt-ex-s.$$ | sed 's/; $//')"
else
fail "exchange /config alt_unit_names" "$(tr '\n' '; ' </tmp/alt-ex-s.$$ | sed 's/; $//')"
fi
rm -f /tmp/alt-ex-s.$$
fi
code=$(http_body "$EXCHANGE_PUBLIC/keys" "$tmp/ex-keys.json")
if [ "$code" = "200" ]; then
python3 - "$tmp/ex-keys.json" <<'PY'
import json,sys,time
d=json.load(open(sys.argv[1]))
sk=d.get("signkeys") or []
acc=d.get("accounts") or []
den=d.get("denominations") or []
# count denoms roughly
n=0
for g in den:
if isinstance(g, dict):
n += len(g.get("denoms") or [])
print(f"signkeys={len(sk)} accounts={len(acc)} denom_groups={len(den)} denoms~={n}")
sys.exit(0 if sk and (acc or n) else 1)
PY
ec=$?
detail=$(python3 - "$tmp/ex-keys.json" <<'PY'
import json,sys
d=json.load(open(sys.argv[1]))
sk=d.get("signkeys") or []
acc=d.get("accounts") or []
den=d.get("denominations") or []
n=sum(len(g.get("denoms") or []) for g in den if isinstance(g, dict))
print(f"signkeys={len(sk)} accounts={len(acc)} denoms~={n}")
PY
)
[ "$ec" -eq 0 ] && ok "exchange /keys usable ($detail)" || fail "exchange /keys usable" "$detail"
fi
# /wire if exposed
wcode=$(http_code "$EXCHANGE_PUBLIC/wire")
if [ "$wcode" = "200" ]; then
ok "exchange public /wire"
else
warn "exchange public /wire" "HTTP $wcode (accounts may only be in /keys)"
fi
if koopa_ssh_ok; then
EOUT=$(koopa_ssh_bash 40 <<'REMOTE' || true
set +e
EX=$(podman ps --format '{{.Names}}' | grep -i exchange | head -1)
echo "CTR=$EX"
code=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:9011/config 2>/dev/null || echo 000)
echo "LOCAL_CONFIG=$code"
codek=$(curl -sS -m 8 -o /dev/null -w '%{http_code}' http://127.0.0.1:9011/keys 2>/dev/null || echo 000)
echo "LOCAL_KEYS=$codek"
if [ -n "$EX" ]; then
if podman exec "$EX" bash -c 'pgrep -f taler-exchange-httpd >/dev/null' 2>/dev/null; then
echo "HTTPD=1"
else
echo "HTTPD=0"
fi
for p in taler-exchange-secmod-rsa taler-exchange-secmod-eddsa taler-exchange-wirewatch taler-exchange-aggregator; do
if podman exec "$EX" bash -c "pgrep -f $p >/dev/null" 2>/dev/null; then
echo "PROC_$p=1"
else
echo "PROC_$p=0"
fi
done
if podman exec "$EX" test -x /usr/local/bin/check_exchange-health.sh 2>/dev/null; then
SKIP_ENSURE=1 podman exec -e SKIP_ENSURE=1 "$EX" /usr/local/bin/check_exchange-health.sh 2>&1 | sed 's/^/HEALTH /' | tail -25
fi
fi
REMOTE
)
echo "$EOUT" | grep -q '^CTR=.\+' && ok "exchange container $(echo "$EOUT" | sed -n 's/^CTR=//p' | head -1)" || fail "exchange container"
echo "$EOUT" | grep -q 'LOCAL_CONFIG=200' && ok "exchange local :9011/config" || fail "exchange local :9011/config"
echo "$EOUT" | grep -q 'LOCAL_KEYS=200' && ok "exchange local :9011/keys" || fail "exchange local :9011/keys"
echo "$EOUT" | grep -q 'HTTPD=1' && ok "exchange-httpd process" || warn "exchange-httpd process" "not detected"
echo "$EOUT" | grep -q 'PROC_taler-exchange-wirewatch=1' && ok "exchange wirewatch" || warn "exchange wirewatch" "not running"
echo "$EOUT" | grep -q 'PROC_taler-exchange-aggregator=1' && ok "exchange aggregator" || warn "exchange aggregator" "not running"
else
warn "exchange server-side" "ssh unavailable"
fi
# ---------------------------------------------------------------------------
set_group merchant
section "sanity · merchant"
# ---------------------------------------------------------------------------
expect_code "merchant public /config" 200 "$MERCHANT_PUBLIC/config"
expect_code "merchant public /webui/" 200 "$MERCHANT_PUBLIC/webui/"
code=$(http_body "$MERCHANT_PUBLIC/config" "$tmp/mer-config.json")
if [ "$code" = "200" ]; then
if python3 - "$tmp/mer-config.json" "$EXCHANGE_PUBLIC" <<'PY'
import json,sys
d=json.load(open(sys.argv[1]))
want=sys.argv[2].rstrip("/")
curs=list((d.get("currencies") or {}).keys())
ex=d.get("exchanges") or []
urls=[]
for e in ex:
if isinstance(e, dict):
u=e.get("base_url") or e.get("url") or e.get("exchange_base_url") or ""
if u: urls.append(u.rstrip("/"))
elif isinstance(e, str):
urls.append(e.rstrip("/"))
ok_goa = "GOA" in curs or any((e.get("currency") if isinstance(e, dict) else None)=="GOA" for e in ex)
ok_ex = any(want in u or "exchange.hacktivism.ch" in u for u in urls)
print("currencies", curs)
print("exchanges", urls[:5])
sys.exit(0 if ok_goa and ok_ex else 1)
PY
then
ok "merchant config GOA + exchange.hacktivism.ch"
else
fail "merchant config GOA + exchange" "see currencies/exchanges"
fi
if json_has_alt_unit_names "$tmp/mer-config.json" >/tmp/alt-mer-s.$$ 2>&1; then
ok "merchant currencies alt_unit_names" "$(tr '\n' '; ' </tmp/alt-mer-s.$$ | sed 's/; $//')"
else
fail "merchant currencies alt_unit_names" "$(tr '\n' '; ' </tmp/alt-mer-s.$$ | sed 's/; $//')"
fi
rm -f /tmp/alt-mer-s.$$
# Each exchange listed on merchant /config must publish alt_unit_names on its own /config
check_merchant_listed_exchanges_alt_units "$tmp/mer-config.json"
fi
# demo instance reachable?
INST="${MERCHANT_INSTANCE}"
icode=$(http_code "$MERCHANT_PUBLIC/instances/${INST}/config")
if [ "$icode" = "200" ]; then
ok "merchant instance ${INST} /config"
else
# some deployments use private only
warn "merchant instance ${INST} /config" "HTTP $icode"
fi
if koopa_ssh_ok; then
MOUT=$(koopa_ssh_bash 40 <<'REMOTE' || true
set +e
MER=$(podman ps --format '{{.Names}}' | grep -E '^taler-hacktivism$' | head -1)
[ -z "$MER" ] && MER=$(podman ps --format '{{.Names}}' | grep -iE 'merchant|hacktivism' | grep -viE 'bank|exchange' | head -1)
echo "CTR=$MER"
code=$(curl -skS -m 5 -o /dev/null -w '%{http_code}' https://127.0.0.1:9010/config 2>/dev/null || echo 000)
echo "LOCAL_CONFIG=$code"
if [ -n "$MER" ]; then
if podman exec "$MER" bash -c 'pgrep -f taler-merchant-httpd >/dev/null' 2>/dev/null; then
echo "HTTPD=1"
else
echo "HTTPD=0"
fi
if podman exec "$MER" test -x /usr/local/bin/check_merchant-health.sh 2>/dev/null; then
SKIP_ENSURE=1 podman exec -e SKIP_ENSURE=1 "$MER" /usr/local/bin/check_merchant-health.sh 2>&1 | sed 's/^/HEALTH /' | tail -30
fi
fi
REMOTE
)
echo "$MOUT" | grep -q '^CTR=.\+' && ok "merchant container $(echo "$MOUT" | sed -n 's/^CTR=//p' | head -1)" || fail "merchant container"
echo "$MOUT" | grep -q 'LOCAL_CONFIG=200' && ok "merchant local :9010/config" || fail "merchant local :9010/config"
echo "$MOUT" | grep -q 'HTTPD=1' && ok "merchant-httpd process" || warn "merchant-httpd process" "not detected"
if echo "$MOUT" | grep -q 'HEALTH '; then
if echo "$MOUT" | grep -qiE 'ALL CRITICAL|HEALTH_EC=0|\[OK\]'; then
ok "merchant check_merchant-health.sh (sample OK)"
else
warn "merchant check_merchant-health.sh" "see remote output"
fi
fi
else
warn "merchant server-side" "ssh unavailable"
fi
summary

158
check_server.sh Executable file
View file

@ -0,0 +1,158 @@
#!/usr/bin/env bash
# Server-side component checks on koopa (via SSH).
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
# Area server.* — host/container ports via SSH
set_area server
set_group ssh
section "server · ssh ${KOOPA_SSH}"
if ! koopa_ssh_ok; then
fail "ssh ${KOOPA_SSH}" "unreachable within ${SSH_CONNECT_TIMEOUT}s (SKIP_SSH=1 to skip)"
summary
exit 1
fi
ok "ssh ${KOOPA_SSH}"
# Run a remote script; collect structured lines
REMOTE=$(koopa_ssh_bash "${SSH_CMD_TIMEOUT}" <<'REMOTE'
set +e
report() { printf 'R|%s|%s|%s\n' "$1" "$2" "$3"; }
# containers
for name in taler-hacktivism-bank taler-hacktivism-exchange-ansible taler-hacktivism; do
if podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$name"; then
st=$(podman ps --filter "name=^${name}$" --format '{{.Status}}' | head -1)
report OK "container $name" "$st"
else
# soft match
hit=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -E "$name|bank|exchange|hacktivism" | head -3 | tr '\n' ' ')
if podman ps --format '{{.Names}}' 2>/dev/null | grep -q bank && [ "$name" = taler-hacktivism-bank ]; then
bn=$(podman ps --format '{{.Names}}' | grep -i bank | head -1)
report OK "container bank ($bn)" "$(podman ps --filter name="$bn" --format '{{.Status}}' | head -1)"
elif podman ps --format '{{.Names}}' 2>/dev/null | grep -qi exchange && echo "$name" | grep -qi exchange; then
en=$(podman ps --format '{{.Names}}' | grep -i exchange | head -1)
report OK "container exchange ($en)" "$(podman ps --filter name="$en" --format '{{.Status}}' | head -1)"
elif podman ps --format '{{.Names}}' 2>/dev/null | grep -qx taler-hacktivism && [ "$name" = taler-hacktivism ]; then
report OK "container taler-hacktivism" "$(podman inspect -f '{{.State.Status}}' taler-hacktivism 2>/dev/null)"
else
report FAIL "container $name" "not running (seen: $hit)"
fi
fi
done
# local HTTP on pasta ports
for spec in \
"bank-api|http://127.0.0.1:9012/config|200" \
"bank-integration|http://127.0.0.1:9012/taler-integration/config|200" \
"landing|http://127.0.0.1:9013/intro/|200" \
"exchange|http://127.0.0.1:9011/config|200" \
"merchant|https://127.0.0.1:9010/config|200"
do
IFS='|' read -r id url want <<<"$spec"
code=$(curl -skS -m 5 -o /tmp/mon-s.out -w '%{http_code}' "$url" 2>/dev/null || echo 000)
if [ "$code" = "$want" ]; then
report OK "local $id : ${url#*//}" "HTTP $code"
else
report FAIL "local $id" "HTTP $code want $want"
fi
done
# processes inside bank
BANK=$(podman ps --format '{{.Names}}' | grep -iE 'bank|hacktivism-bank' | head -1)
if [ -n "$BANK" ]; then
if podman exec "$BANK" bash -c 'pgrep -f "MainKt serve|libeufin-bank serve" >/dev/null' 2>/dev/null; then
report OK "libeufin-bank process" "in $BANK"
else
report FAIL "libeufin-bank process" "not running in $BANK"
fi
if podman exec "$BANK" bash -c 'pg_isready -q' 2>/dev/null; then
report OK "postgres (bank)" "ready"
else
report WARN "postgres (bank)" "pg_isready failed"
fi
if podman exec "$BANK" bash -c 'pgrep -x nginx >/dev/null' 2>/dev/null; then
report OK "nginx landing" "in $BANK"
else
report WARN "nginx landing" "not seen in $BANK"
fi
else
report FAIL "bank container" "none"
fi
# exchange process / systemd if any
EX=$(podman ps --format '{{.Names}}' | grep -i exchange | head -1)
if [ -n "$EX" ]; then
if podman exec "$EX" bash -c 'pgrep -f taler-exchange-httpd >/dev/null || systemctl is-active taler-exchange-httpd 2>/dev/null | grep -q active' 2>/dev/null; then
report OK "exchange-httpd" "in $EX"
else
# config answering is enough
report WARN "exchange-httpd process" "not detected; port check above"
fi
fi
MER=$(podman ps --format '{{.Names}}' | grep -E '^taler-hacktivism$' | head -1)
[ -z "$MER" ] && MER=$(podman ps --format '{{.Names}}' | grep -i merchant | head -1)
if [ -n "$MER" ]; then
if podman exec "$MER" bash -c 'pgrep -f taler-merchant-httpd >/dev/null || true; curl -sk -m 3 -o /dev/null -w %{http_code} https://127.0.0.1:9010/config' 2>/dev/null | grep -q 200; then
report OK "merchant-httpd" "responds in $MER"
else
report WARN "merchant-httpd" "check manually in $MER"
fi
fi
# caddy on host
if systemctl is-active caddy >/dev/null 2>&1 || pgrep -x caddy >/dev/null 2>&1; then
report OK "caddy" "active"
else
report WARN "caddy" "not detected as active"
fi
# disk free (host + containers) — lines for mon_disk_check_remote_text on laptop side
echo "###DISK_HOST###"
df -Pk / /var /home /tmp 2>/dev/null || df -Pk
echo "###DISK_CTRS###"
for c in $(podman ps --format '{{.Names}}' 2>/dev/null); do
echo "###CTR $c###"
podman exec "$c" df -Pk / /var /tmp 2>/dev/null || podman exec "$c" df -Pk 2>/dev/null
done
REMOTE
)
while IFS= read -r line; do
case "$line" in
R\|*)
IFS='|' read -r _ st label detail <<<"$line"
case "$st" in
OK) ok "$label${detail:+ ($detail)}" ;;
FAIL) fail "$label" "$detail" ;;
WARN) warn "$label" "$detail" ;;
esac
;;
esac
done <<<"$REMOTE"
set_group disk
section "server · disk free space"
_host_df=$(printf '%s\n' "$REMOTE" | sed -n '/^###DISK_HOST###$/,/^###DISK_CTRS###$/p' | sed '1d;$d')
mon_disk_check_remote_text "ssh:${KOOPA_SSH}" "$_host_df" || true
_ctr=""; _buf=""
while IFS= read -r _line || [ -n "$_line" ]; do
case "$_line" in
'###CTR '*)
if [ -n "$_ctr" ]; then mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true; fi
_ctr=${_line####CTR }; _ctr=${_ctr%###}; _buf=""
;;
'###DISK_CTRS###'|'###DISK_HOST###') ;;
*)
# only buffer after we entered CTR section
if [ -n "$_ctr" ]; then _buf="${_buf}${_line}"$'\n'; fi
;;
esac
done <<<"$REMOTE"
[ -n "$_ctr" ] && mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true
summary

692
check_surface.sh Executable file
View file

@ -0,0 +1,692 @@
#!/usr/bin/env bash
# check_surface.sh — REMOTE-ONLY public surface / ecosystem scan.
#
# NOT in default phases. Explicit:
# ./taler-monitoring.sh surface
# ./taler-monitoring.sh -d hacktivism.ch surface
#
# Without -d / with generic ecosystem: catalog of taler.net, gnunet.org,
# taler-systems.com, taler-ops, mattermost, …
# With -d DOMAIN: expand domains.conf stack hosts + catalog entries matching
# that domain + common subdomain guesses; port/protocol probes only remote.
#
# Rules:
# - no SSH into targets, no local podman on targets
# - ICMP/port alone never proves "down" — confirm via expected protocol
# - catalogued host unreachable via protocol → ERROR
# - version unknown after probes → WARN
# - CVE hit (OSV) for identified software+version → ERROR
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
set_area surface
section "surface · remote ecosystem / domain inventory (outside-in only)"
CATALOG="${SURFACE_CATALOG:-$ROOT/surface-catalog.conf}"
# Common ports when scanning a host discovered without explicit list
DEFAULT_SCAN_PORTS="${SURFACE_SCAN_PORTS:-22,80,443,993,8443,9418}"
PORT_TIMEOUT="${SURFACE_PORT_TIMEOUT:-2}"
HTTP_TIMEOUT="${SURFACE_HTTP_TIMEOUT:-12}"
# CVE via OSV (public, no key). Disable: SURFACE_CVE=0
: "${SURFACE_CVE:=1}"
# Extra ports when -d domain mode
DOMAIN_EXTRA_PORTS="${SURFACE_DOMAIN_PORTS:-22,80,443,8443}"
declare -A HOST_PORTS # host -> comma ports
declare -A HOST_PROTO # host -> expect proto
declare -A HOST_LABEL # host -> short label
declare -A HOST_EXPECT # host -> 1 if catalogued (must respond)
ORDERED_HOSTS=()
add_host() {
local h="$1" ports="${2:-}" proto="${3:-https}" label="${4:-}" expect="${5:-1}"
h=$(printf '%s' "$h" | tr 'A-Z' 'a-z' | sed 's#^https\?://##;s#/.*##;s/\.$//')
[ -n "$h" ] || return 0
if [ -z "${HOST_PORTS[$h]+x}" ]; then
ORDERED_HOSTS+=("$h")
HOST_PORTS[$h]="${ports:-443}"
HOST_PROTO[$h]="${proto:-https}"
HOST_LABEL[$h]="${label:-$h}"
HOST_EXPECT[$h]="$expect"
else
# merge ports
local p
for p in ${ports//,/ }; do
case ",${HOST_PORTS[$h]}," in
*",$p,"*) ;;
*) HOST_PORTS[$h]="${HOST_PORTS[$h]},$p" ;;
esac
done
[ "${HOST_EXPECT[$h]}" = "1" ] || HOST_EXPECT[$h]="$expect"
fi
}
load_catalog() {
local line h ports proto label
[ -f "$CATALOG" ] || {
warn "catalog" "missing $CATALOG — using empty base list"
return 0
}
while IFS= read -r line || [ -n "$line" ]; do
line=${line%%#*}
line=$(echo "$line" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
[ -z "$line" ] && continue
# host ports proto label
h=$(echo "$line" | awk '{print $1}')
ports=$(echo "$line" | awk '{print $2}')
proto=$(echo "$line" | awk '{print $3}')
label=$(echo "$line" | awk '{print $4}')
[ "$ports" = "-" ] && ports=""
add_host "$h" "$ports" "${proto:-https}" "${label:-}" 1
done <"$CATALOG"
info "catalog" "loaded $CATALOG · ${#ORDERED_HOSTS[@]} hosts"
}
# Clear inventory (used when switching to domain-only scope)
clear_hosts() {
ORDERED_HOSTS=()
unset HOST_PORTS HOST_PROTO HOST_LABEL HOST_EXPECT
declare -gA HOST_PORTS HOST_PROTO HOST_LABEL HOST_EXPECT
}
# Expand for -d domain: ONLY that domain (remote). No whole-ecosystem catalog.
expand_domain_scope() {
local d="$1" h
d=${d#https://}; d=${d%/}
d=${d#http://}
info "scope" "domain mode −d $d (remote only · not full ecosystem catalog)"
clear_hosts
# stack endpoints from applied profile (always expected)
for h in \
"${BANK_PUBLIC:-}" \
"${EXCHANGE_PUBLIC:-}" \
"${MERCHANT_PUBLIC:-}" \
"${PAIVANA_PUBLIC:-}"
do
[ -n "$h" ] || continue
add_host "$h" "80,443,$DOMAIN_EXTRA_PORTS" https "stack" 1
done
# apex + common subdomains (expected if DNS exists — set expect after DNS in scan,
# but catalogued stack already expected; guesses start optional)
for sub in "" www bank exchange taler merchant backend shop shops stage \
git docs www2 api static media landing mon401
do
if [ -z "$sub" ]; then
h="$d"
else
h="${sub}.${d}"
fi
# stack hosts already added as expect=1; guesses optional until we promote
if [ -z "${HOST_PORTS[$h]+x}" ]; then
add_host "$h" "$DOMAIN_EXTRA_PORTS" https "guess" 0
fi
done
# also pick catalog lines that belong to this domain only
if [ -f "$CATALOG" ]; then
local line ports proto label
while IFS= read -r line || [ -n "$line" ]; do
line=${line%%#*}
line=$(echo "$line" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
[ -z "$line" ] && continue
h=$(echo "$line" | awk '{print $1}')
h=$(printf '%s' "$h" | tr 'A-Z' 'a-z' | sed 's#^https\?://##;s#/.*##')
if [ "$h" = "$d" ] || [[ "$h" == *".$d" ]]; then
ports=$(echo "$line" | awk '{print $2}')
proto=$(echo "$line" | awk '{print $3}')
label=$(echo "$line" | awk '{print $4}')
[ "$ports" = "-" ] && ports=""
add_host "$h" "$ports" "${proto:-https}" "${label:-dom}" 1
fi
done <"$CATALOG"
fi
}
# Ecosystem mode: full catalog; all entries expected
expand_ecosystem_scope() {
info "scope" "ecosystem mode (taler.net / gnunet.org / taler-systems.com / taler-ops / mattermost, … · remote only)"
local h
for h in "${ORDERED_HOSTS[@]}"; do
HOST_EXPECT[$h]=1
done
}
# --- remote probes (no SSH) ---
dns_ok() {
local h="$1"
getent ahosts "$h" >/dev/null 2>&1 || getent hosts "$h" >/dev/null 2>&1
}
# TCP connect only (not proof of service)
tcp_open() {
local h="$1" port="$2"
if command -v timeout >/dev/null 2>&1; then
timeout "$PORT_TIMEOUT" bash -c "echo >/dev/tcp/${h}/${port}" 2>/dev/null
else
bash -c "echo >/dev/tcp/${h}/${port}" 2>/dev/null
fi
}
# ICMP optional — never sole grounds for ERROR
ping_host() {
local h="$1"
ping -c 1 -W 2 "$h" >/dev/null 2>&1 || ping -c 1 -w 2 "$h" >/dev/null 2>&1
}
# HTTPS probe: code, server header, version hints, cert subject/dates
probe_https() {
local h="$1" port="${2:-443}"
local url hdr body code server via cert_end subj san
url="https://${h}/"
[ "$port" != "443" ] && url="https://${h}:${port}/"
body=$(mktemp)
hdr=$(mktemp)
code=$(curl -skS -m "$HTTP_TIMEOUT" -D "$hdr" -o "$body" -w '%{http_code}' \
--connect-timeout "$PORT_TIMEOUT" "$url" 2>/dev/null || echo 000)
server=$(awk 'BEGIN{IGNORECASE=1} /^server:/{sub(/\r$/,""); sub(/^server:[[:space:]]*/,""); print; exit}' "$hdr" 2>/dev/null || true)
via=$(awk 'BEGIN{IGNORECASE=1} /^x-powered-by:/{sub(/\r$/,""); sub(/^[^:]+:[[:space:]]*/,""); print; exit}' "$hdr" 2>/dev/null || true)
# cert
cert_end=$(echo | openssl s_client -servername "$h" -connect "${h}:${port}" 2>/dev/null \
| openssl x509 -noout -enddate 2>/dev/null | sed 's/notAfter=//')
subj=$(echo | openssl s_client -servername "$h" -connect "${h}:${port}" 2>/dev/null \
| openssl x509 -noout -subject 2>/dev/null | head -1)
# taler version hints in body
local taler_hint
taler_hint=$(grep -oE 'taler[^"[:space:]]{0,40}|GNU Taler|libeufin|gnunet' "$body" 2>/dev/null | head -3 | tr '\n' ' ' || true)
# config JSON version if /config works
local cfg_ver=""
local ccode
ccode=$(curl -skS -m "$HTTP_TIMEOUT" -o "$body" -w '%{http_code}' \
"https://${h}/config" 2>/dev/null || echo 000)
if [ "$ccode" = "200" ]; then
cfg_ver=$(python3 -c 'import json,sys
try:
d=json.load(open(sys.argv[1]))
print(d.get("version") or d.get("name") or d.get("currency") or "config-json")
except Exception:
print("")' "$body" 2>/dev/null || true)
fi
rm -f "$hdr" "$body"
printf 'code=%s server=%s powered=%s cert_end=%s cfg=%s hint=%s' \
"$code" "${server:-}" "${via:-}" "${cert_end:-}" "${cfg_ver:-}" "${taler_hint:-}"
# return 0 if HTTP answered (any 2xx/3xx/4xx — service is up)
case "$code" in
2??|3??|4??) return 0 ;;
*) return 1 ;;
esac
}
probe_http() {
local h="$1" port="${2:-80}"
local url code
url="http://${h}/"
[ "$port" != "80" ] && url="http://${h}:${port}/"
code=$(curl -sS -m "$HTTP_TIMEOUT" -o /dev/null -w '%{http_code}' \
--connect-timeout "$PORT_TIMEOUT" "$url" 2>/dev/null || echo 000)
printf 'code=%s' "$code"
case "$code" in 2??|3??|4??) return 0 ;; *) return 1 ;; esac
}
probe_ssh_banner() {
local h="$1" port="${2:-22}"
local ban
ban=$(timeout "$PORT_TIMEOUT" bash -c "exec 3<>/dev/tcp/${h}/${port}; dd bs=256 count=1 <&3 2>/dev/null" 2>/dev/null \
| tr -d '\r' | head -1 || true)
printf 'banner=%s' "${ban:-}"
[ -n "$ban" ]
}
# Map Server header → package name guess for OSV
guess_package() {
local server="$1"
local s
s=$(printf '%s' "$server" | tr 'A-Z' 'a-z')
case "$s" in
*nginx*) echo "nginx" ;;
*apache*|*httpd*) echo "apache" ;;
*caddy*) echo "caddy" ;;
*openbsd\ httpd*) echo "openbsd-httpd" ;;
*) echo "" ;;
esac
}
extract_version() {
local server="$1"
# nginx/1.22.1 → 1.22.1
printf '%s' "$server" | sed -n 's/.*\/\([0-9][0-9.]*\).*/\1/p' | head -1
}
# OSV / CVE check for software version taken from Server headers.
#
# Important: headers only expose upstream versions (nginx/1.26.3), never the
# Debian package revision (1.26.3-3+deb13u7). Querying OSV ecosystem=Debian
# with the bare version yields massive false positives (ancient DEBIAN-CVE-*
# with introduced:0 and no fixed event, plus every package revision that
# merely *starts with* 1.26.3).
#
# Policy:
# - bare upstream version → upstream SEMVER signal only (WARN by default)
# - full Debian package version (contains '-') → Debian OSV, filtered
# - SURFACE_CVE=0 disables; SURFACE_CVE_LEVEL=error|warn (default warn for bare)
check_cves() {
local pkg="$1" ver="$2" host="$3"
[ "$SURFACE_CVE" = "1" ] || return 0
[ -n "$pkg" ] && [ -n "$ver" ] || return 0
local level out rc
# bare header versions are approximate → WARN unless overridden
if [[ "$ver" == *-* ]] || [[ "$ver" == *+* ]]; then
level="${SURFACE_CVE_LEVEL:-error}"
else
level="${SURFACE_CVE_LEVEL:-warn}"
fi
out=$(
SURFACE_CVE_PKG="$pkg" SURFACE_CVE_VER="$ver" SURFACE_CVE_HOST="$host" python3 - <<'PY' 2>/dev/null || true
import json, os, re, urllib.request
pkg = os.environ.get("SURFACE_CVE_PKG", "")
ver = os.environ.get("SURFACE_CVE_VER", "")
host = os.environ.get("SURFACE_CVE_HOST", "")
def ver_tuple(s: str):
nums = [int(x) for x in re.findall(r"\d+", s or "")[:5]]
return tuple(nums) if nums else ()
def vt_cmp(a, b):
n = max(len(a), len(b))
a = a + (0,) * (n - len(a))
b = b + (0,) * (n - len(b))
return (a > b) - (a < b)
def in_semver_events(ver, events):
"""True if ver is in [introduced, fixed) for SEMVER-like event list."""
vt = ver_tuple(ver)
if not vt:
return False
# process sequential introduced/fixed pairs
intro = None
for e in events or []:
if "introduced" in e:
intro = e.get("introduced")
elif "fixed" in e or "last_affected" in e:
fixed = e.get("fixed")
last = e.get("last_affected")
lo = ver_tuple("0" if intro in (None, "0") else str(intro))
if vt_cmp(vt, lo) < 0:
intro = None
continue
if fixed is not None:
if vt_cmp(vt, ver_tuple(str(fixed))) < 0:
return True
elif last is not None:
if vt_cmp(vt, ver_tuple(str(last))) <= 0:
return True
intro = None
# open-ended introduced without fixed → ignore (never clears; Debian noise)
return False
def osv_post(body):
req = urllib.request.Request(
"https://api.osv.dev/v1/query",
data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=20) as r:
return json.load(r)
except Exception:
return {}
def osv_get(vid):
try:
with urllib.request.urlopen(f"https://api.osv.dev/v1/vulns/{vid}", timeout=15) as r:
return json.load(r)
except Exception:
return {}
bare = not (("-" in ver) or ("+" in ver))
hits = []
if bare:
# Upstream signal via CVE records that carry extracted_events for this package
# (Server header has no Debian revision — do NOT use ecosystem=Debian).
seed = {
"nginx": [
"CVE-2025-23419",
"CVE-2025-53859",
"CVE-2024-7347",
"CVE-2024-34161",
"CVE-2024-32760",
"CVE-2024-31079",
"CVE-2024-24989",
"CVE-2024-24990",
],
"apache": [
"CVE-2024-38474",
"CVE-2024-38476",
"CVE-2024-38477",
"CVE-2023-31122",
"CVE-2023-43622",
],
"caddy": [
"CVE-2022-29718",
"CVE-2023-50463",
],
}.get(pkg, [])
for vid in seed:
doc = osv_get(vid)
if not doc:
continue
ok_hit = False
for a in doc.get("affected") or []:
# prefer extracted_events (human SEMVER) on GIT/nginx ranges
for rg in a.get("ranges") or []:
db = rg.get("database_specific") or {}
extracted = db.get("extracted_events") or []
if extracted and in_semver_events(ver, extracted):
# package name filter when present
cpes = db.get("cpe") or []
if isinstance(cpes, str):
cpes = [cpes]
blob = json.dumps(a).lower() + json.dumps(cpes).lower()
if pkg == "nginx" and "nginx" not in blob and "f5" not in blob:
continue
ok_hit = True
break
if rg.get("type") == "SEMVER" and in_semver_events(ver, rg.get("events") or []):
ok_hit = True
break
if ok_hit:
break
if ok_hit:
hits.append(vid)
else:
# Full package version — Debian ecosystem is meaningful
data = osv_post({"package": {"name": pkg, "ecosystem": "Debian"}, "version": ver})
for v in data.get("vulns") or []:
vid = v.get("id") or "?"
actionable = False
for a in v.get("affected") or []:
for rg in a.get("ranges") or []:
events = rg.get("events") or []
has_end = any(("fixed" in e) or ("last_affected" in e) for e in events)
if not has_end:
continue # open-ended Debian noise
if rg.get("type") in ("ECOSYSTEM", "SEMVER"):
# version string is full deb version; trust OSV query match
# but only if a fixed event exists (actionable)
actionable = True
break
if actionable:
break
# exact version listed
versions = a.get("versions") or []
if ver in versions:
actionable = True
break
if actionable:
hits.append(vid)
# de-dup preserve order
seen = set()
uniq = []
for h in hits:
if h not in seen:
seen.add(h)
uniq.append(h)
if uniq:
print(f"HIT {len(uniq)} " + ",".join(uniq[:12]))
else:
print("CLEAN")
PY
)
rc=0
case "$out" in
HIT\ *)
local n ids
n=$(printf '%s' "$out" | awk '{print $2}')
ids=$(printf '%s' "$out" | cut -d' ' -f3-)
if [ "$level" = "error" ]; then
err "cve" "$host $pkg $ver — $n actionable vuln(s)" "$ids"
rc=1
else
warn "cve" "$host $pkg $ver — $n actionable vuln(s) (header version · not Debian pkg)" "$ids"
rc=0
fi
;;
CLEAN)
if [[ "$ver" == *-* ]] || [[ "$ver" == *+* ]]; then
info "cve" "$host $pkg $ver — OSV clean (Debian package version)"
else
info "cve" "$host $pkg $ver — no actionable upstream CVE for bare Server-header version"
fi
;;
*)
info "cve" "$host $pkg $ver — CVE probe skipped/unavailable"
;;
esac
return "$rc"
}
cert_expiry_check() {
local h="$1" end="$2"
[ -n "$end" ] || return 0
local end_epoch now left days
end_epoch=$(date -d "$end" +%s 2>/dev/null || date -j -f "%b %e %T %Y %Z" "$end" +%s 2>/dev/null || echo 0)
now=$(date +%s)
[ "$end_epoch" -gt 0 ] || return 0
left=$((end_epoch - now))
days=$((left / 86400))
if [ "$left" -le 0 ]; then
err "tls" "$h certificate EXPIRED" "notAfter=$end"
return 1
fi
if [ "$days" -le 14 ]; then
warn "tls" "$h certificate expires in ${days}d" "notAfter=$end"
else
info "tls" "$h cert ok · ${days}d left · notAfter=$end"
fi
return 0
}
scan_host() {
local h="$1"
local ports proto label expect
local p open_ports=() any_proto_ok=0 ping_ok=0 dns=0
local probe_detail server_hdr pkg ver
ports=${HOST_PORTS[$h]:-443}
proto=${HOST_PROTO[$h]:-https}
label=${HOST_LABEL[$h]:-$h}
expect=${HOST_EXPECT[$h]:-0}
set_group "$label"
if dns_ok "$h"; then
dns=1
ok "dns" "$h resolves"
else
if [ "$expect" = "1" ]; then
err "dns" "$h does not resolve (catalogued)"
else
info "dns" "$h no resolve (optional guess) — skip"
fi
return 0
fi
if ping_host "$h"; then
ping_ok=1
info "ping" "$h ICMP ok"
else
info "ping" "$h ICMP no reply (not decisive)"
fi
# port scan (unique ports)
local _seen_ports=" "
for p in ${ports//,/ }; do
[ -n "$p" ] || continue
case "$_seen_ports" in *" $p "*) continue ;; esac
_seen_ports="$_seen_ports$p "
if tcp_open "$h" "$p"; then
open_ports+=("$p")
info "port" "$h:$p open (TCP)"
else
info "port" "$h:$p closed/filtered (TCP)"
fi
done
# Protocol verification on open ports (and always try 443/80 for https/http expect)
local try_ports=("${open_ports[@]}")
if [ ${#try_ports[@]} -eq 0 ]; then
# still try expected protocol ports even if scan said closed (scan false negatives)
case "$proto" in
https) try_ports=(443) ;;
http) try_ports=(80) ;;
ssh) try_ports=(22) ;;
*) try_ports=(443 80) ;;
esac
fi
server_hdr=""
ver=""
for p in "${try_ports[@]}"; do
case "$p" in
443|8443)
if probe_detail=$(probe_https "$h" "$p"); then
any_proto_ok=1
ok "https" "$h:$p up · $probe_detail"
server_hdr=$(printf '%s' "$probe_detail" | sed -n 's/.*server=\([^ ]*\).*/\1/p')
# cert
local cend
cend=$(printf '%s' "$probe_detail" | sed -n 's/.*cert_end=\([^ ]*\).*/\1/p')
cert_expiry_check "$h" "$cend" || true
local cfg
cfg=$(printf '%s' "$probe_detail" | sed -n 's/.*cfg=\([^ ]*\).*/\1/p')
if [ -n "$cfg" ]; then
info "version" "$h config/api hint: $cfg"
ver="$cfg"
fi
else
info "https" "$h:$p no HTTP response · ${probe_detail:-}"
fi
;;
80|8080)
if probe_detail=$(probe_http "$h" "$p"); then
any_proto_ok=1
ok "http" "$h:$p up · $probe_detail"
else
info "http" "$h:$p no HTTP response"
fi
;;
22)
if probe_detail=$(probe_ssh_banner "$h" "$p"); then
any_proto_ok=1
ok "ssh" "$h:$p banner · $probe_detail"
ver=$(printf '%s' "$probe_detail" | sed 's/banner=//')
else
info "ssh" "$h:$p no SSH banner"
fi
;;
*)
if tcp_open "$h" "$p"; then
info "tcp" "$h:$p open · protocol unknown"
fi
;;
esac
done
# Version summary
if [ -n "$server_hdr" ]; then
info "server-header" "$h · $server_hdr"
pkg=$(guess_package "$server_hdr")
ver_soft=$(extract_version "$server_hdr")
if [ -n "$pkg" ] && [ -n "$ver_soft" ]; then
info "version" "$h software $pkg $ver_soft (from Server header)"
check_cves "$pkg" "$ver_soft" "$h" || true
elif [ -n "$server_hdr" ]; then
warn "version" "$h Server header present but version unknown · $server_hdr"
fi
elif [ -n "$ver" ]; then
info "version" "$h · $ver"
else
if [ "$any_proto_ok" = "1" ]; then
warn "version" "$h reachable but software version unknown"
fi
fi
# Expected service must answer protocol (not just ping/port)
# Note: ${array[*]:-x} is NOT valid default syntax (bash treats : as slice).
local ports_txt="${open_ports[*]}"
ports_txt=${ports_txt:-none}
if [ "$expect" = "1" ]; then
if [ "$any_proto_ok" = "1" ]; then
ok "reachability" "$h catalogued service OK (protocol confirmed)"
else
err "reachability" "$h catalogued but not reachable via ${proto}/protocol" \
"dns=$dns ping=$ping_ok open_ports=${ports_txt} (ICMP/TCP alone not enough; protocol failed)"
fi
else
if [ "$any_proto_ok" = "1" ]; then
info "reachability" "$h optional host responds"
else
info "reachability" "$h optional · no protocol response (ok)"
fi
fi
}
# --- main ---
# Scope (remote only):
# ./taler-monitoring.sh surface → full ecosystem catalog
# ./taler-monitoring.sh -d hacktivism.ch surface → only that domain
if [ "${SURFACE_SCOPE:-}" = "ecosystem" ]; then
load_catalog
expand_ecosystem_scope
elif [ "${SURFACE_SCOPE:-}" = "domain" ] || [ "${TALER_DOMAIN_FROM_CLI:-0}" = "1" ]; then
# domain mode: do not load whole ecosystem first
CATALOG="${SURFACE_CATALOG:-$ROOT/surface-catalog.conf}"
expand_domain_scope "${TALER_DOMAIN:-hacktivism.ch}"
elif [ "${SURFACE_SCOPE:-auto}" = "auto" ]; then
# no -d → ecosystem; with -d → domain (TALER_DOMAIN_FROM_CLI)
if [ "${TALER_DOMAIN_FROM_CLI:-0}" = "1" ]; then
CATALOG="${SURFACE_CATALOG:-$ROOT/surface-catalog.conf}"
expand_domain_scope "$TALER_DOMAIN"
else
load_catalog
expand_ecosystem_scope
fi
else
load_catalog
expand_ecosystem_scope
fi
info "inventory" "${#ORDERED_HOSTS[@]} hosts to probe (remote only · no SSH · no local podman on targets)"
info "flags" "SURFACE_CVE=${SURFACE_CVE:-1} PORT_TIMEOUT=${PORT_TIMEOUT:-2} HTTP_TIMEOUT=${HTTP_TIMEOUT:-12}"
for h in "${ORDERED_HOSTS[@]}"; do
[ -n "$h" ] || continue
scan_host "$h" || true
done
# mon_disk on the *runner* only (optional) — not remote servers' disks
if [ "${SURFACE_CHECK_RUNNER_DISK:-0}" = "1" ]; then
set_group disk
if declare -F mon_disk_check_host >/dev/null 2>&1; then
mon_disk_check_host "runner" || true
else
warn "disk" "mon_disk_check_host not available"
fi
fi
# summary returns non-zero when FAIL_N>0; do not trip set -e mid-script
summary || true
if [ "${FAIL_N:-0}" -eq 0 ]; then
exit 0
fi
exit 1

1595
check_urls.sh Executable file
View file

File diff suppressed because it is too large Load diff

563
check_versions.sh Executable file
View file

@ -0,0 +1,563 @@
#!/usr/bin/env bash
# Area versions-### — Taler packages vs deb.taler.net (trixie) + repo availability.
#
# Checks:
# 1) deb.taler.net apt endpoints reachable (InRelease / Packages / sample .deb)
# 2) containers can reach deb.taler.net (install path from inside)
# 3) installed taler*/libeufin*/libtaler*/libdonau* versions vs suite index
#
# Env:
# TALER_APT_SUITE=trixie
# TALER_APT_BASE=https://deb.taler.net/apt/debian
# TALER_APT_INDEX=…/dists/trixie/main/binary-amd64/Packages
# TALER_APT_TESTING_INDEX=…/dists/trixie-testing/…
# TALER_PKG_BEHIND=warn|error (default: core packages ERROR if behind, else warn)
# SKIP_SSH=1 skip container install checks
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
set_area versions
set_group outside
SUITE="${TALER_APT_SUITE:-trixie}"
APT_BASE="${TALER_APT_BASE:-https://deb.taler.net/apt/debian}"
APT_BASE="${APT_BASE%/}"
INDEX="${TALER_APT_INDEX:-${APT_BASE}/dists/${SUITE}/main/binary-amd64/Packages}"
INRELEASE="${TALER_APT_INRELEASE:-${APT_BASE}/dists/${SUITE}/InRelease}"
TESTING_INDEX="${TALER_APT_TESTING_INDEX:-${APT_BASE}/dists/${SUITE}-testing/main/binary-amd64/Packages}"
BEHIND_MODE="${TALER_PKG_BEHIND:-warn}"
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
http_get() {
# usage: http_get URL [outfile] → prints http_code; body to outfile or /dev/null
local url="$1" out="${2:-/dev/null}"
curl -sS --max-time 30 -L -o "$out" -w '%{http_code}' "$url" 2>/dev/null || echo 000
}
# ---------------------------------------------------------------------------
# 1) OUTSIDE — deb.taler.net install source (runner / laptop, no SSH)
# ---------------------------------------------------------------------------
section "versions · outside · deb.taler.net (${SUITE})"
# DNS
HOST_APT="${APT_BASE#https://}"; HOST_APT="${HOST_APT#http://}"; HOST_APT="${HOST_APT%%/*}"
if getent hosts "$HOST_APT" >/dev/null 2>&1 \
|| python3 -c "import socket; socket.getaddrinfo('${HOST_APT}', 443)" >/dev/null 2>&1; then
ip=$(python3 -c "import socket; print(socket.getaddrinfo('${HOST_APT}',443)[0][4][0])" 2>/dev/null || true)
ok "outside DNS ${HOST_APT}" "${ip:-resolved}"
else
fail "outside DNS ${HOST_APT}" "unresolvable — cannot use as apt source"
fi
# HTTPS portal / apt root
code=$(http_get "https://${HOST_APT}/")
[ "$code" = "200" ] && ok "outside https://${HOST_APT}/" "HTTP $code" \
|| warn "outside https://${HOST_APT}/" "HTTP $code"
code=$(http_get "${APT_BASE}/")
case "$code" in
200|301|302) ok "outside apt base" "${APT_BASE}/ HTTP $code" ;;
*) fail "outside apt base" "${APT_BASE}/ HTTP $code" ;;
esac
# Suite metadata (what apt update needs)
code=$(http_get "$INRELEASE" "$tmp/InRelease")
if [ "$code" = "200" ] && [ -s "$tmp/InRelease" ]; then
ok "outside InRelease" "${SUITE} HTTP 200 · $(wc -c <"$tmp/InRelease" | tr -d ' ') bytes"
else
fail "outside InRelease" "HTTP $code — $INRELEASE (apt update will fail)"
fi
RELEASE_URL="${APT_BASE}/dists/${SUITE}/Release"
code=$(http_get "$RELEASE_URL" "$tmp/Release")
[ "$code" = "200" ] && ok "outside Release" "${SUITE} HTTP 200" \
|| warn "outside Release" "HTTP $code — $RELEASE_URL"
# Packages (plain) — required for version compare
code=$(http_get "$INDEX" "$tmp/Packages")
if [ "$code" = "200" ] && [ -s "$tmp/Packages" ] && grep -q '^Package: ' "$tmp/Packages"; then
n_pkg=$(grep -c '^Package: ' "$tmp/Packages" || true)
ok "outside Packages" "${SUITE} HTTP 200 · ${n_pkg} packages"
else
fail "outside Packages" "HTTP $code — $INDEX"
# still try more probes, but cannot compare versions without index
fi
# Packages.gz (apt often prefers this)
PKGZ_URL="${APT_BASE}/dists/${SUITE}/main/binary-amd64/Packages.gz"
code=$(http_get "$PKGZ_URL" "$tmp/Packages.gz")
if [ "$code" = "200" ] && [ -s "$tmp/Packages.gz" ]; then
ok "outside Packages.gz" "${SUITE} HTTP 200 · $(wc -c <"$tmp/Packages.gz" | tr -d ' ') bytes"
else
warn "outside Packages.gz" "HTTP $code — apt may still use plain Packages"
fi
# Sample pool .deb downloadable (proves packages are installable, not just listed)
if [ -s "$tmp/Packages" ]; then
DEB_PATH=$(awk '
/^Package: taler-exchange$/ { p=1 }
p && /^Filename: / { sub(/^Filename: /,""); print; exit }
p && /^$/ { p=0 }
' "$tmp/Packages")
[ -z "$DEB_PATH" ] && DEB_PATH=$(awk '
/^Package: taler-merchant$/ { p=1 }
p && /^Filename: / { sub(/^Filename: /,""); print; exit }
p && /^$/ { p=0 }
' "$tmp/Packages")
if [ -n "$DEB_PATH" ]; then
DEB_URL="${APT_BASE}/${DEB_PATH}"
code=$(curl -sS --max-time 30 -o /dev/null -w '%{http_code}' -r 0-128 "$DEB_URL" 2>/dev/null || echo 000)
case "$code" in
200|206)
ok "outside pool .deb" "$(basename "$DEB_PATH") HTTP $code"
;;
*)
fail "outside pool .deb" "HTTP $code — $DEB_URL (index ok but debs not fetchable)"
;;
esac
else
warn "outside pool .deb" "no Filename for taler-exchange/merchant in index"
fi
for pkg in taler-exchange taler-merchant libeufin-bank; do
ver=$(awk -v p="$pkg" '
$0=="Package: "p { hit=1; next }
hit && /^Version: / { sub(/^Version: /,""); print; exit }
hit && /^$/ { hit=0 }
' "$tmp/Packages")
if [ -n "$ver" ]; then
ok "outside suite offers ${pkg}" "$ver"
else
fail "outside suite offers ${pkg}" "missing from ${SUITE} Packages"
fi
done
fi
HAVE_TESTING=0
code=$(http_get "$TESTING_INDEX" "$tmp/Packages-testing")
if [ "$code" = "200" ] && [ -s "$tmp/Packages-testing" ] && grep -q '^Package: ' "$tmp/Packages-testing"; then
HAVE_TESTING=1
ok "outside ${SUITE}-testing Packages" "HTTP 200 (optional compare)"
else
info "outside ${SUITE}-testing Packages" "HTTP ${code:-000} (optional)"
fi
# TLS: certificate verify (curl default) already used above; explicit openssl probe if available
if command -v openssl >/dev/null 2>&1; then
if echo | openssl s_client -servername "$HOST_APT" -connect "${HOST_APT}:443" 2>/dev/null \
| grep -q 'Verify return code: 0'; then
ok "outside TLS ${HOST_APT}" "verify ok"
else
# curl succeeded with default CA — soft warn only
warn "outside TLS ${HOST_APT}" "openssl verify not clean (curl may still work)"
fi
fi
# Need Packages index for later compare
if [ ! -s "$tmp/Packages" ] || ! grep -q '^Package: ' "$tmp/Packages"; then
fail "outside Packages usable" "cannot continue version compare without suite index"
summary
exit 1
fi
# ---------------------------------------------------------------------------
# 2) INSIDE — containers can reach deb.taler.net (pasta / install path)
# host-podman: INSIDE_PODMAN=1 on GOA host · ssh: KOOPA_SSH / INSIDE_SSH from laptop
# ---------------------------------------------------------------------------
_VERS_LOCAL=0
_VERS_SSH_HOST=""
_VERS_BANK="taler-hacktivism-bank"
_VERS_EX="taler-hacktivism-exchange-ansible"
_VERS_MER="taler-hacktivism"
if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; then
_VERS_LOCAL=1
elif [ "${INSIDE_PROFILE:-}" = "stage-lfp" ] \
|| [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ] \
|| [ -n "${INSIDE_SSH:-}" ]; then
_VERS_SSH_HOST="${INSIDE_SSH:-francpaysan-stage-user}"
_VERS_BANK="${INSIDE_BANK_CTR:-stage-lfp-bank}"
_VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}"
_VERS_MER="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}"
elif [ "${LOCAL_STACK:-0}" = "1" ] && [ "${SKIP_SSH:-0}" != "1" ]; then
# laptop → koopa via SSH (not host-podman)
_VERS_SSH_HOST="${KOOPA_SSH}"
fi
if [ "$_VERS_LOCAL" = "1" ]; then
VERS_ACCESS=host-podman
set_group host
else
VERS_ACCESS=ssh
set_group ssh
fi
section "versions · inside · containers → deb.taler.net (access=${VERS_ACCESS})"
info "flags" "VERS_ACCESS=${VERS_ACCESS} INSIDE_PODMAN=${INSIDE_PODMAN:-0} INSIDE_MODE=${INSIDE_MODE:-} LOCAL_STACK=${LOCAL_STACK:-0} SKIP_SSH=${SKIP_SSH:-0} KOOPA_SSH=${KOOPA_SSH:-} INSIDE_SSH=${INSIDE_SSH:-}"
if [ "$_VERS_LOCAL" != "1" ] && [ -z "$_VERS_SSH_HOST" ]; then
info "access" "no host-podman and no SSH target — outside-only package index"
info "outside-only" "deb.taler.net public checks completed above"
summary
exit 0
fi
if [ "$_VERS_LOCAL" = "1" ]; then
if ! command -v podman >/dev/null 2>&1; then
err "host" "INSIDE_PODMAN/host-podman but podman missing"
info "outside-only" "deb.taler.net public checks completed above"
summary
exit 1
fi
ok "host→container" "podman exec on this host (VERS_ACCESS=host-podman · no SSH)"
elif [ "$_VERS_SSH_HOST" = "${KOOPA_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "koopa" ]; then
if ! koopa_ssh_ok; then
err "ssh" "cannot reach ${KOOPA_SSH} — from laptop set KOOPA_SSH=; on host set INSIDE_PODMAN=1"
info "outside-only" "deb.taler.net public checks completed above"
summary
exit 1
fi
ok "ssh ${KOOPA_SSH}" "remote then podman (VERS_ACCESS=ssh)"
else
if ! mon_ssh_ok "$_VERS_SSH_HOST"; then
err "ssh" "cannot reach ${_VERS_SSH_HOST} (stagepaysan) — skip container package compare"
info "outside-only" "deb.taler.net public checks completed above"
summary
exit 1
fi
ok "ssh ${_VERS_SSH_HOST}" "stagepaysan (VERS_ACCESS=ssh)"
fi
INRELEASE_URL="${APT_BASE}/dists/${SUITE}/InRelease"
# Write remote script to a file (avoids bash 3.2 parse bugs with case/;; inside $(…)<<heredoc).
{
printf 'INRELEASE_URL=%q\n' "$INRELEASE_URL"
printf 'WANT_BANK=%q; WANT_EX=%q; WANT_MER=%q\n' "$_VERS_BANK" "$_VERS_EX" "$_VERS_MER"
cat <<'REMOTE'
set +e
resolve_ctr() {
local want="$1" c
c=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -Fx "$want" | head -1)
if [ -n "$c" ]; then echo "$c"; return; fi
case "$want" in
*exchange*) podman ps --format '{{.Names}}' 2>/dev/null | grep -i exchange | head -1 ;;
*bank*) podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*bank|hacktivism-bank|taler-bank|lfp-bank' | head -1 ;;
*)
c=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -E '^taler-hacktivism$|^stage-lfp-merchant$' | head -1)
if [ -n "$c" ]; then echo "$c"; return; fi
podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'merchant|hacktivism' | grep -viE 'bank|exchange' | head -1
;;
esac
}
for want in "$WANT_EX" "$WANT_BANK" "$WANT_MER"; do
c=$(resolve_ctr "$want")
[ -n "$c" ] || continue
role=other
case "$c" in
*exchange*) role=exchange ;;
*bank*) role=bank ;;
*) role=merchant ;;
esac
code=$(podman exec "$c" curl -sS -m 12 -o /dev/null -w '%{http_code}' "$INRELEASE_URL" 2>/dev/null || echo 000)
echo "R|${role}|${c}|${code}"
if podman exec "$c" bash -lc 'grep -Rqs deb.taler.net /etc/apt/sources.list /etc/apt/sources.list.d 2>/dev/null'; then
echo "S|${role}|${c}|yes"
else
echo "S|${role}|${c}|no"
fi
podman exec "$c" dpkg-query -W -f '${Package}\t${Version}\n' 2>/dev/null \
| awk -v role="$role" -v ctr="$c" '
$1 ~ /^(taler-|libeufin-|libtaler|libdonau)/ {
printf "P|%s|%s|%s|%s\n", role, ctr, $1, $2
}'
done
REMOTE
} >"$tmp/remote-versions.sh"
if [ "$_VERS_LOCAL" = "1" ]; then
REMOTE_OUT=$(bash "$tmp/remote-versions.sh" 2>/dev/null || true)
elif [ "$_VERS_SSH_HOST" = "${KOOPA_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "koopa" ]; then
REMOTE_OUT=$(koopa_ssh_bash 60 <"$tmp/remote-versions.sh" || true)
else
REMOTE_OUT=$(mon_ssh_bash "$_VERS_SSH_HOST" 60 <"$tmp/remote-versions.sh" || true)
fi
printf '%s\n' "$REMOTE_OUT" | grep -E '^R\|' >"$tmp/reach.tsv" || true
printf '%s\n' "$REMOTE_OUT" | grep -E '^S\|' >"$tmp/sources.tsv" || true
printf '%s\n' "$REMOTE_OUT" | grep -E '^P\|' >"$tmp/installed.tsv" || true
if [ -s "$tmp/reach.tsv" ]; then
while IFS='|' read -r _ role ctr code; do
if [ "$code" = "200" ]; then
ok "container ${role} → deb.taler.net" "${ctr} InRelease HTTP $code"
else
fail "container ${role} → deb.taler.net" "${ctr} InRelease HTTP $code — apt install/update will fail"
fi
done <"$tmp/reach.tsv"
else
warn "container → deb.taler.net" "no reachability rows"
fi
if [ -s "$tmp/sources.tsv" ]; then
while IFS='|' read -r _ role ctr has; do
if [ "$has" = "yes" ]; then
ok "container ${role} apt source" "${ctr} lists deb.taler.net"
else
warn "container ${role} apt source" "${ctr} no deb.taler.net in sources.list*"
fi
done <"$tmp/sources.tsv"
fi
if [ ! -s "$tmp/installed.tsv" ]; then
fail "installed packages" "none found in taler containers"
summary
exit 1
fi
n_inst=$(wc -l <"$tmp/installed.tsv" | tr -d ' ')
ok "collected installed packages" "${n_inst} rows"
# ---------------------------------------------------------------------------
# 3) compare installed vs trixie (and note testing)
# ---------------------------------------------------------------------------
set_group compare
section "versions · compare installed vs ${SUITE}"
export BEHIND_MODE HAVE_TESTING
export TALER_APT_SUITE="$SUITE"
CORE_PKGS="taler-exchange taler-exchange-database taler-merchant taler-merchant-webui libeufin-bank libeufin-common libtalerexchange libtalermerchant taler-terms-generator"
export CORE_PKGS
python3 - "$tmp/Packages" "$tmp/Packages-testing" "$tmp/installed.tsv" <<'PY' >"$tmp/py.out"
import sys, os
from collections import defaultdict
main_idx, test_idx, inst_path = sys.argv[1:4]
have_testing = os.environ.get("HAVE_TESTING", "0") == "1"
core = set(os.environ.get("CORE_PKGS", "").split())
suite = os.environ.get("TALER_APT_SUITE", "trixie")
def _ord_ch(c):
if c == "~":
return -1
if c.isdigit():
return 0
if c.isalpha():
return ord(c)
return ord(c) + 256
def _split_ver(v):
epoch = 0
if ":" in v:
e, v = v.split(":", 1)
try:
epoch = int(e)
except ValueError:
epoch = 0
if "-" in v:
upstream, deb = v.rsplit("-", 1)
else:
upstream, deb = v, ""
return epoch, upstream, deb
def _cmp_part(a, b):
ia = ib = 0
while ia < len(a) or ib < len(b):
while True:
ca = a[ia] if ia < len(a) and not a[ia].isdigit() else ""
cb = b[ib] if ib < len(b) and not b[ib].isdigit() else ""
if ca == "" and cb == "":
break
if ca == "" and cb:
return -1 if _ord_ch(cb) > 0 else (1 if cb == "~" else -1)
if cb == "" and ca:
return 1 if _ord_ch(ca) > 0 else (-1 if ca == "~" else 1)
oa, ob = _ord_ch(ca), _ord_ch(cb)
ia += 1
ib += 1
if oa != ob:
return (oa > ob) - (oa < ob)
sa = sb = ""
while ia < len(a) and a[ia].isdigit():
sa += a[ia]; ia += 1
while ib < len(b) and b[ib].isdigit():
sb += b[ib]; ib += 1
na = int(sa) if sa else 0
nb = int(sb) if sb else 0
if na != nb:
return (na > nb) - (na < nb)
return 0
def deb_cmp(a, b):
if a == b:
return 0
ea, ua, da = _split_ver(a)
eb, ub, db = _split_ver(b)
if ea != eb:
return (ea > eb) - (ea < eb)
c = _cmp_part(ua, ub)
if c:
return c
return _cmp_part(da, db)
def parse_packages(path):
pkgs = {}
if not path or not os.path.isfile(path) or os.path.getsize(path) == 0:
return pkgs
cur = ver = None
with open(path, encoding="utf-8", errors="replace") as f:
for line in f:
line = line.rstrip("\n")
if line.startswith("Package: "):
if cur and ver and (cur not in pkgs or deb_cmp(ver, pkgs[cur]) > 0):
pkgs[cur] = ver
cur = line[9:].strip()
ver = None
elif line.startswith("Version: ") and cur:
ver = line[9:].strip()
elif line == "" and cur:
if ver and (cur not in pkgs or deb_cmp(ver, pkgs[cur]) > 0):
pkgs[cur] = ver
cur = ver = None
if cur and ver and (cur not in pkgs or deb_cmp(ver, pkgs[cur]) > 0):
pkgs[cur] = ver
return pkgs
main = parse_packages(main_idx)
testing = parse_packages(test_idx) if have_testing else {}
by_pkg = defaultdict(list)
with open(inst_path, encoding="utf-8") as f:
for line in f:
line = line.strip()
if not line.startswith("P|"):
continue
parts = line.split("|")
if len(parts) < 5:
continue
_, role, ctr, pkg, ver = parts[:5]
by_pkg[pkg].append((role, ctr, ver))
for pkg in sorted(by_pkg.keys()):
versions = sorted({v for _, _, v in by_pkg[pkg]}, key=lambda v: v)
# pick "highest" via deb_cmp
inst = versions[0]
for v in versions[1:]:
if deb_cmp(v, inst) > 0:
inst = v
roles = ",".join(sorted({r for r, _, _ in by_pkg[pkg]}))
multi = len(set(versions)) > 1
trixie = main.get(pkg)
testv = testing.get(pkg)
status = "ok"
detail = ""
if multi:
status = "warn"
detail = "multiple installed: " + ", ".join(sorted(set(versions)))
if trixie is None:
if status == "ok":
status = "info"
detail = (detail + "; " if detail else "") + f"not in {suite} main index"
else:
c = deb_cmp(inst, trixie)
if c == 0:
detail = (detail + "; " if detail else "") + f"= {suite} {trixie}"
elif c < 0:
status = "behind"
detail = f"installed {inst} < {suite} {trixie}"
else:
status = "ahead"
detail = f"installed {inst} > {suite} {trixie}"
if testv and deb_cmp(inst, testv) == 0:
detail += f" (={suite}-testing)"
elif testv:
detail += f" (testing has {testv})"
core_flag = "1" if pkg in core else "0"
print(f"{status}|{pkg}|{inst}|{trixie or '-'}|{roles}|{core_flag}|{detail}")
PY
ok_n=0; ahead_n=0; behind_n=0
while IFS='|' read -r status pkg inst trixie roles core detail; do
[ -n "${status:-}" ] || continue
label="pkg ${pkg} (${roles})"
case "$status" in
ok)
ok "$label" "${inst}"
ok_n=$((ok_n + 1))
;;
ahead)
info "$label" "$detail"
ahead_n=$((ahead_n + 1))
;;
behind)
behind_n=$((behind_n + 1))
if [ "$core" = "1" ] || [ "$BEHIND_MODE" = "error" ]; then
fail "$label" "$detail"
else
warn "$label" "$detail"
fi
;;
warn)
warn "$label" "$detail"
;;
info)
info "$label" "installed ${inst}${detail:+ — $detail}"
;;
*)
info "$label" "$status $detail"
;;
esac
done <"$tmp/py.out"
# Core packages must exist somewhere
for need in taler-exchange libeufin-bank taler-merchant; do
if grep -qE "^P\|[^|]+\|[^|]+\|${need}\|" "$tmp/installed.tsv"; then
ok "core installed ${need}"
else
fail "core installed ${need}" "not present in any taler container"
fi
done
info "suite" "deb.taler.net ${SUITE} (testing_index=${HAVE_TESTING})"
info "tally" "match=${ok_n} ahead=${ahead_n} behind=${behind_n}"
# Disk free on the machine(s) we just probed (host-podman or SSH target)
set_group disk
section "versions · disk free space"
if [ "${_VERS_LOCAL:-0}" = "1" ]; then
mon_disk_check_host "host" || true
while read -r _vc; do
[ -n "$_vc" ] || continue
mon_disk_check_podman "$_vc" || true
done < <(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'hacktivism|taler-|stage-lfp|lfp-' || true)
elif [ -n "${_VERS_SSH_HOST:-}" ]; then
_vdisk=$(mon_ssh_bash "$_VERS_SSH_HOST" 20 <<'DISK' || true
set +e
echo "###HOST###"
df -Pk / /var /home /tmp 2>/dev/null || df -Pk
echo "###CTRS###"
for c in $(podman ps --format '{{.Names}}' 2>/dev/null); do
echo "###CTR $c###"
podman exec "$c" df -Pk / /var /tmp 2>/dev/null || true
done
DISK
)
mon_disk_check_remote_text "ssh:${_VERS_SSH_HOST}" \
"$(printf '%s\n' "$_vdisk" | sed -n '/^###HOST###$/,/^###CTRS###$/p' | sed '1d;$d')" || true
_ctr=""; _buf=""
while IFS= read -r _line || [ -n "$_line" ]; do
case "$_line" in
'###CTR '*)
[ -n "$_ctr" ] && mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true
_ctr=${_line####CTR }; _ctr=${_ctr%###}; _buf=""
;;
'###CTRS###'|'###HOST###') ;;
*) [ -n "$_ctr" ] && _buf="${_buf}${_line}"$'\n' ;;
esac
done <<<"$_vdisk"
[ -n "$_ctr" ] && mon_disk_check_remote_text "ctr:${_ctr}" "$_buf" || true
fi
summary

46
domains.conf Normal file
View file

@ -0,0 +1,46 @@
# taler-monitoring domain profiles
#
# When you add a stack, declare the three public endpoints here:
# name bank exchange merchant-backend currency local landing [canonical]
#
# - name: what -d / TALER_DOMAIN matches (aliases allowed as extra lines)
# - bank / exchange / merchant: hostname or full https:// URL
# - currency: GOA | KUDOS | CHF | … (empty = report only)
# - local: 1 = koopa SSH/inside/e2e stack; 0 = public-only
# - landing: 1 = check /intro landings + assets; 0 = skip (no GOA-style landings)
# - canonical: optional TALER_DOMAIN label after alias match
#
# CLI overrides after load: --bank --exchange --merchant --currency
# Alternate file: TALER_DOMAINS_CONF=/path/to/domains.conf
# Local GOA (koopa) — landings matter
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1
# Public demo (KUDOS) — has intro-style pages; keep soft landing checks
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 demo.taler.net
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1
# Public test stack (TESTKUDOS) — no GOA-style landings
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0
# TOPS CHF — multi-tenant merchant my.taler-ops.ch; no public GOA landings
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 taler-ops.ch
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0 stage.taler-ops.ch
# Franc Paysan (Infomaniak) — merchant host is monnaie.* (not taler./backend.)
# Prod: no monetary launch yet; stage uses TESTPAYSAN (public HTTPS landings + shops)
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
exchange.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
# Stage TESTPAYSAN — landings on / + farmer shops under stage.monnaie…/shops/
# name bank exchange merchant currency local landing [canonical]
stage.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1
stage.bank.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
stage.exchange.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
stage.monnaie.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch

7
harness-live.txt Normal file
View file

@ -0,0 +1,7 @@
# Optional: taler-harness against *our* exchange (needs a harness version
# that accepts current /config JSON). Older installs often FAIL codec checks
# even when the exchange is fine — trust `./taler-monitoring.sh urls` first.
#
# id<TAB>command
lint-exchange-goa taler-harness deployment lint-exchange-url https://exchange.hacktivism.ch/

41
harness-tests.txt Normal file
View file

@ -0,0 +1,41 @@
# Real-life–relevant taler-harness integration tests
# (from: taler-harness list-integrationtests)
#
# Criteria: everyday user/merchant/bank paths — withdraw, pay, deposit, refund,
# libeufin bank, templates/paywall. Excludes KYC edge cases, experimental,
# timetravel, perf, backup, mailbox, most fault-injection.
#
# These spin up a local TESTKUDOS stack (same client code paths as production).
# For tests against *public* demos, see harness-live.txt / `./taler-monitoring.sh live`.
# --- Core path: get money + spend it (the GOA story) ---
simple-payment
withdrawal-bank-integrated
payment
# --- Withdraw variants users actually hit ---
withdrawal-manual
withdrawal-external
withdrawal-idempotent
# --- Pay variants shops use ---
payment-template
paywall-flow
payment-idempotency
payment-zero
payment-abort
otp
# --- After pay: coins / merchant ops ---
deposit
refund
refund-auto
wallet-refresh
# --- Bank stack we run (libeufin / regional) ---
libeufin-bank
bank-api
# --- Client hygiene ---
wallet-config
term-of-service-format

183
host-agent/README.md Normal file
View file

@ -0,0 +1,183 @@
# host-agent — reporting generation (GOA + FP + any stack)
**Global pipeline** lives in **`run-host-report.sh`** (all stacks share it):
| Behaviour | Default |
|-----------|---------|
| Suite refresh | `update-suite.sh` → `origin/main` |
| Wall clock | `RUN_TIMEOUT=600` |
| Log | line-buffered `run-*.log` (continuous write) |
| HTML | always (first run / empty tree too) + commit URL |
| Timeout notice | end of log + top jump on `/monitoring_err/` |
Thin wrappers only set stack defaults:
| Wrapper | Stack | Typical user |
|---------|-------|----------------|
| `run-hacktivism-monitoring.sh` | GOA `hacktivism.ch` | `hernani` @ koopa |
| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` |
| `run-fp-prod-monitoring.sh` | FP prod | `francpaysan` |
| `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh koopa-external** |
| `run-aptdeploy-monitoring.sh` | **aptdeploy** → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h timer, taler only) |
| `run-surface-monitoring.sh` | **surface** → `/taler-monitoring-surface*` | `hernani` @ koopa (hourly, taler only) |
### apt-src deploy tests (hacktivism `/monitoring`)
| Container | Mode |
|-----------|------|
| `koopa-taler-deploy-test-apt-src-trixie` | **fresh**: recreate when repo package fingerprint changes |
| `koopa-taler-deploy-test-apt-src-trixie-testing` | **fresh** for `trixie-testing` |
| `koopa-taler-deploy-test-apt-src-trixie-upgrade` | **upgrade**: initial install, then mytops-style `apt install` + `upgrade` |
| `koopa-taler-deploy-test-apt-src-trixie-testing-upgrade` | **upgrade** for testing suite |
```bash
# from laptop / clementine → triggers on koopa-external
./host-agent/run-aptdeploy.sh
./host-agent/run-aptdeploy.sh upgrade
./host-agent/run-aptdeploy.sh fresh
./host-agent/run-aptdeploy.sh check-only
# on koopa
./taler-monitoring.sh -d hacktivism.ch aptdeploy
```
Checks: package table, `httpd --version`, libtalerutil/`ldd` on failure, `systemctl` for
`taler-merchant.target`, basic local `/config` probe. No nginx/HTTPS in containers.
### surface + aptdeploy HTML (taler.hacktivism.ch only)
| URL | Role | Timer |
|-----|------|-------|
| `…/taler-monitoring-surface/` | ecosystem surface scan | 1h |
| `…/taler-monitoring-surface_err/` | errors + jump list | ″ |
| `…/taler-monitoring-aptdeploy/` | apt-src container deploy tests | 4h |
| `…/taler-monitoring-aptdeploy_err/` | errors + jump list | ″ |
Not on bank/exchange. Staging:
`~/monitoring-sites-staging/taler.hacktivism.ch/taler-monitoring-{surface,aptdeploy}*`.
```bash
systemctl --user enable --now taler-monitoring-surface.timer
systemctl --user enable --now taler-monitoring-aptdeploy.timer
systemctl --user start taler-monitoring-surface.service
systemctl --user start taler-monitoring-aptdeploy.service
# Caddy: handles only in taler.hacktivism.ch block (root)
```
Env examples: **`env/*.env.example`**. FP docs: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
## GOA (koopa) — host-podman
Runs **as user `hernani`** (systemd user, linger). **Primary** mode:
- **`INSIDE_PODMAN=1`** / **`INSIDE_MODE=local-podman`**: access mode **`host-podman`** —
`podman exec` on this host into
`taler-hacktivism-bank` / `taler-hacktivism-exchange-ansible` / `taler-hacktivism`
(IDs `inside.host-*`, not `inside.ssh-*`).
From a **laptop**, leave `INSIDE_PODMAN` off → access mode **`ssh`** via `KOOPA_SSH`
(`inside.ssh-*`).
- Public **`urls` + `versions`** for the stack (`-d hacktivism.ch`).
- Writes console HTML for all three sites under
`~/monitoring-sites-staging/{bank,exchange,taler}.hacktivism.ch/`.
Caddy (host root) can serve that tree at `/monitoring` — you configure Caddy.
## Latest software + config isolation
**Source of truth is always Forgejo** (`git.hacktivism.ch`), not a laptop rsync copy.
Every host-agent / aptdeploy run:
1. **`update-suite.sh`**: `git fetch` + **`SUITE_UPDATE_MODE=reset`** (default) → `origin/main`
into `SUITE_DIR` (prefer `~/src/koopa-admin-log`).
2. **Re-exec** host-agent scripts from that tree so the running code matches `COMMIT_URL`.
3. Local overrides only in **`~/.config/taler-monitoring/env`** — never overwritten by git.
4. HTML footer = exact commit:
`https://git.hacktivism.ch/hernani/taler-monitoring/src/commit/<full-sha>`
5. **`RUN_TIMEOUT=600`**, line-buffered logs, HTML always written.
If fetch fails → previous tree + WARN.
**To ship features (aptdeploy, surface, …): push to `origin/main`.** Local-only edits vanish on the next reset.
If pull fails (no network / auth), the previous tree is used and a WARN is logged.
## Install (on koopa as hernani)
```bash
# once: local config (optional)
mkdir -p ~/.config/taler-monitoring
cp ~/src/taler-monitoring/host-agent/env.example \
~/.config/taler-monitoring/env
# laptop:
./host-agent/install-host-agent.sh --remote koopa-external
# or on koopa:
~/src/taler-monitoring/host-agent/install-host-agent.sh
```
Units:
| Unit | Role |
|------|------|
| `taler-monitoring-hacktivism.path` | fires on software stamp + landing/caddy config changes |
| `taler-monitoring-hacktivism.timer` | every **4h** fallback |
| `taler-monitoring-hacktivism.service` | oneshot → `/monitoring*` (urls inside versions) |
| `taler-monitoring-surface.timer` | every **1h** |
| `taler-monitoring-surface.service` | → `/taler-monitoring-surface*` (taler only) |
| `taler-monitoring-aptdeploy.timer` | every **4h** |
| `taler-monitoring-aptdeploy.service` | → `/taler-monitoring-aptdeploy*` (taler only) |
## After package / image / landing upgrades
```bash
~/src/taler-monitoring/host-agent/touch-software-stamp.sh
# → path unit starts the service
```
## Manual
```bash
systemctl --user start taler-monitoring-hacktivism.service
systemctl --user status taler-monitoring-hacktivism.service --no-pager
journalctl --user -u taler-monitoring-hacktivism.service -n 50 --no-pager
ls ~/monitoring-sites-staging/bank.hacktivism.ch/monitoring_err/
```
## Relation to firecuda outside-only timer
| | host-agent (koopa) | firecuda site-gen |
|--|--------------------|-------------------|
| Access | **podman on host** | public HTTPS only |
| Phases | urls **inside** versions | urls versions |
| Trigger | software path + 4h | *could* launchd 4h |
| Primary for GOA | **yes** | **off** (optional only) |
**firecuda-external is not scheduled** right now (launchd disabled). Scripts under
`site-gen/` can still one-shot a public run if needed; see `site-gen/README.md`.
No secrets required for default phases. `secrets.env` optional for e2e later.
## Git commit on HTML pages
Header of each generated page includes a link like:
`https://git.hacktivism.ch/hernani/taler-monitoring/src/commit/<full-sha>`
That SHA is the tree **after** `update-suite.sh` for that run.
## Packages on koopa
See **[../DEPENDENCIES.md](../DEPENDENCIES.md)**.
**Default host-agent needs no new packages** if curl/python3/podman/git are present (they are).
**Root once** only for full QR checks in `urls`:
```bash
zypper in qrencode zbar
```
Without them, monitoring still runs; QR group WARNs and skips encode/decode.
Caddy `/var/www/monitoring-sites` is separate (you configured hernani write access).

57
host-agent/ROOT-APPLY-MONITORING.md Normal file
View file

@ -0,0 +1,57 @@
# Root: make monitoring pages live on koopa (GOA)
## Symptom if not applied
Public URLs return **merchant/bank JSON** like:
```json
{ "code": 21, "hint": "There is no endpoint defined for the URL provided..." }
```
or a **404 HTML** page from the app reverse_proxy.
Cause: live `/etc/caddy/Caddyfile` has **no** `handle /monitoring*` (falls through to
API). Prepared config is only under **`/home/hernani/koopa-caddy/Caddyfile`**.
## One-shot as root (recommended)
```bash
sudo /home/hernani/koopa-caddy/apply-monitoring-live.sh
```
That script:
1. `rsync` staging HTML → `/var/www/monitoring-sites/` (incl. surface + aptdeploy)
2. Makes tree writable for `hernani` (group `caddy` + ACL) so later runs self-sync
3. Installs prepared Caddyfile → `/etc/caddy/Caddyfile`, validates, reloads Caddy
4. Smoke-curls all monitoring URLs (must **not** be JSON `code:21`)
## Paths
| Path | Host | Source timer |
|------|------|----------------|
| `/monitoring/` + `/monitoring_err/` | bank + exchange + taler | `taler-monitoring-hacktivism.timer` (4h) |
| `/taler-monitoring-surface/` + `/_err/` | **taler.hacktivism.ch only** | `taler-monitoring-surface.timer` (1h) |
| `/taler-monitoring-aptdeploy/` + `/_err/` | **taler.hacktivism.ch only** | `taler-monitoring-aptdeploy.timer` (4h) |
HTML is generated as **hernani** into
`/home/hernani/monitoring-sites-staging/`.
## Manual equivalent
```bash
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
chown -R hernani:caddy /var/www/monitoring-sites && chmod -R g+rwX /var/www/monitoring-sites
sudo /home/hernani/koopa-caddy/apply.sh # or apply-monitoring-live.sh
```
Expect **200** (HTML with sticky bar). Not merchant JSON `code:21`.
## After each monitoring run (if www is not hernani-writable)
```bash
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
chown -R caddy:caddy /var/www/monitoring-sites
```
Or grant `hernani` write on `/var/www/monitoring-sites` once so the host-agent can rsync itself.

373
host-agent/ensure-apt-deploy-test-containers.sh Executable file
View file

@ -0,0 +1,373 @@
#!/usr/bin/env bash
# ensure-apt-deploy-test-containers.sh
#
# Four rootless podman containers on koopa (no nginx / no HTTPS):
#
# Fresh rebuild when repo fingerprint changes:
# koopa-taler-deploy-test-apt-src-trixie
# koopa-taler-deploy-test-apt-src-trixie-testing
#
# Persistent upgrade path (mytops upgrade.sh style):
# koopa-taler-deploy-test-apt-src-trixie-upgrade
# koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
#
# Run as hernani on koopa. From elsewhere use run-aptdeploy.sh (ssh koopa-external).
#
set -euo pipefail
PODMAN_BIN="${APT_DEPLOY_PODMAN:-podman}"
IMG="${APT_DEPLOY_BASE_IMAGE:-docker.io/library/debian:trixie}"
STATE_DIR="${XDG_STATE_HOME:-$HOME/.local/state}/taler-apt-deploy"
URI_BASE="${APT_DEPLOY_URI:-https://deb.taler.net/apt/debian}"
mkdir -p "$STATE_DIR"
# Packages we track for "new version → rebuild fresh containers"
TRACK_PKGS="${APT_DEPLOY_TRACK_PKGS:-taler-merchant libtalermerchant libtalerexchange libdonau}"
# Canonical four containers (keep). Everything else matching legacy names is removed.
CANONICAL_CONTAINERS=(
koopa-taler-deploy-test-apt-src-trixie
koopa-taler-deploy-test-apt-src-trixie-testing
koopa-taler-deploy-test-apt-src-trixie-upgrade
koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
)
# Legacy / typo names from earlier iterations — always delete if present
LEGACY_CONTAINERS=(
koopa-taler-build-test-apt-trixie
koopa-taler-build-test-apt-trixie-testing
koopa-taler-deploy-test-apt-trixie
koopa-taler-deploy-test-apt-trixie-testing
koopa-taler-buidl-test-apt-trixie
koopa-taler-buidl-test-apt-trixie-testing
)
cleanup_legacy() {
local n img
echo "======== cleanup legacy containers / images ========"
for n in "${LEGACY_CONTAINERS[@]}"; do
if "$PODMAN_BIN" container exists "$n" 2>/dev/null; then
echo "remove legacy container $n"
"$PODMAN_BIN" rm -f "$n" 2>/dev/null || true
fi
# old setup images for that name
if "$PODMAN_BIN" image exists "localhost/${n}:setup" 2>/dev/null; then
echo "remove legacy image localhost/${n}:setup"
"$PODMAN_BIN" rmi -f "localhost/${n}:setup" 2>/dev/null || true
fi
done
# dangling setup images not in canonical set
while read -r img; do
[ -n "$img" ] || continue
local base keep=0
base=${img#localhost/}
base=${base%:setup}
for n in "${CANONICAL_CONTAINERS[@]}"; do
[ "$base" = "$n" ] && keep=1 && break
done
if [ "$keep" -eq 0 ] && [[ "$img" == localhost/koopa-taler-*-test-apt* ]]; then
echo "remove non-canonical image $img"
"$PODMAN_BIN" rmi -f "$img" 2>/dev/null || true
fi
done < <("$PODMAN_BIN" images --format '{{.Repository}}:{{.Tag}}' 2>/dev/null | grep -E 'localhost/koopa-taler-.*-test-apt' || true)
}
# Prefer single suite tree: if both ~/src/koopa-admin-log and ~/koopa-admin-log are
# full git clones, keep the newer (by HEAD commit time) and replace the older with a symlink.
dedupe_suite_repos() {
local a="$HOME/src/koopa-admin-log" b="$HOME/koopa-admin-log"
local ta tb
[ "${APT_DEPLOY_DEDUPE_REPOS:-1}" = "1" ] || return 0
if [ -d "$a/.git" ] && [ -d "$b/.git" ] && [ ! -L "$a" ] && [ ! -L "$b" ]; then
ta=$(git -C "$a" log -1 --format=%ct 2>/dev/null || echo 0)
tb=$(git -C "$b" log -1 --format=%ct 2>/dev/null || echo 0)
echo "======== dedupe suite repos (both are git clones) ========"
echo " $a HEAD_ts=$ta"
echo " $b HEAD_ts=$tb"
if [ "$ta" -ge "$tb" ]; then
echo "keep $a · remove $b · symlink $b → $a"
rm -rf "$b"
ln -sfn "$a" "$b"
else
echo "keep $b · remove $a · symlink $a → $b"
rm -rf "$a"
mkdir -p "$(dirname "$a")"
ln -sfn "$b" "$a"
fi
elif [ -d "$a/scripts/taler-monitoring" ] && [ -d "$b/scripts/taler-monitoring" ] \
&& [ ! -d "$a/.git" ] && [ -d "$b/.git" ] && [ ! -L "$a" ]; then
echo "remove non-git tree $a · symlink → $b"
rm -rf "$a"
mkdir -p "$(dirname "$a")"
ln -sfn "$b" "$a"
elif [ -d "$b/scripts/taler-monitoring" ] && [ -d "$a/scripts/taler-monitoring" ] \
&& [ ! -d "$b/.git" ] && [ -d "$a/.git" ] && [ ! -L "$b" ]; then
echo "remove non-git tree $b · symlink → $a"
rm -rf "$b"
ln -sfn "$a" "$b"
fi
}
cleanup_legacy
dedupe_suite_repos
# ---------------------------------------------------------------------------
# Repo fingerprint: Version: lines from suite Packages index
# ---------------------------------------------------------------------------
repo_pkg_version() {
local suite=$1 pkg=$2
curl -fsS --max-time 45 \
"${URI_BASE}/dists/${suite}/main/binary-amd64/Packages" 2>/dev/null \
| awk -v p="$pkg" '
$0 == "Package: " p { want=1; next }
/^Package: / { want=0 }
want && /^Version: / { print $2; exit }
'
}
repo_fingerprint() {
local suite=$1 pkg v line=""
for pkg in $TRACK_PKGS; do
v=$(repo_pkg_version "$suite" "$pkg" || true)
[ -n "$v" ] || v="?"
line="${line}${pkg}=${v};"
done
printf '%s' "$line"
}
container_fingerprint() {
local name=$1
"$PODMAN_BIN" exec "$name" bash -lc '
for p in '"$TRACK_PKGS"'; do
v=$(dpkg-query -W -f="\${Version}" "$p" 2>/dev/null || echo missing)
printf "%s=%s;" "$p" "$v"
done
' 2>/dev/null || echo "missing"
}
# ---------------------------------------------------------------------------
# Bootstrap: install base + taler source + packages (shared)
# ---------------------------------------------------------------------------
install_base_and_merchant() {
local name=$1 suite=$2 srcfile=$3
"$PODMAN_BIN" exec -u root "$name" bash -lc '
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq ca-certificates curl wget gnupg systemd systemd-sysv \
dbus procps iproute2
'
"$PODMAN_BIN" exec -u root "$name" bash -lc "
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive
mkdir -p /etc/apt/keyrings
wget -q -O /etc/apt/keyrings/taler-systems.gpg https://taler.net/taler-systems.gpg
cat >/etc/apt/sources.list.d/${srcfile} <<EOF
Architectures: amd64
Components: main
X-Repolib-Name: Taler
Signed-By: /etc/apt/keyrings/taler-systems.gpg
Suites: ${suite}
Types: deb
URIs: ${URI_BASE}
EOF
apt-get update -qq
apt-get install -y \
taler-merchant \
taler-merchant-typst \
taler-merchant-webui \
taler-terms-generator \
postgresql \
s-nail \
jq
"
}
commit_and_systemd() {
local name=$1
"$PODMAN_BIN" stop "$name"
"$PODMAN_BIN" commit "$name" "localhost/${name}:setup"
"$PODMAN_BIN" rm "$name"
"$PODMAN_BIN" run -d --name "$name" \
--hostname "$name" \
--systemd=always \
--cgroupns=host \
"localhost/${name}:setup" \
/lib/systemd/systemd
sleep 5
# enable merchant stack if unit exists
"$PODMAN_BIN" exec "$name" bash -lc '
set +e
systemctl daemon-reload 2>/dev/null
systemctl enable taler-merchant.target 2>/dev/null
systemctl start taler-merchant.target 2>/dev/null
sleep 2
systemctl is-active taler-merchant-httpd.service 2>/dev/null || true
' || true
}
create_from_scratch() {
local name=$1 suite=$2 srcfile=$3
echo "======== CREATE (fresh) $name Suites: $suite ========"
"$PODMAN_BIN" pull "$IMG" >/dev/null
"$PODMAN_BIN" rm -f "$name" 2>/dev/null || true
# drop previous setup image so we do not accumulate duplicates
"$PODMAN_BIN" rmi -f "localhost/${name}:setup" 2>/dev/null || true
"$PODMAN_BIN" run -d --name "$name" --hostname "$name" "$IMG" sleep infinity
install_base_and_merchant "$name" "$suite" "$srcfile"
commit_and_systemd "$name"
printf '%s\n' "$(repo_fingerprint "$suite")" >"$STATE_DIR/${name}.fp"
printf '%s\n' "$(container_fingerprint "$name")" >"$STATE_DIR/${name}.installed"
echo "created $name fp=$(cat "$STATE_DIR/${name}.fp")"
}
# ---------------------------------------------------------------------------
# Upgrade path (mytops-admin-log upgrade.sh merchant packages, no restic/nginx)
# Prints a markdown-ish table of before → after versions.
# ---------------------------------------------------------------------------
upgrade_inside() {
local name=$1
local before after pkg v0 v1
echo "======== UPGRADE $name (mytops-style apt install + upgrade) ========"
before=$("$PODMAN_BIN" exec "$name" bash -lc '
for p in taler-merchant libtalermerchant libtalerexchange libdonau libgnunet \
taler-merchant-webui taler-merchant-typst taler-terms-generator; do
v=$(dpkg-query -W -f="\${Version}" "$p" 2>/dev/null || echo "-")
printf "%s %s\n" "$p" "$v"
done
' 2>/dev/null || true)
# capture upgrade output
set +e
"$PODMAN_BIN" exec -u root "$name" bash -lc '
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive
systemctl stop taler-merchant.target 2>/dev/null || true
systemctl stop taler-merchant-donaukeyupdate.service 2>/dev/null || true
apt-get update -qq
apt-get install -y \
taler-merchant \
taler-merchant-typst \
taler-merchant-webui \
taler-terms-generator \
|| true
if dpkg -l challenger-httpd 2>/dev/null | grep -q "^ii"; then
apt-get remove -y challenger-httpd || true
fi
apt-get upgrade -y
systemctl daemon-reload 2>/dev/null || true
systemctl enable taler-merchant.target 2>/dev/null || true
# try dbconfig if present (may no-op without full config)
command -v taler-merchant-dbconfig >/dev/null && taler-merchant-dbconfig 2>/dev/null || true
systemctl reset-failed "taler-merchant*" 2>/dev/null || true
systemctl start taler-merchant.target 2>/dev/null || true
sleep 2
' 2>&1 | tee "$STATE_DIR/${name}.upgrade.log" | tail -30
set -e
after=$("$PODMAN_BIN" exec "$name" bash -lc '
for p in taler-merchant libtalermerchant libtalerexchange libdonau libgnunet \
taler-merchant-webui taler-merchant-typst taler-terms-generator; do
v=$(dpkg-query -W -f="\${Version}" "$p" 2>/dev/null || echo "-")
printf "%s %s\n" "$p" "$v"
done
' 2>/dev/null || true)
echo
echo "| package | before | after | change |"
echo "|---------|--------|-------|--------|"
while read -r pkg v0; do
[ -n "${pkg:-}" ] || continue
v1=$(printf '%s\n' "$after" | awk -v p="$pkg" '$1==p {print $2; exit}')
v1=${v1:--}
if [ "$v0" = "$v1" ]; then
ch="="
else
ch="UPGRADED"
fi
echo "| $pkg | $v0 | $v1 | $ch |"
done <<<"$before"
printf '%s\n' "$(container_fingerprint "$name")" >"$STATE_DIR/${name}.installed"
}
# ---------------------------------------------------------------------------
# Fresh mode: recreate if missing / not running / repo fp changed
# ---------------------------------------------------------------------------
ensure_fresh() {
local name=$1 suite=$2 srcfile=$3
local want have running
want=$(repo_fingerprint "$suite")
echo "repo fingerprint [$suite]: $want"
if ! "$PODMAN_BIN" container exists "$name" 2>/dev/null; then
create_from_scratch "$name" "$suite" "$srcfile"
return
fi
running=$("$PODMAN_BIN" inspect -f '{{.State.Running}}' "$name" 2>/dev/null || echo false)
have=$(container_fingerprint "$name")
echo "installed in $name: $have"
if [ "$running" != "true" ] \
|| ! "$PODMAN_BIN" exec "$name" bash -lc 'command -v taler-merchant-httpd' >/dev/null 2>&1 \
|| [ "$want" != "$have" ]; then
echo "rebuild $name (running=$running fp_match=$([ "$want" = "$have" ] && echo yes || echo NO))"
create_from_scratch "$name" "$suite" "$srcfile"
else
echo "OK $name up-to-date (fresh container matches repo)"
fi
}
# ---------------------------------------------------------------------------
# Upgrade mode: create once, then always upgrade_inside
# ---------------------------------------------------------------------------
ensure_upgrade() {
local name=$1 suite=$2 srcfile=$3
local running
if ! "$PODMAN_BIN" container exists "$name" 2>/dev/null; then
create_from_scratch "$name" "$suite" "$srcfile"
echo "(initial install for upgrade-track container $name)"
return
fi
running=$("$PODMAN_BIN" inspect -f '{{.State.Running}}' "$name" 2>/dev/null || echo false)
if [ "$running" != "true" ]; then
echo "start $name"
"$PODMAN_BIN" start "$name" || {
echo "recreate broken $name"
create_from_scratch "$name" "$suite" "$srcfile"
return
}
sleep 3
fi
if ! "$PODMAN_BIN" exec "$name" bash -lc 'command -v taler-merchant-httpd' >/dev/null 2>&1; then
echo "recreate $name (no httpd after start)"
create_from_scratch "$name" "$suite" "$srcfile"
return
fi
upgrade_inside "$name"
}
MODE="${1:-all}" # all | fresh | upgrade
case "$MODE" in
all|fresh)
ensure_fresh koopa-taler-deploy-test-apt-src-trixie trixie taler.sources
ensure_fresh koopa-taler-deploy-test-apt-src-trixie-testing trixie-testing taler-testing.sources
;;&
all|upgrade)
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-upgrade trixie taler.sources
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-testing-upgrade trixie-testing taler-testing.sources
;;
*)
echo "usage: $0 [all|fresh|upgrade]" >&2
exit 2
;;
esac
echo "done ensure-apt-deploy-test-containers mode=$MODE"

26
host-agent/env.example Normal file
View file

@ -0,0 +1,26 @@
# Copy to: ~/.config/taler-monitoring/env
# This file is NOT inside the git tree and is never overwritten by update-suite.sh.
#
# Full stack examples (same reporting generation for all):
# env/hacktivism.env.example
# env/stagepaysan.env.example
# env/francpaysan.env.example
#
# SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
# SUITE_GIT_REF=main
# SUITE_DIR=$HOME/src/taler-monitoring
# SUITE_UPDATE_MODE=reset
#
# TALER_DOMAIN=hacktivism.ch
# INSIDE_PODMAN=1
# INSIDE_MODE=local-podman
# CONTINUE_ON_ERROR=1
# PHASES for the *default* hacktivism timer only (urls inside versions).
# Dedicated wrappers always override PHASES + HTML paths:
# run-surface-monitoring.sh → /taler-monitoring-surface*
# run-aptdeploy-monitoring.sh → /taler-monitoring-aptdeploy*
# PHASES="urls inside versions"
# RUN_TIMEOUT=600
# HTML_OUT=$HOME/monitoring-sites-staging
# MON_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch"
# SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring

24
host-agent/env/francpaysan.env.example vendored Normal file
View file

@ -0,0 +1,24 @@
# Copy to: ~/.config/taler-monitoring/env (user francpaysan)
# Never inside the git tree — update-suite never overwrites this file.
#
# Shared reporting generation (run-host-report.sh / run-fp-prod-monitoring.sh):
# RUN_TIMEOUT, line-buffered log, always HTML, commit link on git.hacktivism.ch
TALER_DOMAIN=lefrancpaysan.ch
INSIDE_PODMAN=1
INSIDE_MODE=local-podman
LOCAL_STACK=0
CONTINUE_ON_ERROR=1
PHASES="urls inside versions"
RUN_TIMEOUT=600
HTML_OUT=$HOME/monitoring-sites-staging
MON_HOSTS="bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch"
SUITE_DIR=$HOME/src/taler-monitoring
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
SUITE_GIT_REF=main
SUITE_UPDATE_MODE=reset
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
# DEPLOY_WWW_ROOT=/var/www/monitoring-sites

24
host-agent/env/hacktivism.env.example vendored Normal file
View file

@ -0,0 +1,24 @@
# Copy to: ~/.config/taler-monitoring/env (user hernani on koopa)
# Shared reporting generation: RUN_TIMEOUT, line-buffered log, always HTML.
TALER_DOMAIN=hacktivism.ch
INSIDE_PODMAN=1
INSIDE_MODE=local-podman
LOCAL_STACK=1
CONTINUE_ON_ERROR=1
PHASES="urls inside versions"
# aptdeploy HTML: run-aptdeploy-monitoring.sh → /taler-monitoring-aptdeploy*
RUN_TIMEOUT=600
# APT_DEPLOY_ENSURE=1
# APT_DEPLOY_SKIP=0
HTML_OUT=$HOME/monitoring-sites-staging
MON_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch"
SUITE_DIR=$HOME/src/taler-monitoring
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
SUITE_GIT_REF=main
SUITE_UPDATE_MODE=reset
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
DEPLOY_WWW_ROOT=/var/www/monitoring-sites

27
host-agent/env/stagepaysan.env.example vendored Normal file
View file

@ -0,0 +1,27 @@
# Copy to: ~/.config/taler-monitoring/env (user stagepaysan)
# Never inside the git tree — update-suite never overwrites this file.
#
# Shared reporting generation (run-host-report.sh / run-fp-stage-monitoring.sh):
# RUN_TIMEOUT, line-buffered log, always HTML, commit link on git.hacktivism.ch
TALER_DOMAIN=stage.lefrancpaysan.ch
INSIDE_PODMAN=1
INSIDE_MODE=local-podman
INSIDE_PROFILE=stage-lfp
LOCAL_STACK=0
CONTINUE_ON_ERROR=1
PHASES="urls inside versions"
RUN_TIMEOUT=600
HTML_OUT=$HOME/monitoring-sites-staging
MON_HOSTS="stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch"
# Git clone of koopa-admin-log (repo root, not …/scripts/taler-monitoring)
SUITE_DIR=$HOME/src/taler-monitoring
SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git
SUITE_GIT_REF=main
SUITE_UPDATE_MODE=reset
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
# Optional public docroot if this user can write it
# DEPLOY_WWW_ROOT=/var/www/monitoring-sites

107
host-agent/install-host-agent.sh Executable file
View file

@ -0,0 +1,107 @@
#!/usr/bin/env bash
# install-host-agent.sh — install hernani user systemd units on koopa
#
# # on koopa as hernani:
# ~/src/taler-monitoring/host-agent/install-host-agent.sh
# # or from laptop:
# ./install-host-agent.sh --remote koopa-external
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
REMOTE="${1:-}"
[ "${1:-}" = "--remote" ] && REMOTE="${2:-koopa-external}"
install_local() {
local mon home_unit
mon=$(cd "$ROOT/.." && pwd)
# Prefer ~/koopa-admin-log on live host
if [ -d "$HOME/src/taler-monitoring" ]; then
mon="$HOME/src/taler-monitoring"
fi
home_unit="$HOME/.config/systemd/user"
mkdir -p "$home_unit" \
"$HOME/.local/state/taler-hacktivism-monitoring" \
"$HOME/monitoring-sites-staging" \
"$HOME/.config/taler-monitoring" \
"$HOME/src"
# Local env (never in git) — create from example if missing
if [ ! -f "$HOME/.config/taler-monitoring/env" ] && [ -f "$ROOT/env.example" ]; then
cp "$ROOT/env.example" "$HOME/.config/taler-monitoring/env"
echo "created ~/.config/taler-monitoring/env"
fi
# Prefer a real git clone for update-suite.sh
if [ ! -d "$HOME/src/koopa-admin-log/.git" ] && [ ! -d "$HOME/koopa-admin-log/.git" ]; then
if [ -d "$HOME/src/taler-monitoring" ] && [ ! -d "$HOME/koopa-admin-log/.git" ]; then
echo "note: $HOME/koopa-admin-log has no .git — update-suite will clone to ~/src/koopa-admin-log"
fi
git clone https://git.hacktivism.ch/hernani/taler-monitoring.git "$HOME/src/koopa-admin-log" 2>/dev/null \
|| echo "WARN: clone failed (auth/network) — will use existing tree if any"
fi
# Ensure suite is executable
chmod +x "$mon"/taler-monitoring.sh "$mon"/check_*.sh 2>/dev/null || true
chmod +x "$mon/host-agent"/*.sh 2>/dev/null || true
chmod +x "$mon/site-gen"/*.sh "$mon/site-gen/console_to_html.py" 2>/dev/null || true
# Units reference %h/koopa-admin-log — prefer git clone or symlink
if [ -d "$HOME/src/koopa-admin-log/.git" ] && [ ! -e "$HOME/koopa-admin-log" ]; then
ln -sfn "$HOME/src/koopa-admin-log" "$HOME/koopa-admin-log"
echo "symlink $HOME/koopa-admin-log → $HOME/src/koopa-admin-log"
elif [ ! -e "$HOME/koopa-admin-log" ]; then
admin_root=$(cd "$mon/../.." && pwd)
ln -sfn "$admin_root" "$HOME/koopa-admin-log"
echo "symlink $HOME/koopa-admin-log → $admin_root"
fi
install -m 644 "$ROOT/taler-monitoring-hacktivism.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-hacktivism.path" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-hacktivism.timer" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-surface.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-surface.timer" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-aptdeploy.service" "$home_unit/"
install -m 644 "$ROOT/taler-monitoring-aptdeploy.timer" "$home_unit/"
# Fix ExecStart if install path differs (units use %h/koopa-admin-log)
systemctl --user daemon-reload
systemctl --user enable --now taler-monitoring-hacktivism.path
systemctl --user enable --now taler-monitoring-hacktivism.timer
systemctl --user enable --now taler-monitoring-surface.timer
systemctl --user enable --now taler-monitoring-aptdeploy.timer
# initial stamp so path exists
"$ROOT/touch-software-stamp.sh"
systemctl --user start taler-monitoring-hacktivism.service || true
systemctl --user start taler-monitoring-surface.service || true
systemctl --user start taler-monitoring-aptdeploy.service || true
echo "--- status ---"
systemctl --user status taler-monitoring-hacktivism.path --no-pager -l | head -15
systemctl --user status taler-monitoring-hacktivism.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-surface.timer --no-pager -l | head -12
systemctl --user status taler-monitoring-aptdeploy.timer --no-pager -l | head -12
systemctl --user list-timers --all | grep taler-monitoring || true
echo "OK host-agent (hacktivism 4h + surface 1h + aptdeploy 4h)"
echo "manual: systemctl --user start taler-monitoring-hacktivism.service"
echo "manual surface: systemctl --user start taler-monitoring-surface.service"
echo "manual aptdeploy: systemctl --user start taler-monitoring-aptdeploy.service"
echo "after upgrades: $ROOT/touch-software-stamp.sh"
}
if [ -n "$REMOTE" ] && [ "$REMOTE" != "--remote" ]; then
echo "install on $REMOTE …"
rsync -az --delete \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
"$ROOT/../../" \
"${REMOTE}:src/taler-monitoring/"
# ensure full tree for %h/koopa-admin-log if missing configs etc.
ssh -o BatchMode=yes -o ConnectTimeout=20 "$REMOTE" \
'test -d $HOME/koopa-admin-log || git clone https://git.hacktivism.ch/hernani/taler-monitoring.git $HOME/koopa-admin-log 2>/dev/null || true'
rsync -az "$ROOT/" "${REMOTE}:src/taler-monitoring/host-agent/"
rsync -az "$ROOT/../site-gen/" "${REMOTE}:src/taler-monitoring/site-gen/" 2>/dev/null || true
ssh -o BatchMode=yes "$REMOTE" \
'bash $HOME/src/taler-monitoring/host-agent/install-host-agent.sh'
else
install_local
fi

43
host-agent/run-aptdeploy-monitoring.sh Executable file
View file

@ -0,0 +1,43 @@
#!/usr/bin/env bash
# run-aptdeploy-monitoring.sh — apt-src deploy tests → public HTML
# Public only on taler.hacktivism.ch:
# https://taler.hacktivism.ch/taler-monitoring-aptdeploy/
# https://taler.hacktivism.ch/taler-monitoring-aptdeploy_err/
#
# CLI one-shot without HTML remains: run-aptdeploy.sh [all|fresh|upgrade|check-only]
#
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-aptdeploy-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-aptdeploy-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
# containers are local on koopa
export INSIDE_PODMAN=1
export INSIDE_MODE=local-podman
export LOCAL_STACK=1
export SKIP_SSH=0
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
# four containers + checks; allow long wall
export RUN_TIMEOUT="${RUN_TIMEOUT:-1800}"
# dedicated job: always aptdeploy only
export PHASES=aptdeploy
# Only merchant public front — not bank/exchange
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
export HTML_OK_DIR="taler-monitoring-aptdeploy"
export HTML_ERR_DIR="taler-monitoring-aptdeploy_err"
export HTML_URL_OK="/taler-monitoring-aptdeploy/"
export HTML_URL_ERR="/taler-monitoring-aptdeploy_err/"
export PAGE_LABEL="taler-monitoring-aptdeploy"
# Ensure all four apt-src containers (fresh rebuild + upgrade track)
if [ "${APT_DEPLOY_ENSURE:-1}" = "1" ] \
&& [ -x "$AGENT_DIR/ensure-apt-deploy-test-containers.sh" ]; then
bash "$AGENT_DIR/ensure-apt-deploy-test-containers.sh" all \
|| echo "WARN: ensure-apt-deploy-test-containers failed (aptdeploy may ERROR)" >&2
fi
exec bash "$AGENT_DIR/run-host-report.sh" "$@"

150
host-agent/run-aptdeploy.sh Executable file
View file

@ -0,0 +1,150 @@
#!/usr/bin/env bash
# run-aptdeploy.sh — ensure containers + aptdeploy phase only.
#
# Keep monitoring current:
# 1) Prefer Forgejo origin/main (update-suite reset)
# 2) If Forgejo is behind (no aptdeploy yet), re-apply suite-overlay saved
# from this agent tree before reset (rsync from laptop fills that tree)
# 3) Run ensure + taler-monitoring.sh aptdeploy
#
# Host: on koopa → local; else ssh APT_DEPLOY_SSH + rsync mon tree first.
#
set -euo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
BOOT_MON=$(cd "$AGENT_DIR/.." && pwd)
MODE="${1:-all}"
APT_DEPLOY_SSH="${APT_DEPLOY_SSH:-koopa-external}"
OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay"
on_koopa() {
local h
h=$(hostname -s 2>/dev/null || hostname 2>/dev/null || echo "")
case "$h" in koopa|koopa.*) return 0 ;; esac
[ "${KOOPA_FORCE_LOCAL:-0}" = "1" ] && return 0
return 1
}
mon_has_aptdeploy() {
local mon=$1
[ -f "$mon/taler-monitoring.sh" ] && grep -qE 'aptdeploy\|apt-deploy' "$mon/taler-monitoring.sh"
}
save_overlay_from() {
local src=$1
mon_has_aptdeploy "$src" || return 0
mkdir -p "$OVERLAY_DIR"
rsync -a --delete \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
--exclude 'android-test/apks' \
--exclude 'android-test/out*' \
--exclude '__pycache__' \
"$src/" "$OVERLAY_DIR/"
echo "saved suite-overlay from $src → $OVERLAY_DIR"
}
apply_overlay_to() {
local dest=$1
[ -f "$OVERLAY_DIR/taler-monitoring.sh" ] || return 1
mon_has_aptdeploy "$OVERLAY_DIR" || return 1
mkdir -p "$dest"
rsync -a \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
--exclude 'android-test/apks' \
--exclude 'android-test/out*' \
--exclude '__pycache__' \
"$OVERLAY_DIR/" "$dest/"
echo "applied suite-overlay → $dest (Forgejo lag; push origin/main when possible)"
return 0
}
pick_mon_tree() {
# Preserve working mon before reset wipes tracked files
save_overlay_from "$BOOT_MON"
# shellcheck source=update-suite.sh
source "$AGENT_DIR/update-suite.sh"
local suite_mon="$SUITE_DIR"
mkdir -p "$suite_mon"
if mon_has_aptdeploy "$suite_mon"; then
MON_DIR="$suite_mon"
echo "using Forgejo suite mon @ ${COMMIT_SHORT:-?} · $MON_DIR"
elif apply_overlay_to "$suite_mon" && mon_has_aptdeploy "$suite_mon"; then
MON_DIR="$suite_mon"
echo "using suite mon + overlay @ ${COMMIT_SHORT:-?} · $MON_DIR"
elif mon_has_aptdeploy "$BOOT_MON"; then
MON_DIR="$BOOT_MON"
echo "WARN: using boot mon $MON_DIR (overlay apply failed)" >&2
else
echo "ERROR: no mon tree with aptdeploy (Forgejo + overlay empty)" >&2
echo " rsync suite from laptop or push to git.hacktivism.ch" >&2
exit 2
fi
AGENT_DIR="$MON_DIR/host-agent"
export MON_DIR AGENT_DIR SUITE_DIR COMMIT COMMIT_SHORT COMMIT_URL
chmod +x "$AGENT_DIR"/*.sh "$MON_DIR"/taler-monitoring.sh "$MON_DIR"/check_*.sh 2>/dev/null || true
}
run_local() {
export PATH="${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
export PYTHONUNBUFFERED=1
export PROGRESS_OFF="${PROGRESS_OFF:-0}"
echo "======== aptdeploy local on $(hostname) mode=$MODE ========"
pick_mon_tree
if [ "$MODE" != "check-only" ]; then
bash "$AGENT_DIR/ensure-apt-deploy-test-containers.sh" "$MODE"
fi
cd "$MON_DIR"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
export LOCAL_STACK="${LOCAL_STACK:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
echo "run: $MON_DIR/taler-monitoring.sh -d hacktivism.ch aptdeploy · suite=${COMMIT_SHORT:-?}"
./taler-monitoring.sh -d hacktivism.ch aptdeploy
}
run_remote() {
echo "======== aptdeploy remote via ssh ${APT_DEPLOY_SSH} mode=$MODE ========"
if [ "${APT_DEPLOY_RSYNC:-1}" = "1" ]; then
echo "rsync mon suite → ${APT_DEPLOY_SSH}:src/src/taler-monitoring/"
rsync -az \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
--exclude 'android-test/apks' \
--exclude 'android-test/out*' \
--exclude '__pycache__' \
"$BOOT_MON/" \
"${APT_DEPLOY_SSH}:src/src/taler-monitoring/" \
|| echo "WARN: rsync failed" >&2
fi
ssh -o BatchMode=yes -o ConnectTimeout=25 "$APT_DEPLOY_SSH" bash -s -- "$MODE" <<'EOS'
set -euo pipefail
MODE=${1:-all}
export PATH="${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
export KOOPA_FORCE_LOCAL=1
AGENT="$HOME/src/taler-monitoring/host-agent"
if [ ! -x "$AGENT/run-aptdeploy.sh" ]; then
AGENT="$HOME/src/taler-monitoring/host-agent"
fi
if [ ! -x "$AGENT/run-aptdeploy.sh" ]; then
echo "ERROR: run-aptdeploy.sh missing on remote" >&2
exit 1
fi
echo "remote AGENT=$AGENT"
exec bash "$AGENT/run-aptdeploy.sh" "$MODE"
EOS
}
if on_koopa; then
run_local
else
run_remote
fi

20
host-agent/run-fp-prod-monitoring.sh Executable file
View file

@ -0,0 +1,20 @@
#!/usr/bin/env bash
# FrancPaysan PROD host-agent (user francpaysan on francpaysan-host).
# Shared reporting: run-host-report.sh (timeout, line-buffered log, first-run HTML).
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-fp-prod-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-monitoring-lfp-prod}"
export TALER_DOMAIN="${TALER_DOMAIN:-lefrancpaysan.ch}"
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
export LOCAL_STACK="${LOCAL_STACK:-0}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
export PHASES="${PHASES:-urls inside versions}"
export MON_HOSTS="${MON_HOSTS:-bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
exec bash "$AGENT_DIR/run-host-report.sh" "$@"

21
host-agent/run-fp-stage-monitoring.sh Executable file
View file

@ -0,0 +1,21 @@
#!/usr/bin/env bash
# FrancPaysan STAGE host-agent (user stagepaysan on francpaysan-host).
# Shared reporting: run-host-report.sh (timeout, line-buffered log, first-run HTML).
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-fp-stage-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-monitoring-stage-lfp}"
export TALER_DOMAIN="${TALER_DOMAIN:-stage.lefrancpaysan.ch}"
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
export INSIDE_PROFILE="${INSIDE_PROFILE:-stage-lfp}"
export LOCAL_STACK="${LOCAL_STACK:-0}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
export PHASES="${PHASES:-urls inside versions}"
export MON_HOSTS="${MON_HOSTS:-stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
exec bash "$AGENT_DIR/run-host-report.sh" "$@"

23
host-agent/run-hacktivism-monitoring.sh Executable file
View file

@ -0,0 +1,23 @@
#!/usr/bin/env bash
# GOA / hacktivism host-agent on koopa (hernani).
# Stack defaults only — reporting pipeline is shared: run-host-report.sh
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-hacktivism-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-hacktivism-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
export INSIDE_PODMAN="${INSIDE_PODMAN:-1}"
export INSIDE_MODE="${INSIDE_MODE:-local-podman}"
export LOCAL_STACK="${LOCAL_STACK:-1}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
# stack checks only — aptdeploy has its own HTML path + timer:
# run-aptdeploy-monitoring.sh → /taler-monitoring-aptdeploy* (taler.hacktivism.ch)
export PHASES="${PHASES:-urls inside versions}"
export MON_HOSTS="${MON_HOSTS:-bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
export APT_DEPLOY_ENSURE=0
exec bash "$AGENT_DIR/run-host-report.sh" "$@"

358
host-agent/run-host-report.sh Executable file
View file

@ -0,0 +1,358 @@
#!/usr/bin/env bash
# run-host-report.sh — global reporting generation for all stacks
# (GOA/hacktivism, FP stage, FP prod, …).
#
# Shared behaviour (always):
# 1) update-suite → latest git.hacktivism.ch (local ~/.config never overwritten)
# 2) line-buffered run log (written continuously)
# 3) RUN_TIMEOUT wall clock for taler-monitoring.sh (default 600s)
# 4) always write HTML (first run / empty staging too) + Forgejo commit link
# 5) optional rsync to DEPLOY_WWW_ROOT if writable
#
# Configure via env file (not in git):
# ${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env
# or TALER_MONITORING_ENV=/path/to/env
#
# Thin wrappers only set stack defaults then exec this script:
# run-hacktivism-monitoring.sh
# run-fp-stage-monitoring.sh
# run-fp-prod-monitoring.sh
#
set -uo pipefail
export PATH="${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
export PYTHONUNBUFFERED=1
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
AGENT_MON=$(cd "$AGENT_DIR/.." && pwd)
CFG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring"
ENV_FILE="${TALER_MONITORING_ENV:-$CFG_DIR/env}"
mkdir -p "$CFG_DIR"
# Thin wrappers (surface, aptdeploy-only, …) export stack knobs before exec.
# Snapshot them so ~/.config/taler-monitoring/env cannot clobber dedicated jobs
# (env often has PHASES="urls inside versions aptdeploy" for the main timer).
_wrap_keys=(
PHASES HTML_OUT HTML_OK_DIR HTML_ERR_DIR HTML_URL_OK HTML_URL_ERR PAGE_LABEL
RUN_TIMEOUT STATE_NAME AGENT_LABEL MON_HOSTS TALER_DOMAIN DEPLOY_WWW_ROOT
INSIDE_PODMAN INSIDE_MODE LOCAL_STACK SKIP_SSH CONTINUE_ON_ERROR AUTH401_CONTINUE
APT_DEPLOY_ENSURE SURFACE_SCOPE TALER_DOMAIN_FROM_CLI STRICT_EXIT SOURCE_SUITE_PATH
)
for _k in "${_wrap_keys[@]}"; do
if [ -n "${!_k+x}" ]; then
printf -v "_WRAP_SET_${_k}" '%s' 1
printf -v "_WRAP_VAL_${_k}" '%s' "${!_k}"
fi
done
unset _k
if [ -f "$ENV_FILE" ]; then
# shellcheck disable=SC1090
set -a
# shellcheck source=/dev/null
source "$ENV_FILE"
set +a
fi
# Tell update-suite.sh not to re-source env (would clobber wrapper PHASES etc.)
export _TALER_MON_ENV_LOADED=1
# Restore wrapper overrides (dedicated jobs win over shared env file)
for _k in "${_wrap_keys[@]}"; do
_set_var="_WRAP_SET_${_k}"
_val_var="_WRAP_VAL_${_k}"
if [ "${!_set_var:-}" = "1" ]; then
printf -v "$_k" '%s' "${!_val_var}"
export "$_k"
fi
done
unset _k _set_var _val_var
# --- global report defaults (env file / wrapper wins if already set) ---
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
export AUTH401_CONTINUE="${AUTH401_CONTINUE:-${CONTINUE_ON_ERROR}}"
export RUN_TIMEOUT="${RUN_TIMEOUT:-600}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
PHASES="${PHASES:-urls inside versions}"
HTML_BASE="${HTML_OUT:-$HOME/monitoring-sites-staging}"
MON_HOSTS="${MON_HOSTS:-}"
SUITE_PATH="${SOURCE_SUITE_PATH:-.}"
AGENT_LABEL="${AGENT_LABEL:-host-report}"
STATE_NAME="${STATE_NAME:-taler-monitoring-${TALER_DOMAIN}}"
STATE_DIR="${XDG_STATE_HOME:-$HOME/.local/state}/${STATE_NAME}"
LOG_DIR="$STATE_DIR/logs"
DEPLOY_WWW="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
# HTML path layout under each host (monitoring / surface / …)
HTML_OK_DIR="${HTML_OK_DIR:-monitoring}"
HTML_ERR_DIR="${HTML_ERR_DIR:-monitoring_err}"
HTML_URL_OK="${HTML_URL_OK:-/monitoring/}"
HTML_URL_ERR="${HTML_URL_ERR:-/monitoring_err/}"
PAGE_LABEL="${PAGE_LABEL:-monitoring}"
# Sensible MON_HOSTS from domain if not set
if [ -z "$MON_HOSTS" ]; then
case "$TALER_DOMAIN" in
hacktivism.ch|koopa)
MON_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch"
;;
stage.lefrancpaysan.ch|stage.*lefrancpaysan*)
MON_HOSTS="stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch"
;;
lefrancpaysan.ch)
MON_HOSTS="bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch"
;;
*)
MON_HOSTS="${TALER_DOMAIN}"
;;
esac
fi
mkdir -p "$LOG_DIR" "$HTML_BASE"
STAMP=$(date +%Y%m%d-%H%M%S)
LOG="$LOG_DIR/run-${STAMP}.log"
: >"$LOG"
if command -v stdbuf >/dev/null 2>&1; then
exec > >(stdbuf -oL -eL tee -a "$LOG") 2>&1
else
exec > >(tee -a "$LOG") 2>&1
fi
echo "======== $(date -Iseconds 2>/dev/null || date) ${AGENT_LABEL} ========"
echo "env_file=$ENV_FILE"
echo "domain=$TALER_DOMAIN · phases=$PHASES · RUN_TIMEOUT=${RUN_TIMEOUT}s"
echo "hosts=$MON_HOSTS"
echo "flags INSIDE_PODMAN=${INSIDE_PODMAN:-} INSIDE_MODE=${INSIDE_MODE:-} LOCAL_STACK=${LOCAL_STACK:-} SKIP_SSH=${SKIP_SSH:-} INSIDE_PROFILE=${INSIDE_PROFILE:-}"
# --- refresh from Forgejo; re-apply suite-overlay if Forgejo lacks needed phases ---
OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay"
# preserve capable mon before reset wipes tracked files
if [ -f "$AGENT_MON/taler-monitoring.sh" ]; then
mkdir -p "$OVERLAY_DIR"
rsync -a \
--exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \
--exclude 'android-test/out*' --exclude '__pycache__' \
"$AGENT_MON/" "$OVERLAY_DIR/" 2>/dev/null || true
fi
# shellcheck source=update-suite.sh
source "$AGENT_DIR/update-suite.sh"
SUITE_MON="${SUITE_DIR}"
ROOT="$SUITE_MON"
_need_overlay=0
# Detect missing features in Forgejo tree (untracked helper files alone are not enough —
# taler-monitoring.sh must accept the phase name).
case " $PHASES " in
*" aptdeploy "*|*" apt-deploy "*|*" apt_src "*)
if ! grep -qE 'aptdeploy\|apt-deploy' "$SUITE_MON/taler-monitoring.sh" 2>/dev/null; then
_need_overlay=1
fi
;;
esac
case " $PHASES " in
*" surface "*|*" ecosystem "*)
if ! grep -qE 'surface\|ecosystem' "$SUITE_MON/taler-monitoring.sh" 2>/dev/null \
|| [ ! -f "$SUITE_MON/check_surface.sh" ]; then
_need_overlay=1
fi
;;
esac
# Wrapper-preserve + HTML path knobs may live only in local overlay until pushed
if [ -f "$OVERLAY_DIR/host-agent/run-host-report.sh" ] \
&& ! grep -q '_wrap_keys\|_WRAP_SET_' "$SUITE_MON/host-agent/run-host-report.sh" 2>/dev/null \
&& grep -q '_wrap_keys\|_WRAP_SET_' "$OVERLAY_DIR/host-agent/run-host-report.sh" 2>/dev/null; then
_need_overlay=1
fi
if [ "$_need_overlay" = "1" ] && [ -f "$OVERLAY_DIR/taler-monitoring.sh" ]; then
echo "WARN: Forgejo @ ${COMMIT_SHORT:-?} lagging local suite-overlay (phases=[$PHASES])" >&2
echo " re-applying suite-overlay → $SUITE_MON" >&2
mkdir -p "$SUITE_MON"
rsync -a \
--exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \
--exclude 'android-test/out*' --exclude '__pycache__' \
"$OVERLAY_DIR/" "$SUITE_MON/"
ROOT="$SUITE_MON"
elif [ "$_need_overlay" = "1" ]; then
echo "WARN: Forgejo missing features and no suite-overlay — using agent mon $AGENT_MON" >&2
ROOT="$AGENT_MON"
else
echo "using suite mon @ ${COMMIT_SHORT:-?} · $ROOT"
fi
# Always restore host-agent + site-gen from pre-reset snapshot when present
# (systemd ExecStart points at koopa-admin-log which may be the reset suite tree).
if [ -d "$OVERLAY_DIR/host-agent" ]; then
mkdir -p "$SUITE_MON/host-agent" 2>/dev/null || true
rsync -a "$OVERLAY_DIR/host-agent/" "$SUITE_MON/host-agent/" 2>/dev/null || true
fi
if [ -d "$OVERLAY_DIR/site-gen" ]; then
mkdir -p "$SUITE_MON/site-gen"
rsync -a "$OVERLAY_DIR/site-gen/" "$SUITE_MON/site-gen/" 2>/dev/null || true
fi
unset _need_overlay
if [ -f "$ROOT/site-gen/console_to_html.py" ]; then
SITE_GEN="$ROOT/site-gen"
elif [ -f "$AGENT_MON/site-gen/console_to_html.py" ]; then
SITE_GEN="$AGENT_MON/site-gen"
echo "note: using agent site-gen at $SITE_GEN"
elif [ -f "$SUITE_MON/site-gen/console_to_html.py" ]; then
SITE_GEN="$SUITE_MON/site-gen"
else
SITE_GEN=""
fi
if [ ! -x "$ROOT/taler-monitoring.sh" ]; then
if [ -x "$AGENT_MON/taler-monitoring.sh" ]; then
ROOT="$AGENT_MON"
echo "note: using agent mon tree $ROOT"
else
echo "ERROR: suite missing at $ROOT" >&2
COMMIT="${COMMIT:-unknown}"
COMMIT_SHORT="${COMMIT_SHORT:-unknown}"
COMMIT_URL="${COMMIT_URL:-}"
fi
fi
echo "commit=${COMMIT_SHORT:-?} · ${COMMIT_URL:-}"
ec=1
if [ -x "${ROOT:-}/taler-monitoring.sh" ]; then
cd "$ROOT" || exit 1
chmod +x taler-monitoring.sh check_*.sh host-agent/*.sh site-gen/*.sh site-gen/*.py 2>/dev/null || true
# Overlay newer inside/versions host-podman bits if suite clone is stale
for f in check_inside.sh check_versions.sh taler-monitoring.sh lib.sh; do
if [ -f "$AGENT_MON/$f" ] && [ -f "$ROOT/$f" ]; then
if ! grep -q 'host-podman\|INSIDE_ACCESS\|RUN_TIMEOUT' "$ROOT/$f" 2>/dev/null \
&& grep -q 'host-podman\|INSIDE_ACCESS\|RUN_TIMEOUT' "$AGENT_MON/$f" 2>/dev/null; then
echo "note: overlay $f from agent install"
cp -f "$AGENT_MON/$f" "$ROOT/$f"
chmod +x "$ROOT/$f" 2>/dev/null || true
fi
fi
done
# shellcheck disable=SC2086
./taler-monitoring.sh -d "$TALER_DOMAIN" $PHASES
ec=$?
fi
# --- always generate HTML (first run / missing pages / every run) ---
write_bootstrap_html() {
local out="$1" host="$2" mode="$3"
mkdir -p "$(dirname "$out")"
local cshort="${COMMIT_SHORT:-unknown}"
local curl="${COMMIT_URL:-}"
local clink="$cshort"
local gen iso
gen=$(date -u +"%Y-%m-%d %H:%M:%SZ" 2>/dev/null || date)
iso=$(date -u +"%Y-%m-%dT%H:%M:%SZ" 2>/dev/null || date -Iseconds)
if [ -n "$curl" ]; then
clink="<a href=\"$(printf '%s' "$curl" | sed 's/&/\&amp;/g')\">$cshort</a>"
fi
cat >"$out" <<EOF
<!DOCTYPE html>
<html lang="en"><head><meta charset="utf-8"/>
<meta name="generated" content="${iso}"/>
<title>monitoring ${mode} · ${host}</title>
<style>
body{margin:0;background:#0c0c0c;color:#c8c8c8;font-family:ui-monospace,monospace}
a{color:#61afef}.err{color:#ff5c5c}
.sticky-bar{position:sticky;top:0;z-index:100;padding:10px 14px;background:#2a1010f2;border-bottom:2px solid #a33}
.sticky-bar .pill{font-weight:800;color:#ff5c5c;margin-right:8px}
main{padding:1.5rem}
</style></head><body>
<div class="sticky-bar" id="status-bar">
<span class="pill">BOOTSTRAP</span>
${PAGE_LABEL:-monitoring} · ${host} ·
<span data-generated-iso="${iso}">generated ${gen}</span>
· source ${clink}
</div>
<main>
<p class="err">Bootstrap page (converter missing or first install). See host-agent log.</p>
<pre style="white-space:pre-wrap;font-size:12px">$(tail -n 80 "$LOG" 2>/dev/null | sed 's/&/\&amp;/g;s/</\&lt;/g' || true)</pre>
</main>
</body></html>
EOF
echo "bootstrap html → $out"
}
htmlify_host() {
local host="$1"
local mon="$HTML_BASE/$host/${HTML_OK_DIR}/index.html"
local mon_err="$HTML_BASE/$host/${HTML_ERR_DIR}/index.html"
mkdir -p "$HTML_BASE/$host/${HTML_OK_DIR}" "$HTML_BASE/$host/${HTML_ERR_DIR}"
if [ -n "$SITE_GEN" ] && [ -f "$SITE_GEN/console_to_html.py" ]; then
python3 "$SITE_GEN/console_to_html.py" \
--log "$LOG" \
--out "$mon_err" \
--hostname "$host" \
--mode err \
--commit "${COMMIT:-}" \
--commit-url "${COMMIT_URL:-}" \
--suite-path "$SUITE_PATH" \
--page-label "$PAGE_LABEL" \
--path-err "$HTML_URL_ERR" \
--link-other "$HTML_URL_OK"
if [ "$ec" -eq 0 ]; then
python3 "$SITE_GEN/console_to_html.py" \
--log "$LOG" \
--out "$mon" \
--hostname "$host" \
--mode ok \
--commit "${COMMIT:-}" \
--commit-url "${COMMIT_URL:-}" \
--suite-path "$SUITE_PATH" \
--page-label "$PAGE_LABEL" \
--path-err "$HTML_URL_ERR" \
--link-other ""
rm -rf "$HTML_BASE/$host/${HTML_ERR_DIR}"
echo "html $host → ${HTML_URL_OK} only (clean · ${COMMIT_SHORT:-?})"
else
python3 "$SITE_GEN/console_to_html.py" \
--log "$LOG" \
--out "$mon" \
--hostname "$host" \
--mode redirect \
--commit "${COMMIT:-}" \
--commit-url "${COMMIT_URL:-}" \
--suite-path "$SUITE_PATH" \
--page-label "$PAGE_LABEL" \
--path-err "$HTML_URL_ERR"
echo "html $host → ${HTML_URL_OK} stub + ${HTML_URL_ERR} (ec=$ec · ${COMMIT_SHORT:-?})"
fi
else
echo "WARN: console_to_html.py missing — bootstrap pages"
write_bootstrap_html "$mon_err" "$host" "err"
if [ "$ec" -eq 0 ]; then
write_bootstrap_html "$mon" "$host" "ok"
rm -rf "$HTML_BASE/$host/${HTML_ERR_DIR}"
else
write_bootstrap_html "$mon" "$host" "redirect"
fi
fi
if [ ! -f "$mon" ]; then
write_bootstrap_html "$mon" "$host" "err"
fi
printf '%s\n' "${COMMIT:-unknown}" >"$HTML_BASE/$host/COMMIT"
printf '%s\n' "${COMMIT_URL:-}" >"$HTML_BASE/$host/COMMIT_URL"
}
for host in $MON_HOSTS; do
htmlify_host "$host"
done
if [ -n "$DEPLOY_WWW" ] && [ -w "$DEPLOY_WWW" ] 2>/dev/null; then
rsync -a --delete "$HTML_BASE/" "$DEPLOY_WWW/" && \
echo "synced → $DEPLOY_WWW/"
fi
ls -1t "$LOG_DIR"/run-*.log 2>/dev/null | tail -n +30 | xargs rm -f 2>/dev/null || true
echo "======== done ec=$ec · log=$LOG · ${COMMIT_URL:-} ========"
if [ "${STRICT_EXIT:-0}" = "1" ]; then
exit "$ec"
fi
exit 0

41
host-agent/run-surface-monitoring.sh Executable file
View file

@ -0,0 +1,41 @@
#!/usr/bin/env bash
# run-surface-monitoring.sh — hourly remote-only ecosystem surface scan
# Public only on taler.hacktivism.ch:
# https://taler.hacktivism.ch/taler-monitoring-surface/
# https://taler.hacktivism.ch/taler-monitoring-surface_err/
#
set -uo pipefail
AGENT_DIR=$(cd "$(dirname "$0")" && pwd)
export AGENT_LABEL="${AGENT_LABEL:-surface-host-agent}"
export STATE_NAME="${STATE_NAME:-taler-surface-monitoring}"
export TALER_DOMAIN="${TALER_DOMAIN:-hacktivism.ch}"
# surface is remote-only — force flags (ignore shared env)
export INSIDE_PODMAN=0
export LOCAL_STACK=0
export SKIP_SSH=1
export INSIDE_MODE="${INSIDE_MODE:-none}"
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
# many hosts; allow up to 45 min wall (phase also has RUN_TIMEOUT)
export RUN_TIMEOUT="${RUN_TIMEOUT:-2400}"
# dedicated job: always surface only (never inherit PHASES from shared env)
export PHASES=surface
# Only merchant public front — not bank/exchange
export MON_HOSTS="${MON_HOSTS:-taler.hacktivism.ch}"
export HTML_OUT="${HTML_OUT:-$HOME/monitoring-sites-staging}"
export DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
export HTML_OK_DIR="taler-monitoring-surface"
export HTML_ERR_DIR="taler-monitoring-surface_err"
export HTML_URL_OK="/taler-monitoring-surface/"
export HTML_URL_ERR="/taler-monitoring-surface_err/"
export PAGE_LABEL="taler-monitoring-surface"
# ecosystem catalog by default (not domain-only)
export SURFACE_SCOPE="${SURFACE_SCOPE:-ecosystem}"
export TALER_DOMAIN_FROM_CLI=0
# Do not run apt-deploy ensure for surface-only jobs
export APT_DEPLOY_ENSURE=0
exec bash "$AGENT_DIR/run-host-report.sh" "$@"

18
host-agent/taler-monitoring-aptdeploy.service Normal file
View file

@ -0,0 +1,18 @@
[Unit]
Description=Taler monitoring aptdeploy (apt-src container deploy tests)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
Environment=CONTINUE_ON_ERROR=1
Environment=RUN_TIMEOUT=1800
WorkingDirectory=%h/src/taler-monitoring
ExecStart=%h/src/taler-monitoring/host-agent/run-aptdeploy-monitoring.sh
Nice=10
TimeoutStartSec=45min
[Install]
WantedBy=default.target

14
host-agent/taler-monitoring-aptdeploy.timer Normal file
View file

@ -0,0 +1,14 @@
[Unit]
Description=Periodic taler-monitoring aptdeploy (apt-src deploy tests)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
[Timer]
OnBootSec=35min
OnUnitActiveSec=4h
AccuracySec=5min
Persistent=true
RandomizedDelaySec=10min
Unit=taler-monitoring-aptdeploy.service
[Install]
WantedBy=timers.target

19
host-agent/taler-monitoring-hacktivism.path Normal file
View file

@ -0,0 +1,19 @@
[Unit]
Description=Watch hacktivism software/config changes → re-run monitoring
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
[Path]
# Explicit stamp after package/image upgrades (touch-software-stamp.sh)
PathChanged=%h/.local/state/taler-hacktivism-monitoring/software.stamp
PathModified=%h/.local/state/taler-hacktivism-monitoring/software.stamp
# Landing + caddy config in admin-log (when checked out under ~)
PathChanged=%h/koopa-admin-log/configs/bank-landing
PathChanged=%h/koopa-admin-log/configs/exchange-landing
PathChanged=%h/koopa-admin-log/configs/merchant-landing
PathChanged=%h/koopa-admin-log/configs/caddy
# Debounce: wait for quiet period before firing
TriggerLimitIntervalSec=120
TriggerLimitBurst=3
[Install]
WantedBy=default.target

21
host-agent/taler-monitoring-hacktivism.service Normal file
View file

@ -0,0 +1,21 @@
[Unit]
Description=Taler monitoring GOA/hacktivism (host-podman inside + public urls)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
# Primary: host can podman-exec into taler-hacktivism* containers
Environment=INSIDE_PODMAN=1
Environment=CONTINUE_ON_ERROR=1
Environment=RUN_TIMEOUT=600
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
WorkingDirectory=%h/src/taler-monitoring
ExecStart=%h/src/taler-monitoring/host-agent/run-hacktivism-monitoring.sh
Nice=10
# Suite budget is RUN_TIMEOUT (10m) + git/HTML overhead
TimeoutStartSec=15min
[Install]
WantedBy=default.target

14
host-agent/taler-monitoring-hacktivism.timer Normal file
View file

@ -0,0 +1,14 @@
[Unit]
Description=Periodic hacktivism monitoring (fallback if no path events)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
[Timer]
# Every 4h + boot
OnBootSec=10min
OnUnitActiveSec=4h
AccuracySec=5min
Persistent=true
Unit=taler-monitoring-hacktivism.service
[Install]
WantedBy=timers.target

18
host-agent/taler-monitoring-surface.service Normal file
View file

@ -0,0 +1,18 @@
[Unit]
Description=Taler monitoring surface (remote ecosystem inventory)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
Environment=CONTINUE_ON_ERROR=1
Environment=RUN_TIMEOUT=2400
WorkingDirectory=%h/src/taler-monitoring
ExecStart=%h/src/taler-monitoring/host-agent/run-surface-monitoring.sh
Nice=10
TimeoutStartSec=50min
[Install]
WantedBy=default.target

14
host-agent/taler-monitoring-surface.timer Normal file
View file

@ -0,0 +1,14 @@
[Unit]
Description=Hourly taler-monitoring surface scan (ecosystem)
Documentation=file:%h/src/taler-monitoring/host-agent/README.md
[Timer]
OnBootSec=20min
OnUnitActiveSec=1h
AccuracySec=2min
Persistent=true
RandomizedDelaySec=5min
Unit=taler-monitoring-surface.service
[Install]
WantedBy=timers.target

17
host-agent/touch-software-stamp.sh Executable file
View file

@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Call after upgrading hacktivism packages / redeploying landings / image rebuilds.
# systemd path unit watches this stamp → re-runs monitoring.
set -euo pipefail
STATE="${XDG_STATE_HOME:-$HOME/.local/state}/taler-hacktivism-monitoring"
mkdir -p "$STATE"
STAMP="$STATE/software.stamp"
date -Iseconds >"$STAMP"
# also record container image ids for debugging
if command -v podman >/dev/null 2>&1; then
{
echo "# images $(date -Iseconds)"
podman images --format '{{.Repository}}:{{.Tag}} {{.ID}} {{.Digest}}' 2>/dev/null | grep -i hacktivism || true
podman ps --format '{{.Names}} {{.ImageID}} {{.Status}}' 2>/dev/null | grep -i hacktivism || true
} >"$STATE/software-images.txt" || true
fi
echo "touched $STAMP"

45
host-agent/update-suite.sh Executable file
View file

@ -0,0 +1,45 @@
#!/usr/bin/env bash
# update-suite.sh — latest taler-monitoring from Forgejo (standalone repo root).
set -uo pipefail
CFG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring"
ENV_FILE="${TALER_MONITORING_ENV:-$CFG_DIR/env}"
if [ -f "$ENV_FILE" ] && [ "${_TALER_MON_ENV_LOADED:-0}" != "1" ]; then
set -a; source "$ENV_FILE"; set +a
fi
SUITE_GIT_URL="${SUITE_GIT_URL:-https://git.hacktivism.ch/hernani/taler-monitoring.git}"
SUITE_GIT_REF="${SUITE_GIT_REF:-main}"
SUITE_UPDATE_MODE="${SUITE_UPDATE_MODE:-reset}"
if [ -z "${SUITE_DIR:-}" ]; then
if [ -d "$HOME/src/taler-monitoring/.git" ]; then SUITE_DIR="$HOME/src/taler-monitoring"
elif [ -d "$HOME/taler-monitoring/.git" ]; then SUITE_DIR="$HOME/taler-monitoring"
else SUITE_DIR="$HOME/src/taler-monitoring"; fi
fi
export SUITE_DIR
mkdir -p "$(dirname "$SUITE_DIR")" "$CFG_DIR"
if [ ! -d "$SUITE_DIR/.git" ]; then
echo "clone $SUITE_GIT_URL → $SUITE_DIR"
git clone --branch "$SUITE_GIT_REF" "$SUITE_GIT_URL" "$SUITE_DIR" || git clone "$SUITE_GIT_URL" "$SUITE_DIR"
fi
cd "$SUITE_DIR" || exit 1
git remote set-url origin "$SUITE_GIT_URL" 2>/dev/null || true
echo "fetch origin ($SUITE_GIT_REF) …"
if ! git fetch --tags origin 2>&1; then
echo "WARN: git fetch failed — using existing tree" >&2
return 0 2>/dev/null || exit 0
fi
case "$SUITE_UPDATE_MODE" in
reset)
git checkout -B "$SUITE_GIT_REF" "origin/$SUITE_GIT_REF" 2>/dev/null || git checkout -B "$SUITE_GIT_REF" FETCH_HEAD
git reset --hard "origin/$SUITE_GIT_REF" 2>/dev/null || git reset --hard FETCH_HEAD
;;
*) git pull --ff-only origin "$SUITE_GIT_REF" 2>&1 || true ;;
esac
COMMIT=$(git rev-parse HEAD)
COMMIT_SHORT=$(git rev-parse --short=12 HEAD)
SOURCE_REPO_WEB="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-monitoring}"
COMMIT_URL="${SOURCE_REPO_WEB}/src/commit/${COMMIT}"
export COMMIT COMMIT_SHORT COMMIT_URL SOURCE_REPO_WEB
echo "suite @ $COMMIT_SHORT $COMMIT_URL"
echo " dir=$SUITE_DIR"
ln -sfn "$SUITE_DIR" "$HOME/taler-monitoring" 2>/dev/null || true
chmod +x taler-monitoring.sh check_*.sh host-agent/*.sh site-gen/*.sh site-gen/*.py 2>/dev/null || true

1257
lib.sh Executable file
View file

File diff suppressed because it is too large Load diff

1431
metrics.sh Normal file
View file

File diff suppressed because it is too large Load diff

87
secrets.env.example Normal file
View file

@ -0,0 +1,87 @@
# taler-monitoring — local secrets template
#
# Copy to secrets.env (gitignored) and fill values, OR export the same vars:
# cp secrets.env.example secrets.env
# $EDITOR secrets.env
#
# Never commit secrets.env. Prefer file paths under koopa-admin-secrets
# for long-term storage; this file is for laptop one-shots / overrides.
#
# Load order: env already set > secrets.env > SECRETS_ROOT files > SSH koopa
# --- Preferred: point at secrets tree (no passwords in this file) ---
# Sibling of admin-log on this machine:
# KOOPA_ADMIN_SECRETS=$HOME/src/koopa/koopa-admin-secrets
# Or host-root only:
# SECRETS_ROOT=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root
# --- Or paste passwords / tokens (local GOA e2e) ---
# Bank admin (libeufin-bank admin) — funds / creates accounts in e2e
# E2E_BANK_ADMIN_PASS=
# BANK_ADMIN_PASS= # alias, same meaning
# Merchant instance token for private orders (goa-demo-cp4zqk on hacktivism)
# Full "secret-token:…" or raw secret (Bearer prefix added by e2e)
# E2E_MERCHANT_TOKEN=
# MERCHANT_TOKEN=
# MERCHANT_INSTANCE_PASSWORD=
# --- FrancPaysan STAGE (TESTPAYSAN) e2e ---
# Default phases for -d stage.lefrancpaysan.ch: urls + inside + versions + e2e
# inside uses francpaysan-stage-user (stagepaysan); stats from public HTTPS.
# Admin credit (required for ATM withdraw ladder on stage):
# E2E_BANK_ADMIN_PASS=… # or auto-load via SSH francpaysan-host
# FRANCPAYSAN_SSH=francpaysan-host
# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets
# Public shop pays use E2E_USE_TEMPLATES=1 (default on TESTPAYSAN) — no merchant token.
# Optional overrides
# MERCHANT_INSTANCE=goa-demo-cp4zqk
# EXPECT_CURRENCY=GOA
# E2E_TIMEOUT=340
# PROGRESS_TOTAL=120
# PROGRESS_SHOW_EVERY=8
# PROGRESS_OFF=1
# --- Merchant SPA /webui/ fingerprints (urls phase, group www.webui) ---
# EXPECT_WEBUI_VERSION=1.6.11
# EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11
# WEBUI_OVERLAY_DENY=c778af04|master-11340
# CHECK_WEBUI_SPA=0
# --- auth401 phase (WebUI signup/login/pwchange/admin + case matrix) ---
# Default FULL=1 creates throwaway MIX instance + runs all groups.
# Stage also loads durable mon401 for extra durable.* checks:
# AUTH401_FULL=1
# AUTH401_CONTINUE=1 # optional: hard prereqs soft; full ERROR list always via EXIT trap
# CONTINUE_ON_ERROR=1 # alias for AUTH401_CONTINUE
# (auth401 no longer uses set -e — silent mid-run abort fixed)
# AUTH401_CREATE=1
# AUTH401_PREFER_DURABLE=0 # 1 = only mon401, skip throwaway signup
# AUTH401_INSTANCE=mon401
# AUTH401_PASSWORD=
# AUTH401_DURABLE_ID=mon401
# AUTH401_CREATE_PASSWORD=Mon401-Test-Pass!
# AUTH401_SKIP_IF_MFA=1
# AUTH401_BEARER= # default-instance-token (not admin)
# AUTH401_ADMIN_TOKEN= # required for admin-create group
# AUTH401_ADMIN_PASSWORD= # or login as AUTH401_ADMIN_USER=admin
# AUTH401_ADMIN_USER=admin
# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets
# --- Ladder (GOA withdraw ladder · explorer pool) ---
# Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt).
# Overrides only if needed:
# EXP_USER=explorer
# EXP_PW=
# EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt
# CLI_JS=$HOME/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs
# Values that typically appear on a full stack (for orientation only):
# bank-admin-password.txt
# bank-explorer-password.txt
# bank-exchange-password.txt
# bank-goa-demo-cp4zqk-password.txt
# merchant-goa-demo-cp4zqk-password.txt
# exchange-attribute-encryption.secret.conf
# (see ../SECRETS.md and koopa-admin-secrets/MAP.md)

81
site-gen/README.md Normal file
View file

@ -0,0 +1,81 @@
# monitoring site-gen
Build **console-style HTML** from `taler-monitoring` runs and stage them for
**host Caddy** (`/monitoring`, `/monitoring_err`).
Same **global reporting defaults** as host-agents (`run-host-report.sh`):
- **`RUN_TIMEOUT=600`** on each suite run
- **line-buffered** logs (continuous write)
- **always** write HTML (first run / empty logs too)
- Forgejo **commit link** in the page header
Applies to all nine fronts (GOA + FP stage + FP prod) in the default `SITES` list.
| Host | Role |
|------|------|
| **firecuda-external** | optional runner (public DNS) |
| **koopa-external** | deploy staging + Caddy (root service) |
| **host-agents** | primary: GOA + FP on-box (`../host-agent/`) |
No secrets in `settings.conf` — only hosts, paths, site list.
## firecuda-external (outside-only) — **disabled for now**
**firecuda-external could** run public-only monitoring (`SKIP_SSH=1`, phases
`urls versions`) on a schedule (launchd every 4h) and write HTML under
`~/var/taler-monitoring-sites-work/html/`. Scripts for that still live here
(`install-firecuda-timer.sh`, `run-on-firecuda.sh`, plist template).
**Current policy: do not run a real launchd job on firecuda.**
Primary GOA monitoring is **koopa host-agent** (`host-agent/`, user `hernani`,
local `podman exec`). Re-enable firecuda only if you explicitly want a second,
outside-only view.
```bash
# optional one-shot (no timer):
# ssh firecuda-external '~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh'
# if a timer was installed earlier, ensure it is off:
# ssh firecuda-external 'launchctl list | grep taler-monitoring || echo off'
```
Disabled plists may remain as `*.plist.disabled-*` under `~/Library/LaunchAgents/`.
## Quick start (laptop)
```bash
cd site-gen
cp settings.conf.example settings.conf # optional
chmod +x generate-monitoring-sites.sh deploy-monitoring-sites.sh console_to_html.py
ONLY_HOSTS="bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch" \
SKIP_SSH=1 RUNNER_SSH= \
./generate-monitoring-sites.sh
./deploy-monitoring-sites.sh
```
Then on koopa **as root**: see **[ROOT-ON-KOOPA.md](./ROOT-ON-KOOPA.md)**.
## URL rules
| Path | When |
|------|------|
| `/monitoring_err/` | always: full console log + error list on top (CONTINUE_ON_ERROR run) |
| `/monitoring/` | **clean** full log if last strict run exit 0; else **stub** linking to `/monitoring_err/` |
Footer links **Forgejo commit** of the admin-log tree used for that run.
## Layout
```text
site-gen/
settings.conf.example
generate-monitoring-sites.sh
deploy-monitoring-sites.sh
console_to_html.py
caddy-monitoring-handles.snippet
ROOT-ON-KOOPA.md
README.md
```

92
site-gen/ROOT-ON-KOOPA.md Normal file
View file

@ -0,0 +1,92 @@
# Root on koopa (hacktivism monitoring sites)
Host Caddy runs as **root/systemd** (`caddy` user). Static monitoring HTML is
**not** put into Taler containers — only host paths + Caddy `handle`.
After `generate-monitoring-sites.sh` + `deploy-monitoring-sites.sh` (as hernani),
files live in:
```text
/home/hernani/monitoring-sites-staging/<hostname>/monitoring/index.html
/home/hernani/monitoring-sites-staging/<hostname>/monitoring_err/index.html
```
## One-shot as root (copy + Caddy)
```bash
# on koopa (koopa-external), as root:
# 1) web root
install -d -o caddy -g caddy -m 755 /var/www/monitoring-sites
rsync -a --delete /home/hernani/monitoring-sites-staging/ /var/www/monitoring-sites/
chown -R caddy:caddy /var/www/monitoring-sites
# 2) Caddyfile — merge handles from admin-log mirror, then:
# (either edit /etc/caddy/Caddyfile by hand using snippet below,
# or copy full mirror after review)
#
# install -m 644 /home/hernani/koopa-admin-log/configs/caddy/Caddyfile /etc/caddy/Caddyfile
# # or: ~/koopa-caddy/apply.sh after syncing Caddyfile into ~/koopa-caddy/
caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy
systemctl is-active caddy
```
## Snippet per hacktivism site (inside each `*.hacktivism.ch { }` block)
Place **before** the catch-all `reverse_proxy` (same idea as `/intro*`):
```caddy
# Public taler-monitoring console HTML (static; host only)
handle /monitoring_err* {
root * /var/www/monitoring-sites/{host}
rewrite * /monitoring_err{uri}
# uri is /monitoring_err or /monitoring_err/ → serve directory index
file_server
}
handle /monitoring* {
root * /var/www/monitoring-sites/{host}
file_server
}
```
Simpler layout (recommended): path on disk matches URL under host root:
```text
/var/www/monitoring-sites/bank.hacktivism.ch/monitoring/index.html
/var/www/monitoring-sites/bank.hacktivism.ch/monitoring_err/index.html
```
```caddy
handle_path /monitoring_err/* {
root * /var/www/monitoring-sites/{host}/monitoring_err
file_server
}
handle /monitoring_err {
redir /monitoring_err/ 302
}
handle_path /monitoring/* {
root * /var/www/monitoring-sites/{host}/monitoring
file_server
}
handle /monitoring {
redir /monitoring/ 302
}
```
Canonical snippet is also in `caddy-monitoring-handles.snippet` and applied
in `configs/caddy/Caddyfile` for the three GOA hosts.
## Check
```bash
curl -sS -o /dev/null -w '%{http_code}\n' https://bank.hacktivism.ch/monitoring/
curl -sS -o /dev/null -w '%{http_code}\n' https://bank.hacktivism.ch/monitoring_err/
curl -sS https://bank.hacktivism.ch/monitoring_err/ | head
```
## Not needed as root
- Running `taler-monitoring` (use **firecuda-external** / laptop)
- Writing into podman Taler containers

57
site-gen/caddy-monitoring-handles.snippet Normal file
View file

@ -0,0 +1,57 @@
# Host Caddy on koopa (root service). Paste BEFORE catch-all reverse_proxy.
#
# Disk layout:
# /var/www/monitoring-sites/{host}/monitoring/index.html
# /var/www/monitoring-sites/{host}/monitoring_err/index.html
# /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface*/index.html
# /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy*/index.html
#
# /monitoring* → bank + exchange + taler.hacktivism.ch
# /taler-monitoring-surface* → taler.hacktivism.ch only
# /taler-monitoring-aptdeploy* → taler.hacktivism.ch only
# --- only inside taler.hacktivism.ch { ... } ---
handle /taler-monitoring-surface_err {
redir /taler-monitoring-surface_err/ 302
}
handle /taler-monitoring-surface_err/ {
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface_err
file_server
}
handle /taler-monitoring-surface {
redir /taler-monitoring-surface/ 302
}
handle /taler-monitoring-surface/ {
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-surface
file_server
}
handle /taler-monitoring-aptdeploy_err {
redir /taler-monitoring-aptdeploy_err/ 302
}
handle /taler-monitoring-aptdeploy_err/ {
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy_err
file_server
}
handle /taler-monitoring-aptdeploy {
redir /taler-monitoring-aptdeploy/ 302
}
handle /taler-monitoring-aptdeploy/ {
root * /var/www/monitoring-sites/taler.hacktivism.ch/taler-monitoring-aptdeploy
file_server
}
# --- bank + exchange + taler (each site block) ---
handle /monitoring_err {
redir /monitoring_err/ 302
}
handle /monitoring_err/ {
root * /var/www/monitoring-sites/{host}/monitoring_err
file_server
}
handle /monitoring {
redir /monitoring/ 302
}
handle /monitoring/ {
root * /var/www/monitoring-sites/{host}/monitoring
file_server
}

34
site-gen/com.hacktivism.taler-monitoring-sites.plist Normal file
View file

@ -0,0 +1,34 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!--
launchd user agent — firecuda (max): every 4 hours
Install: ./install-firecuda-timer.sh
-->
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.hacktivism.taler-monitoring-sites</string>
<key>ProgramArguments</key>
<array>
<string>/bin/bash</string>
<string>__SITE_GEN__/run-on-firecuda.sh</string>
</array>
<key>StartInterval</key>
<integer>14400</integer>
<key>RunAtLoad</key>
<true/>
<key>StandardOutPath</key>
<string>__HOME__/Library/Logs/taler-monitoring-sites/launchd.out.log</string>
<key>StandardErrorPath</key>
<string>__HOME__/Library/Logs/taler-monitoring-sites/launchd.err.log</string>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
<key>SKIP_SSH</key>
<string>1</string>
<key>CONTINUE_ON_ERROR</key>
<string>1</string>
</dict>
</dict>
</plist>

935
site-gen/console_to_html.py Executable file
View file

@ -0,0 +1,935 @@
#!/usr/bin/env python3
"""Convert taler-monitoring console log to a console-feel HTML page."""
from __future__ import annotations
import argparse
import html
import re
from datetime import datetime, timezone
from pathlib import Path
# ANSI strip
ANSI_RE = re.compile(r"\x1b\[[0-9;]*m")
ERR_BULLET_RE = re.compile(r"^\s*[•·*-]\s+(?P<body>.+)$")
TID_RE = re.compile(r"\b(?P<tid>[a-z0-9_.-]+-\d+)\b", re.I)
def strip_ansi(s: str) -> str:
return ANSI_RE.sub("", s)
def classify(line: str) -> str:
u = line.upper()
# section / summary headers before badge matching (they contain the word ERROR)
if "SUMMARY" in u or "--- ERRORS" in u or "ERRORS ·" in u or "ERRORS (" in u:
return "meta"
if "numbered checks" in line.lower() or "totals:" in line.lower():
return "meta"
if "BLOCKER" in u or "┌ BLOCKER" in u:
return "blocker"
if "ERROR" in u or "┌ ERROR" in u:
return "error"
if "WARN" in u or "┌ WARN" in u:
return "warn"
if "INFO" in u or "┌ INFO" in u:
return "info"
if " OK" in u or "[OK" in u or "┌ OK" in u or u.strip().startswith("OK"):
return "ok"
if u.strip().startswith("-- ") or u.strip().startswith("=="):
return "section"
return "plain"
def is_run_timeout_text(text: str) -> bool:
u = text.upper()
return "RUN.TIMEOUT" in u or "RUN_TIMEOUT" in u or "RUN TIMEOUT" in u
def is_summary_error_line(ln: str) -> bool:
"""Skip summary/header lines that contain ERROR but are not checks."""
low = ln.lower()
if "failed — see" in low or "failed - see" in low:
return True
if "errors · failed" in low or "errors (failed" in low:
return True
if "--- errors" in low:
return True
if "┌ summary" in low or "[ summary" in low:
return True
if "totals:" in low:
return True
# header-ish RUN_TIMEOUT= without error badge
if is_run_timeout_text(ln) and "ERROR" not in ln.upper() and "run.timeout" not in low:
return True
return False
def extract_errors(lines: list[str]) -> list[str]:
errs: list[str] = []
timeout_detail = ""
in_block = False
for ln in lines:
if "--- ERRORS" in ln or "ERRORS (failed" in ln or "RUN TIMEOUT · extraordinary" in ln:
in_block = True
continue
if in_block:
if ln.strip().startswith("---") or "[ SUMMARY" in ln or "totals:" in ln:
in_block = False
continue
m = ERR_BULLET_RE.match(ln)
if m:
body = m.group("body").strip()
if is_run_timeout_text(body):
timeout_detail = body
else:
errs.append(body)
elif "ERROR" in ln.upper() and ln.strip() and not is_run_timeout_text(ln):
errs.append(ln.strip())
if re.search(r"\bERROR\b", ln, re.I) and "--- ERRORS" not in ln:
if is_run_timeout_text(ln):
if "run.timeout-01" in ln.lower() or "RUN_TIMEOUT exceeded" in ln:
timeout_detail = re.sub(r"\s+", " ", strip_ansi(ln)).strip()
continue
if is_summary_error_line(ln):
continue
body = re.sub(r"^.*\bERROR\b\s*\]?\s*", "", ln, flags=re.I).strip(" ·")
if body and body not in errs and "failed — see" not in body.lower():
if re.search(
r"#\d+|www\.|e2e\.|auth401\.|versions\.|inside\.|surface\.|aptdeploy\.",
body,
):
errs.append(body)
seen: set[str] = set()
out: list[str] = []
for e in errs:
if e in seen or is_run_timeout_text(e):
continue
seen.add(e)
out.append(e)
if timeout_detail or any(
is_run_timeout_text(l) and ("ERROR" in l.upper() or "run.timeout" in l.lower())
for l in lines
):
# only if real timeout ERROR badge, not mere RUN_TIMEOUT= header
if timeout_detail or any(
"run.timeout-01" in l.lower() or "RUN_TIMEOUT exceeded" in l for l in lines
):
canon = timeout_detail or "run.timeout-01 [run] RUN_TIMEOUT exceeded"
out = [canon] + out
return out
def count_status(lines: list[str]) -> tuple[int, int, int | None, int | None]:
"""Return (error_count, warn_count, first_error_idx, first_warn_idx)."""
n_err = 0
n_warn = 0
first_err: int | None = None
first_warn: int | None = None
for i, ln in enumerate(lines):
kind = classify(ln)
if kind in ("error", "blocker"):
if is_summary_error_line(ln):
continue
n_err += 1
if first_err is None:
first_err = i
elif kind == "warn":
n_warn += 1
if first_warn is None:
first_warn = i
return n_err, n_warn, first_err, first_warn
def slug_error(i: int, text: str) -> str:
if is_run_timeout_text(text):
return "err-run-timeout"
m = TID_RE.search(text)
if m:
return "err-" + re.sub(r"[^a-z0-9_-]+", "-", m.group("tid").lower())
return f"err-{i:03d}"
def render_line(ln: str, err_slugs: dict[str, str], extra_ids: list[str] | None = None) -> str:
kind = classify(ln)
esc = html.escape(ln)
for tid, slug in err_slugs.items():
if tid in ln:
esc = esc.replace(
html.escape(tid),
f'<a class="jump" href="#{html.escape(slug)}">{html.escape(tid)}</a>',
)
id_attr = ""
if extra_ids:
id_attr = f' id="{" ".join(extra_ids)}"' # invalid multi-id; use first only
# HTML allows one id — join carefully
id_attr = f' id="{html.escape(extra_ids[0])}"'
if len(extra_ids) > 1:
# secondary anchors as empty spans prepended
spans = "".join(f'<span id="{html.escape(x)}"></span>' for x in extra_ids[1:])
return f'{spans}<div{id_attr} class="line {kind}">{esc}</div>\n'
return f'<div{id_attr} class="line {kind}">{esc}</div>\n'
def status_level(n_err: int, n_warn: int) -> str:
if n_err > 0:
return "red"
if n_warn > 0:
return "yellow"
return "green"
def extract_phases(log_text: str) -> str:
"""Best-effort phases string from host-agent / taler-monitoring log header."""
for pat in (
r"phases=([a-z0-9 _.-]+)",
r"· phases=([a-z0-9 _.-]+)",
r"phases\s{2,}([a-z0-9 _.-]+)",
r"^\s*phases\s+([a-z0-9 _.-]+)\s*$",
):
m = re.search(pat, log_text, re.I | re.M)
if m:
return re.sub(r"\s+", " ", m.group(1)).strip(" ·")
return ""
def monitoring_scope(
page_label: str,
hostname: str,
log_text: str = "",
) -> dict[str, object]:
"""
Human-readable description of what this monitoring page covers.
Returned keys: title, summary, items (list[str]), kind
"""
label = (page_label or "monitoring").lower().replace("_", "-")
phases = extract_phases(log_text)
host = hostname or "?"
if "surface" in label:
return {
"kind": "surface",
"title": "Remote ecosystem surface inventory",
"summary": "Public catalog hosts · DNS / TCP / HTTPS / TLS · Server version · CVE",
"items": [
"Outside-in only: no SSH and no podman exec on remote targets",
"Catalog (surface-catalog.conf): taler.net, demo/test, gnunet.org, "
"taler-ops.ch, taler-systems.com, deb.taler.net, ftp.gnu.org, …",
"Per host: DNS resolve, ICMP (info), open ports, protocol probes, "
"TLS cert expiry, Server-header software version, optional CVE check",
f"HTML published only on taler.hacktivism.ch "
f"(this page host: {host})",
"Timer: taler-monitoring-surface.timer (hourly)",
],
}
if "aptdeploy" in label or "apt-deploy" in label or "apt_src" in label:
return {
"kind": "aptdeploy",
"title": "apt-src merchant deploy tests (podman on koopa)",
"summary": "4 containers · fresh + upgrade tracks · trixie & trixie-testing",
"items": [
"koopa-taler-deploy-test-apt-src-trixie (fresh, suite trixie)",
"koopa-taler-deploy-test-apt-src-trixie-testing (fresh, trixie-testing)",
"koopa-taler-deploy-test-apt-src-trixie-upgrade (upgrade track)",
"koopa-taler-deploy-test-apt-src-trixie-testing-upgrade (upgrade track)",
"Checks: packages, taler-merchant-httpd --version / ldd, systemd unit, "
"optional local /config probe — not public nginx/HTTPS",
f"HTML published only on taler.hacktivism.ch (this page host: {host})",
"Timer: taler-monitoring-aptdeploy.timer (4h)",
],
}
# Default: stack /monitoring pages (bank, exchange, merchant)
phase_txt = phases or "urls · inside · versions"
role = "stack"
if host.startswith("bank."):
role = "bank"
focus = "Libeufin bank public HTTPS + container inside checks"
elif host.startswith("exchange.") or host.startswith("stage.exchange."):
role = "exchange"
focus = "Exchange public HTTPS + container inside checks"
elif host.startswith("taler.") or "merchant" in host or host.startswith("monnaie."):
role = "merchant"
focus = "Merchant backend / SPA public HTTPS + container inside checks"
else:
focus = "Taler stack public + inside checks"
items = [
f"Focus: {focus}",
f"Phases this run: {phase_txt}",
"Typical: public URLs (HTTPS, QR, perf), inside (podman/ssh), package versions",
f"HTML host: {host}",
"Timer: taler-monitoring-hacktivism (path + 4h) · /monitoring on bank, exchange, taler",
]
if phases:
items.insert(2, f"Log header phases= {phases}")
return {
"kind": role,
"title": f"Stack monitoring · {host}",
"summary": f"{phase_txt} · public + inside",
"items": items,
}
def sticky_bar_html(
*,
generated_at: str,
generated_iso: str,
n_err: int,
n_warn: int,
hostname: str,
page_label: str,
mode: str,
commit_html: str,
suite_path: str,
link_other: str | None,
first_err_href: str | None,
first_warn_href: str | None,
scope: dict[str, object] | None = None,
) -> str:
level = status_level(n_err, n_warn)
if level == "red":
status_txt = "ERRORS"
elif level == "yellow":
status_txt = "WARNINGS"
else:
status_txt = "OK"
if first_err_href and n_err:
err_stat = (
f'<a class="stat err" href="{html.escape(first_err_href)}" '
f'title="Jump to first error">{n_err} error{"s" if n_err != 1 else ""}</a>'
)
else:
err_stat = f'<span class="stat err muted">{n_err} errors</span>'
if first_warn_href and n_warn:
warn_stat = (
f'<a class="stat warn" href="{html.escape(first_warn_href)}" '
f'title="Jump to first warning">{n_warn} warning{"s" if n_warn != 1 else ""}</a>'
)
else:
warn_stat = f'<span class="stat warn muted">{n_warn} warnings</span>'
other = ""
if link_other:
other = (
f'<a class="nav-link" href="{html.escape(link_other)}">'
f"{html.escape(link_other)}</a>"
)
if scope is None:
scope = monitoring_scope(page_label, hostname, "")
scope_title = html.escape(str(scope.get("title") or "Monitoring"))
scope_summary = html.escape(str(scope.get("summary") or ""))
scope_kind = html.escape(str(scope.get("kind") or "monitoring"))
lis = "\n".join(
f"<li>{html.escape(str(it))}</li>" for it in (scope.get("items") or [])
)
suite_bit = (
f" · <code>{html.escape(suite_path)}</code>" if suite_path else ""
)
# Compact sticky row always visible; "Was geprüft wird" expands inside sticky-bar
return f"""
<div class="sticky-bar sticky-{level}" id="status-bar" role="status"
data-errors="{n_err}" data-warnings="{n_warn}" data-level="{level}"
data-scope-kind="{scope_kind}">
<div class="sticky-bar-row primary">
<span class="status-pill sticky-{level}">{html.escape(status_txt)}</span>
<span class="host">{html.escape(page_label)} · {html.escape(hostname)}</span>
<span class="sep">·</span>
{err_stat}
<span class="sep">·</span>
{warn_stat}
<span class="sep">·</span>
<span class="generated"
data-generated-iso="{html.escape(generated_iso)}"
title="{html.escape(generated_at)}">
generated <time datetime="{html.escape(generated_iso)}">{html.escape(generated_at)}</time>
<span class="age" id="generated-age"></span>
</span>
<button type="button" class="scope-toggle" id="scope-toggle"
aria-expanded="false" aria-controls="sticky-scope-panel"
title="Aufklappen: was diese Seite prüft">
<span class="scope-chevron" aria-hidden="true"></span>
<span class="scope-toggle-label">Was geprüft wird</span>
<span class="scope-toggle-hint">{scope_summary}</span>
</button>
</div>
<div class="sticky-scope-panel" id="sticky-scope-panel" hidden>
<div class="sticky-bar-row secondary">
<span class="badge mode-{html.escape(mode)}">{html.escape(mode.upper())}</span>
source {commit_html}{suite_bit}
{(" · " + other) if other else ""}
</div>
<div class="scope-body">
<h3 class="scope-title">{scope_title}</h3>
<p class="scope-lead">{scope_summary}</p>
<ul class="scope-list">
{lis}
</ul>
</div>
</div>
</div>
"""
STICKY_CSS = """
.sticky-bar {
position: sticky; top: 0; z-index: 100;
border-bottom: 2px solid var(--border);
padding: 10px 14px 8px;
backdrop-filter: blur(8px);
-webkit-backdrop-filter: blur(8px);
box-shadow: 0 2px 12px #0008;
}
.sticky-bar.sticky-green {
background: linear-gradient(180deg, #0d1f14f2, #0a1410ee);
border-bottom-color: #1f6b3a;
}
.sticky-bar.sticky-yellow {
background: linear-gradient(180deg, #2a2210f2, #1a160cee);
border-bottom-color: #a68b2d;
}
.sticky-bar.sticky-red {
background: linear-gradient(180deg, #2a1010f2, #1a0808ee);
border-bottom-color: #a33;
}
.sticky-bar-row {
display: flex; flex-wrap: wrap; align-items: center; gap: 6px 10px;
max-width: 1200px;
}
.sticky-bar-row.secondary {
margin-top: 8px; font-size: 12px; color: var(--dim);
}
.sticky-bar-row.secondary a { color: var(--info); text-decoration: none; }
.sticky-bar-row.secondary a:hover { text-decoration: underline; }
.status-pill {
display: inline-block; font-size: 11px; font-weight: 800;
letter-spacing: 0.06em; padding: 2px 10px; border-radius: 3px;
border: 1px solid;
}
.status-pill.sticky-green { color: var(--ok); border-color: #1f6b3a; background: #0a1a10; }
.status-pill.sticky-yellow { color: var(--warn); border-color: #a68b2d; background: #1a1608; }
.status-pill.sticky-red { color: var(--err); border-color: #a33; background: #1a0a0a; }
.host { font-weight: 600; color: #eee; font-size: 13px; }
.sep { color: var(--dim); }
.stat {
font-weight: 700; font-size: 12px; text-decoration: none;
padding: 1px 8px; border-radius: 3px; border: 1px solid transparent;
}
.stat.err { color: var(--err); border-color: #522; background: #1a0a0a; }
.stat.warn { color: var(--warn); border-color: #664; background: #1a1608; }
.stat.muted { opacity: 0.55; font-weight: 600; }
a.stat:hover { text-decoration: underline; filter: brightness(1.15); }
.generated { color: var(--dim); font-size: 12px; }
.generated time { color: #bbb; }
.generated .age { color: #888; margin-left: 4px; }
.generated .age:not(:empty)::before { content: "("; }
.generated .age:not(:empty)::after { content: ")"; }
.badge {
display: inline-block; padding: 1px 8px; border-radius: 3px;
font-size: 11px; font-weight: 700; margin-right: 4px;
border: 1px solid var(--border);
}
.badge.mode-err { color: var(--err); border-color: #522; background: #1a0a0a; }
.badge.mode-ok { color: var(--ok); border-color: #143; background: #0a1a10; }
.badge.mode-redirect { color: var(--warn); border-color: #664; background: #1a1608; }
/* collapsible block inside sticky-bar */
.scope-toggle {
margin-left: auto;
display: inline-flex;
align-items: center;
gap: 8px;
max-width: min(42rem, 100%);
cursor: pointer;
font: inherit;
font-size: 12px;
color: #ddd;
background: #0006;
border: 1px solid #444;
border-radius: 4px;
padding: 4px 10px;
text-align: left;
}
.scope-toggle:hover { border-color: #888; background: #111a; color: #fff; }
.scope-toggle:focus-visible { outline: 2px solid var(--info); outline-offset: 2px; }
.scope-toggle[aria-expanded="true"] {
border-color: #777;
background: #141414;
}
.scope-chevron {
display: inline-block;
width: 0; height: 0;
border-left: 5px solid #aaa;
border-top: 4px solid transparent;
border-bottom: 4px solid transparent;
transition: transform 0.12s ease;
flex-shrink: 0;
}
.scope-toggle[aria-expanded="true"] .scope-chevron {
transform: rotate(90deg);
border-left-color: #eee;
}
.scope-toggle-label {
font-weight: 700;
letter-spacing: 0.03em;
text-transform: uppercase;
font-size: 11px;
white-space: nowrap;
}
.scope-toggle-hint {
color: #888;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
min-width: 0;
}
.sticky-scope-panel {
margin-top: 8px;
max-width: 1200px;
border: 1px solid #333;
border-radius: 4px;
background: #080808e6;
padding: 8px 10px 10px;
}
.sticky-scope-panel[hidden] { display: none !important; }
.scope-body { padding: 4px 4px 2px 6px; }
.scope-title {
margin: 4px 0 4px;
font-size: 13px;
font-weight: 600;
color: #eee;
}
.scope-lead {
margin: 0 0 6px;
font-size: 12px;
color: #999;
}
.scope-list {
margin: 0;
padding-left: 1.15em;
color: #b0b0b0;
font-size: 12px;
line-height: 1.5;
}
.scope-list li { margin: 3px 0; }
#first-error, #first-warn,
[id^="err-"][id$="-line"] {
scroll-margin-top: 120px;
}
@media (max-width: 720px) {
.scope-toggle { margin-left: 0; width: 100%; }
.scope-toggle-hint { display: none; }
}
"""
STICKY_JS = """
<script>
(function () {
function fmtAge(sec) {
if (sec < 0) sec = 0;
if (sec < 60) return sec + "s ago";
if (sec < 3600) return Math.floor(sec / 60) + "m ago";
if (sec < 86400) return Math.floor(sec / 3600) + "h ago";
return Math.floor(sec / 86400) + "d ago";
}
function tick() {
var el = document.querySelector("[data-generated-iso]");
var age = document.getElementById("generated-age");
if (!el || !age) return;
var iso = el.getAttribute("data-generated-iso");
if (!iso) return;
var t = Date.parse(iso);
if (isNaN(t)) return;
var sec = Math.floor((Date.now() - t) / 1000);
age.textContent = fmtAge(sec);
age.title = "page age · updates every second";
}
tick();
setInterval(tick, 1000);
// Sticky-bar collapsible: what this page monitors
var btn = document.getElementById("scope-toggle");
var panel = document.getElementById("sticky-scope-panel");
if (btn && panel) {
var key = "taler-mon-scope-open:" + (location.pathname || "");
function setOpen(open) {
btn.setAttribute("aria-expanded", open ? "true" : "false");
if (open) panel.removeAttribute("hidden");
else panel.setAttribute("hidden", "");
try { localStorage.setItem(key, open ? "1" : "0"); } catch (e) {}
}
btn.addEventListener("click", function () {
setOpen(btn.getAttribute("aria-expanded") !== "true");
});
try {
if (localStorage.getItem(key) === "1") setOpen(true);
} catch (e) {}
}
})();
</script>
"""
def build_html(
*,
title: str,
hostname: str,
mode: str,
log_text: str,
commit: str,
commit_url: str,
suite_path: str,
generated_at: str,
generated_iso: str,
link_other: str | None,
page_label: str = "monitoring",
) -> str:
raw_lines = [strip_ansi(l).rstrip("\n") for l in log_text.splitlines()]
while raw_lines and not raw_lines[-1].strip():
raw_lines.pop()
n_err, n_warn, first_err_i, first_warn_i = count_status(raw_lines)
errors = extract_errors(raw_lines)
err_slugs: dict[str, str] = {}
for i, e in enumerate(errors, 1):
slug = slug_error(i, e)
err_slugs[e] = slug
m = TID_RE.search(e)
if m:
err_slugs[m.group("tid")] = slug
tid_to_slug = {
k: v for k, v in err_slugs.items() if re.match(r"^[a-z0-9_.-]+-\d+$", k, re.I)
}
# Build body with anchors for first error/warn + per-error slugs
body_lines: list[str] = []
claimed_err: set[str] = set()
for idx, ln in enumerate(raw_lines):
extra: list[str] = []
if first_err_i is not None and idx == first_err_i:
extra.append("first-error")
if first_warn_i is not None and idx == first_warn_i:
extra.append("first-warn")
# per-error jump targets (all modes)
if classify(ln) in ("error", "blocker") and not is_summary_error_line(ln):
for i, e in enumerate(errors, 1):
slug = slug_error(i, e)
if slug in claimed_err:
continue
tid_m = TID_RE.search(e)
if is_run_timeout_text(e):
if "run.timeout-01" in ln and "ERROR" in ln.upper():
extra.append(f"{slug}-line")
claimed_err.add(slug)
break
continue
key = tid_m.group(1) if tid_m else e[:40]
if key and key in ln:
extra.append(f"{slug}-line")
claimed_err.add(slug)
break
body_lines.append(render_line(ln, tid_to_slug, extra or None))
err_nav = ""
if errors and mode == "err":
items = []
timeout_items = []
for i, e in enumerate(errors, 1):
slug = slug_error(i, e)
li = (
f'<li id="{html.escape(slug)}">'
f'<a href="#{html.escape(slug)}-line">{html.escape(e)}</a></li>'
)
if is_run_timeout_text(e):
timeout_items.append(li)
else:
items.append(li)
if timeout_items:
err_nav += (
'<section class="err-top extraordinary" id="run-timeout-top">\n'
"<h2>EXTRAORDINARY · RUN TIMEOUT</h2>\n"
"<p>Whole-run wall clock exceeded. "
'<a href="#err-run-timeout-line">Jump to detail →</a></p>\n'
'<ol class="err-list">\n'
+ "\n".join(timeout_items[:1])
+ "\n</ol>\n</section>\n"
)
rest = items if timeout_items else (timeout_items + items)
if rest or not timeout_items:
err_nav += (
'<section class="err-top">\n'
f"<h2>ERRORS ({len(errors)}) — jump to detail</h2>\n"
'<ol class="err-list">\n'
+ "\n".join(rest if rest else items)
+ "\n</ol>\n</section>\n"
)
commit_short = commit[:12] if commit else "unknown"
commit_html = (
f'<a href="{html.escape(commit_url)}">{html.escape(commit_short)}</a>'
if commit_url
else html.escape(commit_short)
)
first_err_href = "#first-error" if first_err_i is not None else None
first_warn_href = "#first-warn" if first_warn_i is not None else None
scope = monitoring_scope(page_label, hostname, log_text)
bar = sticky_bar_html(
generated_at=generated_at,
generated_iso=generated_iso,
n_err=n_err,
n_warn=n_warn,
hostname=hostname,
page_label=page_label,
mode=mode,
commit_html=commit_html,
suite_path=suite_path,
link_other=link_other,
first_err_href=first_err_href,
first_warn_href=first_warn_href,
scope=scope,
)
return f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="generated" content="{html.escape(generated_iso)}"/>
<title>{html.escape(title)}</title>
<style>
:root {{
--bg: #0c0c0c;
--fg: #c8c8c8;
--dim: #666;
--ok: #3ddc84;
--err: #ff5c5c;
--warn: #e6c07b;
--info: #61afef;
--blocker: #c678dd;
--sec: #56b6c2;
--border: #222;
--panel: #121212;
font-family: "JetBrains Mono", "Fira Code", "Cascadia Code", ui-monospace, Menlo, Consolas, monospace;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0; background: var(--bg); color: var(--fg);
font-size: 13px; line-height: 1.45;
}}
{STICKY_CSS}
main {{ padding: 12px 14px 48px; max-width: 1200px; }}
.err-top {{
background: var(--panel); border: 1px solid #422;
padding: 10px 12px; margin-bottom: 14px; border-radius: 4px;
}}
.err-top.extraordinary {{
border-color: #a44; background: #1a0808;
box-shadow: 0 0 0 1px #622 inset;
}}
.err-top.extraordinary h2 {{ color: #ff8a8a; }}
.err-top.extraordinary p {{ margin: 0 0 8px; color: var(--fg); font-size: 12px; }}
.err-top.extraordinary p a {{ color: #ff8a8a; font-weight: 600; }}
.err-top h2 {{ margin: 0 0 8px; font-size: 13px; color: var(--err); }}
.err-list {{ margin: 0; padding-left: 1.2em; }}
.err-list li {{ margin: 4px 0; }}
.err-list a {{ color: var(--err); text-decoration: none; }}
.err-list a:hover {{ text-decoration: underline; }}
#err-run-timeout-line {{
outline: 1px solid #a44; background: #1a0808;
padding: 4px 6px; margin: 4px 0; border-radius: 3px;
}}
.console {{
background: #0a0a0a; border: 1px solid var(--border);
border-radius: 4px; padding: 8px 10px;
white-space: pre-wrap; word-break: break-word;
}}
.line {{ padding: 1px 0; }}
.line.ok {{ color: var(--ok); }}
.line.error {{ color: var(--err); }}
.line.warn {{ color: var(--warn); }}
.line.info {{ color: var(--info); }}
.line.blocker {{ color: var(--blocker); }}
.line.section {{ color: var(--sec); margin-top: 8px; font-weight: 600; }}
.line.meta {{ color: var(--dim); }}
.line.plain {{ color: var(--fg); }}
a.jump {{ color: inherit; text-decoration: underline dotted; }}
footer {{
margin-top: 18px; color: var(--dim); font-size: 12px;
border-top: 1px solid var(--border); padding-top: 10px;
}}
</style>
</head>
<body>
{bar}
<main>
{err_nav}
<div class="console">
{"".join(body_lines)}
</div>
<footer>
Console-style render of taler-monitoring output.
Sticky bar: green = clean · yellow = warnings · red = errors.
Commit pins the exact tree used for this run.
</footer>
</main>
{STICKY_JS}
</body>
</html>
"""
def build_redirect_html(
*,
hostname: str,
commit: str,
commit_url: str,
path_err: str = "/monitoring_err/",
page_label: str = "monitoring",
generated_at: str = "",
generated_iso: str = "",
n_err: int = 0,
n_warn: int = 0,
log_text: str = "",
) -> str:
commit_short = commit[:12] if commit else "unknown"
link = path_err if path_err.endswith("/") else path_err + "/"
c = (
f'<a href="{html.escape(commit_url)}">{html.escape(commit_short)}</a>'
if commit_url
else html.escape(commit_short)
)
if not generated_at:
now = datetime.now(timezone.utc)
generated_iso = now.strftime("%Y-%m-%dT%H:%M:%SZ")
generated_at = now.strftime("%Y-%m-%d %H:%M:%SZ")
# Redirect pages exist only when the run failed → treat as red if n_err unknown
if n_err <= 0:
n_err = max(n_err, 1)
scope = monitoring_scope(page_label, hostname, log_text)
bar = sticky_bar_html(
generated_at=generated_at,
generated_iso=generated_iso,
n_err=n_err,
n_warn=n_warn,
hostname=hostname,
page_label=page_label,
mode="redirect",
commit_html=c,
suite_path="",
link_other=link,
first_err_href=link,
first_warn_href=None,
scope=scope,
)
return f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta http-equiv="refresh" content="2; url={html.escape(link)}"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="generated" content="{html.escape(generated_iso)}"/>
<title>{html.escape(page_label)} · errors · {html.escape(hostname)}</title>
<style>
:root {{
--bg: #0c0c0c; --fg: #c8c8c8; --dim: #666;
--ok: #3ddc84; --err: #ff5c5c; --warn: #e6c07b; --info: #61afef; --border: #222;
font-family: ui-monospace, Menlo, Consolas, monospace;
}}
body {{ margin: 0; background: var(--bg); color: var(--fg); font-size: 13px; }}
{STICKY_CSS}
.box {{
max-width: 520px; margin: 12vh auto; padding: 20px;
border: 1px solid #422; background: #121212; border-radius: 4px;
}}
.box a {{ color: var(--err); }}
</style>
</head>
<body>
{bar}
<div class="box">
<p><strong>{html.escape(page_label)}</strong> has failures.</p>
<p>See <a href="{html.escape(link)}">{html.escape(link)}</a> for the full console log,
error index, and sticky status bar.</p>
<p style="color:#666;font-size:12px">generated {html.escape(generated_at)} · source {c}</p>
</div>
{STICKY_JS}
</body>
</html>
"""
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--log", required=True, type=Path)
ap.add_argument("--out", required=True, type=Path)
ap.add_argument("--hostname", required=True)
ap.add_argument("--mode", choices=("ok", "err", "redirect"), required=True)
ap.add_argument("--commit", default="")
ap.add_argument("--commit-url", default="")
ap.add_argument("--suite-path", default=".")
ap.add_argument("--title", default="")
ap.add_argument("--link-other", default="")
ap.add_argument(
"--path-err",
default="/monitoring_err/",
help="URL path for error page (redirect target)",
)
ap.add_argument(
"--page-label",
default="monitoring",
help="Short name shown in titles/redirect box",
)
args = ap.parse_args()
log_text = args.log.read_text(errors="replace") if args.log.is_file() else ""
now = datetime.now(timezone.utc)
generated_iso = now.strftime("%Y-%m-%dT%H:%M:%SZ")
generated_at = now.strftime("%Y-%m-%d %H:%M:%SZ")
title = args.title or f"{args.page_label} {args.mode} · {args.hostname}"
raw_for_counts = [strip_ansi(l).rstrip("\n") for l in log_text.splitlines()]
n_err, n_warn, _, _ = count_status(raw_for_counts)
if args.mode == "redirect":
html_out = build_redirect_html(
hostname=args.hostname,
commit=args.commit,
commit_url=args.commit_url,
path_err=args.path_err,
page_label=args.page_label,
generated_at=generated_at,
generated_iso=generated_iso,
n_err=n_err,
n_warn=n_warn,
log_text=log_text,
)
else:
html_out = build_html(
title=title,
hostname=args.hostname,
mode=args.mode,
log_text=log_text,
commit=args.commit,
commit_url=args.commit_url,
suite_path=args.suite_path,
generated_at=generated_at,
generated_iso=generated_iso,
link_other=args.link_other or None,
page_label=args.page_label,
)
args.out.parent.mkdir(parents=True, exist_ok=True)
args.out.write_text(html_out, encoding="utf-8")
print(f"wrote {args.out} (errors={n_err} warnings={n_warn} level={status_level(n_err, n_warn)})")
if __name__ == "__main__":
main()

44
site-gen/deploy-monitoring-sites.sh Executable file
View file

@ -0,0 +1,44 @@
#!/usr/bin/env bash
# deploy-monitoring-sites.sh — rsync generated HTML to DEPLOY_WWW_ROOT on koopa.
# Does NOT reload Caddy (needs root — see ROOT-ON-KOOPA.md).
set -uo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
SETTINGS="${SITE_GEN_SETTINGS:-$ROOT/settings.conf}"
[ -f "$SETTINGS" ] || SETTINGS="$ROOT/settings.conf.example"
# shellcheck disable=SC1090
set -a
source <(grep -v '^\s*#' "$SETTINGS" | grep -v '^\s*$' | sed 's/\r$//')
set +a
WORK_ROOT="${WORK_ROOT:-/tmp/taler-monitoring-sites-work}"
HTML_DIR="$WORK_ROOT/html"
DEPLOY_SSH="${DEPLOY_SSH:-koopa-external}"
DEPLOY_WWW_ROOT="${DEPLOY_WWW_ROOT:-/var/www/monitoring-sites}"
if [ ! -d "$HTML_DIR" ]; then
echo "no HTML_DIR $HTML_DIR — run generate-monitoring-sites.sh first" >&2
exit 1
fi
echo "deploy $HTML_DIR → ${DEPLOY_SSH}:${DEPLOY_WWW_ROOT}/"
# Prefer writing to a hernani-owned staging dir if /var/www not writable
STAGE_REMOTE="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
ssh -o BatchMode=yes -o ConnectTimeout=20 "$DEPLOY_SSH" \
"mkdir -p '$STAGE_REMOTE' '$DEPLOY_WWW_ROOT' 2>/dev/null || mkdir -p '$STAGE_REMOTE'"
rsync -az --delete "$HTML_DIR/" "${DEPLOY_SSH}:${STAGE_REMOTE}/"
echo "staged on $DEPLOY_SSH:$STAGE_REMOTE"
echo
echo "If $DEPLOY_WWW_ROOT is root-owned, as root on koopa:"
echo " mkdir -p $DEPLOY_WWW_ROOT"
echo " rsync -a --delete $STAGE_REMOTE/ $DEPLOY_WWW_ROOT/"
echo " chown -R caddy:caddy $DEPLOY_WWW_ROOT # or root:caddy — match other www"
echo " # then Caddyfile handles + reload — see ROOT-ON-KOOPA.md"
echo
# try direct rsync to DEPLOY_WWW_ROOT if writable
if ssh -o BatchMode=yes "$DEPLOY_SSH" "test -w '$DEPLOY_WWW_ROOT' 2>/dev/null || test -w \"\$(dirname '$DEPLOY_WWW_ROOT')\""; then
rsync -az --delete "$HTML_DIR/" "${DEPLOY_SSH}:${DEPLOY_WWW_ROOT}/" && \
echo "also synced directly to $DEPLOY_WWW_ROOT"
fi

273
site-gen/generate-monitoring-sites.sh Executable file
View file

@ -0,0 +1,273 @@
#!/usr/bin/env bash
# generate-monitoring-sites.sh — run taler-monitoring, build console HTML for
# /monitoring and /monitoring_err (static files for host Caddy later).
#
# Usage:
# ./generate-monitoring-sites.sh # all SITES (or ONLY_HOSTS)
# ONLY_HOSTS="bank.hacktivism.ch taler.hacktivism.ch" ./generate-monitoring-sites.sh
# ./generate-monitoring-sites.sh --dry-run # HTML from empty/missing logs only
# ./generate-monitoring-sites.sh --skip-run # only convert existing logs
#
# Settings: settings.conf (from settings.conf.example). No secrets in settings.
# Secrets for e2e: ../secrets.env (optional; default phases are urls versions).
#
set -uo pipefail
export PYTHONUNBUFFERED=1
# Global reporting defaults (same as host-agent run-host-report.sh)
: "${RUN_TIMEOUT:=600}"
export RUN_TIMEOUT
ROOT=$(cd "$(dirname "$0")" && pwd)
SETTINGS="${SITE_GEN_SETTINGS:-$ROOT/settings.conf}"
EXAMPLE="$ROOT/settings.conf.example"
if [ ! -f "$SETTINGS" ]; then
if [ -f "$EXAMPLE" ]; then
echo "note: no $SETTINGS — using example defaults (copy to settings.conf to customize)"
SETTINGS="$EXAMPLE"
else
echo "missing settings: $EXAMPLE" >&2
exit 2
fi
fi
# Load KEY=value lines; allow SITES already set in environment to win.
# Multiline SITES in the conf file: use SITES_FILE= or env SITES= instead.
while IFS= read -r _line || [ -n "$_line" ]; do
_line=${_line%%#*}
_line=$(echo "$_line" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
[ -z "$_line" ] && continue
case "$_line" in
SITES=*|SITES_FILE=*) continue ;; # handled below / env
*=*)
_k=${_line%%=*}
_v=${_line#*=}
# do not override pre-set env
if [ -z "${!_k+x}" ] 2>/dev/null || [ -z "${!_k}" ]; then
export "$_k=$_v"
fi
;;
esac
done < "$SETTINGS"
# SITES from dedicated file or keep env; else parse heredoc-style from example list
if [ -n "${SITES_FILE:-}" ] && [ -f "$SITES_FILE" ]; then
SITES=$(grep -v '^\s*#' "$SITES_FILE" | grep -v '^\s*$')
export SITES
elif [ -z "${SITES:-}" ]; then
# default 9 fronts
SITES="bank.hacktivism.ch|hacktivism.ch
exchange.hacktivism.ch|hacktivism.ch
taler.hacktivism.ch|hacktivism.ch
stage.bank.lefrancpaysan.ch|stage.lefrancpaysan.ch
stage.exchange.lefrancpaysan.ch|stage.lefrancpaysan.ch
stage.monnaie.lefrancpaysan.ch|stage.lefrancpaysan.ch
bank.lefrancpaysan.ch|lefrancpaysan.ch
exchange.lefrancpaysan.ch|lefrancpaysan.ch
monnaie.lefrancpaysan.ch|lefrancpaysan.ch"
export SITES
fi
SKIP_RUN=0
DRY=0
while [ $# -gt 0 ]; do
case "$1" in
--skip-run) SKIP_RUN=1; shift ;;
--dry-run) DRY=1; SKIP_RUN=1; shift ;;
-h|--help) sed -n '1,20p' "$0"; exit 0 ;;
*) echo "unknown arg: $1" >&2; exit 2 ;;
esac
done
MONITORING_ROOT=$(cd "$ROOT/${MONITORING_ROOT:-..}" && pwd)
WORK_ROOT="${WORK_ROOT:-/tmp/taler-monitoring-sites-work}"
LOG_DIR="$WORK_ROOT/logs"
HTML_DIR="$WORK_ROOT/html"
mkdir -p "$LOG_DIR" "$HTML_DIR"
# commit of monitoring suite used for this run
if git -C "$MONITORING_ROOT/../.." rev-parse HEAD >/dev/null 2>&1; then
REPO_ROOT=$(cd "$MONITORING_ROOT/../.." && pwd)
elif git -C "$MONITORING_ROOT" rev-parse HEAD >/dev/null 2>&1; then
REPO_ROOT=$(cd "$MONITORING_ROOT" && pwd)
else
REPO_ROOT=$(cd "$ROOT/../../.." && pwd)
fi
COMMIT=$(git -C "$REPO_ROOT" rev-parse HEAD 2>/dev/null || echo unknown)
COMMIT_SHORT=$(git -C "$REPO_ROOT" rev-parse --short=12 HEAD 2>/dev/null || echo unknown)
SOURCE_REPO_WEB="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-monitoring}"
TMPL="${SOURCE_COMMIT_URL_TMPL:-\{repo\}/src/commit/\{commit\}}"
COMMIT_URL=$(printf '%s' "$TMPL" | sed "s|{repo}|$SOURCE_REPO_WEB|g; s|{commit}|$COMMIT|g")
SUITE_PATH="${SOURCE_SUITE_PATH:-.}"
# Outside-only public checks (no SSH into koopa/stage containers)
PHASES_ERR="${PHASES_ERR:-urls versions}"
PHASES_OK="${PHASES_OK:-urls versions}"
CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}"
SKIP_SSH="${SKIP_SSH:-1}"
# Drop SSH phases if someone passes "all" / inside / server / e2e by mistake
filter_phases_outside() {
local out=() p
for p in "$@"; do
case "$p" in
inside|server|e2e|ladder|goa-ladder|auth401) continue ;;
all) out+=(urls versions);;
*) out+=("$p") ;;
esac
done
printf '%s\n' "${out[@]}"
}
run_mon() {
local domain="$1" phases="$2" logfile="$3" cont="$4"
# shellcheck disable=SC2206
local ph=( $phases )
mapfile -t ph < <(filter_phases_outside "${ph[@]}")
[ "${#ph[@]}" -gt 0 ] || ph=(urls versions)
: >"$logfile"
echo "+ -d $domain · phases=${ph[*]} · CONTINUE=$cont · SKIP_SSH=$SKIP_SSH · RUN_TIMEOUT=$RUN_TIMEOUT"
if [ -n "${RUNNER_SSH:-}" ]; then
echo "+ via ssh $RUNNER_SSH"
rsync -az --delete \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
--exclude '.git' \
"$MONITORING_ROOT/" \
"${RUNNER_SSH}:${RUNNER_REMOTE_WORKDIR}/" \
|| return 1
# shellcheck disable=SC2029
if command -v stdbuf >/dev/null 2>&1; then
ssh -o BatchMode=yes -o ConnectTimeout=30 "$RUNNER_SSH" \
"cd '${RUNNER_REMOTE_WORKDIR}' && chmod +x taler-monitoring.sh check_*.sh 2>/dev/null; \
export CONTINUE_ON_ERROR='$cont' AUTH401_CONTINUE='$cont' SKIP_SSH='$SKIP_SSH' RUN_TIMEOUT='$RUN_TIMEOUT'; \
stdbuf -oL -eL ./taler-monitoring.sh -d '$domain' ${ph[*]}" \
2>&1 | stdbuf -oL -eL tee -a "$logfile"
else
ssh -o BatchMode=yes -o ConnectTimeout=30 "$RUNNER_SSH" \
"cd '${RUNNER_REMOTE_WORKDIR}' && chmod +x taler-monitoring.sh check_*.sh 2>/dev/null; \
export CONTINUE_ON_ERROR='$cont' AUTH401_CONTINUE='$cont' SKIP_SSH='$SKIP_SSH' RUN_TIMEOUT='$RUN_TIMEOUT'; \
./taler-monitoring.sh -d '$domain' ${ph[*]}" \
2>&1 | tee -a "$logfile"
fi
return "${PIPESTATUS[0]}"
fi
if command -v stdbuf >/dev/null 2>&1; then
( cd "$MONITORING_ROOT" && \
CONTINUE_ON_ERROR="$cont" AUTH401_CONTINUE="$cont" SKIP_SSH="$SKIP_SSH" \
RUN_TIMEOUT="$RUN_TIMEOUT" \
stdbuf -oL -eL ./taler-monitoring.sh -d "$domain" "${ph[@]}" ) \
2>&1 | stdbuf -oL -eL tee -a "$logfile"
else
( cd "$MONITORING_ROOT" && \
CONTINUE_ON_ERROR="$cont" AUTH401_CONTINUE="$cont" SKIP_SSH="$SKIP_SSH" \
RUN_TIMEOUT="$RUN_TIMEOUT" \
./taler-monitoring.sh -d "$domain" "${ph[@]}" ) \
2>&1 | tee -a "$logfile"
fi
return "${PIPESTATUS[0]}"
}
htmlify() {
local host="$1" mode="$2" log="$3" out="$4" other="$5"
mkdir -p "$(dirname "$out")"
if [ ! -f "$ROOT/console_to_html.py" ]; then
echo "WARN: console_to_html.py missing — bootstrap $out"
{
echo "<!DOCTYPE html><html><head><meta charset=utf-8><title>$host $mode</title></head>"
echo "<body><h1>$host · $mode</h1><p>commit <a href=\"$COMMIT_URL\">$COMMIT_SHORT</a></p>"
echo "<pre>"
tail -n 100 "$log" 2>/dev/null | sed 's/&/\&amp;/g;s/</\&lt;/g' || true
echo "</pre></body></html>"
} >"$out"
return 0
fi
python3 "$ROOT/console_to_html.py" \
--log "$log" \
--out "$out" \
--hostname "$host" \
--mode "$mode" \
--commit "$COMMIT" \
--commit-url "$COMMIT_URL" \
--suite-path "$SUITE_PATH" \
--link-other "$other"
}
# Parse SITES
mapfile -t SITE_LINES < <(printf '%s\n' "$SITES" | sed '/^\s*$/d' | sed 's/#.*//')
ONLY="${ONLY_HOSTS:-}"
for line in "${SITE_LINES[@]}"; do
line=$(echo "$line" | tr -d ' \t')
[ -z "$line" ] && continue
host=${line%%|*}
domain=${line#*|}
if [ -n "$ONLY" ]; then
echo " $ONLY " | grep -q " $host " || continue
fi
echo "======== $host (domain=$domain) commit=$COMMIT_SHORT ========"
base_html="$HTML_DIR/$host"
mkdir -p "$base_html/monitoring" "$base_html/monitoring_err"
log_err="$LOG_DIR/${host}.err.log"
log_ok="$LOG_DIR/${host}.ok.log"
ec_err=1
ec_ok=1
if [ "$SKIP_RUN" != "1" ]; then
run_mon "$domain" "$PHASES_ERR" "$log_err" "$CONTINUE_ON_ERROR"
ec_err=$?
# ok-run without continue (strict)
run_mon "$domain" "$PHASES_OK" "$log_ok" "0"
ec_ok=$?
else
[ -f "$log_err" ] || : >"$log_err"
[ -f "$log_ok" ] || : >"$log_ok"
# infer exit from log if present
if grep -q '\[ ERROR \]' "$log_err" 2>/dev/null || grep -q 'ERROR' "$log_err" 2>/dev/null; then
ec_err=1
else
ec_err=0
fi
ec_ok=$ec_err
fi
# Always write pages (first run / empty logs too)
[ -f "$log_err" ] || : >"$log_err"
[ -f "$log_ok" ] || : >"$log_ok"
htmlify "$host" "err" "$log_err" \
"$base_html/monitoring_err/index.html" "/monitoring/"
if [ "$ec_ok" -eq 0 ] && [ "$ec_err" -eq 0 ]; then
htmlify "$host" "ok" "$log_ok" \
"$base_html/monitoring/index.html" "/monitoring_err/"
echo " → /monitoring (clean) + /monitoring_err"
else
htmlify "$host" "redirect" "$log_err" \
"$base_html/monitoring/index.html" "/monitoring_err/"
echo " → /monitoring (redirect stub) + /monitoring_err (errors) ec_err=$ec_err ec_ok=$ec_ok"
fi
# first-run safety
if [ ! -f "$base_html/monitoring/index.html" ]; then
htmlify "$host" "err" "$log_err" \
"$base_html/monitoring/index.html" "/monitoring_err/"
fi
printf '%s\n' "$COMMIT" >"$base_html/COMMIT"
printf '%s\n' "$COMMIT_URL" >"$base_html/COMMIT_URL"
printf 'host=%s domain=%s ec_err=%s ec_ok=%s commit=%s run_timeout=%s\n' \
"$host" "$domain" "$ec_err" "$ec_ok" "$COMMIT" "$RUN_TIMEOUT" | tee "$base_html/STATUS.txt"
done
echo
echo "HTML under $HTML_DIR"
echo "commit $COMMIT_SHORT $COMMIT_URL"
echo "RUN_TIMEOUT=${RUN_TIMEOUT}s (global; 0=unlimited)"
echo "Deploy (as hernani, may need sudo for /var/www):"
echo " ./deploy-monitoring-sites.sh"
echo "Root on koopa: see ROOT-ON-KOOPA.md"

79
site-gen/install-firecuda-timer.sh Executable file
View file

@ -0,0 +1,79 @@
#!/usr/bin/env bash
# install-firecuda-timer.sh — OPTIONAL: rsync suite to firecuda + launchd every 4h
#
# POLICY (2026-07): firecuda launchd is **disabled**. Prefer koopa host-agent.
# This script remains for documentation / future re-enable only.
# Do not run unless you intentionally want a second outside-only scheduler.
#
# ./install-firecuda-timer.sh
# ./install-firecuda-timer.sh --run-now
#
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
MON_ROOT=$(cd "$ROOT/.." && pwd)
FIRECUDA="${FIRECUDA_SSH:-firecuda-external}"
REMOTE_BASE="${FIRECUDA_INSTALL_DIR:-taler-monitoring-site-gen}"
RUN_NOW=0
[ "${1:-}" = "--run-now" ] && RUN_NOW=1
echo "install → $FIRECUDA:~/$REMOTE_BASE (outside-only, every 4h)"
ssh -o BatchMode=yes -o ConnectTimeout=20 "$FIRECUDA" \
"mkdir -p \"\$HOME/$REMOTE_BASE\" \"\$HOME/Library/Logs/taler-monitoring-sites\" \"\$HOME/var/taler-monitoring-sites-work\" \"\$HOME/Library/LaunchAgents\""
# Sync full taler-monitoring suite (no secrets.env)
rsync -az --delete \
--exclude secrets.env \
--exclude 'android-test/artifacts' \
--exclude '.git' \
--exclude 'site-gen/settings.conf' \
"$MON_ROOT/" \
"${FIRECUDA}:$REMOTE_BASE/"
# settings on firecuda: outside only, run locally on firecuda, deploy to koopa
ssh -o BatchMode=yes "$FIRECUDA" "cat > \"\$HOME/$REMOTE_BASE/site-gen/settings.conf\" <<'EOF'
# auto-installed by install-firecuda-timer.sh — no secrets
RUNNER_SSH=
SKIP_SSH=1
CONTINUE_ON_ERROR=1
PHASES_ERR=urls versions
PHASES_OK=urls versions
DEPLOY_SSH=koopa-external
DEPLOY_STAGING=/home/hernani/monitoring-sites-staging
WORK_ROOT=
MONITORING_ROOT=..
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
SOURCE_COMMIT_URL_TMPL={repo}/src/commit/{commit}
SOURCE_SUITE_PATH=.
EOF
# WORK_ROOT expanded on remote
sed -i.bak \"s|^WORK_ROOT=.*|WORK_ROOT=\$HOME/var/taler-monitoring-sites-work|\" \"\$HOME/$REMOTE_BASE/site-gen/settings.conf\" 2>/dev/null || \
sed -i '' \"s|^WORK_ROOT=.*|WORK_ROOT=\$HOME/var/taler-monitoring-sites-work|\" \"\$HOME/$REMOTE_BASE/site-gen/settings.conf\"
"
# launchd plist with absolute paths
ssh -o BatchMode=yes "$FIRECUDA" bash -s <<REMOTE
set -euo pipefail
HOME_DIR=\$HOME
SITE_GEN="\$HOME_DIR/$REMOTE_BASE/site-gen"
PLIST_SRC="\$SITE_GEN/com.hacktivism.taler-monitoring-sites.plist"
PLIST_DST="\$HOME_DIR/Library/LaunchAgents/com.hacktivism.taler-monitoring-sites.plist"
sed -e "s|__SITE_GEN__|\${SITE_GEN}|g" -e "s|__HOME__|\${HOME_DIR}|g" "\$PLIST_SRC" > "\$PLIST_DST"
chmod +x "\$SITE_GEN"/*.sh "\$SITE_GEN"/console_to_html.py 2>/dev/null || true
chmod +x "\$SITE_GEN/run-on-firecuda.sh"
launchctl bootout "gui/\$(id -u)/com.hacktivism.taler-monitoring-sites" 2>/dev/null || true
launchctl unload "\$PLIST_DST" 2>/dev/null || true
launchctl bootstrap "gui/\$(id -u)" "\$PLIST_DST" 2>/dev/null || launchctl load "\$PLIST_DST"
launchctl enable "gui/\$(id -u)/com.hacktivism.taler-monitoring-sites" 2>/dev/null || true
launchctl print "gui/\$(id -u)/com.hacktivism.taler-monitoring-sites" 2>/dev/null | head -20 || launchctl list | grep taler-monitoring || true
echo "OK: launchd com.hacktivism.taler-monitoring-sites (StartInterval=14400)"
REMOTE
if [ "$RUN_NOW" = "1" ]; then
echo "run once now…"
ssh -o BatchMode=yes "$FIRECUDA" "\"\$HOME/$REMOTE_BASE/site-gen/run-on-firecuda.sh\"" || true
fi
echo "done. logs: $FIRECUDA:~/Library/Logs/taler-monitoring-sites/"
echo "manual: ssh $FIRECUDA '~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh'"

15
site-gen/pull-firecuda-to-koopa.sh Executable file
View file

@ -0,0 +1,15 @@
#!/usr/bin/env bash
# pull-firecuda-to-koopa.sh — copy HTML from firecuda work dir → koopa staging
# Run on laptop (hernani) where both SSH aliases work.
set -euo pipefail
FIRECUDA="${FIRECUDA_SSH:-firecuda-external}"
KOOPA="${DEPLOY_SSH:-koopa-external}"
REMOTE_HTML="${FIRECUDA_HTML:-var/taler-monitoring-sites-work/html/}"
STAGE="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
echo "firecuda:$REMOTE_HTML → $KOOPA:$STAGE"
ssh -o BatchMode=yes -o ConnectTimeout=20 "$KOOPA" "mkdir -p '$STAGE'"
rsync -az --delete \
"${FIRECUDA}:${REMOTE_HTML}" \
"${KOOPA}:${STAGE}/"
echo "OK staged. Root: rsync to /var/www/monitoring-sites (see ROOT-ON-KOOPA.md)"

59
site-gen/run-on-firecuda.sh Executable file
View file

@ -0,0 +1,59 @@
#!/usr/bin/env bash
# run-on-firecuda.sh — entrypoint for launchd/cron on firecuda-external.
# Outside-only monitoring (SKIP_SSH=1), all 9 sites, then stage HTML to koopa.
#
# Installed path on firecuda (default):
# ~/taler-monitoring-site-gen/site-gen/run-on-firecuda.sh
#
set -uo pipefail
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}"
ROOT=$(cd "$(dirname "$0")" && pwd)
LOG_DIR="${FIRECUDA_LOG_DIR:-$HOME/Library/Logs/taler-monitoring-sites}"
mkdir -p "$LOG_DIR"
STAMP=$(date +%Y%m%d-%H%M%S)
LOG="$LOG_DIR/run-${STAMP}.log"
exec >>"$LOG" 2>&1
echo "======== $(date -Iseconds 2>/dev/null || date) firecuda monitoring sites ========"
# Always: public DNS only, no SSH checks into stacks
export SKIP_SSH=1
export CONTINUE_ON_ERROR=1
export AUTH401_CONTINUE=1
export RUNNER_SSH=
export PHASES_ERR="${PHASES_ERR:-urls versions}"
export PHASES_OK="${PHASES_OK:-urls versions}"
export WORK_ROOT="${WORK_ROOT:-$HOME/var/taler-monitoring-sites-work}"
export MONITORING_ROOT="${MONITORING_ROOT:-$ROOT/..}"
export DEPLOY_SSH="${DEPLOY_SSH:-koopa-external}"
export DEPLOY_STAGING="${DEPLOY_STAGING:-/home/hernani/monitoring-sites-staging}"
# Prefer full 9-site list from settings or defaults inside generate script
unset ONLY_HOSTS
cd "$ROOT" || exit 1
chmod +x generate-monitoring-sites.sh deploy-monitoring-sites.sh console_to_html.py 2>/dev/null || true
./generate-monitoring-sites.sh
ec=$?
# HTML stays on firecuda under WORK_ROOT/html.
# Optional push to koopa if this host can SSH there (often only from laptop):
if [ -n "${DEPLOY_SSH:-}" ] && [ "${DEPLOY_FROM_FIRECUDA:-0}" = "1" ]; then
if [ -x ./deploy-monitoring-sites.sh ]; then
./deploy-monitoring-sites.sh || echo "WARN: deploy staging failed"
fi
else
echo "note: HTML at ${WORK_ROOT}/html — pull from laptop:"
echo " rsync -az firecuda-external:var/taler-monitoring-sites-work/html/ \\"
echo " koopa-external:monitoring-sites-staging/"
echo " # or: ./pull-firecuda-to-koopa.sh"
fi
# keep last 20 logs
ls -1t "$LOG_DIR"/run-*.log 2>/dev/null | tail -n +21 | xargs rm -f 2>/dev/null || true
echo "======== done ec=$ec ========"
exit "$ec"

55
site-gen/settings.conf.example Normal file
View file

@ -0,0 +1,55 @@
# monitoring site-gen — non-secret settings (safe as example)
# Copy: cp settings.conf.example settings.conf
# Never put passwords/tokens here.
# On firecuda the suite runs *locally* (RUNNER_SSH empty). From a laptop you
# can set RUNNER_SSH=firecuda-external to drive a one-off remote run instead.
RUNNER_SSH=
RUNNER_SSH_FALLBACKS=
RUNNER_REMOTE_WORKDIR=/tmp/taler-monitoring-site-gen
MONITORING_ROOT=..
# Outside-only: never SSH into bank/exchange/merchant containers
SKIP_SSH=1
# Deploy staging on koopa (host Caddy later — root copies to /var/www)
DEPLOY_SSH=koopa-external
DEPLOY_SSH_FALLBACKS=koopa
# Static files for Caddy file_server (created per hostname)
DEPLOY_WWW_ROOT=/var/www/monitoring-sites
DEPLOY_SUDO=sudo
CADDY_CONFIG=/etc/caddy/Caddyfile
CADDY_MIRROR_REL=configs/caddy/Caddyfile
# Working tree on deploy host for apply (optional)
KOOPA_CADDY_DIR=/home/hernani/koopa-caddy
# Source link for HTML footer (Forgejo)
SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring
SOURCE_COMMIT_URL_TMPL={repo}/src/commit/{commit}
SOURCE_SUITE_PATH=.
CONTINUE_ON_ERROR=1
PHASES_ERR=urls versions
PHASES_OK=urls versions
SKIP_E2E=1
# Local work dir for logs + HTML
WORK_ROOT=/tmp/taler-monitoring-sites-work
# Sites: hostname|domain_profile (9 fronts)
# Start with hacktivism; stage + prod LFP follow
SITES="
bank.hacktivism.ch|hacktivism.ch
exchange.hacktivism.ch|hacktivism.ch
taler.hacktivism.ch|hacktivism.ch
stage.bank.lefrancpaysan.ch|stage.lefrancpaysan.ch
stage.exchange.lefrancpaysan.ch|stage.lefrancpaysan.ch
stage.monnaie.lefrancpaysan.ch|stage.lefrancpaysan.ch
bank.lefrancpaysan.ch|lefrancpaysan.ch
exchange.lefrancpaysan.ch|lefrancpaysan.ch
monnaie.lefrancpaysan.ch|lefrancpaysan.ch
"
# Only process this subset (space-separated hostnames). Empty = all SITES.
# Example for first deploy: ONLY_HOSTS=bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch
ONLY_HOSTS=

52
surface-catalog.conf Normal file
View file

@ -0,0 +1,52 @@
# surface-catalog.conf — known public Taler / GNUnet / Taler Systems hosts
# Format (whitespace-separated; # comments):
# host ports expect_proto label
# expect_proto: https | http | ssh | tcp | any
# ports: comma-separated (empty = defaults for proto)
#
# This list is expanded at runtime (DNS discovery, cert SANs, -d domain profile).
# Phase `surface` is NOT in default phases — run explicitly:
# ./taler-monitoring.sh surface
# ./taler-monitoring.sh -d hacktivism.ch surface
# --- taler.net ecosystem ---
www.taler.net 443 https taler-www
docs.taler.net 443 https taler-docs
git.taler.net 443,22 https taler-git
bugs.taler.net 443 https taler-bugs
lists.taler.net 443 https taler-lists
tutorials.taler.net 443 https taler-tutorials
bank.demo.taler.net 443 https demo-bank
exchange.demo.taler.net 443 https demo-exchange
backend.demo.taler.net 443 https demo-merchant
shop.demo.taler.net 443 https demo-shop
bank.test.taler.net 443 https test-bank
exchange.test.taler.net 443 https test-exchange
backend.test.taler.net 443 https test-merchant
# --- taler-ops / TOPS ---
www.taler-ops.ch 443 https tops-www
taler-ops.ch 443 https tops-apex
bank.taler-ops.ch 443 https tops-bank
exchange.taler-ops.ch 443 https tops-exchange
my.taler-ops.ch 443 https tops-merchant
stage.taler-ops.ch 443 https tops-stage-www
stage.my.taler-ops.ch 443 https tops-stage-merchant
exchange.stage.taler-ops.ch 443 https tops-stage-exchange
nexus.stage.taler-ops.ch 443 https tops-stage-nexus
shops.taler-ops.ch 443 https tops-shops
mattermost.taler.net 443 https mattermost
# --- gnunet.org ---
www.gnunet.org 443,80 https gnunet-www
gnunet.org 443,80 https gnunet-apex
git.gnunet.org 443,22 https gnunet-git
bugs.gnunet.org 443 https gnunet-bugs
# --- taler-systems.com ---
www.taler-systems.com 443 https tsa-www
taler-systems.com 443 https tsa-apex
# --- common infra hostnames (remote only; may 404/redirect) ---
deb.taler.net 443 https deb-taler
ftp.gnu.org 443 https gnu-ftp

477
taler-monitoring.sh Executable file
View file

@ -0,0 +1,477 @@
#!/usr/bin/env bash
# taler-monitoring — public URL / stack checks for a Taler domain
#
# ./taler-monitoring.sh # local GOA (urls + inside + e2e)
# ./taler-monitoring.sh -d taler.net urls # public demo, no SSH
# ./taler-monitoring.sh --domain taler-ops.ch # public ops, no SSH
# TALER_DOMAIN=demo.taler.net ./taler-monitoring.sh urls
#
# Tags: [OK] [INFO] [WARN] [ERROR] [BLOCKER]
# Exit 0 only if every selected phase exits 0.
set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd)
# Line-buffered stdout/stderr so redirected logs (tee/host-agent) flush each line.
if [ "${_TALER_MON_STDBUF:-0}" != "1" ] && command -v stdbuf >/dev/null 2>&1; then
export _TALER_MON_STDBUF=1
exec stdbuf -oL -eL bash "$0" "$@"
fi
usage() {
cat <<'EOF'
taler-monitoring — bank / exchange / merchant checks
Usage:
./taler-monitoring.sh [options] [phases...]
Phases:
urls public HTTPS (no SSH) ← default with --domain
(includes merchant /webui/ SPA fingerprints: version.txt, overlay, assets)
inside container status via SSH (local stack only)
versions taler packages vs deb.taler.net trixie + repo availability
sanity public + optional server
server server-side only (SSH)
e2e withdraw + pay (small amounts; remote aborts on login/KYC)
ladder withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire)
auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
aptdeploy koopa podman apt-src smoke: taler-merchant in
koopa-taler-deploy-test-apt-src-trixie{,-testing}
surface REMOTE-ONLY public inventory (NOT in default/all/full):
ecosystem hosts (taler.net, gnunet.org, taler-systems.com, mattermost, …)
or -d DOMAIN → that domain’s surface; port/protocol/TLS/CVE (OSV)
all urls + inside + versions + sanity + e2e (SSH phases only on koopa)
full all + server + ladder + auth401 (maximum; long-running, needs secrets)
NOTE: surface is never included in all/full — pass it explicitly
Options:
-d, --domain DOMAIN load profile from domains.conf (bank/exchange/merchant)
presets: koopa | hacktivism.ch | taler.net | taler-ops.ch
| my.taler-ops.ch | stage.taler-ops.ch
unknown domain → heuristic hosts (see README)
local (koopa) allows SSH; others public-only
--bank URL bank base (hostname or https://…) — overrides profile
--exchange URL exchange base — overrides profile
--merchant URL merchant-backend base — overrides profile
--currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only
--no-probe do not probe alternate hosts (unknown domains only)
-h, --help
Add a stack: edit domains.conf (name + bank + exchange + merchant + currency).
Examples:
./taler-monitoring.sh -d taler.net
./taler-monitoring.sh -d taler-ops.ch urls
./taler-monitoring.sh -d my.taler-ops.ch urls
./taler-monitoring.sh -d taler-ops.ch --merchant https://my.taler-ops.ch urls
./taler-monitoring.sh -d demo.taler.net --currency KUDOS
./taler-monitoring.sh --exchange https://exchange.taler-ops.ch \
--merchant https://my.taler-ops.ch --bank https://bank.taler-ops.ch urls
Env (same meaning):
TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC EXPECT_CURRENCY
TALER_DOMAINS_CONF SKIP_SSH=1
NO_COLOR=1 / CLICOLOR=0 disable green/yellow/red tags (default: coloured)
SKIP_SSH=1 NO_COLOR=1
RUN_TIMEOUT=600 whole-run wall clock seconds (default 600; 0=unlimited)
long phases (ladder/full/e2e) need a higher value or 0
DISK_WARN_USED_PCT=85 disk free: WARN when used ≥ this %
DISK_ERR_USED_PCT=95 disk free: ERROR when used ≥ this % (or avail=0 / 100%)
PERF_WARN_MS PERF_FAIL_MS (urls latency; default 8000 / 20000)
QR_CHECK=0 skip QR form + qrencode/zbarimg (urls phase)
QR_ECC=M qrencode ECC level (default M)
KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external)
METRICS_LOAD=0 skip host/container RAM/CPU probes (e2e/ladder/inside)
EXPECT_WEBUI_VERSION=1.6.11 pin merchant /webui/version.txt (urls)
EXPECT_WEBUI_OVERLAY=selfbuild require substr in version-overlay.txt
WEBUI_OVERLAY_DENY='master-11340' fail if overlay matches regex
CHECK_WEBUI_SPA=0 skip SPA fingerprint block in urls
AUTH401_* see check_auth401.sh / secrets.env.example
AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1
auth401: run all groups, collect every ERROR (no mid-run halt)
APT_DEPLOY_SKIP=1 skip aptdeploy phase
APT_DEPLOY_CONTAINERS="name:suite …" default trixie + trixie-testing deploy-test pods
SURFACE_CVE=0 disable CVE queries in surface phase
SURFACE_CVE_LEVEL=warn|error bare Server-header versions default warn
(Debian package versions default error)
SURFACE_CATALOG=path override surface-catalog.conf
Full load (GOA / hacktivism):
./taler-monitoring.sh -d hacktivism.ch full
# or explicit:
./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401
# hacktivism host-agent also runs aptdeploy (apt-src merchant containers on koopa)
Remote surface / ecosystem (explicit only — never default):
./taler-monitoring.sh surface
./taler-monitoring.sh -d hacktivism.ch surface
./taler-monitoring.sh -d lefrancpaysan.ch surface
SPA pin after selfbuild:
EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \\
WEBUI_OVERLAY_DENY='c778af04|master-11340' \\
./taler-monitoring.sh -d hacktivism.ch urls
EOF
}
# shellcheck source=lib.sh
source "$ROOT/lib.sh"
# Optional gitignored secrets.env next to this suite (or MONITORING_SECRETS_ENV=)
load_monitoring_secrets_env 2>/dev/null || true
# Persist #NNN + progress across check_*.sh child processes
TALER_MON_STATE=$(mktemp "${TMPDIR:-/tmp}/taler-mon-state.XXXXXX")
export TALER_MON_STATE
trap 'rm -f "${TALER_MON_STATE:-}"' EXIT
PHASES=()
DOMAIN_SET=0
BANK_OVERRIDE=""
EXCHANGE_OVERRIDE=""
MERCHANT_OVERRIDE=""
CURRENCY_OVERRIDE=""
NO_PROBE=0
while [ $# -gt 0 ]; do
case "$1" in
-h|--help) usage; exit 0 ;;
-d|--domain)
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
apply_taler_domain "$2"
DOMAIN_SET=1
TALER_DOMAIN_FROM_CLI=1
shift 2
;;
--bank)
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
BANK_OVERRIDE="${2%/}"; shift 2
;;
--exchange)
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
EXCHANGE_OVERRIDE="${2%/}"; shift 2
;;
--merchant)
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
MERCHANT_OVERRIDE="${2%/}"; shift 2
;;
--currency)
[ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; }
CURRENCY_OVERRIDE="$2"; shift 2
;;
--no-probe) NO_PROBE=1; shift ;;
urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|all|full) PHASES+=("$1"); shift ;;
*)
# bare domain shorthand: ./taler-monitoring.sh taler.net
if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then
apply_taler_domain "$1"
DOMAIN_SET=1
TALER_DOMAIN_FROM_CLI=1
shift
else
echo "Unknown: $1" >&2; usage >&2; exit 2
fi
;;
esac
done
if [ "$NO_PROBE" = "1" ]; then
TALER_DOMAIN_PROBE=0
fi
if [ -n "$CURRENCY_OVERRIDE" ]; then
EXPECT_CURRENCY="$CURRENCY_OVERRIDE"
fi
if [ -n "$BANK_OVERRIDE" ]; then BANK_PUBLIC="$BANK_OVERRIDE"; fi
if [ -n "$EXCHANGE_OVERRIDE" ]; then EXCHANGE_PUBLIC="$EXCHANGE_OVERRIDE"; fi
if [ -n "$MERCHANT_OVERRIDE" ]; then MERCHANT_PUBLIC="$MERCHANT_OVERRIDE"; fi
# Only koopa may use SSH. Remote domains: public + optional e2e (no SSH).
if [ "${LOCAL_STACK:-1}" != "1" ]; then
SKIP_SSH=1
fi
# Remote: no SSH; ATM withdraw ladder set in check_e2e (smaller notes)
if [ "${LOCAL_STACK}" != "1" ]; then
E2E_FAKE_INCOMING=0
E2E_REMOTE=1
: "${E2E_WITHDRAW_VALUES:=10 20 50}"
: "${E2E_PAY_VALUES:=0.01 0.05 0.1 1}"
fi
# Export so check_*.sh (re-source lib) see the same targets via env
export TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC
export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE CHECK_LANDING
export TALER_DOMAIN_FROM_CLI="${TALER_DOMAIN_FROM_CLI:-0}"
export DOMAIN_SET="${DOMAIN_SET:-0}"
export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE
export E2E_FAKE_INCOMING E2E_REMOTE E2E_VARIABLE E2E_ATM_MAX
export E2E_WITHDRAW_VALUES E2E_PAY_VALUES E2E_USE_TEMPLATES E2E_TEMPLATE_MAP
export PAIVANA_PUBLIC E2E_PAIVANA E2E_PAIVANA_TEMPLATE E2E_PAIVANA_AMOUNT E2E_PAIVANA_INSTANCE
export INSIDE_PROFILE INSIDE_SSH
export INSIDE_BANK_CTR INSIDE_EXCHANGE_CTR INSIDE_MERCHANT_CTR
export INSIDE_BANK_PORT INSIDE_EXCHANGE_PORT INSIDE_MERCHANT_PORT
export INSIDE_DNS_BANK INSIDE_DNS_EXCHANGE INSIDE_DNS_MERCHANT
# Ladder: withdraw then pay — 0 + random mids + max-1 + max (see check_goa_ladder.sh).
# Defaults so set -u export is safe when vars were never set by caller.
: "${LADDER_STEPS:=23}"
: "${LADDER_MIN_AMOUNT:=0.000001}"
: "${LADDER_CONFIRM_POLLS:=40}"
: "${LADDER_MAX_RUNGS:=99}"
: "${LADDER_TIMEOUT_S:=3600}"
: "${LADDER_REPORT_DIR:=}"
: "${LADDER_SETTLE_ROUNDS:=18}"
: "${LADDER_SETTLE_SLEEP:=2}"
: "${LADDER_MAX_AMOUNT:=4503599627370496}"
: "${LADDER_INCLUDE_ZERO:=1}"
: "${LADDER_INCLUDE_MAX:=1}"
: "${LADDER_HIGH_FROM:=1000000}"
: "${LADDER_HIGH_RUNGS:=12}"
: "${LADDER_LOAD:=1}"
: "${LADDER_PAY:=1}"
: "${LADDER_WITHDRAW_SCALE:=1.5}"
: "${LADDER_PAY_SETTLE_ROUNDS:=6}"
export LADDER_STEPS LADDER_MIN_AMOUNT LADDER_CONFIRM_POLLS LADDER_MAX_RUNGS LADDER_TIMEOUT_S LADDER_REPORT_DIR
export LADDER_SETTLE_ROUNDS LADDER_SETTLE_SLEEP EXP_PW_FILE EXP_USER
export LADDER_MAX_AMOUNT LADDER_INCLUDE_ZERO LADDER_INCLUDE_MAX
export LADDER_HIGH_FROM LADDER_HIGH_RUNGS LADDER_LOAD
export LADDER_PAY LADDER_WITHDRAW_SCALE LADDER_PAY_SETTLE_ROUNDS
export TALER_DOMAIN_APPLIED=1
# Optional: auth401 and other phases may honor CONTINUE_ON_ERROR
export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-}"
export AUTH401_CONTINUE="${AUTH401_CONTINUE:-${CONTINUE_ON_ERROR:-}}"
# Default phases
# - local GOA (koopa): urls + inside + versions + e2e
# - FrancPaysan STAGE: urls + inside (stagepaysan) + versions + e2e
# - other remote: urls only (pass e2e/inside explicitly)
if [ "${#PHASES[@]}" -eq 0 ]; then
if [ "${LOCAL_STACK}" = "1" ]; then
PHASES=(urls inside versions e2e)
elif [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ] \
|| [[ "${TALER_DOMAIN:-}" == stage.*lefrancpaysan* ]] \
|| [[ "${TALER_DOMAIN:-}" == *stage.lefrancpaysan* ]]; then
# Like hacktivism breadth: public urls + inside (stagepaysan) + versions + e2e
PHASES=(urls inside versions e2e)
else
PHASES=(urls)
fi
fi
OUT_PHASES=()
for p in "${PHASES[@]}"; do
if [ "$p" = "all" ]; then
if [ "${LOCAL_STACK}" = "1" ]; then
OUT_PHASES+=(urls inside versions sanity e2e)
elif [ -n "${INSIDE_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "stage-lfp" ]; then
OUT_PHASES+=(urls inside versions e2e)
else
# remote without inside SSH: public + optional e2e
OUT_PHASES+=(urls versions e2e)
fi
elif [ "$p" = "full" ]; then
# Maximum coverage. Long. Needs secrets for e2e/ladder/auth401.
if [ "${LOCAL_STACK}" = "1" ]; then
OUT_PHASES+=(urls inside versions sanity server e2e ladder auth401)
elif [ -n "${INSIDE_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "stage-lfp" ]; then
OUT_PHASES+=(urls inside versions e2e ladder auth401)
else
OUT_PHASES+=(urls versions e2e auth401)
fi
else
OUT_PHASES+=("$p")
fi
done
PHASES=()
seen=" "
for p in "${OUT_PHASES[@]}"; do
# server = koopa-only. inside allowed on koopa OR stagepaysan (INSIDE_SSH).
case "$p" in
server)
if [ "${LOCAL_STACK}" != "1" ]; then
echo "[INFO] skip phase 'server' (koopa only)" >&2
continue
fi
;;
inside)
if [ "${LOCAL_STACK}" != "1" ] && [ -z "${INSIDE_SSH:-}" ] && [ "${INSIDE_PROFILE:-}" != "stage-lfp" ]; then
echo "[INFO] skip phase 'inside' (no INSIDE_SSH / not local)" >&2
continue
fi
;;
esac
case "$seen" in *" $p "*) ;; *) PHASES+=("$p"); seen="$seen$p " ;; esac
done
if [ "${#PHASES[@]}" -eq 0 ]; then
PHASES=(urls)
fi
# Rough expected check counts for progress % (override: PROGRESS_TOTAL=N).
# Prefer slightly high — lib.sh rebalances if still short (never shows done>total).
if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then
_pt=0
for p in "${PHASES[@]}"; do
case "$p" in
urls) _pt=$((_pt + 95)) ;; # +qr + webui SPA fingerprints
inside) _pt=$((_pt + 30)) ;;
versions) _pt=$((_pt + 25)) ;;
sanity) _pt=$((_pt + 35)) ;;
server) _pt=$((_pt + 20)) ;;
# e2e: ATM ladder emits many INFO lines (coins before/after each note)
e2e) _pt=$((_pt + 240)) ;;
ladder|goa-ladder) _pt=$((_pt + 120)) ;;
auth401) _pt=$((_pt + 70)) ;;
aptdeploy) _pt=$((_pt + 20)) ;;
surface|ecosystem) _pt=$((_pt + 80)) ;;
esac
done
set_progress_total "$_pt"
unset _pt
fi
# Whole-run wall clock (default 10 min). Override: RUN_TIMEOUT=0 (unlimited).
: "${RUN_TIMEOUT:=600}"
export RUN_TIMEOUT
MON_T0=$(date +%s)
export MON_T0
RUN_TIMED_OUT=0
RUN_TIMEOUT_AT_PHASE=""
RUN_SKIPPED_PHASES=()
mon_seconds_left() {
if [ "${RUN_TIMEOUT:-0}" -eq 0 ]; then
printf '%s' "999999"
return 0
fi
local now left
now=$(date +%s)
left=$((RUN_TIMEOUT - (now - MON_T0)))
[ "$left" -lt 0 ] && left=0
printf '%s' "$left"
}
# Run one phase script under remaining RUN_TIMEOUT budget.
# Exit 124 (timeout utility) → mark RUN_TIMED_OUT.
run_phase() {
local phase="$1" script="$2" left rc
left=$(mon_seconds_left)
if [ "${RUN_TIMEOUT:-0}" -gt 0 ] && [ "$left" -le 0 ]; then
RUN_TIMED_OUT=1
RUN_TIMEOUT_AT_PHASE="${RUN_TIMEOUT_AT_PHASE:-$phase}"
return 1
fi
if [ "${RUN_TIMEOUT:-0}" -eq 0 ]; then
"$script"
return $?
fi
# Keep at least 2s so timeout(1) can start the child.
[ "$left" -lt 2 ] && left=2
set +e
with_timeout "$left" "$script"
rc=$?
set -e
if [ "$rc" -eq 124 ]; then
RUN_TIMED_OUT=1
RUN_TIMEOUT_AT_PHASE="$phase"
return 1
fi
return "$rc"
}
# How inside/versions will reach containers (host-podman vs ssh)
_INSIDE_ACCESS_HINT=ssh
if [ "${INSIDE_PODMAN:-0}" = "1" ] || [ "${INSIDE_MODE:-}" = "local-podman" ]; then
_INSIDE_ACCESS_HINT=host-podman
elif [ "${SKIP_SSH:-0}" = "1" ]; then
_INSIDE_ACCESS_HINT=skipped
elif [ "${LOCAL_STACK:-0}" = "1" ] && command -v podman >/dev/null 2>&1 \
&& podman ps --format '{{.Names}}' 2>/dev/null | grep -qE 'taler-hacktivism'; then
_INSIDE_ACCESS_HINT=host-podman
fi
printf 'target domain=%s\n' "${TALER_DOMAIN}"
printf ' bank %s\n' "$BANK_PUBLIC"
printf ' exchange %s\n' "$EXCHANGE_PUBLIC"
printf ' merchant %s\n' "$MERCHANT_PUBLIC"
printf ' currency expect=%s\n' "${EXPECT_CURRENCY:-any}"
printf ' phases %s\n' "${PHASES[*]}"
printf ' flags LOCAL_STACK=%s SKIP_SSH=%s INSIDE_PODMAN=%s INSIDE_MODE=%s\n' \
"${LOCAL_STACK:-}" "${SKIP_SSH:-0}" "${INSIDE_PODMAN:-0}" "${INSIDE_MODE:-}"
printf ' flags KOOPA_SSH=%s INSIDE_SSH=%s INSIDE_PROFILE=%s\n' \
"${KOOPA_SSH:-}" "${INSIDE_SSH:-}" "${INSIDE_PROFILE:-}"
printf ' access inside/versions → %s' "$_INSIDE_ACCESS_HINT"
case "$_INSIDE_ACCESS_HINT" in
host-podman) printf ' (podman exec on this host; IDs inside.host-*)\n' ;;
ssh) printf ' (SSH then podman; IDs inside.ssh-*)\n' ;;
skipped) printf ' (SKIP_SSH=1)\n' ;;
*) printf '\n' ;;
esac
if [ "${RUN_TIMEOUT:-0}" -eq 0 ]; then
printf ' run_timeout unlimited (RUN_TIMEOUT=0)\n'
else
printf ' run_timeout %ss wall clock (RUN_TIMEOUT=; 0=unlimited)\n' "$RUN_TIMEOUT"
fi
printf ' progress total≈%s (set PROGRESS_TOTAL= to override; PROGRESS_OFF=1 to hide)\n' "${PROGRESS_TOTAL:-0}"
unset _INSIDE_ACCESS_HINT
chmod +x "$ROOT"/check_*.sh 2>/dev/null || true
ec=0
for p in "${PHASES[@]}"; do
if [ "$RUN_TIMED_OUT" = "1" ]; then
RUN_SKIPPED_PHASES+=("$p")
continue
fi
left=$(mon_seconds_left)
if [ "${RUN_TIMEOUT:-0}" -gt 0 ] && [ "$left" -le 0 ]; then
RUN_TIMED_OUT=1
RUN_TIMEOUT_AT_PHASE="${RUN_TIMEOUT_AT_PHASE:-$p}"
RUN_SKIPPED_PHASES+=("$p")
continue
fi
case "$p" in
urls) run_phase urls "$ROOT/check_urls.sh" || ec=1 ;;
inside) run_phase inside "$ROOT/check_inside.sh" || ec=1 ;;
versions) run_phase versions "$ROOT/check_versions.sh" || ec=1 ;;
sanity) run_phase sanity "$ROOT/check_sanity.sh" || ec=1 ;;
server) run_phase server "$ROOT/check_server.sh" || ec=1 ;;
e2e) run_phase e2e "$ROOT/check_e2e.sh" || ec=1 ;;
ladder|goa-ladder) run_phase ladder "$ROOT/check_goa_ladder.sh" || ec=1 ;;
auth401) run_phase auth401 "$ROOT/check_auth401.sh" || ec=1 ;;
aptdeploy|apt-deploy|apt_src) run_phase aptdeploy "$ROOT/check_apt_deploy.sh" || ec=1 ;;
surface|ecosystem) run_phase surface "$ROOT/check_surface.sh" || ec=1 ;;
esac
done
# Extraordinary run-budget failure: always report at end; HTML links top → here.
if [ "$RUN_TIMED_OUT" = "1" ]; then
ec=1
elapsed=$(( $(date +%s) - MON_T0 ))
skipped="${RUN_SKIPPED_PHASES[*]:-}"
# Stable id for jump links (console HTML + err-top banner)
printf '\n'
printf '╔══════════════════════════════════════════════════════════╗\n'
printf '║ RUN TIMEOUT · extraordinary (see jump target below) ║\n'
printf '╚══════════════════════════════════════════════════════════╝\n'
printf '┌ ERROR ┐ #run.timeout-01 RUN_TIMEOUT exceeded · budget %ss · elapsed %ss\n' \
"$RUN_TIMEOUT" "$elapsed"
printf ' detail: wall-clock limit hit'
if [ -n "${RUN_TIMEOUT_AT_PHASE:-}" ]; then
printf ' during/after phase "%s"' "$RUN_TIMEOUT_AT_PHASE"
fi
if [ -n "$skipped" ]; then
printf ' · skipped: %s' "$skipped"
fi
printf '\n'
printf ' id=run.timeout-01\n'
printf ' hint: raise RUN_TIMEOUT= (seconds) or set RUN_TIMEOUT=0 for unlimited\n'
printf -- '┌ RUN TIMEOUT · extraordinary ┐\n'
printf -- ' • run.timeout-01 [run] RUN_TIMEOUT=%ss exceeded (elapsed %ss)%s%s\n' \
"$RUN_TIMEOUT" "$elapsed" \
"${RUN_TIMEOUT_AT_PHASE:+ · phase $RUN_TIMEOUT_AT_PHASE}" \
"${skipped:+ · skipped $skipped}"
printf -- '--- ERRORS (failed checks) ---\n'
printf -- ' • run.timeout-01 [run] RUN_TIMEOUT=%ss exceeded (elapsed %ss)\n' \
"$RUN_TIMEOUT" "$elapsed"
fi
exit "$ec"