hernani/koopa-admin-log
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: hernani:d6f4f649c1d151e4250a75999dc39320d163879f
Branches Tags
hernani:main
..
compare: hernani:78bbd80d1fb604bea9e578e7eb3df8d4dff80734
Branches Tags
hernani:main

No commits in common. "d6f4f649c1d151e4250a75999dc39320d163879f" and "78bbd80d1fb604bea9e578e7eb3df8d4dff80734" have entirely different histories.
d6f4f649c1 ... 78bbd80d1f

11 changed files with 199 additions and 1054 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -5,4 +5,3 @@ __pycache__/
*.swp *.swp
*~ *~
.tmp/ .tmp/
scripts/taler-monitoring/settings.conf

145
2026/2026-07-16--taler-package-upgrade-goa.md
View file

@ -1,145 +0,0 @@
# 2026-07-16 — GOA container package upgrade (trixie debs)
## Why
`taler-monitoring.sh versions` reported packages **behind** `deb.taler.net` **trixie**:
| Component | Package | Was | Target (trixie index) |
|-----------|---------|-----|------------------------|
| bank | libeufin-bank / common | 1.6.6 | **1.6.7** |
| exchange | taler-exchange* / libtalerexchange | 1.6.6 | **1.6.7~dev2** |
| merchant | taler-merchant-webui | 1.6.9 | **1.6.11** |
| merchant | taler-merchant / libtalermerchant | 1.6.9 | still 1.6.9 (no newer in suite for those) |
Host is openSUSE Tumbleweed — **irrelevant** for these versions. Debs live **inside** Debian-based podman images.
## Containers
| Role | Name |
|------|------|
| bank | `taler-hacktivism-bank` |
| exchange | `taler-hacktivism-exchange-ansible` |
| merchant | `taler-hacktivism` |
**No systemd as PID 1** in these containers. Do **not** rely on `systemctl restart` after apt (policy-rc.d / no bus). Restart with:
```bash
# bank
podman exec -u root taler-hacktivism-bank \
runuser -u libeufin-bank -- /usr/local/bin/start_bank.sh --restart
# merchant
podman exec -u root taler-hacktivism \
runuser -u taler-merchant-httpd -- /usr/local/bin/start_merchant.sh --restart
# exchange: base (root) then start_exchange as httpd user
podman exec -u root taler-hacktivism-exchange-ansible \
bash -c '/root/start_base_services_for_taler_exchange.sh --no-shell 2>/dev/null; \
runuser -u taler-exchange-httpd -- /usr/local/bin/start_exchange.sh --restart'
```
## Procedure (manual)
```bash
# as root inside each container (example bank)
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y libeufin-bank libeufin-common # bank
# exchange: taler-exchange taler-exchange-database libtalerexchange …
# merchant: taler-merchant-webui (and stack as needed)
```
Then restart with `start_*.sh` as above.
**Scripted (host):** `scripts/taler-shared/upgrade-goa-debs.sh`
Copy to koopa or run from a checkout: `./upgrade-goa-debs.sh` / `bank|exchange|merchant`.
## Lessons from 2026-07-16 run
1. **`apt-get install` as root works** (`podman exec -u root`); suite `trixie` (+ merchant may also have `trixie-testing`).
2. **`taler-exchange-dbinit` as root fails** (`role "root" does not exist`) — run as `taler-exchange-httpd` if needed.
3. **Bank after upgrade:** if postgres socket was down, bank dies with pool init error; `pg_ctlcluster 17 main start` then `start_bank.sh --restart`.
4. **Merchant `taler-merchant-dbinit` / merchant-0041.sql** can noise-fail (psql cluster path); after `start_merchant.sh --restart`, health check can still be green — verify `https://127.0.0.1:9010/config` and public `taler.hacktivism.ch`.
5. Exchange **postinst** may warn about missing SPA files under `/usr/share/taler-exchange/{aml,kyc}-spa/`; packages `taler-exchange-aml-webui` / `kyc-webui` pull in SPAs — re-check if AML UI is used.
6. **Images are live-writable** (not immutable rebuild): upgraded debs are in the running container layers until next image rebuild/snapshot.
## Smoke after upgrade
```bash
curl -sS -o /dev/null -w "%{http_code}\n" http://127.0.0.1:9012/config
curl -sS -o /dev/null -w "%{http_code}\n" http://127.0.0.1:9011/config
curl -skS -o /dev/null -w "%{http_code}\n" https://127.0.0.1:9010/config
# public
curl -skS -o /dev/null -w "%{http_code}\n" https://bank.hacktivism.ch/config
curl -skS -o /dev/null -w "%{http_code}\n" https://exchange.hacktivism.ch/config
curl -skS -o /dev/null -w "%{http_code}\n" https://taler.hacktivism.ch/config
```
Laptop: `./scripts/taler-monitoring/taler-monitoring.sh versions` (or `urls`).
## Result (this day)
- bank **1.6.7**, exchange **1.6.7~dev2**, merchant-webui **1.6.11**
- public `/config` **200** for bank / exchange / merchant after restarts
- `taler-monitoring.sh -d hacktivism.ch versions` → **behind=0** (all package rows OK)
## Specific residual problems (re-checked after upgrade)
Severity: **P1** = breaks settlement/ops soon · **P2** = noise / tooling · **P3** = hygiene.
### P1 — none known after restarts
Settlement path: bank/exchange/merchant `/config` 200; merchant health can pass including wirewatch (see below).
### P2 — `taler-merchant-dbinit` fails when run by hand
```text
WARNING Could not run PSQL on file …/global_procedures.sql: psql exit code was 1
ERROR Failed to initialize tables
```
- **Schema is actually current:** `_v.patches` includes **`merchant-0041`** (applied 2026-07-16, by role `postgres` during upgrade noise).
- **Service works:** `check_merchant-health.sh` → ALL CRITICAL CHECKS PASSED when helpers are up.
- **Cause (likely):** service-user `psql` / cluster path (`Error: Invalid data directory for cluster 17 main` when run poorly), not missing migrations.
- **Action:** do not treat hand-run `dbinit` exit≠0 as deploy failure; verify `_v.patches` + `/config`. Optional follow-up: fix `pg_wrapper`/`.postgresqlrc` for `taler-merchant-httpd` so dbinit is clean.
### P2 — merchant health vs wirewatch (race / detect)
- `check_merchant-health.sh` uses `pgrep` for `taler-merchant-wirewatch`.
- Wirewatch is supervised by **`taler-merchant-wirewatch-supervise.sh`** (restarts on PG NOTIFY exit).
- During restart windows the helper can be absent for seconds → **false FAIL**.
- When wirewatch + supervise are both live, health reports **OK**.
- **Action:** re-run health after 5s; ensure supervise is started after package upgrades (start_merchant / ensure_helpers).
### P2 — `check_exchange-health.sh` missing in live exchange container
```text
exec: "/usr/local/bin/check_exchange-health.sh": no such file or directory
```
- Script exists in **admin-log** (`scripts/taler-exchange/check_exchange-health.sh`) but was **not installed** into `taler-hacktivism-exchange-ansible`.
- **Action:** copy into image/live `/usr/local/bin/` on next deploy (same as bank/merchant health scripts).
### P3 — zombie processes inside bank + merchant containers
- **bank:** defunct `java`, `python3`, occasional `postgres` / `dpkg-preconfigu`.
- **merchant:** many old `taler-merchant-*` defunct (pre-restart leftovers).
- **Cause:** no proper init/reaper (not systemd PID 1); supervise/start scripts leave zombies.
- **Impact:** mostly cosmetic / PID table clutter unless extreme.
- **Action:** periodic container restart or install a tiny reaper; not urgent.
### P3 — package skew (informational)
| Package | Installed | Note |
|---------|-----------|------|
| taler-merchant / libtalermerchant / typst | 1.6.9 | no newer in trixie index at check time |
| taler-merchant-webui | 1.6.11 | intentionally newer SPA |
| exchange aml/kyc webui | 1.6.8~dev3 | pulled with exchange upgrade |
Monitoring does **not** ERROR on this skew when suite index matches installed.
### Resolved during upgrade (do not re-open without evidence)
- Version **behind** bank/exchange/webui — fixed.
- Exchange postinst “missing aml-spa/forms.json” — paths **`/usr/share/taler-exchange/{aml,kyc}-spa`** now present after webui packages.
- Bank down after apt — fixed with postgres socket + `start_bank.sh --restart`.

5
scripts/README.md
View file

@ -9,7 +9,6 @@
| `taler-shared/` | host **ensure-taler-apps** (post-container app start + auto-confirm) | | `taler-shared/` | host **ensure-taler-apps** (post-container app start + auto-confirm) |
| `taler-sanity/` | host root checks (stack, settlement, helpers) | | `taler-sanity/` | host root checks (stack, settlement, helpers) |
| `taler-monitoring/` | **outside-in** public URL walk (`/config` → keys/terms/integration/webui) | | `taler-monitoring/` | **outside-in** public URL walk (`/config` → keys/terms/integration/webui) |
| `taler-shared/` | shared helpers (`upgrade-goa-debs.sh` = apt upgrade **inside** bank/exchange/merchant containers) |
| `taler-landing/` | **hernani user systemd**: central landing `stats.json` (full bank scan + alt units) | | `taler-landing/` | **hernani user systemd**: central landing `stats.json` (full bank scan + alt units) |
| `monitoring/` | host `/home/hernani/scripts` (tor relay stats) | | `monitoring/` | host `/home/hernani/scripts` (tor relay stats) |
| `nym/` | **koopa-nym** build/up/status (nym.com nym-node) | | `nym/` | **koopa-nym** build/up/status (nym.com nym-node) |
@ -62,8 +61,4 @@ Exchange one-shots (root, offline / wire):
`runuser -u USER -- bash` (never `-u` with `-s` on util-linux). `runuser -u USER -- bash` (never `-u` with `-s` on util-linux).
### Package upgrades (trixie debs inside containers)
Containers are **not** systemd. After `apt-get install` of Taler/libeufin packages, restart with `start_*.sh` only — see **`taler-shared/upgrade-goa-debs.sh`** and day note `2026/2026-07-16--taler-package-upgrade-goa.md`. Host `zypper` does not update these.
SMS helper symlinks into `/var/taler-src/...` are not copied (merchant only). SMS helper symlinks into `/var/taler-src/...` are not copied (merchant only).

69
scripts/taler-monitoring/README.md
View file

@ -33,68 +33,16 @@ End of each phase: totals + list of **BLOCKERS** and **ERRORS**.
# Other domains — public HTTPS only, never SSH # Other domains — public HTTPS only, never SSH
./taler-monitoring.sh -d taler.net ./taler-monitoring.sh -d taler.net
./taler-monitoring.sh --domain taler-ops.ch ./taler-monitoring.sh --domain taler-ops.ch
./taler-monitoring.sh -d my.taler-ops.ch urls
./taler-monitoring.sh -d demo.taler.net urls ./taler-monitoring.sh -d demo.taler.net urls
./taler-monitoring.sh taler.net # bare domain = same as -d ./taler-monitoring.sh taler.net # bare domain = same as -d
# Explicit endpoints (any mix; override profile or skip -d)
./taler-monitoring.sh --bank https://bank.demo.taler.net \
--exchange https://exchange.demo.taler.net \
--merchant https://backend.demo.taler.net --currency KUDOS urls
``` ```
## Domain profiles (`domains.conf`) | Domain | Bank | Exchange | Merchant | Currency |
|--------|------|----------|----------|----------|
Each monitored stack must say **what is bank, exchange, and merchant-backend**. | `hacktivism.ch` (default) | bank.hacktivism.ch | exchange.hacktivism.ch | taler.hacktivism.ch | GOA |
Edit **[domains.conf](./domains.conf)** (or set `TALER_DOMAINS_CONF=`):
```text
# name bank exchange merchant-backend currency local [canonical]
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0
```
| Field | Meaning |
|-------|---------|
| **name** | `-d NAME` / `TALER_DOMAIN` match (add alias lines as needed) |
| **bank** | libeufin/bank public host or `https://…` |
| **exchange** | exchange public host or URL |
| **merchant** | merchant **backend** (SPA/API), e.g. `my.taler-ops.ch` |
| **currency** | `GOA` / `KUDOS` / `CHF` / … |
| **local** | `1` = koopa SSH stack; `0` = public only |
| **landing** | `1` = check `/intro` landings + assets; `0` = skip (TOPS / mytops) |
| **canonical** | optional label after alias (e.g. `my.taler-ops.ch` → `taler-ops.ch`) |
Then:
```bash
./taler-monitoring.sh -d taler-ops.ch urls
```
Overrides (always win after profile load):
```bash
./taler-monitoring.sh -d taler-ops.ch \
--merchant https://my.taler-ops.ch \
--exchange https://exchange.taler-ops.ch \
--bank https://bank.taler-ops.ch urls
```
Helpers in `lib.sh`: `set_taler_stack`, `load_domain_profile`, `apply_taler_domain`.
### Built-in profiles (see `domains.conf`)
| Domain / alias | Bank | Exchange | Merchant | Currency |
|----------------|------|----------|----------|----------|
| `hacktivism.ch` / `koopa` | bank.hacktivism.ch | exchange.hacktivism.ch | taler.hacktivism.ch | GOA |
| `taler.net` / `demo.taler.net` | bank.demo.taler.net | exchange.demo.taler.net | backend.demo.taler.net | KUDOS | | `taler.net` / `demo.taler.net` | bank.demo.taler.net | exchange.demo.taler.net | backend.demo.taler.net | KUDOS |
| `test.taler.net` | bank.test… | exchange.test… | backend.test… | **TESTKUDOS** · no landings | | `taler-ops.ch` | bank.* (probe) | exchange.taler-ops.ch | backend.* (probe) | CHF |
| `lefrancpaysan.ch` | bank.… | exchange.… | **monnaie.…** (merchant) | report-only · no landings | | other | bank.DOMAIN | exchange.DOMAIN | backend/taler/merchant (probe) | any |
| `taler-ops.ch` / `my.taler-ops.ch` | bank.taler-ops.ch | exchange.taler-ops.ch | **my.taler-ops.ch** | CHF · **no landings** |
| `stage.taler-ops.ch` | bank.stage… | exchange.stage… | **my.stage…** | CHF · **no landings** |
| unknown | bank.DOMAIN | exchange.DOMAIN | my.DOMAIN then probe | any · no landings |
**Landings:** only GOA (and optionally demo) use public `/intro` pages. **taler-ops.ch does not** — `CHECK_LANDING=0` in the profile skips intro crawl, shop assets, and demo-withdraw checks.
**SSH only for koopa** (`hacktivism.ch` / `-d koopa`). **SSH only for koopa** (`hacktivism.ch` / `-d koopa`).
@ -104,9 +52,8 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC.
| | Local (koopa) | Remote | | | Local (koopa) | Remote |
|--|---------------|--------| |--|---------------|--------|
| **ATM withdraw** | 20 · 50 · 100 · 200 · **4200** (paivana) | 10 · 20 · 50 | | **ATM withdraw** | 20 · 50 · 100 · 200 | 10 · 20 · 50 |
| **Pay ladder** | 0.01 … 10 | 0.01 … 1 | | **Pay ladder** | 0.01 … 10 | 0.01 … 1 |
| **Paivana** | HTTP paywall + template pay **GOA:4200** (`goa-shop` / `paivana`) | skipped |
```bash ```bash
./taler-monitoring.sh e2e ./taler-monitoring.sh e2e
@ -119,10 +66,6 @@ E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # s
# E2E_SHOP_PRODUCTS lines: id|Product name|GOA:amount # E2E_SHOP_PRODUCTS lines: id|Product name|GOA:amount
# E2E_SHOP_PICK_N=2 # E2E_SHOP_PICK_N=2
# (landing QR = taler://pay-template/…/{id}; popup = live taler://pay after POST templates/{id}) # (landing QR = taler://pay-template/…/{id}; popup = live taler://pay after POST templates/{id})
# Paivana paywall (local GOA only):
# E2E_PAIVANA=1 (default) PAIVANA_PUBLIC=https://paivana.hacktivism.ch
# E2E_PAIVANA_TEMPLATE=paivana E2E_PAIVANA_AMOUNT=GOA:4200 E2E_PAIVANA_INSTANCE=goa-shop
# E2E_PAIVANA=0 # skip
# remote secrets: # remote secrets:
# E2E_BANK_ADMIN_PASS=… E2E_MERCHANT_TOKEN=… # E2E_BANK_ADMIN_PASS=… E2E_MERCHANT_TOKEN=…
``` ```

9
scripts/taler-monitoring/TESTS.md
View file

@ -43,17 +43,12 @@ IDs are assigned **in run order** within the area (`set_area` resets the counter
| www-… | **merchant `/terms`** body (dual-currency notice) | | www-… | **merchant `/terms`** body (dual-currency notice) |
| www-… | **merchant `/privacy`** body (must not be `not configured`) | | www-… | **merchant `/privacy`** body (must not be `not configured`) |
| www-… | merchant `/terms/` redirect | | www-… | merchant `/terms/` redirect |
| www-… | **landing (aggregated per site)**: one line for bank/merchant/exchange (`/intro` + assets + own/external link counts); failures listed as samples; soft external/shop assets compact |
| www-… | cross-links: one line (4 intros) on local stack |
| www-… | bank demo-withdraw + shop assets: one line (+ soft shop-pay if missing) |
| www-… | **bank `/intro/demo-withdraw.json`** → `taler://withdraw/HOST:PORT/taler-integration/…` + integration op HTTP 200 |
| www-… | **landing exposed links** (bank / merchant / exchange): parse each `/intro/` HTML, probe every own-stack `https://` + root-relative `href`/`src`/`content`, soft-check external stores/docs | | www-… | **landing exposed links** (bank / merchant / exchange): parse each `/intro/` HTML, probe every own-stack `https://` + root-relative `href`/`src`/`content`, soft-check external stores/docs |
| www-… | landing static: `qrcode.min.js`, `og-goa-shop.png`, `qr-logo.png`, shop-pay.js/css | | www-… | landing static: `qrcode.min.js`, `og-goa-shop.png`, `qr-logo.png`, shop-pay.js/css |
| www-… | cross-links between bank ↔ merchant ↔ exchange intros (local stack) | | www-… | cross-links between bank ↔ merchant ↔ exchange intros (local stack) |
| www-… | **bank `/intro/demo-withdraw.json`** → `taler://withdraw/HOST/taler-integration/…` (no default `:443`/`:80`; non-default port OK) + integration op HTTP 200 | | www-… | **bank `/intro/demo-withdraw.json`** → `taler://withdraw/HOST/taler-integration/…` (no default `:443`/`:80`; non-default port OK) + integration op HTTP 200 |
| www-… | bank `/intro/auto-account.json` (earlier) → same withdraw shape, **no payto_uri**, login at `/webui/` | | www-… | bank `/intro/auto-account.json` (earlier) → same withdraw shape, **no payto_uri**, login at `/webui/` |
| www-… | **performance** (outside-in RTT): bank/exchange/merchant `/config`, keys, webui, intro — ms; WARN ≥ `PERF_WARN_MS` (8000); ERROR ≥ `PERF_FAIL_MS` (20000) | | www-… | **performance** (outside-in): public HTTPS RTT for bank `/config`, `/taler-integration/config`, `/webui/`, `/intro/`, `stats.json`; exchange `/config`, `/keys`, `/intro/`; merchant `/config`, `/webui/`, `/intro/` — report ms; WARN ≥ `PERF_WARN_MS` (default 8000); **ERROR ≥ `PERF_FAIL_MS` (default 20000)** |
| www-… | **landing load stats**: `/intro/stats.json` → loadavg, container RSS, in-container probe ms (public; SSH container cat fallback); warn if stale/missing |
**Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. On local stack, optional content needle (terms/privacy/FADP/GOA…). **Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. On local stack, optional content needle (terms/privacy/FADP/GOA…).
@ -143,8 +138,6 @@ Without SSH (`SKIP_SSH=1` or remote domain): still runs outside-in repo checks;
## e2e — payment path (`./taler-monitoring.sh e2e`) ## e2e — payment path (`./taler-monitoring.sh e2e`)
Local GOA also: **ATM includes GOA:4200**, then **paivana** (HTTP 302 on `PAIVANA_PUBLIC` + public template pay `paivana` / GOA:4200 on `goa-shop`). Disable with `E2E_PAIVANA=0`.
| ID | Step (approx.) | | ID | Step (approx.) |
|----|----------------| |----|----------------|
| e2e-001 | budget info | | e2e-001 | budget info |

155
scripts/taler-monitoring/check_e2e.sh
View file

@ -216,14 +216,14 @@ except Exception: print("")' 2>/dev/null || true)
# E2E_ATM_MAX=N # try at most N ATM withdraws (budget) # E2E_ATM_MAX=N # try at most N ATM withdraws (budget)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
: "${E2E_VARIABLE:=1}" : "${E2E_VARIABLE:=1}"
: "${E2E_ATM_MAX:=5}" : "${E2E_ATM_MAX:=4}"
if [ "$E2E_REMOTE" = "1" ]; then if [ "$E2E_REMOTE" = "1" ]; then
# remote: still ATM-shaped but smaller notes; keep cheap # remote: still ATM-shaped but smaller notes; keep cheap
: "${E2E_WITHDRAW_VALUES:=10 20 50}" : "${E2E_WITHDRAW_VALUES:=10 20 50}"
: "${E2E_PAY_VALUES:=0.01 0.05 0.1 1}" : "${E2E_PAY_VALUES:=0.01 0.05 0.1 1}"
else else
# local GOA: classic ATM notes + 4200 for paivana paywall template # local GOA: classic ATM denominations
: "${E2E_WITHDRAW_VALUES:=20 50 100 200 4200}" : "${E2E_WITHDRAW_VALUES:=20 50 100 200}"
: "${E2E_PAY_VALUES:=0.01 0.05 0.1 0.5 1 2 5 10}" : "${E2E_PAY_VALUES:=0.01 0.05 0.1 0.5 1 2 5 10}"
fi fi
@ -275,24 +275,12 @@ PAY_AMT=$(printf '%s' "$PAY_LIST" | awk '{print $1}')
if [ "$E2E_VARIABLE" = "1" ]; then if [ "$E2E_VARIABLE" = "1" ]; then
n_w=$(printf '%s' "$WITHDRAW_LIST" | wc -w | tr -d ' ') n_w=$(printf '%s' "$WITHDRAW_LIST" | wc -w | tr -d ' ')
n_p=$(printf '%s' "$PAY_LIST" | wc -w | tr -d ' ') n_p=$(printf '%s' "$PAY_LIST" | wc -w | tr -d ' ')
need=$(( 50 + n_w * 40 + n_p * 20 + 60 )) # + room for paivana pay need=$(( 40 + n_w * 35 + n_p * 20 ))
if [ "${E2E_TIMEOUT}" -lt "$need" ]; then if [ "${E2E_TIMEOUT}" -lt "$need" ]; then
E2E_TIMEOUT=$need E2E_TIMEOUT=$need
fi fi
# cap ATM attempts but always keep CUR:4200 if listed (paivana) # cap ATM attempts
WITHDRAW_LIST=$( WITHDRAW_LIST=$(printf '%s' "$WITHDRAW_LIST" | tr ' ' '\n' | head -n "$E2E_ATM_MAX" | tr '\n' ' ' | sed 's/ *$//')
printf '%s' "$WITHDRAW_LIST" | tr ' ' '\n' | awk -v max="$E2E_ATM_MAX" -v big="${CUR}:4200" '
NF==0 { next }
$0==big { has=1; next }
{ small[++n]=$0 }
END {
keep = max - (has ? 1 : 0)
if (keep < 0) keep = 0
for (i = 1; i <= n && i <= keep; i++) printf "%s ", small[i]
if (has) printf "%s", big
print ""
}' | sed 's/ *$//'
)
fi fi
BANK_HOST=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$BANK_PUBLIC"'").hostname or "bank")' 2>/dev/null || echo bank) BANK_HOST=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$BANK_PUBLIC"'").hostname or "bank")' 2>/dev/null || echo bank)
@ -302,7 +290,7 @@ WDB="$SCRATCH/wallet.sqlite3"
METRICS_DIR="${METRICS_DIR:-$SCRATCH/metrics}" METRICS_DIR="${METRICS_DIR:-$SCRATCH/metrics}"
mkdir -p "$METRICS_DIR" mkdir -p "$METRICS_DIR"
export METRICS_DIR CUR="${CUR:-${EXPECT_CURRENCY:-GOA}}" WDB export METRICS_DIR CUR="${CUR:-${EXPECT_CURRENCY:-GOA}}" WDB
export PATH="${HOME}/.local/bin:${HOME}/taler/opt/taler-wallet-cli/usr/bin:/opt/homebrew/bin:/usr/local/bin:$PATH" export PATH="/opt/homebrew/bin:/usr/local/bin:$PATH"
# Always dump balances on any exit (timeout, blocker, signal, success) # Always dump balances on any exit (timeout, blocker, signal, success)
trap 'ec=$?; e2e_finish "$ec"; rm -rf "$SCRATCH"; exit "$ec"' EXIT trap 'ec=$?; e2e_finish "$ec"; rm -rf "$SCRATCH"; exit "$ec"' EXIT
@ -331,41 +319,7 @@ WALLET_CLI=$(find_wallet_cli) || {
blocker "prereq" "taler-wallet-cli not found (set WALLET_CLI=)" blocker "prereq" "taler-wallet-cli not found (set WALLET_CLI=)"
exit 1 exit 1
} }
# Implementation version (--version) + wallet-core API ranges (version command) ok "wallet-cli ($WALLET_CLI)"
WALLET_IMPL_VER=$(
node "$WALLET_CLI" --version 2>/dev/null | head -1 | tr -d '\r' || true
)
WALLET_CORE_VER=$(
_wver_db=$(mktemp "${TMPDIR:-/tmp}/wver.XXXXXX.sqlite3" 2>/dev/null || echo "${TMPDIR:-/tmp}/wver-$$.sqlite3")
node "$WALLET_CLI" --wallet-db="$_wver_db" --no-throttle --skip-defaults version 2>/dev/null \
| python3 -c '
import json,re,sys
raw=sys.stdin.read()
d=None
for m in re.finditer(r"\{", raw):
try:
d=json.loads(raw[m.start():])
if isinstance(d, dict) and "version" in d:
break
except Exception:
d=None
if not isinstance(d, dict):
sys.exit(0)
parts=[]
# wallet-core libversion + protocol ranges used against exchange/merchant/bank
for k in ("version","implementationSemver","exchange","merchant","bank","bankIntegrationApiRange","corebankApiRange"):
if k in d and d[k] is not None:
parts.append("%s=%s" % (k, d[k]))
print(" ".join(parts))
' 2>/dev/null || true
rm -f "$_wver_db" "$_wver_db"-* 2>/dev/null || true
)
ok "wallet-cli" "${WALLET_IMPL_VER:-?} · ${WALLET_CLI}"
if [ -n "${WALLET_CORE_VER:-}" ]; then
info "wallet-core" "$WALLET_CORE_VER"
else
info "wallet-core" "version details unavailable (impl ${WALLET_IMPL_VER:-?})"
fi
info "e2e mode" "$([ "$E2E_REMOTE" = "1" ] && echo "remote/public domain (no SSH)" || echo "local koopa stack")" info "e2e mode" "$([ "$E2E_REMOTE" = "1" ] && echo "remote/public domain (no SSH)" || echo "local koopa stack")"
info "currency" "$CUR" info "currency" "$CUR"
info "ATM withdraw ladder" "$WITHDRAW_LIST (credit $CREDIT_AMT)" info "ATM withdraw ladder" "$WITHDRAW_LIST (credit $CREDIT_AMT)"
@ -1296,104 +1250,11 @@ EOF
fi fi
fi fi
# ---------------------------------------------------------------------------
section "e2e · paivana paywall (template GOA:4200)"
# ---------------------------------------------------------------------------
# Public paywall: https://paivana.hacktivism.ch → 302 to merchant template "paivana".
# Requires spendable ≥ 4200 GOA (ATM ladder includes GOA:4200 by default).
: "${PAIVANA_PUBLIC:=https://paivana.hacktivism.ch}"
: "${E2E_PAIVANA:=1}"
: "${E2E_PAIVANA_TEMPLATE:=paivana}"
: "${E2E_PAIVANA_AMOUNT:=GOA:4200}"
: "${E2E_PAIVANA_INSTANCE:=goa-shop}"
PAIVANA_OK=0
PAIVANA_REPORT=""
if [ "$E2E_REMOTE" = "1" ] || [ "${CUR:-}" != "GOA" ] || [ "${E2E_PAIVANA}" = "0" ]; then
info "paivana" "SKIPPED (remote, non-GOA, or E2E_PAIVANA=0)"
PAIVANA_REPORT="SKIP"
else
PAIVANA_PUBLIC="${PAIVANA_PUBLIC%/}"
# HTTP: paywall front must redirect into template flow (or serve well-known)
hdr=$(curl -skS -m 12 -D - -o /dev/null "$PAIVANA_PUBLIC/" 2>/dev/null || true)
pcode=$(printf '%s' "$hdr" | awk 'BEGIN{c="000"} /^HTTP/{c=$2} END{print c}')
loc=$(printf '%s' "$hdr" | awk 'BEGIN{IGNORECASE=1} /^location:/{sub(/\r$/,""); sub(/^location:[[:space:]]*/,""); print; exit}')
case "$pcode" in
301|302|303|307|308)
if printf '%s' "$loc" | grep -qiE 'paivana|templates|well-known'; then
ok "paivana HTTP" "HTTP $pcode → ${loc:0:120}"
else
ok "paivana HTTP" "HTTP $pcode redirect (Location: ${loc:0:80})"
fi
;;
200)
warn "paivana HTTP" "HTTP 200 without paywall redirect — unexpected for locked site"
;;
*)
fail "paivana HTTP" "HTTP ${pcode:-000} on $PAIVANA_PUBLIC/ (want 302 to template)"
PAIVANA_REPORT="HTTP_FAIL"
;;
esac
# Ensure wallet can cover 4200 (extra ATM if ladder did not fund enough)
pay_need=$(python3 -c 'import sys; print(float(sys.argv[1].split(":",1)[-1]))' "$E2E_PAIVANA_AMOUNT" 2>/dev/null || echo 4200)
av_now=$(wallet_avail_num)
if ! python3 -c "import sys; sys.exit(0 if float(sys.argv[1]) + 1e-9 >= float(sys.argv[2]) else 1)" "$av_now" "$pay_need" 2>/dev/null; then
info "paivana" "avail ${CUR}:${av_now} < ${pay_need} — ATM withdraw ${E2E_PAIVANA_AMOUNT}"
e2e_over && warn "paivana" "budget low before 4200 withdraw"
set +e
e2e_one_withdraw "$E2E_PAIVANA_AMOUNT"
wc=$?
set -e
if [ "$wc" = "0" ]; then
WITHDRAW_OK=1
WITHDRAW_OK_N=$((WITHDRAW_OK_N + 1))
WITHDRAW_REPORT="${WITHDRAW_REPORT}${WITHDRAW_REPORT:+ }${E2E_PAIVANA_AMOUNT}=OK(paivana)"
elif [ "$wc" = "2" ]; then
WITHDRAW_LAG_N=$((WITHDRAW_LAG_N + 1))
WITHDRAW_REPORT="${WITHDRAW_REPORT}${WITHDRAW_REPORT:+ }${E2E_PAIVANA_AMOUNT}=LAG(paivana)"
else
WITHDRAW_FAIL_N=$((WITHDRAW_FAIL_N + 1))
WITHDRAW_REPORT="${WITHDRAW_REPORT}${WITHDRAW_REPORT:+ }${E2E_PAIVANA_AMOUNT}=FAIL(paivana)"
fi
wait_wallet_balance "$(python3 -c "print(max(0, $pay_need - 1))" 2>/dev/null || echo 4199)" 12 3 || true
av_now=$(wallet_avail_num)
fi
if ! python3 -c "import sys; sys.exit(0 if float(sys.argv[1]) + 1e-9 >= float(sys.argv[2]) else 1)" "$av_now" "$pay_need" 2>/dev/null; then
warn "paivana" "skip pay — insufficient avail ${CUR}:${av_now} (need ${pay_need})"
PAIVANA_REPORT="${PAIVANA_REPORT:+$PAIVANA_REPORT+}SKIP(bal)"
else
_E2E_INST_SAVE="$INST"
INST="${E2E_PAIVANA_INSTANCE}"
set +e
e2e_one_pay_public_template \
"${E2E_PAIVANA_TEMPLATE}" \
"Paivana paywall" \
"${E2E_PAIVANA_AMOUNT}"
pc=$?
set -e
INST="$_E2E_INST_SAVE"
unset _E2E_INST_SAVE
if [ "$pc" = "0" ]; then
PAIVANA_OK=1
PAY_OK=1
PAY_OK_N=$((PAY_OK_N + 1))
PAIVANA_REPORT="${PAIVANA_REPORT:+$PAIVANA_REPORT+}PAY_OK"
ok "paivana" "template ${E2E_PAIVANA_TEMPLATE} paid ${E2E_PAIVANA_AMOUNT}"
else
PAIVANA_REPORT="${PAIVANA_REPORT:+$PAIVANA_REPORT+}PAY_FAIL"
warn "paivana" "template pay failed (${E2E_PAIVANA_TEMPLATE} · ${E2E_PAIVANA_AMOUNT})"
fi
fi
info "paivana summary" "${PAIVANA_REPORT:-?} · instance ${E2E_PAIVANA_INSTANCE}"
fi
section "e2e · report" section "e2e · report"
info "user" "$USER" info "user" "$USER"
info "ATM withdraws" "$WITHDRAW_REPORT" info "ATM withdraws" "$WITHDRAW_REPORT"
info "payments" "$PAY_REPORT" info "payments" "$PAY_REPORT"
info "goa-shop" "${SHOP_REPORT:-(n/a)}" info "goa-shop" "${SHOP_REPORT:-(n/a)}"
info "paivana" "${PAIVANA_REPORT:-(n/a)}"
info "final balance" "$(fmt_bal "$SCRATCH/bal-live.out" 2>/dev/null || echo "(n/a)")" info "final balance" "$(fmt_bal "$SCRATCH/bal-live.out" 2>/dev/null || echo "(n/a)")"
info "scratch" "$SCRATCH" info "scratch" "$SCRATCH"
# Success if we had coins and at least one pay, OR coins + only pay skips (nothing affordable) # Success if we had coins and at least one pay, OR coins + only pay skips (nothing affordable)

404
scripts/taler-monitoring/check_urls.sh
View file

@ -207,13 +207,9 @@ PY
fi fi
fi fi
fi fi
if [ "${CHECK_LANDING:-1}" = "1" ]; then check_url_soft "exchange /intro/" 200 "$EXCHANGE_PUBLIC/intro/"
check_url_soft "exchange /intro/" 200 "$EXCHANGE_PUBLIC/intro/" # Root should land on intro (302/301 then 200 on follow is checked separately)
# Root should land on intro (302/301 then 200 on follow is checked separately) check_url_soft "exchange /" 302,301,200 "$EXCHANGE_PUBLIC/"
check_url_soft "exchange /" 302,301,200 "$EXCHANGE_PUBLIC/"
else
info "exchange /intro" "skipped (CHECK_LANDING=0 · no public landings for this stack)"
fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Performance — outside-in public HTTPS latency (this runner, not loopback) # Performance — outside-in public HTTPS latency (this runner, not loopback)
@ -267,137 +263,20 @@ check_perf() {
# Bank first (wallet-critical paths before UI chrome) # Bank first (wallet-critical paths before UI chrome)
check_perf "perf bank /taler-integration/config" "$BANK_PUBLIC/taler-integration/config" check_perf "perf bank /taler-integration/config" "$BANK_PUBLIC/taler-integration/config"
check_perf "perf bank /config" "$BANK_PUBLIC/config" check_perf "perf bank /config" "$BANK_PUBLIC/config"
if [ "${CHECK_LANDING:-1}" = "1" ]; then check_perf "perf bank /intro/" "$BANK_PUBLIC/intro/"
check_perf "perf bank /intro/" "$BANK_PUBLIC/intro/" check_perf "perf bank /intro/stats.json" "$BANK_PUBLIC/intro/stats.json" 200
check_perf "perf bank /intro/stats.json" "$BANK_PUBLIC/intro/stats.json" 200
fi
check_perf "perf bank /webui/" "$BANK_PUBLIC/webui/" 200,301,302 check_perf "perf bank /webui/" "$BANK_PUBLIC/webui/" 200,301,302
# Exchange # Exchange
check_perf "perf exchange /config" "$EXCHANGE_PUBLIC/config" check_perf "perf exchange /config" "$EXCHANGE_PUBLIC/config"
check_perf "perf exchange /keys" "$EXCHANGE_PUBLIC/keys" check_perf "perf exchange /keys" "$EXCHANGE_PUBLIC/keys"
if [ "${CHECK_LANDING:-1}" = "1" ]; then check_perf "perf exchange /intro/" "$EXCHANGE_PUBLIC/intro/"
check_perf "perf exchange /intro/" "$EXCHANGE_PUBLIC/intro/"
fi
# Merchant # Merchant
check_perf "perf merchant /config" "$MERCHANT_PUBLIC/config" check_perf "perf merchant /config" "$MERCHANT_PUBLIC/config"
check_perf "perf merchant /webui/" "$MERCHANT_PUBLIC/webui/" 200,301,302 check_perf "perf merchant /webui/" "$MERCHANT_PUBLIC/webui/" 200,301,302
if [ "${CHECK_LANDING:-1}" = "1" ]; then check_perf "perf merchant /intro/" "$MERCHANT_PUBLIC/intro/"
check_perf "perf merchant /intro/" "$MERCHANT_PUBLIC/intro/"
fi
info "perf note" "RTT measured from this host (outside-in)"
# ---------------------------------------------------------------------------
# Landing load stats (stats.json written *inside* containers, usually public).
# Prefer HTTPS /intro/stats.json (no SSH). If missing and SSH works, cat from
# the container. Report loadavg + RSS + in-container probe ms — one line each.
# ---------------------------------------------------------------------------
: "${STATS_STALE_SECS:=900}" # warn if generated_at older than 15m
_fetch_landing_stats_json() {
# $1=name bank|exchange|merchant $2=base URL → writes $tmp/stats-$1.json, exit 0 if ok
local name="$1" base="$2"
local f="$tmp/stats-${name}.json"
local code ctr path
rm -f "$f"
code=$(http_body "${base}/intro/stats.json" "$f" 2>/dev/null || echo 000)
if [ "$code" = "200" ] && [ -s "$f" ]; then
return 0
fi
# Optional inside-container fallback (no public path or empty)
if [ "${LOCAL_STACK:-0}" = "1" ] && [ "${SKIP_SSH:-0}" != "1" ] && koopa_ssh_ok 2>/dev/null; then
case "$name" in
bank) ctr=taler-hacktivism-bank; path=/var/www/bank-landing/stats.json ;;
exchange) ctr=taler-hacktivism-exchange-ansible; path=/var/www/exchange-landing/stats.json ;;
merchant) ctr=taler-hacktivism; path=/var/www/merchant-landing/stats.json ;;
*) return 1 ;;
esac
if koopa_ssh_run 12 "podman exec ${ctr} cat ${path} 2>/dev/null" >"$f" 2>/dev/null \
&& [ -s "$f" ]; then
return 0
fi
fi
return 1
}
report_landing_load_stats() {
local name="$1" base="$2"
local f="$tmp/stats-${name}.json"
local line age_s
if ! _fetch_landing_stats_json "$name" "$base"; then
if [ "${CHECK_LANDING:-1}" = "1" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
warn "perf ${name} landing-stats" "stats.json not available (public or SSH)"
fi
return
fi
line=$(python3 - "$f" "${STATS_STALE_SECS}" <<'PY' 2>/dev/null || true
import json, sys, time
path, stale = sys.argv[1], int(sys.argv[2])
try:
d = json.load(open(path))
except Exception as e:
print("ERR parse:" + str(e)[:80])
sys.exit(0)
if not d.get("ok", True):
print("ERR ok=false " + str(d.get("error") or "")[:80])
sys.exit(0)
p = d.get("performance") or {}
if not isinstance(p, dict) or not p:
print("ERR no performance block")
sys.exit(0)
mem = p.get("memory") if isinstance(p.get("memory"), dict) else {}
rss = mem.get("container_rss_human") or mem.get("proc_sum_rss_human") or "?"
load = p.get("loadavg") or "?"
# latency fields differ by site
bits = []
for k in ("config_ms", "integration_ms", "webui_ms", "keys_ms", "terms_ms"):
if k in p and p[k] is not None:
bits.append("%s=%sms" % (k.replace("_ms", ""), p[k]))
lat = " ".join(bits) if bits else "latency=?"
gen = d.get("generated_at_human") or d.get("generated_at") or "?"
# staleness
age = ""
try:
gu = d.get("generated_at_unix")
if gu is not None:
age_s = int(time.time()) - int(gu)
age = " age=%ss" % age_s
if age_s > stale:
print("STALE loadavg=%s RSS=%s %s gen=%s%s" % (load, rss, lat, gen, age))
sys.exit(0)
except Exception:
pass
print("OK loadavg=%s RSS=%s %s gen=%s%s" % (load, rss, lat, gen, age))
PY
)
case "$line" in
OK\ *)
ok "perf ${name} landing-stats" "${line#OK }"
;;
STALE\ *)
warn "perf ${name} landing-stats" "stale · ${line#STALE }"
;;
ERR\ *)
warn "perf ${name} landing-stats" "${line#ERR }"
;;
*)
warn "perf ${name} landing-stats" "unreadable stats.json"
;;
esac
}
if [ "${CHECK_LANDING:-1}" = "1" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
section "www · performance · landing load stats (stats.json · in-container probes)"
report_landing_load_stats "bank" "$BANK_PUBLIC"
report_landing_load_stats "exchange" "$EXCHANGE_PUBLIC"
report_landing_load_stats "merchant" "$MERCHANT_PUBLIC"
info "perf landing-stats note" "from /intro/stats.json (public); SSH container fallback if needed"
fi
# Rollup + optional JSON for metrics_print_overall # Rollup + optional JSON for metrics_print_overall
if [ -s "$PERF_TSV" ]; then if [ -s "$PERF_TSV" ]; then
PERF_JSON="$tmp/perf-summary.json" PERF_JSON="$tmp/perf-summary.json"
@ -590,9 +469,7 @@ PY
rm -f /tmp/alt-mer.$$ rm -f /tmp/alt-mer.$$
# Follow each exchange listed in merchant /config and require its /config alt_unit_names # Follow each exchange listed in merchant /config and require its /config alt_unit_names
check_merchant_listed_exchanges_alt_units "$tmp/mc.json" check_merchant_listed_exchanges_alt_units "$tmp/mc.json"
if [ "${CHECK_LANDING:-1}" = "1" ]; then
check_url_soft "merchant /intro/" 200 "$MERCHANT_PUBLIC/intro/" check_url_soft "merchant /intro/" 200 "$MERCHANT_PUBLIC/intro/"
fi
check_url_soft "merchant /webui/" 200 "$MERCHANT_PUBLIC/webui/" check_url_soft "merchant /webui/" 200 "$MERCHANT_PUBLIC/webui/"
check_url_soft "merchant /" 302,301,200 "$MERCHANT_PUBLIC/" check_url_soft "merchant /" 302,301,200 "$MERCHANT_PUBLIC/"
fi fi
@ -606,92 +483,49 @@ case "$code" in
*) warn "merchant /terms/" "HTTP $code (expect 302 → /terms)" ;; *) warn "merchant /terms/" "HTTP $code (expect 302 → /terms)" ;;
esac esac
# ---------------------------------------------------------------------------
# Paivana paywall (local GOA stack) — public front only; pay path is e2e
# ---------------------------------------------------------------------------
if [ "${LOCAL_STACK:-1}" = "1" ] && [ "${E2E_PAIVANA:-1}" != "0" ]; then
section "www · paivana paywall"
: "${PAIVANA_PUBLIC:=https://paivana.hacktivism.ch}"
PAIVANA_PUBLIC="${PAIVANA_PUBLIC%/}"
hdr=$(curl -skS -m "${TIMEOUT}" -D - -o /dev/null "${PAIVANA_PUBLIC}/" 2>/dev/null || true)
pcode=$(printf '%s' "$hdr" | awk 'BEGIN{c="000"} /^HTTP/{c=$2} END{print c}')
loc=$(printf '%s' "$hdr" | awk 'BEGIN{IGNORECASE=1} /^location:/{sub(/\r$/,""); sub(/^location:[[:space:]]*/,""); print; exit}')
case "$pcode" in
301|302|303|307|308)
if printf '%s' "$loc" | grep -qiE 'paivana|templates|well-known'; then
ok "paivana /" "HTTP $pcode → template flow"
else
ok "paivana /" "HTTP $pcode redirect"
fi
info "paivana Location" "${loc:0:140}"
;;
200)
warn "paivana /" "HTTP 200 (expected paywall redirect to template)"
;;
*)
warn "paivana /" "HTTP ${pcode:-000} — ${PAIVANA_PUBLIC}/ (e2e pay may still work via template)"
;;
esac
fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Landing pages: every HTTPS link exposed on bank / merchant / exchange intros # Landing pages: every HTTPS link exposed on bank / merchant / exchange intros
# + required static assets + bank withdraw mint (taler://withdraw only) # + required static assets + bank withdraw mint (taler://withdraw only)
# Skipped when CHECK_LANDING=0 (e.g. taler-ops.ch — no GOA-style landings).
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
if [ "${CHECK_LANDING:-1}" != "1" ]; then
section "www · landing pages"
info "landing checks" "skipped (CHECK_LANDING=0 · stack has no public /intro landings)"
summary
exit 0
fi
section "www · landing exposed links · bank / merchant / exchange" section "www · landing exposed links · bank / merchant / exchange"
# Probe one URL: print code to stdout (200 after following redirects counts as 200). # Known-good landing static paths (relative to each host base)
# Sets _landing_code. Exit 0 if OK (200 or redirect→200), 1 otherwise. # Checked even if HTML parse misses them.
_landing_probe() { check_landing_asset() {
local url="$1" local label="$1" url="$2" soft="${3:-0}"
local code local code
code=$(http_code "$url") code=$(http_code "$url")
case "$code" in case "$code" in
200) _landing_code=200; return 0 ;; 200) ok "$label" "HTTP 200 · $url" ;;
301|302|303|307|308) 301|302|303|307|308)
code=$(curl -skS --max-redirs 5 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || echo 000) # follow once for assets that redirect
_landing_code="$code" code=$(curl -skS --max-redirs 3 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || echo 000)
[ "$code" = "200" ] && return 0 if [ "$code" = "200" ]; then
return 1 ok "$label" "HTTP redirect→200 · $url"
elif [ "$soft" = "1" ]; then
warn "$label" "HTTP $code — $url"
else
fail "$label" "HTTP $code after redirect — $url"
fi
;; ;;
*) *)
_landing_code="$code" if [ "$soft" = "1" ]; then
return 1 warn "$label" "HTTP $code — $url"
else
fail "$label" "HTTP $code — $url"
fi
;; ;;
esac esac
} }
# Known-good landing static paths — one report line via caller aggregate, or # Soft external (app stores / upstream docs): WARN if down, never ERROR
# soft=1 single warn. Returns 0 if ok.
check_landing_asset() {
local label="$1" url="$2" soft="${3:-0}"
if _landing_probe "$url"; then
return 0
fi
if [ "$soft" = "1" ]; then
warn "$label" "HTTP ${_landing_code:-?} — $url"
else
fail "$label" "HTTP ${_landing_code:-?} — $url"
fi
return 1
}
# Soft external: never ERROR; used only for failures in aggregated external probe.
check_external_soft() { check_external_soft() {
local label="$1" url="$2" local label="$1" url="$2"
local code local code
code=$(curl -skS --max-redirs 5 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || echo 000) code=$(curl -skS --max-redirs 5 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || echo 000)
case "$code" in case "$code" in
200|204|301|302|303|307|308) return 0 ;; 200|204|301|302|303|307|308) ok "$label" "HTTP $code · $url" ;;
*) warn "$label" "HTTP $code · $url"; return 1 ;; *) warn "$label" "HTTP $code (external soft) · $url" ;;
esac esac
} }
@ -788,98 +622,76 @@ check_one_landing() {
local html="$tmp/landing-${name}.html" local html="$tmp/landing-${name}.html"
local pref="$tmp/urls-${name}" local pref="$tmp/urls-${name}"
local code n own_n ext_n local code n own_n ext_n
local a_ok=0 a_fail=0 a_soft=0
local own_ok=0 own_fail=0
local ext_ok=0 ext_fail=0
local fail_sample="" soft_sample=""
code=$(http_body "${base}/intro/" "$html") code=$(http_body "${base}/intro/" "$html")
if [ "$code" != "200" ]; then if [ "$code" != "200" ]; then
if [ "${LOCAL_STACK:-1}" = "1" ]; then if [ "${LOCAL_STACK:-1}" = "1" ]; then
fail "landing ${name}" "/intro/ HTTP $code — skip assets/links" fail "landing ${name} /intro/" "HTTP $code"
else else
warn "landing ${name}" "/intro/ HTTP $code — skip assets/links" warn "landing ${name} /intro/" "HTTP $code"
fi fi
return return
fi fi
ok "landing ${name} /intro/" "HTTP 200 · $(wc -c <"$html" | tr -d ' ') bytes"
# Required static assets (hard: qrcode + og; soft: qr-logo) # Required static assets (hard on local)
for url in \ check_landing_asset "landing ${name} qrcode.min.js" "${base}/intro/qrcode.min.js"
"${base}/intro/qrcode.min.js" \ check_landing_asset "landing ${name} og-goa-shop.png" "${base}/intro/og-goa-shop.png"
"${base}/intro/og-goa-shop.png" check_landing_asset "landing ${name} qr-logo.png" "${base}/intro/qr-logo.png" 1
do
if _landing_probe "$url"; then
a_ok=$((a_ok + 1))
else
a_fail=$((a_fail + 1))
fail_sample="${fail_sample}${fail_sample:+; }HTTP ${_landing_code} $url"
fi
done
if _landing_probe "${base}/intro/qr-logo.png"; then
a_ok=$((a_ok + 1))
else
a_soft=$((a_soft + 1))
soft_sample="${soft_sample}${soft_sample:+; }HTTP ${_landing_code} qr-logo.png"
fi
n=$(extract_landing_urls "$base" "$html" "$pref" 2>/dev/null || echo "own=0 ext=0") n=$(extract_landing_urls "$base" "$html" "$pref" 2>/dev/null || echo "own=0 ext=0")
info "landing ${name} link extract" "$n"
own_n=0 own_n=0
ext_n=0 ext_n=0
[ -f "${pref}.own" ] && own_n=$(grep -c . "${pref}.own" 2>/dev/null || echo 0) [ -f "${pref}.own" ] && own_n=$(grep -c . "${pref}.own" 2>/dev/null || echo 0)
[ -f "${pref}.ext" ] && ext_n=$(grep -c . "${pref}.ext" 2>/dev/null || echo 0) [ -f "${pref}.ext" ] && ext_n=$(grep -c . "${pref}.ext" 2>/dev/null || echo 0)
# strip newlines from grep -c edge cases if [ "${own_n:-0}" -lt 1 ]; then
own_n=${own_n//[^0-9]/} fail "landing ${name} own-stack links" "none extracted from HTML"
ext_n=${ext_n//[^0-9]/} else
own_n=${own_n:-0} ok "landing ${name} own-stack links" "${own_n} URLs to probe"
ext_n=${ext_n:-0} fi
# Probe own-stack URLs — count only; list failures # Probe every own-stack URL from the page
if [ -f "${pref}.own" ]; then if [ -f "${pref}.own" ]; then
while IFS= read -r u; do while IFS= read -r u; do
[ -n "$u" ] || continue [ -n "$u" ] || continue
if _landing_probe "$u"; then # skip mint endpoints that create resources on GET if any (auto-account creates accounts)
own_ok=$((own_ok + 1)) case "$u" in
else */intro/auto-account.json)
own_fail=$((own_fail + 1)) # shape checked separately; still require 200 GET
# keep a few samples (max ~3) ;;
if [ "$own_fail" -le 3 ]; then esac
fail_sample="${fail_sample}${fail_sample:+; }own HTTP ${_landing_code} $u" code=$(http_code "$u")
fi
fi
done < "${pref}.own"
fi
# External: soft counts
if [ -f "${pref}.ext" ]; then
while IFS= read -r u; do
[ -n "$u" ] || continue
code=$(curl -skS --max-redirs 5 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$u" 2>/dev/null || echo 000)
case "$code" in case "$code" in
200|204|301|302|303|307|308) ext_ok=$((ext_ok + 1)) ;; 200) ok "landing ${name} link" "HTTP 200 · $u" ;;
301|302|303|307|308)
code=$(curl -skS --max-redirs 5 -L -m "${TIMEOUT}" -o /dev/null -w '%{http_code}' "$u" 2>/dev/null || echo 000)
if [ "$code" = "200" ]; then
ok "landing ${name} link" "redirect→200 · $u"
else
fail "landing ${name} link" "HTTP $code after redirect · $u"
fi
;;
405|501)
# some APIs reject wrong method — try GET already failed; soft note
fail "landing ${name} link" "HTTP $code · $u"
;;
*) *)
ext_fail=$((ext_fail + 1)) if [ "${LOCAL_STACK:-1}" = "1" ]; then
if [ "$ext_fail" -le 3 ]; then fail "landing ${name} link" "HTTP $code · $u"
soft_sample="${soft_sample}${soft_sample:+; }ext HTTP $code $u" else
warn "landing ${name} link" "HTTP $code · $u"
fi fi
;; ;;
esac esac
done < "${pref}.ext" done < "${pref}.own"
fi fi
# One primary line per landing # External store / docs: soft
local detail if [ -f "${pref}.ext" ]; then
detail="/intro $(wc -c <"$html" | tr -d ' ')B · assets ${a_ok}/$((a_ok + a_fail + a_soft)) · own-links ${own_ok}/${own_n} · external ${ext_ok}/${ext_n}" while IFS= read -r u; do
if [ "$a_fail" -gt 0 ] || [ "$own_fail" -gt 0 ] || [ "${own_n:-0}" -lt 1 ]; then [ -n "$u" ] || continue
if [ "${LOCAL_STACK:-1}" = "1" ]; then check_external_soft "landing ${name} external" "$u"
fail "landing ${name}" "$detail${fail_sample:+ · $fail_sample}" done < "${pref}.ext"
else
warn "landing ${name}" "$detail${fail_sample:+ · $fail_sample}"
fi
else
ok "landing ${name}" "$detail"
fi
if [ "$a_soft" -gt 0 ] || [ "$ext_fail" -gt 0 ]; then
warn "landing ${name} soft" "${soft_sample:-soft issues}"
fi fi
} }
@ -887,40 +699,18 @@ check_one_landing "bank" "$BANK_PUBLIC"
check_one_landing "merchant" "$MERCHANT_PUBLIC" check_one_landing "merchant" "$MERCHANT_PUBLIC"
check_one_landing "exchange" "$EXCHANGE_PUBLIC" check_one_landing "exchange" "$EXCHANGE_PUBLIC"
# Cross-links: one line # Cross-links between the three landings (always on local stack)
if [ "${LOCAL_STACK:-1}" = "1" ]; then if [ "${LOCAL_STACK:-1}" = "1" ]; then
_cx_ok=0 check_landing_asset "cross bank→merchant intro" "$MERCHANT_PUBLIC/intro/"
_cx_fail=0 check_landing_asset "cross bank→exchange intro" "$EXCHANGE_PUBLIC/intro/"
_cx_detail="" check_landing_asset "cross merchant→bank intro" "$BANK_PUBLIC/intro/"
for pair in \ check_landing_asset "cross exchange→bank intro" "$BANK_PUBLIC/intro/"
"bank→merchant|$MERCHANT_PUBLIC/intro/" \
"bank→exchange|$EXCHANGE_PUBLIC/intro/" \
"merchant→bank|$BANK_PUBLIC/intro/" \
"exchange→bank|$BANK_PUBLIC/intro/"
do
_cx_name="${pair%%|*}"
_cx_url="${pair#*|}"
if _landing_probe "$_cx_url"; then
_cx_ok=$((_cx_ok + 1))
else
_cx_fail=$((_cx_fail + 1))
_cx_detail="${_cx_detail}${_cx_detail:+; }${_cx_name} HTTP ${_landing_code}"
fi
done
if [ "$_cx_fail" -eq 0 ]; then
ok "landing cross-links" "${_cx_ok}/4 intros reachable"
else
fail "landing cross-links" "${_cx_ok}/4 ok · ${_cx_detail}"
fi
fi fi
# Bank withdraw mint + shop assets — compact # Bank-only: shared-pool withdraw mint + static withdraw files + shop assets
if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
_ba_ok=0 check_landing_asset "bank shop-pay.js" "$BANK_PUBLIC/intro/shop-pay.js" 1
_ba_soft=0 check_landing_asset "bank shop-pay.css" "$BANK_PUBLIC/intro/shop-pay.css" 1
_ba_msg=""
if _landing_probe "$BANK_PUBLIC/intro/shop-pay.js"; then _ba_ok=$((_ba_ok + 1)); else _ba_soft=$((_ba_soft + 1)); _ba_msg="${_ba_msg}shop-pay.js; "; fi
if _landing_probe "$BANK_PUBLIC/intro/shop-pay.css"; then _ba_ok=$((_ba_ok + 1)); else _ba_soft=$((_ba_soft + 1)); _ba_msg="${_ba_msg}shop-pay.css; "; fi
dw_code=$(http_body "$BANK_PUBLIC/intro/demo-withdraw.json" "$tmp/dw.json") dw_code=$(http_body "$BANK_PUBLIC/intro/demo-withdraw.json" "$tmp/dw.json")
case "$dw_code" in case "$dw_code" in
200) 200)
@ -930,42 +720,28 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
[ "$wid" = "$dw_out" ] && wid= [ "$wid" = "$dw_out" ] && wid=
ok "bank /intro/demo-withdraw.json" "$dw_detail" ok "bank /intro/demo-withdraw.json" "$dw_detail"
if [ -n "$wid" ]; then if [ -n "$wid" ]; then
if _landing_probe "$BANK_PUBLIC/taler-integration/withdrawal-operation/${wid}"; then check_landing_asset "bank taler-integration withdraw op" \
_ba_ok=$((_ba_ok + 1)) "$BANK_PUBLIC/taler-integration/withdrawal-operation/${wid}"
else
fail "landing bank withdraw-op" "HTTP ${_landing_code} · id=$wid"
fi fi
fi
ok "landing bank withdraw/shop" "demo-withdraw + shop assets ok (${_ba_ok} checks)"
else else
fail "landing bank demo-withdraw" "invalid taler://withdraw shape"
fail "bank /intro/demo-withdraw.json" "invalid taler://withdraw ($(tr '\n' ' ' <"$tmp/dw-val" | sed 's/[[:space:]]*$//'))" fail "bank /intro/demo-withdraw.json" "invalid taler://withdraw ($(tr '\n' ' ' <"$tmp/dw-val" | sed 's/[[:space:]]*$//'))"
fi fi
;; ;;
405|501|404|502|503|000) 405|501|404|502|503|000)
fail "landing bank demo-withdraw" "HTTP $dw_code (want 200)" fail "bank /intro/demo-withdraw.json" "HTTP $dw_code (want 200)"
;; ;;
*) *)
if [ "${LOCAL_STACK:-1}" = "1" ]; then if [ "${LOCAL_STACK:-1}" = "1" ]; then
fail "landing bank demo-withdraw" "HTTP $dw_code want 200" fail "bank /intro/demo-withdraw.json" "HTTP $dw_code want 200"
else else
warn "landing bank demo-withdraw" "HTTP $dw_code" warn "bank /intro/demo-withdraw.json" "HTTP $dw_code"
fi fi
;; ;;
esac esac
if [ "$_ba_soft" -gt 0 ]; then
warn "landing bank shop assets" "${_ba_msg}soft-missing"
fi
fi fi
# Merchant shop assets — one soft line # Merchant landing shop assets
_ma=0 check_landing_asset "merchant shop-pay.js" "$MERCHANT_PUBLIC/intro/shop-pay.js" 1
_landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.js" && _ma=$((_ma + 1)) check_landing_asset "merchant shop-pay.css" "$MERCHANT_PUBLIC/intro/shop-pay.css" 1
_landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.css" && _ma=$((_ma + 1))
if [ "$_ma" -eq 2 ]; then
ok "landing merchant shop assets" "shop-pay.js + .css"
else
warn "landing merchant shop assets" "${_ma}/2 present (soft)"
fi
summary summary

46
scripts/taler-monitoring/domains.conf
View file

@ -1,46 +0,0 @@
# taler-monitoring domain profiles
#
# When you add a stack, declare the three public endpoints here:
# name bank exchange merchant-backend currency local landing [canonical]
#
# - name: what -d / TALER_DOMAIN matches (aliases allowed as extra lines)
# - bank / exchange / merchant: hostname or full https:// URL
# - currency: GOA | KUDOS | CHF | … (empty = report only)
# - local: 1 = koopa SSH/inside/e2e stack; 0 = public-only
# - landing: 1 = check /intro landings + assets; 0 = skip (no GOA-style landings)
# - canonical: optional TALER_DOMAIN label after alias match
#
# CLI overrides after load: --bank --exchange --merchant --currency
# Alternate file: TALER_DOMAINS_CONF=/path/to/domains.conf
# Local GOA (koopa) — landings matter
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 hacktivism.ch
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1
# Public demo (KUDOS) — has intro-style pages; keep soft landing checks
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 demo.taler.net
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1
# Public test stack (TESTKUDOS) — no GOA-style landings
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0
# TOPS CHF — multi-tenant merchant my.taler-ops.ch; no public GOA landings
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 taler-ops.ch
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch my.stage.taler-ops.ch CHF 0 0 stage.taler-ops.ch
# Franc Paysan (Infomaniak) — merchant host is monnaie.* (not taler./backend.)
# Prod: no monetary launch yet; stage uses TESTPAYSAN (public HTTPS landings + shops)
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
exchange.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 lefrancpaysan.ch
# Stage TESTPAYSAN — landings on / + farmer shops under stage.monnaie…/shops/
# name bank exchange merchant currency local landing [canonical]
stage.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1
stage.bank.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
stage.exchange.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch
stage.monnaie.lefrancpaysan.ch stage.bank.lefrancpaysan.ch stage.exchange.lefrancpaysan.ch stage.monnaie.lefrancpaysan.ch TESTPAYSAN 0 1 stage.lefrancpaysan.ch

227
scripts/taler-monitoring/lib.sh
View file

@ -16,13 +16,10 @@
: "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}"
: "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses ATM notes : "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses ATM notes
: "${PAY_AMT:=GOA:0.01}" : "${PAY_AMT:=GOA:0.01}"
: "${CREDIT_AMT:=GOA:4700}" # covers ATM ladder 20+50+100+200+4200 (paivana) : "${CREDIT_AMT:=GOA:400}" # covers ATM ladder 20+50+100+200
: "${TIMEOUT:=12}" : "${TIMEOUT:=12}"
: "${E2E_TIMEOUT:=55}" # whole e2e budget; skip rest when exceeded (e2e raises as needed) : "${E2E_TIMEOUT:=55}" # whole e2e budget; skip rest when exceeded
: "${E2E_PAY_SECS:=22}" # dedicated seconds for pay handle-uri (avoid Alarm clock) : "${E2E_PAY_SECS:=22}" # dedicated seconds for pay handle-uri (avoid Alarm clock)
# Local GOA: public paivana paywall (https://paivana.hacktivism.ch · template GOA:4200)
: "${PAIVANA_PUBLIC:=https://paivana.hacktivism.ch}"
: "${E2E_PAIVANA:=1}" # 0 = skip paivana section in e2e
# Devtest: inject reserve credit via wire-gateway admin/add-incoming (optional). # Devtest: inject reserve credit via wire-gateway admin/add-incoming (optional).
# Default off once wirewatch DNS works; set E2E_FAKE_INCOMING=1 to force. # Default off once wirewatch DNS works; set E2E_FAKE_INCOMING=1 to force.
: "${E2E_FAKE_INCOMING:=0}" : "${E2E_FAKE_INCOMING:=0}"
@ -42,108 +39,17 @@ EXCHANGE_PUBLIC=${EXCHANGE_PUBLIC%/}
MERCHANT_PUBLIC=${MERCHANT_PUBLIC%/} MERCHANT_PUBLIC=${MERCHANT_PUBLIC%/}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Domain profiles → bank / exchange / merchant base URLs # Domain presets → public bank / exchange / merchant base URLs
# #
# Single place to declare a stack: scripts/taler-monitoring/domains.conf # TALER_DOMAIN=hacktivism.ch (default, GOA, local stack)
# (or TALER_DOMAINS_CONF). Each profile names the three public endpoints. # TALER_DOMAIN=taler.net → demo.taler.net (KUDOS)
# TALER_DOMAIN=demo.taler.net
# TALER_DOMAIN=taler-ops.ch → exchange.taler-ops.ch (CHF; bank/merchant if up)
# TALER_DOMAIN=example.org → bank/exchange/backend|taler|merchant.example.org
# #
# CLI still wins after profile load: # Explicit BANK_PUBLIC / EXCHANGE_PUBLIC / MERCHANT_PUBLIC still win if set
# --bank URL --exchange URL --merchant URL --currency CODE # *after* apply_taler_domain, or pass full URLs via --bank/--exchange/--merchant.
# Env: BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC EXPECT_CURRENCY
#
# Unknown domains fall back to heuristics (see apply_taler_domain).
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
_MONITOR_LIB_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
: "${TALER_DOMAINS_CONF:=${_MONITOR_LIB_DIR}/domains.conf}"
# host_or_url → https://… (no trailing slash)
_to_https_base() {
local x="${1%/}"
case "$x" in
https://*|http://*) printf '%s' "$x" ;;
"") printf '' ;;
*) printf 'https://%s' "$x" ;;
esac
}
# 1 = check GOA-style /intro landing pages (assets, link crawl, demo-withdraw).
# 0 = skip (e.g. taler-ops.ch / mytops — no public landings).
: "${CHECK_LANDING:=1}"
# Declare bank + exchange + merchant (+ currency, local stack, landing, domain id).
# Usage:
# set_taler_stack BANK EXCHANGE MERCHANT [CURRENCY] [LOCAL 0|1] [LANDING 0|1] [TALER_DOMAIN]
# BANK/EXCHANGE/MERCHANT: hostname or full URL
set_taler_stack() {
local bank="${1:-}" exchange="${2:-}" merchant="${3:-}"
local currency="${4:-}" local_stack="${5:-0}" landing="${6:-}" domain_id="${7:-}"
BANK_PUBLIC=$(_to_https_base "$bank")
EXCHANGE_PUBLIC=$(_to_https_base "$exchange")
MERCHANT_PUBLIC=$(_to_https_base "$merchant")
# currency "-" or empty → report-only (do not force GOA)
if [ -n "$currency" ] && [ "$currency" != "-" ]; then
EXPECT_CURRENCY="$currency"
elif [ "$currency" = "-" ]; then
EXPECT_CURRENCY=""
fi
LOCAL_STACK="$local_stack"
if [ "$LOCAL_STACK" = "1" ]; then
SKIP_SSH=0
else
SKIP_SSH=1
fi
# default: landings only matter on local GOA stack
if [ -n "$landing" ]; then
CHECK_LANDING="$landing"
elif [ "$LOCAL_STACK" = "1" ]; then
CHECK_LANDING=1
else
CHECK_LANDING=0
fi
[ -n "$domain_id" ] && TALER_DOMAIN="$domain_id"
BANK_PUBLIC=${BANK_PUBLIC%/}
EXCHANGE_PUBLIC=${EXCHANGE_PUBLIC%/}
MERCHANT_PUBLIC=${MERCHANT_PUBLIC%/}
}
# Load first matching profile from domains.conf.
# Fields (whitespace-separated; # comments; blank lines ignored):
# name bank exchange merchant currency local[0|1] landing[0|1] [canonical_domain]
# Returns 0 if found, 1 if not.
load_domain_profile() {
local want="$1" conf="${TALER_DOMAINS_CONF:-}"
local line name bank exchange merchant currency local_stack landing canon
[ -n "$want" ] || return 1
[ -n "$conf" ] && [ -f "$conf" ] || return 1
while IFS= read -r line || [ -n "$line" ]; do
line="${line%%#*}"
# shellcheck disable=SC2086
set -- $line
[ $# -ge 5 ] || continue
name="$1"
[ "$name" = "$want" ] || continue
bank="$2"
exchange="$3"
merchant="$4"
currency="$5"
local_stack="${6:-0}"
landing="${7:-}"
canon="${8:-$name}"
# Backward compat: old 7th field was canonical domain (contains a dot)
if [ -n "$landing" ] && [[ "$landing" == *.* && "$landing" != "0" && "$landing" != "1" ]]; then
canon="$landing"
landing=""
fi
set_taler_stack "$bank" "$exchange" "$merchant" "$currency" "$local_stack" "$landing" "$canon"
return 0
done <"$conf"
return 1
}
_normalize_domain() { _normalize_domain() {
local d="$1" local d="$1"
d="${d#https://}" d="${d#https://}"
@ -154,7 +60,7 @@ _normalize_domain() {
# e.g. bank.demo.taler.net → demo.taler.net, taler.hacktivism.ch → hacktivism.ch # e.g. bank.demo.taler.net → demo.taler.net, taler.hacktivism.ch → hacktivism.ch
# but NOT taler.net → net # but NOT taler.net → net
case "$d" in case "$d" in
bank.*|exchange.*|taler.*|backend.*|merchant.*|shop.*|libeufin.*|my.*|map.*|monnaie.*) bank.*|exchange.*|taler.*|backend.*|merchant.*|shop.*|libeufin.*)
rest="${d#*.}" rest="${d#*.}"
if [[ "$rest" == *.* ]]; then if [[ "$rest" == *.* ]]; then
d="$rest" d="$rest"
@ -178,42 +84,66 @@ apply_taler_domain() {
d=$(_normalize_domain "$raw") d=$(_normalize_domain "$raw")
TALER_DOMAIN="$d" TALER_DOMAIN="$d"
# 1) Explicit profile (domains.conf) — preferred way to add stacks case "$d" in
if load_domain_profile "$d" || load_domain_profile "$raw"; then # Local koopa stack only — SSH / inside / e2e allowed
# Optional tiny defaults for e2e ladders by currency koopa|hacktivism.ch|hacktivism)
case "${EXPECT_CURRENCY:-}" in BANK_PUBLIC="https://bank.hacktivism.ch"
GOA) EXCHANGE_PUBLIC="https://exchange.hacktivism.ch"
MERCHANT_PUBLIC="https://taler.hacktivism.ch"
EXPECT_CURRENCY="GOA"
LOCAL_STACK=1
SKIP_SSH=0
TALER_DOMAIN="hacktivism.ch"
: "${WITHDRAW_AMT:=GOA:20}" : "${WITHDRAW_AMT:=GOA:20}"
: "${PAY_AMT:=GOA:0.01}" : "${PAY_AMT:=GOA:0.01}"
: "${CREDIT_AMT:=GOA:400}" : "${CREDIT_AMT:=GOA:400}"
;; ;;
KUDOS|TESTKUDOS) taler.net|demo.taler.net)
# Official public demo (KUDOS) — public only, never SSH; tiny e2e amounts
BANK_PUBLIC="https://bank.demo.taler.net"
EXCHANGE_PUBLIC="https://exchange.demo.taler.net"
MERCHANT_PUBLIC="https://backend.demo.taler.net"
EXPECT_CURRENCY="KUDOS"
LOCAL_STACK=0
SKIP_SSH=1
MERCHANT_INSTANCE="${MERCHANT_INSTANCE:-sandbox}" MERCHANT_INSTANCE="${MERCHANT_INSTANCE:-sandbox}"
WITHDRAW_AMT="${WITHDRAW_AMT:-${EXPECT_CURRENCY}:20}" WITHDRAW_AMT="${WITHDRAW_AMT:-KUDOS:20}"
PAY_AMT="${PAY_AMT:-${EXPECT_CURRENCY}:0.01}" PAY_AMT="${PAY_AMT:-KUDOS:0.01}"
CREDIT_AMT="${CREDIT_AMT:-${EXPECT_CURRENCY}:100}" CREDIT_AMT="${CREDIT_AMT:-KUDOS:100}"
TALER_DOMAIN="demo.taler.net"
;; ;;
CHF) taler-ops.ch)
# Public CHF exchange; bank/merchant hosts vary — probe common names
EXCHANGE_PUBLIC="https://exchange.taler-ops.ch"
BANK_PUBLIC="https://bank.taler-ops.ch"
MERCHANT_PUBLIC="https://backend.taler-ops.ch"
EXPECT_CURRENCY="CHF"
LOCAL_STACK=0
SKIP_SSH=1
WITHDRAW_AMT="${WITHDRAW_AMT:-CHF:20}" WITHDRAW_AMT="${WITHDRAW_AMT:-CHF:20}"
PAY_AMT="${PAY_AMT:-CHF:0.01}" PAY_AMT="${PAY_AMT:-CHF:0.01}"
CREDIT_AMT="${CREDIT_AMT:-CHF:100}" CREDIT_AMT="${CREDIT_AMT:-CHF:100}"
;;
esac
else
# 2) Unknown domain — heuristics only (prefer profile in domains.conf)
# Override: --bank / --exchange / --merchant / --currency
# Do not keep default GOA: empty currency = report only, no hard fail.
BANK_PUBLIC="https://bank.${d}"
EXCHANGE_PUBLIC="https://exchange.${d}"
# TOPS-style multi-tenant merchant first, then legacy names
MERCHANT_PUBLIC="https://my.${d}"
EXPECT_CURRENCY=""
LOCAL_STACK=0
SKIP_SSH=1
CHECK_LANDING=0
if [ "${TALER_DOMAIN_PROBE}" = "1" ]; then if [ "${TALER_DOMAIN_PROBE}" = "1" ]; then
local h local h
for h in "monnaie.${d}" "backend.${d}" "my.${d}" "taler.${d}" "merchant.${d}" "shop.${d}"; do for h in bank.taler-ops.ch bank.demo.taler-ops.ch; do
_probe_https_config "$h" && { BANK_PUBLIC="https://$h"; break; }
done
for h in backend.taler-ops.ch merchant.taler-ops.ch taler.taler-ops.ch shop.taler-ops.ch; do
_probe_https_config "$h" && { MERCHANT_PUBLIC="https://$h"; break; }
done
fi
;;
*)
# Any other domain — public HTTPS only, never SSH to koopa
BANK_PUBLIC="https://bank.${d}"
EXCHANGE_PUBLIC="https://exchange.${d}"
MERCHANT_PUBLIC="https://backend.${d}"
EXPECT_CURRENCY="${EXPECT_CURRENCY:-}" # unknown — don't hard-fail currency
LOCAL_STACK=0
SKIP_SSH=1
if [ "${TALER_DOMAIN_PROBE}" = "1" ]; then
local h
for h in "backend.${d}" "taler.${d}" "merchant.${d}" "shop.${d}"; do
_probe_https_config "$h" && { MERCHANT_PUBLIC="https://$h"; break; } _probe_https_config "$h" && { MERCHANT_PUBLIC="https://$h"; break; }
done done
for h in "bank.${d}" "libeufin.${d}"; do for h in "bank.${d}" "libeufin.${d}"; do
@ -221,7 +151,8 @@ apply_taler_domain() {
done done
_probe_https_config "exchange.${d}" || true _probe_https_config "exchange.${d}" || true
fi fi
fi ;;
esac
BANK_PUBLIC=${BANK_PUBLIC%/} BANK_PUBLIC=${BANK_PUBLIC%/}
EXCHANGE_PUBLIC=${EXCHANGE_PUBLIC%/} EXCHANGE_PUBLIC=${EXCHANGE_PUBLIC%/}
@ -352,17 +283,10 @@ koopa_ssh_python() {
with_timeout "$t" ssh "${SSH_BASE_OPTS[@]}" "${KOOPA_SSH}" python3 - with_timeout "$t" ssh "${SSH_BASE_OPTS[@]}" "${KOOPA_SSH}" python3 -
} }
# ANSI colours for tags (green OK / yellow WARN / red ERROR·BLOCKER / cyan INFO). if [ "${NO_COLOR:-0}" = "1" ] || [ ! -t 1 ]; then
# Default: always on so pasted logs keep visible tags. Disable: NO_COLOR=1 or CLICOLOR=0.
if [ "${NO_COLOR:-0}" = "1" ] || [ "${CLICOLOR:-1}" = "0" ]; then
G= R= Y= C= N= B= G= R= Y= C= N= B=
else else
G=$'\e[1;32m' # bold green [OK] G=$'\e[32m'; R=$'\e[31m'; Y=$'\e[33m'; C=$'\e[36m'; B=$'\e[1m'; N=$'\e[0m'
R=$'\e[1;31m' # bold red [ERROR] [BLOCKER]
Y=$'\e[1;33m' # bold yellow [WARN]
C=$'\e[1;36m' # bold cyan [INFO]
B=$'\e[1m'
N=$'\e[0m'
fi fi
PASS_N=0 PASS_N=0
@ -667,35 +591,14 @@ read_secret() {
} }
find_wallet_cli() { find_wallet_cli() {
# Must return a path suitable for: node "$WALLET_CLI" …
# Prefer *.mjs / bundled entry; plain /usr/bin/taler-wallet-cli is often a
# shell wrapper and cannot be passed to node.
if [ -n "${WALLET_CLI:-}" ] && [ -f "$WALLET_CLI" ]; then if [ -n "${WALLET_CLI:-}" ] && [ -f "$WALLET_CLI" ]; then
echo "$WALLET_CLI"; return 0 echo "$WALLET_CLI"; return 0
fi fi
local c cand
for c in \ for c in \
"${HOME}/.local/bin/taler-wallet-cli.mjs" \
"${HOME}/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
/home/hernani/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
/Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \
/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \ "$(command -v taler-wallet-cli 2>/dev/null || true)"
/usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs
do do
[ -n "$c" ] && [ -f "$c" ] && { echo "$c"; return 0; } [ -n "$c" ] && [ -f "$c" ] && { echo "$c"; return 0; }
done done
# Debian package wrapper → resolve real .mjs
cand=$(command -v taler-wallet-cli 2>/dev/null || true)
if [ -n "$cand" ] && [ -f "$cand" ]; then
if head -1 "$cand" 2>/dev/null | grep -q 'node\|mjs'; then
# shebang node script or .mjs
echo "$cand"; return 0
fi
# follow symlink into package tree
if [ -L "$cand" ]; then
c=$(readlink -f "$cand" 2>/dev/null || true)
[ -n "$c" ] && [ -f "$c" ] && { echo "$c"; return 0; }
fi
fi
return 1 return 1
} }

28
scripts/taler-monitoring/taler-monitoring.sh
View file

@ -30,33 +30,26 @@ Phases:
all urls + inside + versions + sanity + e2e (SSH phases only on koopa) all urls + inside + versions + sanity + e2e (SSH phases only on koopa)
Options: Options:
-d, --domain DOMAIN load profile from domains.conf (bank/exchange/merchant) -d, --domain DOMAIN target domain
koopa / hacktivism.ch → local stack, SSH ok
everything else → no SSH; e2e optional (tiny amounts)
presets: koopa | hacktivism.ch | taler.net | taler-ops.ch presets: koopa | hacktivism.ch | taler.net | taler-ops.ch
| my.taler-ops.ch | stage.taler-ops.ch generic: bank/exchange/backend.<domain>
unknown domain → heuristic hosts (see README) --bank URL override bank base (https://…)
local (koopa) allows SSH; others public-only --exchange URL override exchange base
--bank URL bank base (hostname or https://…) — overrides profile --merchant URL override merchant base
--exchange URL exchange base — overrides profile
--merchant URL merchant-backend base — overrides profile
--currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only --currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only
--no-probe do not probe alternate hosts (unknown domains only) --no-probe do not probe alternate merchant/bank hostnames
-h, --help -h, --help
Add a stack: edit domains.conf (name + bank + exchange + merchant + currency).
Examples: Examples:
./taler-monitoring.sh -d taler.net ./taler-monitoring.sh -d taler.net
./taler-monitoring.sh -d taler-ops.ch urls ./taler-monitoring.sh -d taler-ops.ch urls
./taler-monitoring.sh -d my.taler-ops.ch urls
./taler-monitoring.sh -d taler-ops.ch --merchant https://my.taler-ops.ch urls
./taler-monitoring.sh -d demo.taler.net --currency KUDOS ./taler-monitoring.sh -d demo.taler.net --currency KUDOS
./taler-monitoring.sh --exchange https://exchange.taler-ops.ch \ ./taler-monitoring.sh --exchange https://exchange.taler-ops.ch urls
--merchant https://my.taler-ops.ch --bank https://bank.taler-ops.ch urls
Env (same meaning): Env (same meaning):
TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC EXPECT_CURRENCY TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC EXPECT_CURRENCY
TALER_DOMAINS_CONF SKIP_SSH=1
NO_COLOR=1 / CLICOLOR=0 disable green/yellow/red tags (default: coloured)
SKIP_SSH=1 NO_COLOR=1 SKIP_SSH=1 NO_COLOR=1
PERF_WARN_MS PERF_FAIL_MS (urls latency; default 8000 / 20000) PERF_WARN_MS PERF_FAIL_MS (urls latency; default 8000 / 20000)
KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external) KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external)
@ -140,11 +133,10 @@ fi
# Export so check_*.sh (re-source lib) see the same targets via env # Export so check_*.sh (re-source lib) see the same targets via env
export TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC export TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC
export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE CHECK_LANDING export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE
export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE
export E2E_FAKE_INCOMING E2E_REMOTE E2E_VARIABLE E2E_ATM_MAX export E2E_FAKE_INCOMING E2E_REMOTE E2E_VARIABLE E2E_ATM_MAX
export E2E_WITHDRAW_VALUES E2E_PAY_VALUES export E2E_WITHDRAW_VALUES E2E_PAY_VALUES
export PAIVANA_PUBLIC E2E_PAIVANA E2E_PAIVANA_TEMPLATE E2E_PAIVANA_AMOUNT E2E_PAIVANA_INSTANCE
# Ladder: withdraw then pay — 0 + random mids + max-1 + max (see check_goa_ladder.sh). # Ladder: withdraw then pay — 0 + random mids + max-1 + max (see check_goa_ladder.sh).
# Defaults so set -u export is safe when vars were never set by caller. # Defaults so set -u export is safe when vars were never set by caller.
: "${LADDER_STEPS:=23}" : "${LADDER_STEPS:=23}"

126
scripts/taler-shared/upgrade-goa-debs.sh
View file

@ -1,126 +0,0 @@
#!/usr/bin/env bash
# Upgrade GNU Taler .deb packages *inside* the three GOA containers on koopa.
# Host openSUSE packages are unrelated (zypper) — do not confuse with this.
#
# Containers (no systemd as PID 1): after apt, restart with start_*.sh only.
#
# Run on koopa host as a user that can `podman exec` the containers:
# ./upgrade-goa-debs.sh
# ./upgrade-goa-debs.sh bank|exchange|merchant
#
# Mirror: scripts/taler-shared/ in koopa-admin-log.
# Live note: 2026/2026-07-16--taler-package-upgrade-goa.md
set -euo pipefail
BANK_CTR="${BANK_CTR:-taler-hacktivism-bank}"
EXCHANGE_CTR="${EXCHANGE_CTR:-taler-hacktivism-exchange-ansible}"
MERCHANT_CTR="${MERCHANT_CTR:-taler-hacktivism}"
only="${1:-all}"
pod_root() {
local ctr="$1"
shift
podman exec -u root "$ctr" bash -c "$*"
}
upgrade_bank() {
echo "======== bank ($BANK_CTR) ========"
pod_root "$BANK_CTR" '
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y libeufin-bank libeufin-common
dpkg -l libeufin-bank libeufin-common | awk "/^ii/{print \$2,\$3}"
'
# restart app only (postgres via start_base if needed)
pod_root "$BANK_CTR" '
set -e
if ! runuser -u postgres -- psql -c "SELECT 1" >/dev/null 2>&1; then
pg_ctlcluster 17 main start || true
sleep 2
fi
mkdir -p /var/log/libeufin-bank
chown libeufin-bank:libeufin-bank /var/log/libeufin-bank 2>/dev/null || true
runuser -u libeufin-bank -- /usr/local/bin/start_bank.sh --restart
'
}
upgrade_exchange() {
echo "======== exchange ($EXCHANGE_CTR) ========"
pod_root "$EXCHANGE_CTR" '
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y \
taler-exchange taler-exchange-database taler-exchange-offline \
taler-exchange-typst libtalerexchange taler-terms-generator \
taler-exchange-aml-webui taler-exchange-kyc-webui || true
dpkg -l "taler-exchange*" "libtalerexchange" "taler-terms-generator" 2>/dev/null \
| awk "/^ii/{print \$2,\$3}"
'
# do not run dbinit as root (role "root" does not exist)
pod_root "$EXCHANGE_CTR" '
set -e
if id taler-exchange-httpd >/dev/null 2>&1; then
runuser -u taler-exchange-httpd -- taler-exchange-dbinit 2>&1 | tail -20 || true
fi
if [ -x /root/start_base_services_for_taler_exchange.sh ]; then
/root/start_base_services_for_taler_exchange.sh --no-shell 2>/dev/null || true
fi
if [ -x /usr/local/bin/start_exchange.sh ]; then
runuser -u taler-exchange-httpd -- /usr/local/bin/start_exchange.sh --restart 2>&1 | tail -30 || true
fi
pgrep -a taler-exchange-httpd | head -3 || true
'
}
upgrade_merchant() {
echo "======== merchant ($MERCHANT_CTR) ========"
pod_root "$MERCHANT_CTR" '
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y \
taler-merchant taler-merchant-webui taler-merchant-typst \
libtalermerchant taler-terms-generator
dpkg -l "taler-merchant*" "libtalermerchant" 2>/dev/null | awk "/^ii/{print \$2,\$3}"
'
pod_root "$MERCHANT_CTR" '
set -e
# schema: prefer tools as service user; failures may be noisy — check /config after
if id taler-merchant-httpd >/dev/null 2>&1; then
runuser -u taler-merchant-httpd -- taler-merchant-dbinit 2>&1 | tail -20 || true
fi
runuser -u taler-merchant-httpd -- /usr/local/bin/start_merchant.sh --restart
'
}
smoke() {
echo "======== smoke (host loopback) ========"
curl -sS -m 8 -o /dev/null -w "bank %{http_code} :9012/config\n" http://127.0.0.1:9012/config || true
curl -sS -m 8 -o /dev/null -w "exchange %{http_code} :9011/config\n" http://127.0.0.1:9011/config || true
curl -skS -m 8 -o /dev/null -w "merchant %{http_code} :9010/config\n" https://127.0.0.1:9010/config || true
}
case "$only" in
all)
upgrade_bank
upgrade_exchange
upgrade_merchant
smoke
;;
bank) upgrade_bank; smoke ;;
exchange) upgrade_exchange; smoke ;;
merchant) upgrade_merchant; smoke ;;
-h|--help|help)
sed -n '2,20p' "$0"
exit 0
;;
*)
echo "usage: $0 [all|bank|exchange|merchant]" >&2
exit 2
;;
esac
echo "DONE — re-run taler-monitoring versions/urls from a laptop if desired."