diff --git a/2026/2026-07-13--paivana-goa.md b/2026/2026-07-13--paivana-goa.md index 9606641..0095b40 100644 --- a/2026/2026-07-13--paivana-goa.md +++ b/2026/2026-07-13--paivana-goa.md @@ -7,13 +7,22 @@ - Image `localhost/koopa-paivana:latest` (built from `git.taler.net/paivana` + deb.taler.net libs; package not on apt yet) - Merchant: instance **`goa-shop`**, template **`paivana`** (`template_type: paivana`, **GOA:4200**) - After payment: site-wide cookie (`-g`) → example upstream free to browse -- `extra_hosts: taler.hacktivism.ch:host-gateway` so pasta can reach merchant - Flags: `-g` (site-wide), `-f` (X-Forwarded-For) - Smoke: `curl -si http://127.0.0.1:9025/` → **302** to `/.well-known/paivana/templates/paivana#…` - Config mirror: `configs/paivana/` - Caddy: `paivana.hacktivism.ch` → `:9025` (`configs/caddy/Caddyfile`) - User unit: `container-koopa-paivana.service` enabled +## Was später kaputt / nachgezogen (klar) + +Siehe **`configs/paivana/README.md` → „Was kaputt war“**. Kurz: + +1. **Extra Upstream-systemd-Unit** race mit Compose → exit **125** / start-limit — **nur** Compose-Unit behalten. +2. **Paivana vor Merchant-Apps** → öffentliches **502** — `order.conf` `After=taler-merchant-apps`. +3. **Pasta DNS/AAAA**: `host-gateway` reichte nicht → `extra_hosts` auf **`172.17.0.1`** pin (`f26bf6a`). + +Gesund: **302** lokal / **402** public paywall + monitoring `www.paivana`. + ## Refs - https://docs.taler.net/taler-paivana-manual.html diff --git a/2026/2026-07-17.md b/2026/2026-07-17.md index de35634..4423c96 100644 --- a/2026/2026-07-17.md +++ b/2026/2026-07-17.md @@ -3,3 +3,10 @@ - Bank: `taler://withdraw` **ohne `:443`** (Wallet-Apps); live deployed - Merchant/bank in-container apps after reboot via `ensure-taler-apps` + user units - Admin-log: 13 Commits (19:04–01:07) + follow-up; `main` force-pushed → Forgejo +- **Paivana — was kaputt war** klar dokumentiert (`configs/paivana/README.md`): + 1. Upstream-only systemd race (exit 125) + 2. Start vor Merchant-Apps → 502 + 3. Pasta AAAA/timeout → pin `172.17.0.1` + Live-Stack war gesund (302/402); Ursache/Fix nicht wiederholen. +- **Merchant portal:** instance ids **lowercase only** — capitals → odd **401**; + documented in taler-hacktivism + stagepaysan config READMEs; monitoring lowercases `MERCHANT_INSTANCE`. diff --git a/configs/bank-landing/shop-pay.js b/configs/bank-landing/shop-pay.js index 2c0573b..5ebc439 100644 --- a/configs/bank-landing/shop-pay.js +++ b/configs/bank-landing/shop-pay.js @@ -10,7 +10,7 @@ (function () { var MERCHANT_HOST = "taler.hacktivism.ch"; /* goa-shop: dedicated instance with fixed-order product templates */ - var INSTANCE = "goa-shop"; + var INSTANCE = String("goa-shop").toLowerCase(); var SHOP_PAYTO = "payto://x-taler-bank/bank.hacktivism.ch/goa-shop?receiver-name=GOA%20Shop"; var SHOP_PAYTO_HTTPS = "https://bank.hacktivism.ch/webui/"; diff --git a/configs/merchant-landing/shop-pay.js b/configs/merchant-landing/shop-pay.js index 2c0573b..af62a30 100644 --- a/configs/merchant-landing/shop-pay.js +++ b/configs/merchant-landing/shop-pay.js @@ -9,8 +9,9 @@ */ (function () { var MERCHANT_HOST = "taler.hacktivism.ch"; - /* goa-shop: dedicated instance with fixed-order product templates */ - var INSTANCE = "goa-shop"; + /* goa-shop: dedicated instance with fixed-order product templates. + * Instance id must stay lowercase (portal/API: capitals → odd 401s). */ + var INSTANCE = String("goa-shop").toLowerCase(); var SHOP_PAYTO = "payto://x-taler-bank/bank.hacktivism.ch/goa-shop?receiver-name=GOA%20Shop"; var SHOP_PAYTO_HTTPS = "https://bank.hacktivism.ch/webui/"; diff --git a/configs/paivana/README.md b/configs/paivana/README.md index e813a82..643fdb4 100644 --- a/configs/paivana/README.md +++ b/configs/paivana/README.md @@ -37,6 +37,65 @@ GNU Taler **paivana-httpd** reverse-proxy paywall (DD 95 / DD 76 style), GOA pay Mirror notes: `koopa-admin-secrets` → `koopa/home-hernani/koopa-paivana/secrets/` (paths only / examples). +## Was kaputt war (klar · 2026-07) + +Drei **unabhängige** Defekte; Symptom oft gleich (Paywall 502 / tot / kein Template). +Stand: behoben auf koopa; nicht wiederholen. + +### 1) Boot-Race: extra systemd-Unit nur für Upstream + +| | | +|--|--| +| **Symptom** | Nach Reboot: `container-koopa-paivana-upstream.service` **failed** / start-limit-hit; Upstream fehlt oder Compose und Unit kämpfen. | +| **Ursache** | Separate User-Unit tat nur `podman start koopa-paivana-upstream` **ohne** dass Compose die Container schon angelegt hatte → **exit 125**. Compose-Unit startet **beide** Services (`paivana` + `upstream`) ohnehin. | +| **Fix** | **Nur** `container-koopa-paivana.service` (compose up). Separate Upstream-Unit: `disable --now`. Nicht neu anlegen. | +| **Nicht verwechseln mit** | Problem 2 (Merchant noch down → 502 trotz laufender Container). | + +### 2) Start-Reihenfolge: Paivana vor Merchant-Apps → **502** + +| | | +|--|--| +| **Symptom** | `paivana.hacktivism.ch` / `:9025` **502** nach Boot; Container laufen, Paywall/Template-Flow tot. | +| **Ursache** | Merchant-Container ist hoch, aber **In-App** (`taler-merchant-httpd` etc.) startet erst über `taler-merchant-apps.service` / `ensure-taler-apps`. Paivana braucht die **private Merchant-API** (Template `paivana` / goa-shop). Zu früh → 502. | +| **Fix** | Drop-in `configs/systemd/user/container-koopa-paivana.service.d/order.conf`: `After=` / `Wants=` `taler-merchant-apps.service` (+ ggf. merchant-Container). Siehe `2026/2026-07-16--reboot-recovery.md`. | +| **Check** | Merchant `https://taler.hacktivism.ch/config` **200**, dann Paivana smoke (unten). | + +### 3) Pasta / DNS: Container erreicht Merchant nicht (Timeout / AAAA) + +| | | +|--|--| +| **Symptom** | Paivana läuft, aber Template-Laden / Merchant-Calls **hängen oder scheitern**; Logs: Timeouts Richtung `taler.hacktivism.ch`. | +| **Ursache** | Rootless **pasta** hairpint öffentliche DNS nicht sauber. `extra_hosts: …:host-gateway` allein reichte nicht: oft gewinnt noch die **öffentliche AAAA**, Connection timeout. Merchant sitzt am Host (**~:9010**), erreichbar über Pasta-IPv4 **`172.17.0.1`**. | +| **Fix** | In `compose.yml` feste IPv4-Hosts (nicht nur host-gateway): | + +```yaml +extra_hosts: + - "taler.hacktivism.ch:172.17.0.1" + - "bank.hacktivism.ch:172.17.0.1" + - "exchange.hacktivism.ch:172.17.0.1" +``` + +| | | +|--|--| +| **Commit** | `f26bf6a` (pin Taler hosts to pasta IPv4) | + +### Kurz: was *nicht* das Problem war + +- Caddy-Vhost an sich (leitet auf **9025**). +- Fehlen des Templates im Merchant (Template `paivana` / GOA:4200 war angelegt; ohne Merchant-Apps oder ohne Routing wirkt es so, als fehlte es). +- „Nur nginx-Upstream“ — Upstream ist unprivileged nginx **:8080**; ohne Problem 1/2/3 ist das stabil. + +### Erwartet gesund + +| Check | Expect | +|-------|--------| +| `curl -si http://127.0.0.1:9025/` | **302** → `/.well-known/paivana/templates/paivana#…` | +| `https://paivana.hacktivism.ch/` | **402** paywall HTML + `paivana: taler://pay-template/…` | +| monitoring `urls` | `www.paivana` OK | +| e2e (GOA, lokal) | `e2e.paivana` Template-Pay **GOA:4200** (wenn `E2E_PAIVANA=1`) | + +--- + ## Ops ```bash @@ -48,25 +107,28 @@ podman logs -f koopa-paivana curl -si http://127.0.0.1:9025/ | head # 302 → /.well-known/paivana/templates/paivana#… ``` -Autostart: +Autostart (**only** the compose unit — starts **both** `koopa-paivana` and +`koopa-paivana-upstream`): ```bash mkdir -p ~/.config/systemd/user cp ~/koopa-paivana/container-koopa-paivana.service ~/.config/systemd/user/ +# required on koopa: After= merchant apps +# configs/systemd/user/container-koopa-paivana.service.d/order.conf systemctl --user daemon-reload systemctl --user enable --now container-koopa-paivana.service ``` +**Do not** enable a separate `container-koopa-paivana-upstream.service` that only +runs `podman start koopa-paivana-upstream` — see failure **#1** above. +If such a unit exists: +`systemctl --user disable --now container-koopa-paivana-upstream.service`. + ## Network notes -Rootless pasta cannot hairpin public DNS for the merchant. Compose sets: - -```yaml -extra_hosts: - - "taler.hacktivism.ch:host-gateway" -``` - -so `paivana-httpd` can load templates from the merchant API. +See failure **#3**. Live compose pins Taler hostnames to pasta IPv4 +`172.17.0.1` so `paivana-httpd` reaches the host merchant private API without +public AAAA timeouts. ## Caddy diff --git a/configs/taler-hacktivism/README.md b/configs/taler-hacktivism/README.md index 127cd32..06a110e 100644 --- a/configs/taler-hacktivism/README.md +++ b/configs/taler-hacktivism/README.md @@ -56,6 +56,23 @@ After conf/hosts changes: `taler-merchant-exchangekeyupdate -t` and `taler-merch | 9010 | nginx TLS in container (podman) | | unix sock | `taler-merchant-httpd` | +## Merchant portal (SPA `/webui/`) — instance IDs **lowercase only** + +The merchant **webui** (and private API paths under `/instances/{id}/…`) must use +**only lowercase** instance ids (`a-z`, digits, hyphens). + +| Do | Do not | +|----|--------| +| `goa-shop`, `goa-demo-cp4zqk`, `fermes-des-collines` | `Goa-Shop`, `GOA-DEMO`, mixed case | + +**Why:** uppercase (or mixed-case) ids produce **odd HTTP 401** failures on +login / private API even when the password/token is correct. Auth looks like +“wrong secret” but the real issue is path / instance string case. + +Create instances, login fields, `Authorization` targets, and `taler://pay…` +paths with the **exact lowercase** id. Display names may use capitals; +**ids must not**. + ## Demo instance (2026-07-09) Self-provisioned shop **`goa-demo-cp4zqk`**, bank account same username, payto linked, seed **GOA:5000**. diff --git a/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md b/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md index 95aa009..109c0bb 100644 --- a/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md +++ b/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md @@ -15,6 +15,10 @@ Created **2026-07-09** via self-provisioning (`POST /instances`, no SMS TAN). Login: SPA → instance id + password (or bearer `secret-token:$PASSWORD`). +**Instance id must be all-lowercase** (`goa-demo-cp4zqk`). Typing capitals in +the portal login field (or calling `/instances/Goa-Demo-…/private/…`) yields +**strange 401s** even with the right password — use exact lowercase only. + ## Bank account (linked) | Field | Value | diff --git a/scripts/git-cadence-commit-push.sh b/scripts/git-cadence-commit-push.sh new file mode 100755 index 0000000..9775bef --- /dev/null +++ b/scripts/git-cadence-commit-push.sh @@ -0,0 +1,157 @@ +#!/usr/bin/env bash +# git-cadence-commit.sh — optional local WIP commit on a soft cadence +# +# Model (koopa-admin-log / francpaysan-admin-log histories + ops preference): +# - message: area: short description +# - size: small logical steps; **prefer more commits over few** +# - interval: soft range **7–13 minutes** while work is open (not a strict timer; +# denser bursts of 0–2 min are fine for tiny follow-ups) +# - push: **only at the end of a work session** (CADENCE_PUSH=1), never by default +# +# Usage: +# ./scripts/git-cadence-commit-push.sh # commit only if dirty +# ./scripts/git-cadence-commit-push.sh /path/to/repo +# CADENCE_MSG='monitoring: wip ladder' ./scripts/git-cadence-commit-push.sh +# CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh +# CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh # explicit push (end of work) +# CADENCE_MIN_AGE_S=420 # skip if last commit younger than this (default 0) +# +# Soft timer (7–13 min, not fixed): fire often, skip until age ∈ [7,13] min target. +# # agent/cron every 7m — script enforces random target in range when CADENCE_JITTER=1 +# CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh +# +set -euo pipefail + +REPO="${1:-}" +if [ -z "$REPO" ]; then + REPO="$(cd "$(dirname "$0")/.." && pwd)" +fi +REPO="$(cd "$REPO" && pwd)" +cd "$REPO" + +CADENCE_DRY="${CADENCE_DRY:-0}" +# Push only when explicitly requested (end of session) — default OFF +CADENCE_PUSH="${CADENCE_PUSH:-0}" +# Soft range 7–13 minutes (seconds) +CADENCE_RANGE_MIN_S="${CADENCE_RANGE_MIN_S:-420}" # 7 min +CADENCE_RANGE_MAX_S="${CADENCE_RANGE_MAX_S:-780}" # 13 min +CADENCE_JITTER="${CADENCE_JITTER:-0}" +CADENCE_MIN_AGE_S="${CADENCE_MIN_AGE_S:-0}" + +SKIP_GLOBS=( + 'secrets.env' + '**/secrets.env' + '**/*password*' + '**/*secret*' + '**/*.pem' + '**/*token*' +) + +if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "not a git repo: $REPO" >&2 + exit 1 +fi + +branch=$(git rev-parse --abbrev-ref HEAD) +remote=$(git remote 2>/dev/null | head -1 || true) +: "${remote:=origin}" + +# Jitter: only proceed if last commit is at least a random target in [7,13] min +if [ "$CADENCE_JITTER" = "1" ]; then + last=$(git log -1 --format=%ct 2>/dev/null || echo 0) + now=$(date +%s) + age=$((now - last)) + # stable-ish target per hour so repeated 7m fires don't all commit + target=$(python3 -c " +import random, time +random.seed(int(time.time()) // 60) # changes every minute +print(random.randint(${CADENCE_RANGE_MIN_S}, ${CADENCE_RANGE_MAX_S})) +" 2>/dev/null || echo 600) + if [ "$age" -lt "$target" ]; then + echo "jitter wait · age=${age}s < target=${target}s (range ${CADENCE_RANGE_MIN_S}-${CADENCE_RANGE_MAX_S}s) · $REPO" + exit 0 + fi +elif [ "${CADENCE_MIN_AGE_S}" -gt 0 ] 2>/dev/null; then + last=$(git log -1 --format=%ct 2>/dev/null || echo 0) + now=$(date +%s) + age=$((now - last)) + if [ "$age" -lt "$CADENCE_MIN_AGE_S" ]; then + echo "min-age wait · age=${age}s < ${CADENCE_MIN_AGE_S}s · $REPO" + exit 0 + fi +fi + +do_push() { + local ahead + ahead=$(git rev-list --count "${remote}/${branch}..HEAD" 2>/dev/null || echo 0) + if [ "${ahead:-0}" -le 0 ]; then + echo "nothing to push · $REPO · $branch" + return 0 + fi + echo "ahead of ${remote}/${branch} by $ahead" + if [ "$CADENCE_DRY" = "1" ]; then + echo "DRY: git push $remote $branch" + return 0 + fi + if [ "$CADENCE_PUSH" != "1" ]; then + echo "skip push (CADENCE_PUSH!=1 — push only at end of work)" + return 0 + fi + git push "$remote" "$branch" || { + echo "WARN: push failed for $REPO ($remote $branch)" >&2 + return 0 + } + echo "pushed · $remote $branch" +} + +# Clean working tree → maybe push only +if git diff --quiet && git diff --cached --quiet \ + && [ -z "$(git ls-files --others --exclude-standard)" ]; then + echo "clean · $REPO · $branch (nothing to commit)" + do_push + exit 0 +fi + +git add -A +for g in "${SKIP_GLOBS[@]}"; do + git reset -q -- "$g" 2>/dev/null || true +done +while IFS= read -r f; do + case "$f" in + *secret*|*password*|*token*|secrets.env) + git reset -q -- "$f" 2>/dev/null || true + ;; + esac +done < <(git diff --cached --name-only 2>/dev/null || true) + +if git diff --cached --quiet; then + echo "only secret/skip paths dirty — not committing · $REPO" + exit 0 +fi + +msg="${CADENCE_MSG:-}" +if [ -z "$msg" ]; then + files=$(git diff --cached --name-only | head -20) + area="wip" + case "$files" in + *taler-monitoring*|*scripts/taler-monitoring*) area="monitoring" ;; + *landing*|*landings*|*configs/*landing*) area="landing" ;; + *docs*|*README*|*TODO*|*NOTES*) area="docs" ;; + *caddy*) area="caddy" ;; + *scripts*) area="scripts" ;; + esac + n=$(git diff --cached --name-only | wc -l | tr -d ' ') + sample=$(git diff --cached --name-only | head -1 | xargs -r basename) + msg="${area}: cadence ${n} file(s) · ${sample}" +fi + +if [ "$CADENCE_DRY" = "1" ]; then + echo "DRY commit: $msg" + git diff --cached --stat + [ "$CADENCE_PUSH" = "1" ] && echo "DRY push: $remote $branch" || echo "DRY: no push" + exit 0 +fi + +git commit -m "$msg" +echo "committed · $msg" +do_push diff --git a/scripts/taler-merchant/README.md b/scripts/taler-merchant/README.md index 339d693..2173a28 100644 --- a/scripts/taler-merchant/README.md +++ b/scripts/taler-merchant/README.md @@ -30,6 +30,34 @@ Automatic import needs: SMS backends are symlinks into `/var/taler-src/...` (not copied). +## Self-build deploy (webui tag + merchant master) + +From a laptop with `ssh koopa` + `pnpm`/`node` (e.g. clementine): + +```bash +cd ~/src/koopa/koopa-admin-log +chmod +x scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +./scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +``` + +What it does: + +1. **Clone** `taler-typescript-core` **tag `v1.6.11`**, build SPA, install into + `taler-hacktivism:/usr/share/taler-merchant-webui` (backup `…bak-selfbuild-*`). +2. **Inside the container:** update `/var/taler-src/merchant` to **master**, + `./bootstrap && ./configure --prefix=/usr && make && make install`. +3. **`start_merchant.sh --restart`** + smoke `/config` + `/webui/`. + +Env knobs: `SKIP_WEBUI=1`, `SKIP_MERCHANT=1`, `SKIP_RESTART=1`, +`WEBUI_TAG=…`, `MERCHANT_REF=master`, `KOOPA_SSH=koopa`. + +## Portal / instance IDs: **lowercase only** + +Merchant SPA (`/webui/`) and `/instances/{id}/…` require **lowercase** instance +ids. Uppercase or mixed case → **weird HTTP 401** (looks like bad password). +Display names may be mixed case; **ids must be lowercase** (`goa-shop`, not +`Goa-Shop`). See `configs/taler-hacktivism/README.md`. + ## Usage ```bash diff --git a/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh b/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh new file mode 100755 index 0000000..bfd0b05 --- /dev/null +++ b/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh @@ -0,0 +1,217 @@ +#!/usr/bin/env bash +# Build taler-merchant-webui from tag v1.6.11 + taler-merchant from master, +# deploy into podman container taler-hacktivism on koopa, restart services. +# +# Run from clementine (or any host with: git, node/pnpm, ssh koopa, podman on koopa): +# ./scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +# +# Env: +# KOOPA_SSH=koopa +# MERCHANT_CTR=taler-hacktivism +# TTC_GIT=git@git.taler.net:taler-typescript-core.git +# MERCHANT_GIT=https://git.taler.net/merchant.git # or git@… +# WEBUI_TAG=v1.6.11 +# MERCHANT_REF=master +# SKIP_WEBUI=0 SKIP_MERCHANT=0 SKIP_RESTART=0 +# PNPM=$HOME/.local/bin/pnpm +# WORK_ROOT=/tmp/taler-selfbuild-$$ +# +set -euo pipefail + +KOOPA_SSH="${KOOPA_SSH:-koopa}" +MERCHANT_CTR="${MERCHANT_CTR:-taler-hacktivism}" +TTC_GIT="${TTC_GIT:-git@git.taler.net:taler-typescript-core.git}" +MERCHANT_GIT="${MERCHANT_GIT:-https://git.taler.net/merchant.git}" +WEBUI_TAG="${WEBUI_TAG:-v1.6.11}" +MERCHANT_REF="${MERCHANT_REF:-master}" +SKIP_WEBUI="${SKIP_WEBUI:-0}" +SKIP_MERCHANT="${SKIP_MERCHANT:-0}" +SKIP_RESTART="${SKIP_RESTART:-0}" +PNPM="${PNPM:-$HOME/.local/bin/pnpm}" +WORK_ROOT="${WORK_ROOT:-/tmp/taler-selfbuild-$$}" +SPA_DIR="${WORK_ROOT}/spa-prod" +TTC_DIR="${WORK_ROOT}/taler-typescript-core" +export PATH="${HOME}/.local/bin:${PATH}" + +log() { printf '+ %s\n' "$*"; } +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } + +need() { command -v "$1" >/dev/null 2>&1 || die "missing command: $1"; } + +ssh_koopa() { + ssh -o BatchMode=yes -o ConnectTimeout=15 "${KOOPA_SSH}" "$@" +} + +ctr_root() { + ssh_koopa podman exec -u root "${MERCHANT_CTR}" bash -c "$*" +} + +log "work root: ${WORK_ROOT}" +mkdir -p "${WORK_ROOT}" +need git +need ssh +need tar +need curl + +# --------------------------------------------------------------------------- +# 1) WebUI from tag +# --------------------------------------------------------------------------- +if [ "${SKIP_WEBUI}" != "1" ]; then + need node + if [ ! -x "${PNPM}" ]; then + log "installing pnpm to ~/.local …" + npm install -g pnpm@10.33.4 --prefix "${HOME}/.local" + fi + need pnpm || [ -x "${PNPM}" ] + PNPM_BIN=$(command -v pnpm || echo "${PNPM}") + + log "clone ${TTC_GIT} @ ${WEBUI_TAG} (full clone, not worktree — build.mjs needs real .git)" + rm -rf "${TTC_DIR}" + git clone --branch "${WEBUI_TAG}" --depth 1 "${TTC_GIT}" "${TTC_DIR}" + cd "${TTC_DIR}" + git submodule update --init contrib/taler-assets + test -f contrib/taler-assets/svg/logo/qr-logo.svg || die "taler-assets submodule incomplete" + + ver=$(python3 -c "import json; print(json.load(open('packages/taler-merchant-webui/package.json'))['version'])") + log "package.json version=${ver} HEAD=$(git rev-parse --short HEAD)" + + log "pnpm install + compile merchant-webui …" + "${PNPM_BIN}" install --frozen-lockfile --filter @gnu-taler/taler-merchant-webui... + "${PNPM_BIN}" run --filter @gnu-taler/taler-merchant-webui... compile + + test -f packages/taler-merchant-webui/dist/prod/index.js || die "dist/prod/index.js missing" + test -f packages/taler-merchant-webui/dist/prod/version.txt || die "version.txt missing" + got=$(tr -d '\n' < packages/taler-merchant-webui/dist/prod/version.txt) + log "built SPA version.txt=${got}" + # expect 1.6.11 for default tag + case "${WEBUI_TAG}" in + v1.6.11|1.6.11) + [ "${got}" = "1.6.11" ] || die "expected version 1.6.11, got ${got}" + ;; + esac + + rm -rf "${SPA_DIR}" + mkdir -p "${SPA_DIR}" + cp -a packages/taler-merchant-webui/dist/prod/. "${SPA_DIR}/" + printf 'selfbuild-%s-%s\n' "${WEBUI_TAG}" "$(git rev-parse --short HEAD)" >"${SPA_DIR}/version-overlay.txt" + + log "deploy SPA into ${MERCHANT_CTR}:/usr/share/taler-merchant-webui" + ts=$(date +%Y%m%d-%H%M%S) + ctr_root " + set -e + if [ -d /usr/share/taler-merchant-webui ]; then + cp -a /usr/share/taler-merchant-webui /usr/share/taler-merchant-webui.bak-selfbuild-${ts} + fi + mkdir -p /usr/share/taler-merchant-webui + # clear previous SPA files (keep directory) + find /usr/share/taler-merchant-webui -mindepth 1 -maxdepth 1 ! -name 'bak-*' -exec rm -rf {} + + " + tar -C "${SPA_DIR}" -czf - . | ssh_koopa "podman exec -i -u root ${MERCHANT_CTR} bash -c ' + set -e + cd /usr/share/taler-merchant-webui + tar -xzf - + chown -R root:root /usr/share/taler-merchant-webui + echo deployed: + cat version.txt + cat version-overlay.txt + wc -c index.js + '" + log "webui deploy done" +else + log "SKIP_WEBUI=1" +fi + +# --------------------------------------------------------------------------- +# 2) Merchant backend from master (build inside container) +# --------------------------------------------------------------------------- +if [ "${SKIP_MERCHANT}" != "1" ]; then + log "build taler-merchant ${MERCHANT_REF} inside ${MERCHANT_CTR}" + # Install build deps if missing; update /var/taler-src/merchant; bootstrap; make install + ssh_koopa "podman exec -u root ${MERCHANT_CTR} bash -s" <&1 | tail -30 || true + +# exchange headers sometimes via package names that differ +apt-get install -y --no-install-recommends libtalerutil-dev 2>/dev/null || true + +echo '=== source tree ===' +mkdir -p /var/taler-src +if [ -d "\$SRC/.git" ]; then + cd "\$SRC" + git remote set-url origin "\$GIT_URL" 2>/dev/null || git remote add origin "\$GIT_URL" || true + git fetch --tags origin + git checkout "\$REF" + git pull --ff-only origin "\$REF" || git reset --hard "origin/\$REF" +else + rm -rf "\$SRC" + git clone "\$GIT_URL" "\$SRC" + cd "\$SRC" + git checkout "\$REF" +fi +# submodules if any +git submodule update --init --recursive 2>/dev/null || true +echo "merchant HEAD=\$(git rev-parse --short HEAD) \$(git log -1 --oneline)" + +echo '=== bootstrap / configure / make ===' +if [ ! -x configure ]; then + ./bootstrap +fi +# Prefer /usr prefix to replace package binaries +./configure --prefix=/usr --disable-doc 2>&1 | tail -40 +make -j"\$(nproc)" 2>&1 | tail -50 +make install 2>&1 | tail -30 + +echo '=== installed httpd ===' +ls -la /usr/bin/taler-merchant-httpd +taler-merchant-httpd --help 2>&1 | head -5 || true +# record build id +git rev-parse HEAD > /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD 2>/dev/null \\ + || mkdir -p /usr/share/taler-merchant && git rev-parse HEAD > /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD +echo "SELFBUILD_MERCHANT_HEAD=\$(cat /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD)" +REMOTE + log "merchant build/install done" +else + log "SKIP_MERCHANT=1" +fi + +# --------------------------------------------------------------------------- +# 3) Restart + health +# --------------------------------------------------------------------------- +if [ "${SKIP_RESTART}" != "1" ]; then + log "restart merchant services" + ssh_koopa "podman exec -u root ${MERCHANT_CTR} bash -c ' + set -e + if id taler-merchant-httpd >/dev/null 2>&1; then + runuser -u taler-merchant-httpd -- taler-merchant-dbinit 2>&1 | tail -15 || true + fi + runuser -u taler-merchant-httpd -- /usr/local/bin/start_merchant.sh --restart + '" + log "smoke" + ssh_koopa 'curl -skS -m 8 -o /dev/null -w "merchant_local %{http_code}\n" https://127.0.0.1:9010/config || true' + curl -skS -m 12 -o /dev/null -w "merchant_public %{http_code}\n" https://taler.hacktivism.ch/config || true + curl -skS -m 12 -o /dev/null -w "webui %{http_code}\n" https://taler.hacktivism.ch/webui/ || true + pubver=$(curl -skS -m 12 https://taler.hacktivism.ch/webui/version.txt 2>/dev/null || true) + pubov=$(curl -skS -m 12 https://taler.hacktivism.ch/webui/version-overlay.txt 2>/dev/null || true) + log "public version.txt=${pubver}" + log "public version-overlay.txt=${pubov}" +else + log "SKIP_RESTART=1" +fi + +log "DONE" +log " webui: tag ${WEBUI_TAG} → container /usr/share/taler-merchant-webui" +log " merchant: ${MERCHANT_REF} → /usr/bin/taler-merchant-*" +log " work left at ${WORK_ROOT} (delete when satisfied)" diff --git a/scripts/taler-merchant/selfbuild-webui.sh b/scripts/taler-merchant/selfbuild-webui.sh new file mode 100755 index 0000000..bd1cfda --- /dev/null +++ b/scripts/taler-merchant/selfbuild-webui.sh @@ -0,0 +1,197 @@ +#!/usr/bin/env bash +# Build taler-merchant-webui SPA *inside* this container and install it. +# +# Usage (as root in taler-hacktivism): +# selfbuild-webui.sh [REF] +# selfbuild-webui.sh v1.6.11 +# selfbuild-webui.sh 0e4807845fc06f7c705711974650d67ef30e7919 +# WEBUI_REF=master selfbuild-webui.sh +# +# Env: +# WEBUI_REF tag / branch / commit (default: first arg or v1.6.11) +# TTC_GIT default https://git.taler.net/taler-typescript-core.git +# TTC_DIR default /var/taler-src/taler-typescript-core +# SPA_DEST default /usr/share/taler-merchant-webui +# PNPM_VERSION default 10.33.4 +# SKIP_NODE_INSTALL 1 = do not apt/npm install node/pnpm +# SKIP_DEPLOY 1 = build only, leave dist/prod in tree +# EXPECT_VERSION if set, die unless dist version.txt matches +# +set -euo pipefail + +export DEBIAN_FRONTEND=noninteractive +export PATH="/usr/local/bin:/usr/bin:/bin:${HOME}/.local/bin:${PATH:-}" + +WEBUI_REF="${WEBUI_REF:-${1:-v1.6.11}}" +TTC_GIT="${TTC_GIT:-https://git.taler.net/taler-typescript-core.git}" +TTC_DIR="${TTC_DIR:-/var/taler-src/taler-typescript-core}" +SPA_DEST="${SPA_DEST:-/usr/share/taler-merchant-webui}" +PNPM_VERSION="${PNPM_VERSION:-10.33.4}" +SKIP_NODE_INSTALL="${SKIP_NODE_INSTALL:-0}" +SKIP_DEPLOY="${SKIP_DEPLOY:-0}" +EXPECT_VERSION="${EXPECT_VERSION:-}" + +log() { printf '+ %s\n' "$*"; } +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } + +[ "$(id -u)" -eq 0 ] || die "run as root in the merchant container" + +# --------------------------------------------------------------------------- +# Toolchain: node + pnpm +# --------------------------------------------------------------------------- +ensure_node() { + if command -v node >/dev/null 2>&1 && command -v npm >/dev/null 2>&1; then + log "node $(node -v) npm $(npm -v)" + return 0 + fi + [ "${SKIP_NODE_INSTALL}" = "1" ] && die "node/npm missing and SKIP_NODE_INSTALL=1" + + log "install nodejs + npm via apt" + apt-get update -qq + apt-get install -y --no-install-recommends nodejs npm ca-certificates git \ + python3 curl ca-certificates 2>&1 | tail -20 + command -v node >/dev/null 2>&1 || die "node still missing after apt" + log "node $(node -v) npm $(npm -v)" +} + +ensure_pnpm() { + if command -v pnpm >/dev/null 2>&1; then + log "pnpm $(pnpm -v)" + return 0 + fi + [ "${SKIP_NODE_INSTALL}" = "1" ] && die "pnpm missing and SKIP_NODE_INSTALL=1" + + log "install pnpm@${PNPM_VERSION} globally" + npm install -g "pnpm@${PNPM_VERSION}" 2>&1 | tail -15 + # npm global bin may be /usr/local/bin or under /usr/lib + hash -r 2>/dev/null || true + export PATH="$(npm prefix -g)/bin:${PATH}" + command -v pnpm >/dev/null 2>&1 || die "pnpm still missing after npm install -g" + log "pnpm $(pnpm -v)" +} + +# --------------------------------------------------------------------------- +# Source tree at REF +# --------------------------------------------------------------------------- +checkout_ttc() { + mkdir -p /var/taler-src + if [ -d "${TTC_DIR}/.git" ]; then + log "update existing ${TTC_DIR}" + cd "${TTC_DIR}" + git remote set-url origin "${TTC_GIT}" 2>/dev/null \ + || git remote add origin "${TTC_GIT}" || true + git fetch --tags --force origin + else + log "clone ${TTC_GIT} → ${TTC_DIR}" + rm -rf "${TTC_DIR}" + git clone "${TTC_GIT}" "${TTC_DIR}" + cd "${TTC_DIR}" + fi + + log "checkout ${WEBUI_REF}" + # Accept tag, branch, or raw commit + if git rev-parse --verify --quiet "refs/tags/${WEBUI_REF}" >/dev/null; then + git checkout -f "refs/tags/${WEBUI_REF}" + elif git rev-parse --verify --quiet "origin/${WEBUI_REF}" >/dev/null; then + git checkout -f -B "${WEBUI_REF}" "origin/${WEBUI_REF}" + elif git rev-parse --verify --quiet "${WEBUI_REF}" >/dev/null; then + git checkout -f "${WEBUI_REF}" + else + # maybe short hash only available after fetch + git fetch origin "${WEBUI_REF}" 2>/dev/null || true + git checkout -f "${WEBUI_REF}" || die "cannot checkout WEBUI_REF=${WEBUI_REF}" + fi + + # assets submodule (needed by build.mjs) + if [ -f .gitmodules ] && grep -q 'taler-assets' .gitmodules 2>/dev/null; then + log "submodule contrib/taler-assets" + git submodule update --init contrib/taler-assets \ + || git submodule update --init --recursive contrib/taler-assets \ + || true + fi + if [ ! -f contrib/taler-assets/svg/logo/qr-logo.svg ]; then + # fallback: shallow clone assets next to known relative url + log "assets missing — try direct clone" + rm -rf contrib/taler-assets + mkdir -p contrib + git clone --depth 1 https://git.taler.net/taler-assets.git contrib/taler-assets \ + || die "taler-assets incomplete" + fi + test -f contrib/taler-assets/svg/logo/qr-logo.svg || die "taler-assets still incomplete" + + HEAD_SHORT=$(git rev-parse --short HEAD) + HEAD_FULL=$(git rev-parse HEAD) + PKG_VER=$(python3 -c "import json; print(json.load(open('packages/taler-merchant-webui/package.json'))['version'])") + log "HEAD=${HEAD_SHORT} (${HEAD_FULL})" + log "package.json version=${PKG_VER}" +} + +# --------------------------------------------------------------------------- +# Build + install +# --------------------------------------------------------------------------- +build_spa() { + cd "${TTC_DIR}" + log "pnpm install + compile @gnu-taler/taler-merchant-webui" + pnpm install --frozen-lockfile --filter @gnu-taler/taler-merchant-webui... + pnpm run --filter @gnu-taler/taler-merchant-webui... compile + + PROD="${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + test -f "${PROD}/index.js" || die "dist/prod/index.js missing" + test -f "${PROD}/version.txt" || die "dist/prod/version.txt missing" + GOT=$(tr -d ' \n\r' <"${PROD}/version.txt") + log "built version.txt=${GOT}" + if [ -n "${EXPECT_VERSION}" ] && [ "${GOT}" != "${EXPECT_VERSION}" ]; then + die "expected version ${EXPECT_VERSION}, got ${GOT}" + fi + # sensible default when building known tag + case "${WEBUI_REF}" in + v1.6.11|1.6.11) + [ "${GOT}" = "1.6.11" ] || die "expected 1.6.11 for ref ${WEBUI_REF}, got ${GOT}" + ;; + esac + BUILT_VER="${GOT}" +} + +deploy_spa() { + cd "${TTC_DIR}" + PROD="${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + HEAD_SHORT=$(git rev-parse --short HEAD) + ts=$(date +%Y%m%d-%H%M%S) + + if [ -d "${SPA_DEST}" ] && [ "$(ls -A "${SPA_DEST}" 2>/dev/null || true)" ]; then + bak="${SPA_DEST}.bak-selfbuild-${ts}" + log "backup ${SPA_DEST} → ${bak}" + cp -a "${SPA_DEST}" "${bak}" + fi + mkdir -p "${SPA_DEST}" + find "${SPA_DEST}" -mindepth 1 -maxdepth 1 ! -name 'bak-*' -exec rm -rf {} + + + log "install SPA → ${SPA_DEST}" + cp -a "${PROD}/." "${SPA_DEST}/" + printf 'selfbuild-%s-%s\n' "${WEBUI_REF}" "${HEAD_SHORT}" >"${SPA_DEST}/version-overlay.txt" + chown -R root:root "${SPA_DEST}" + + echo "=== deployed ===" + cat "${SPA_DEST}/version.txt" + echo + cat "${SPA_DEST}/version-overlay.txt" + echo + wc -c "${SPA_DEST}/index.js" +} + +# --------------------------------------------------------------------------- +main() { + log "selfbuild-webui REF=${WEBUI_REF}" + ensure_node + ensure_pnpm + checkout_ttc + build_spa + if [ "${SKIP_DEPLOY}" = "1" ]; then + log "SKIP_DEPLOY=1 — left at ${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + else + deploy_spa + fi + log "DONE webui ${WEBUI_REF} → version ${BUILT_VER:-?} @ ${SPA_DEST}" +} + +main "$@" diff --git a/scripts/taler-merchant/setup_credit_facade.sh b/scripts/taler-merchant/setup_credit_facade.sh index 3ab0403..bf5ffb4 100755 --- a/scripts/taler-merchant/setup_credit_facade.sh +++ b/scripts/taler-merchant/setup_credit_facade.sh @@ -16,7 +16,8 @@ # - restarts taler-merchant-wirewatch (once, after facade is set) set -euo pipefail -INST="${MERCHANT_INSTANCE:-goa-demo-cp4zqk}" +# Instance id must be lowercase (portal/API: capitals → odd 401s). +INST=$(printf '%s' "${MERCHANT_INSTANCE:-goa-demo-cp4zqk}" | tr '[:upper:]' '[:lower:]') BANK_USER="${BANK_USER:-$INST}" MERCHANT_PW_FILE="${MERCHANT_PW_FILE:-/root/merchant-${INST}-password.txt}" BANK_PW_FILE="${BANK_PW_FILE:-/root/bank-${BANK_USER}-password.txt}" diff --git a/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md new file mode 100644 index 0000000..3cd311a --- /dev/null +++ b/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md @@ -0,0 +1,228 @@ +# CLI automation notes (`taler-wallet-cli` + monitoring) + +Living list of **recurring issues** seen while driving GOA (hacktivism) and +stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring +e2e/ladder). Ordered roughly by **how often they hurt automation** and how +useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be. + +Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md). + +Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`. + +Legend: + +| Tag | Meaning | +|-----|---------| +| **cli** | wallet-cli UX / flags / hang behaviour | +| **core** | wallet-core protocol / state machine | +| **bank** | libeufin-bank / integration API | +| **ops** | secrets, paths, multi-host | +| **doc** | missing or wrong documentation | + +--- + +## 1. `run-until-done` is unusable for automation + +| | | +|--|--| +| **Seen** | e2e, ladder, Android notes, macOS comments: *hangs / banned in monitoring* | +| **Area** | **cli** / **core** | +| **Problem** | `taler-wallet-cli run-until-done` (or long shepherd runs) often **never returns** or blocks CI-length timeouts. Monitoring **forbids** it and polls balance + bank `transfer_done` instead. | +| **Workaround** | `advanced serve` + socket (`WSOCK`) or pure poll loops (`LADDER_SETTLE_*`, e2e settle wait). | +| **Wanted for wallet-cli** | Bounded wait: e.g. `run-until-done --timeout=Ns --exit-on=withdrawn|paid|idle`; stable non-zero exit on timeout; progress on stdout/JSON. | + +--- + +## 2. Withdraw completion is not observable from CLI alone + +| | | +|--|--| +| **Seen** | e2e “coins missing after withdraw”; ladder `OK_BANK` vs wallet avail | +| **Area** | **cli** / **core** / **bank** | +| **Problem** | After `accept-uri` + bank confirm, coins may lag (wirewatch). CLI has no single “withdraw settled” command with clear success/failure; automation reimplements status from bank integration JSON + balance polls. | +| **Workaround** | Poll `balance` + `…/withdrawal-operation/{id}` (`status`, `transfer_done`, `selected_reserve_pub`). | +| **Wanted** | `wallet-cli withdrawals wait --id=… --timeout=` or JSON event stream: `selected → confirmed → coins-available`. | + +--- + +## 3. `reserve_pub` / force-select is fragile (5114) + +| | | +|--|--| +| **Seen** | ladder force-select, e2e confirm path, Android spinner issues | +| **Area** | **cli** / **core** / **bank** | +| **Problem** | Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). Automation scrapes accept/tx dumps for candidates. | +| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. | +| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. | + +--- + +## 4. Exchange not auto-known for bank deep links / withdraw URIs + +| | | +|--|--| +| **Seen** | Android endless spinner; GOA `bank.hacktivism.ch` withdraw; fixed in app branch `fix/bank-withdraw-auto-exchange` | +| **Area** | **core** / **cli** | +| **Problem** | Withdraw URI names an exchange the wallet has never added → stuck loading / empty exchange list. CLI requires explicit `exchanges add` + `update` + `accept-tos` every fresh DB. | +| **Workaround** | e2e always adds exchange before ATM ladder; Android needs ensureExchange-style fix. | +| **Wanted** | CLI/core: on `accept-uri` withdraw, **auto-add** exchange from URI/bank details (`allowCompletion`), then surface ToS if needed in one command. | + +--- + +## 5. Default ports in URIs (`:443` / `:80`) break round-trips + +| | | +|--|--| +| **Seen** | ladder/e2e strip ports; QR checks; mint URIs with `:443` | +| **Area** | **cli** / **bank** / **doc** | +| **Problem** | Some bank/wallet outputs include `taler://…host:443/…`. QR validators and some clients reject or double-normalize inconsistently. | +| **Workaround** | `sed 's/:443\//\//g'` everywhere in automation. | +| **Wanted** | Canonical form without default ports in **all** wallet-cli and bank integration outputs; document as invariant. | + +--- + +## 6. No first-class “demo withdraw” / explorer mint in wallet-cli + +| | | +|--|--| +| **Seen** | e2e uses bank admin + ATM create; ladder/gui-chain reimplement explorer mint in Python | +| **Area** | **cli** / **ops** | +| **Problem** | Automation always hand-rolls Basic auth → token → `POST …/withdrawals` → `taler_withdraw_uri`. Easy to get wrong (auth header, amount currency). | +| **Workaround** | Shared Python in `goa-chain` / ladder / e2e; secrets files. | +| **Wanted** | Optional helper: `wallet-cli testing mint-withdraw --bank=URL --user=explorer --password-file=… --amount=GOA:10` (or bank-side only tool in libeufin) emitting a clean URI. | + +--- + +## 7. Long-lived wallet process required for pay/withdraw progress + +| | | +|--|--| +| **Seen** | e2e `advanced serve` / shepherd socket | +| **Area** | **cli** | +| **Problem** | One-shot CLI invocations do not keep background work alive; without serve/shepherd, withdraw/pay may stall mid-flight. | +| **Workaround** | Start serve once per e2e run; route `wcli` through the socket. | +| **Wanted** | Documented, supported “session mode”: start/stop serve; or each mutating command optionally `--background-until=…` with timeout. | + +--- + +## 8. ToS accept is a separate step that fails silently or blocks + +| | | +|--|--| +| **Seen** | e2e `exchanges accept-tos`; ladder after add; Android GUI ToS screens | +| **Area** | **cli** / **core** | +| **Problem** | Fresh exchange → operations need ToS; CLI must call `accept-tos` explicitly. Failures are easy to miss in multi-step scripts. | +| **Workaround** | Always `accept-tos` after `update` in e2e/ladder. | +| **Wanted** | `accept-uri --accept-tos` or auto-prompt with non-interactive `--yes` that covers exchange ToS for that URI’s exchange. | + +--- + +## 9. Pay template / public order path is merchant-shaped, not CLI-shaped + +| | | +|--|--| +| **Seen** | e2e shop templates, stage farmer shops, ladder private orders | +| **Area** | **cli** / **doc** | +| **Problem** | Wallet-cli pays via `handle-uri` on `taler://pay/…`; creating the order is always custom curl (template POST or private order + token). No unified “pay this amount to instance” for public templates. | +| **Workaround** | Monitoring builds URI externally then `handle-uri --yes`. | +| **Wanted** | Documented recipe only, or `wallet-cli testing pay-template --base=… --instance=… --id=…` for demos. | + +--- + +## 10. Amount / currency parsing edge cases + +| | | +|--|--| +| **Seen** | ladder min denom 0.01 vs GOA 1e-6; zero withdraw rejected (HTTP 409 / amount too low); max wire ceilings | +| **Area** | **cli** / **bank** | +| **Problem** | `CURRENCY:0` and sub-min amounts fail differently per stack; CLI error strings are not machine-stable. | +| **Workaround** | Soft-skip zero rung; clamp ladder min to exchange denoms / bank max_wire. | +| **Wanted** | Structured errors (`AMOUNT_too_small`, `currency_unknown`) in JSON mode; `wallet-cli amount validate --exchange=`. | + +--- + +## 11. Finding the right `taler-wallet-cli` binary + +| | | +|--|--| +| **Seen** | `find_wallet_cli` in `lib.sh`; hardcoded laptop paths; wrapper vs `.mjs` | +| **Area** | **ops** / **cli** | +| **Problem** | Debian package is a shell wrapper; some tools need `node …/taler-wallet-cli.mjs`. Hardcoded `/Users/…` paths break other hosts. | +| **Workaround** | `find_wallet_cli` search list; `WALLET_CLI=` override. | +| **Wanted** | Single install story: `wallet-cli --version` JSON with path + libversion; no need to pass `.mjs` to node by hand. | + +--- + +## 12. Secrets and multi-stack confusion + +| | | +|--|--| +| **Seen** | explorer vs admin password; GOA secrets used on stage; ladder EXP_PW | +| **Area** | **ops** | +| **Problem** | CLI does not know “which stack”; wrong password → 401 mid-ladder. Not a wallet-cli bug, but every CLI automation hits it. | +| **Workaround** | `SECRETS_ROOT`, stage SSH, `STACK=` profiles. | +| **Wanted** | Optional `~/.config/taler/stacks.d/goa.env` convention documented next to wallet-cli; still no secrets in repo. | + +--- + +## 13. No stable machine-readable “step result” for scripts + +| | | +|--|--| +| **Seen** | e2e greps accept output; ladder scrapes JSON from mixed stdout | +| **Area** | **cli** | +| **Problem** | Human logs + occasional JSON blobs; hard to parse reliably. | +| **Workaround** | Python scrapers, temp files, `tee`. | +| **Wanted** | Global `--json` / `--ndjson` for all commands; one object per completed operation with `ok`, `op`, `ids`, `amounts`. | + +--- + +## 14. Pay settlement wait is symmetric to withdraw pain + +| | | +|--|--| +| **Seen** | e2e pay settle loops; ladder pay settle rounds | +| **Area** | **cli** / **core** | +| **Problem** | After `handle-uri` pay, success is “order paid” / balance drop / merchant order status — not one CLI wait. | +| **Workaround** | Short poll loops; never run-until-done. | +| **Wanted** | Same as §1–2: bounded wait on transaction id / order id. | + +--- + +## Priority shortlist for `taler-wallet-cli` / core + +If only a few changes land, these unlock the most automation: + +1. **Bounded `run-until-done` / wait-for-state** (§1, §2, §14) +2. **Structured JSON on every command** (§13) +3. **Auto-add exchange + ToS on accept-uri** (§4, §8) +4. **Canonical URIs without :443** (§5) +5. **Emit reserve_pub for last withdraw** (§3) + +--- + +## How we work around today (monitoring) + +| Issue | Monitoring behaviour | +|-------|----------------------| +| run-until-done | Disabled; balance + bank poll | +| serve | Optional long-lived socket in e2e | +| exchange | Explicit add/update/accept-tos | +| ladder explorer | `read_secret` / EXP_PW_FILE | +| stage maxima | bank `max_wire` + keys min denom | +| force-select | multi-rpub try; soft-skip confirm | + +--- + +## How to add an issue + +```markdown +### N. short title +| | | +|--|--| +| **Seen** | where / which stack | +| **Area** | cli / core / bank / ops / doc | +| **Problem** | … | +| **Workaround** | … | +| **Wanted** | concrete CLI/core behaviour | +``` diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md new file mode 100644 index 0000000..4ad80b3 --- /dev/null +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -0,0 +1,198 @@ +# Git / automation notes (taler-monitoring + mobile) + +Notes for **automated checks** around the Taler stacks (GOA, stage +TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet +UI automation is separate tooling** and is **not** installed on the +default ops laptop yet. + +--- + +## Git commit / push cadence (existing model) + +Observed in `koopa-admin-log` / `francpaysan-admin-log` histories, plus ops preference: + +| Rule | Practice | +|------|----------| +| **Commits** | **Prefer frequent** small commits over rare large ones | +| **Interval** | Soft range **7–13 minutes** while a topic is open — **not** a fixed 9‑minute metronome; denser 0–2 min bursts for tiny follow-ups are fine | +| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …` | +| **Push** | **Only at the end of a work session** (`CADENCE_PUSH=1` or manual `git push`) — not after every commit | + +### Agent / human workflow + +1. Finish a small slice → commit (`area: …`). Repeat often. +2. Do **not** push after each commit. +3. When the topic/session is done → one `git push` (or `CADENCE_PUSH=1`). +4. Soft next-commit window: about **7–13 min** (jitter), sooner if the next slice is ready. + +### Helper script + +```bash +# dry-run +CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh + +# commit only if dirty (default: no push; skips secrets) +./scripts/git-cadence-commit-push.sh +./scripts/git-cadence-commit-push.sh ~/taler/src/francpaysan-admin-log + +# end of session — push commits that are already local +CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh + +# soft timer: only commit if last commit age ≥ random target in 7–13 min +CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh +``` + +Optional user cron (check every 7 min; **jitter** keeps effective gaps in **7–13 min**): + +```bash +*/7 * * * * CADENCE_JITTER=1 $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1 +``` + +**Auth note:** `francpaysan-admin-log` (gnunet SSH) pushes OK. +`koopa-admin-log` (hacktivism HTTPS) needs credentials/SSH key before end-of-session push works from this host. + +## What runs without Android + +| Phase | Needs | +|-------|--------| +| `urls` / `sanity` | public HTTPS only | +| `inside` / `versions` | SSH (`koopa` or `francpaysan-stage-user`) | +| `e2e` / `ladder` | `taler-wallet-cli` + bank/merchant secrets | + +```bash +./taler-monitoring.sh -d stage.lefrancpaysan.ch urls +./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env +``` + +Mobile app checks (manual withdraw/pay, then automation) are **out of +band** until the software below is present. + +--- + +## Android automation software — **required** (not installed yet) + +To automate **GNU Taler Wallet** on Android (stable + master builds; +manual then auto withdraw; fixed + variable pay) against stage +TESTPAYSAN, the following must be available on the runner host. + +### Status on this host (2026-07-17) + +| Component | Status | +|-----------|--------| +| Java (OpenJDK 21) | present (`/usr/bin/java`) | +| `aapt` / `aapt2` | present | +| **`adb`** (platform-tools) | **missing** (package was removed: `android-tools-adb` rc) | +| **Android SDK** (`ANDROID_HOME` / `ANDROID_SDK_ROOT`) | **missing** / empty | +| **Emulator + AVD** | **missing** | +| Appium / Maestro / `uiautomator2` | **missing** | +| scrcpy | missing (optional) | + +### Must install before mobile automation + +1. **Android platform-tools** — at least `adb` + - Debian: `android-tools-adb` (and preferably full SDK platform-tools) +2. **Android SDK command-line tools** — `sdkmanager`, platforms, build-tools + - set `ANDROID_HOME` (or `ANDROID_SDK_ROOT`) permanently +3. **System image + AVD** — for emulator runs (x86_64 or host-matching ABI) + - `emulator`, `avdmanager`, KVM/acceleration if available +4. **UI automation stack** (pick one, prefer the leaner path for wallet flows): + - **Maestro**, or + - **Appium** + **UiAutomator2** (+ Node/Python bindings as needed) +5. **Optional:** `scrcpy` for live view during manual debugging + +### Explicitly out of scope until freigabe + +- Installing production wallet APKs and driving real stage withdraw/pay + without a human go-ahead +- Committing secrets, device serials, or APK binaries into git + +### Operator still needs to supply + +- Stable / master **APK** (or store/build URLs) for the wallet +- USB debugging consent on a physical device, **or** agreement to use AVD only +- Which flows are in-scope first (withdraw communal QR → shop pay, …) + +### Suggested freigabe prompt (for an agent) + +See session notes: full copy-paste block under “Android-Automatisierung +vollständig einrichten” — install tooling, verify `adb devices` + emulator, +document paths; **no** real wallet payments until approved. + +--- + +## Related docs + +| Doc | Role | +|-----|------| +| [README.md](./README.md) | how to run `taler-monitoring.sh` | +| [TESTS.md](./TESTS.md) | grouped check IDs | +| [secrets.env.example](./secrets.env.example) | e2e secrets layout | +| `francpaysan-admin-log/TODO.md` | stage wallet mobile test backlog | + +When Android tooling is installed, append a short “installed on …” +section here (date, `adb version`, AVD name, `ANDROID_HOME`). + +--- + +## Android pay smoke (published APK, no root preferred) + +Script: [`android-test/run-android-pay-smoke.sh`](./android-test/run-android-pay-smoke.sh) + +| Step | Detail | +|------|--------| +| APK | F-Droid **Taler Wallet 1.6.1** (`net.taler.wallet.fdroid_854.apk`) | +| Install | `adb install -r` (no root) | +| Withdraw | `taler://withdraw/…` from `…/intro/demo-withdraw.json` (GOA or stage) | +| Pay | stage: shop template order → `taler://pay/…`; GOA: `taler://pay-template/…/goa-shop/paivana` | +| Evidence | screenshots + logcat under `android-test/out/` | + +```bash +# device or emulator already booted +./android-test/run-android-pay-smoke.sh +STACK=goa ./android-test/run-android-pay-smoke.sh +STACK=stage ./android-test/run-android-pay-smoke.sh +``` + +**User-local emulator (no apt root if SDK already under `$HOME/Android/Sdk`):** + +```bash +# Default = headless (no host window); guest GLES via SwiftShader. +# Prefer the helper — same flags smokes use: +./android-test/start-android-emulator.sh --wait +./android-test/run-android-pay-smoke.sh +# Or let smoke auto-start the AVD (AUTO_START_EMULATOR=1). +# Windowed laptop: WINDOWED=1 ./android-test/start-android-emulator.sh --wait +``` + +**Limits (clementine 2026-07-17):** host RAM ~3.7 GiB → emulator often ANRs System UI; +wallet-core still receives withdraw intents (`prepareBankIntegratedWithdrawal` in logcat). +For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 GiB** free RAM. + +### Status snapshot (2026-07-17) + +| Item | Status | +|------|--------| +| `adb` | present (`android-sdk-platform-tools`) | +| F-Droid APK | downloaded under `android-test/apks/` | +| User SDK + emulator + AVD `TalerWallet34` | installed under `$HOME/Android/Sdk` (~5 GiB) | +| `adb install` wallet | **Success** on emulator-5554 | +| Deep-link withdraw | **delivered**; wallet-core started `prepareBankIntegratedWithdrawal` for GOA | +| Unattended UI confirm | flaky (ANR) on this host’s RAM | +| Source build `assembleFdroidDebug` | **OK** (2026-07-17, `taler-android@f65a976c0d4a`, ~12 min, APK ~59 MiB) | + +### From-source smoke (same tests as F-Droid) + +```bash +# single ref (default: inference fix branch) +./android-test/run-android-build-and-smoke.sh +BRANCH=master ./android-test/run-android-build-and-smoke.sh +# self-build published stable tag (parity with F-Droid 1.6.1 source) +BRANCH=wallet-1.6.1 LABEL=stable-self ./android-test/run-android-build-and-smoke.sh +# matrix: published APK + stable self-build + master (anti–weird-build) +./android-test/run-android-variant-matrix.sh +STACK=goa ./android-test/run-android-build-and-smoke.sh +# already built: +APK_PATH=$HOME/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ + PKG=net.taler.wallet.fdroid.debug \ + ./android-test/run-android-pay-smoke.sh +``` diff --git a/scripts/taler-monitoring/README.md b/scripts/taler-monitoring/README.md index e627a35..578b683 100644 --- a/scripts/taler-monitoring/README.md +++ b/scripts/taler-monitoring/README.md @@ -13,7 +13,10 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs** IDs reset per **group** inside an area — so “too many www tests” become `www.exchange-*`, `www.bank-*`, `www.landing-*`, etc. -Catalog: **[TESTS.md](./TESTS.md)**. +Catalog: **[TESTS.md](./TESTS.md)**. +Android / tooling: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**. +GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**. +CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**. ```text ┌ OK ┐ www.exchange-01 exchange /config · HTTP 200 @@ -106,6 +109,26 @@ Each check prints **global** and **grouped** ids: # Landing stats.json freshness (outside-in; age via generated_at_unix) STATS_STALE_SECS=900 STATS_FAIL_SECS=3600 ./taler-monitoring.sh -d stage.lefrancpaysan.ch urls + +# --- Full load (GOA / hacktivism) --- +# all = urls + inside + versions + sanity + e2e +# full = all + server + ladder + auth401 (long; needs secrets.env) +./taler-monitoring.sh -d hacktivism.ch full +./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401 + +# SPA pin after selfbuild-webui.sh (container): +EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \ + WEBUI_OVERLAY_DENY='c778af04|master-11340' \ + ./taler-monitoring.sh -d hacktivism.ch urls + +# Only 401 matrix (may create throwaway instance): +./taler-monitoring.sh -d hacktivism.ch auth401 + +# Stage FrancPaysan — durable mon401 account (password in francpaysan-secrets): +./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401 +# secrets: ~/francpaysan-secrets/stage/instances/mon401.password +# ~/francpaysan-secrets/stage/default-instance-token.txt +# force throwaway create instead: AUTH401_PREFER_DURABLE=0 AUTH401_CREATE=1 … ``` ### Landing `stats.json` (www.stats) @@ -197,7 +220,13 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC. ```bash ./taler-monitoring.sh e2e ./taler-monitoring.sh -d taler.net urls e2e -# customize: +# Amount ladder (withdraw 0 → … → max, then pay): same script for GOA + stage +./taler-monitoring.sh ladder # GOA / current domain +./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima +# Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01). +# Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=). +# Override: LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 … +# customize e2e: E2E_WITHDRAW_VALUES="20 50 100" E2E_PAY_VALUES="0.05 1 5" ./taler-monitoring.sh e2e E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # single fixed # GOA shop catalog (local hacktivism): full list in E2E_SHOP_PRODUCTS; each run diff --git a/scripts/taler-monitoring/TESTS.md b/scripts/taler-monitoring/TESTS.md index 7bd51c1..fb829d0 100644 --- a/scripts/taler-monitoring/TESTS.md +++ b/scripts/taler-monitoring/TESTS.md @@ -52,8 +52,9 @@ Numbering follows **executed** checks (early skip may shift later NN inside the | **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three | | **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` | | **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** | +| **www.webui-** | SPA fingerprints: **`/webui/version.txt`**, **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` | | **www.paivana-** | local GOA paywall front (redirect to template) | -| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; cross-links local | +| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local | | **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` | **QR rule:** `taler://withdraw/HOST/taler-integration/UUID` (no default `:443`/`:80`); `taler://pay/` / `pay-template/`; `payto://…` shape OK; decoded PNG must equal payload. Form errors on withdraw/pay/payto are **ERROR** on local/GOA. @@ -106,7 +107,7 @@ When filing an issue, quote the full id + label, e.g. --- -## versions / sanity / ladder / server +## versions / sanity / ladder / server / auth401 | Area.group | Meaning | |------------|---------| @@ -114,8 +115,36 @@ When filing an issue, quote the full id + label, e.g. | **versions.inside-** | packages in containers | | **versions.compare-** | installed vs suite | | **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component | -| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | GOA amount ladder | +| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder (GOA ceiling or stage TESTPAYSAN `max_wire`) | | **server-** | SSH host ports / processes (flat unless grouped later) | +| **auth401.*** | merchant 401 / case matrix (WebUI APIs): **signup** `POST /instances` MIX id, **login** Basic user low vs MIX/UPPER, full **case** matrix, **password** bad/empty, **pwchange** `POST private/auth` + re-login, **admin-create** `POST /management/instances` (if admin token), **bearer**, **durable mon401**, **webui** SPA `toLowerCase` probe (`check_auth401.sh`) | + +### SPA pin (after selfbuild-webui) + +```bash +EXPECT_WEBUI_VERSION=1.6.11 \ +EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \ +WEBUI_OVERLAY_DENY='c778af04|master-11340' \ + ./taler-monitoring.sh -d hacktivism.ch urls +``` + +Skip SPA block: `CHECK_WEBUI_SPA=0`. + +### Full load (maximum phases) + +| Alias | Expands to (local GOA) | +|-------|-------------------------| +| **`all`** | `urls inside versions sanity e2e` | +| **`full`** | `urls inside versions sanity server e2e ladder auth401` | + +```bash +cd ~/src/koopa/koopa-admin-log/scripts/taler-monitoring +# secrets.env with bank admin + merchant tokens (see secrets.env.example) +./taler-monitoring.sh -d hacktivism.ch full +# same, explicit: +./taler-monitoring.sh -d hacktivism.ch \ + urls inside versions sanity server e2e ladder auth401 +``` --- diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore new file mode 100644 index 0000000..49b08e5 --- /dev/null +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -0,0 +1,11 @@ +apks/*.apk +out/ +*.png +out-source/ +out-source-*/ +out-gui/ +out-source-gui/ +out-gui-chain/ +out-matrix-*/ +out-stack-run-*/ +out-survey-*/ diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md new file mode 100644 index 0000000..18fe398 --- /dev/null +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -0,0 +1,362 @@ +# GUI automation notes (Android wallet) + +Canonical notes for **graphical** Android wallet tests against GOA / stage. + +--- + +## Minimal variant (implemented first) — status 2026-07 + +This is the **minimal vanilla** path we actually shipped; full dual-platform +GUI is planned via flags (below), not required to use these scripts today. + +| Piece | Status | Notes | +|-------|--------|--------| +| F-Droid published APK install | **done** | `net.taler.wallet.fdroid` 1.6.1 / 854 | +| Source build + same smoke | **done** | `:wallet:assembleFdroidDebug` → `net.taler.wallet.fdroid.debug` | +| Default source branch | **done** | `dev/hernani-inference/fix-bank-withdraw-auto-exchange` (minimal GOA exchange auto-add) | +| Deep-link entry (≡ QR) | **done** | `adb` `VIEW` **withdraw then pay** (`taler://withdraw/…` → `taler://pay…`) | +| Explorer API mint (chain) | **done** | `run-goa-gui-chain.sh` (GOA + stage) | +| UI taps via uiautomator | **done** | best-effort Confirm/ToS/Pay; ANR → prefer **Wait** | +| Screenshots / logcat artifacts | **done** | under `out*`, `out-gui-chain/` (gitignored) | +| Host CLI e2e settlement | **separate** | `taler-monitoring.sh e2e` / `ladder` (not Android UI) | +| Reliable unattended Confirm on 4 GiB Linux host | **limited** | System UI ANR; use phone or more RAM | +| iOS GUI | **not in this tree** | see taler-ios `dev/hernani-inference/gui-workflows` | +| macOS full dual AVD helpers | **upstream** | taler-android `gui-workflows` (Homebrew); ported ideas here for Linux | + +**Ground rule:** bring it to run; app diffs only if needed and **minimal**; prefer +extending existing `dev/hernani-inference/*` branches over new trees. + +### Minimalvariante: was geht / was nicht + +Einschränkungen der **aktuell implementierten** Minimalvariante (deep-link + +uiautomator best-effort). Kein voller Dual-Platform-GUI-Stack, kein Ersatz für +Host-CLI-Settlement. + +#### Geht (grün) + +| Fähigkeit | Wie / Hinweis | +|-----------|----------------| +| APK installieren | F-Droid `net.taler.wallet.fdroid` **oder** Source `assembleFdroidDebug` (`.debug`) | +| GOA-Exchange ohne manuelles Add | Source-Build von `fix-bank-withdraw-auto-exchange` (Default in `run-android-build-and-smoke.sh`) | +| **Abheben + Bezahlen** | Beide Beine: erst Withdraw-URI + Taps, dann Pay-URI + Taps (`DO_WITHDRAW`/`DO_PAY`) | +| Light UI-Drive | Confirm / Accept / Pay-Taps über Label-Suche (`lib_ui.py`, gui-smoke, chain) | +| Multi-Runden-Taps | `run-android-gui-smoke.sh` / `run-goa-gui-chain.sh` (best-effort) | +| Explorer-Mint (Kette) | `run-goa-gui-chain.sh` + `EXP_PW_FILE` (GOA + stage) | +| Stack-Wahl | `STACK=goa` / `stage` (Bank/Merchant-URLs) | +| Artefakte | Screenshots, UI-XML, logcat unter `out*` / `out-gui-chain/` (gitignored) | +| Hybrid-Smoke | Install + Intent + leichte Taps: `run-android-pay-smoke.sh` | +| Settlement **nachweisen** | **Host-seitig**: `taler-monitoring.sh e2e` / `ladder` (nicht Android-UI) | +| Flags abschalten | `AUTO_ANDROID=0` skip; `AUTO_GUI=0` → nur deep-link/pay-smoke | + +#### Geht **nicht** oder nur eingeschränkt (rot / gelb) + +| Beschränkung | Folge / Workaround | +|--------------|--------------------| +| **Unattended Confirm → settled im Emulator (~4 GiB Host)** | Häufig **System UI ANR**; Taps bleiben hängen. Prefer **Wait** auf ANR-Dialog, echtes Gerät, oder Host mit mehr RAM. Kein zuverlässiges grünes E2E nur über GUI auf dem schwachen Emulator. | +| **Vollständige Wallet-Settlement-Assertion in der GUI** | Minimalvariante prüft **nicht** „Balance final / transfer_done“ in der App-Oberfläche. Proof bleibt CLI/e2e. | +| **Kamera / physischer QR-Scan** | Absichtlich nicht automatisiert; deep-link ist der Shortcut. Landing-QR-Pfade sind **HTTP**-Thema, nicht Android-GUI. | +| **Browser-Shop-Checkout-UI** | Kein Chromium/WebView-Drive; Pay-URI kommt per Template-`POST` + Intent. | +| **Volles ToS-Scroll / alle Dialoge** | Best-effort Accept-Tap; lange ToS oder unerwartete Sheets können hängen bleiben. | +| **Published F-Droid allein auf GOA** | Kann Exchange-Add / Spinner-Probleme zeigen → **fix-branch-APK** bauen; bei Spinner-Hang optional `fix-withdraw-spinner-fallback`. | +| **iOS-GUI** | Nicht in diesem Tree; siehe taler-ios `gui-workflows`. | +| **macOS dual AVD / Homebrew-Helpers** | Upstream `gui-workflows`; hier nur Linux-taugliche Port-Ideen. | +| **`AUTO_*` in jedem Script erzwungen** | Design + teilweise verdrahtet; Wrapper-Checkliste noch offen. | +| **`run-until-done` / CLI-Wallet in GUI-Skripten** | Gehört zu CLI-Automation, nicht GUI; siehe `CLI-AUTOMATION-NOTES.md`. | +| **CI grün = „User hat bezahlt“** | Smoke = Intent + best-effort UI. Wirtschaftlicher Erfolg = Host-e2e / ladder / Explorer. | + +#### Erwartetes Ergebnis pro Schicht + +| Schicht | Erfolgskriterium Minimalvariante | +|---------|----------------------------------| +| A Install | APK installed, App startet | +| B Deep-link | Intent delivered, Wallet öffnet Withdraw/Pay-Flow | +| C GUI taps | Dump+Tap ohne Crash; ANR ggf. mit Wait; **kein** Garant für „Paid“ | +| D Evidence | Artefakte geschrieben | +| Settlement | **Außerhalb** GUI: Host e2e/ladder | + +#### Wann Minimalvariante reicht + +- Smoke nach APK-/Branch-Build (Intent kommt an, App crasht nicht). +- Manuelle Nacharbeit am Gerät nach vorbereiteten URIs. +- GOA/stage **Erreichbarkeit** von Landing/Template + Wallet-Einstieg. + +#### Wann nicht reicht → nächster Schritt + +- Unattended grünes Pay-E2E auf dem Emulator → mehr RAM / physisches Gerät, ggf. `fix-withdraw-spinner-fallback`. +- Beweis „Münzen da / Merchant paid“ → CLI e2e/ladder, nicht GUI-smoke. +- iOS / Dual-Platform → andere Repos/Flags, nicht diese Skripte. + +```bash +cd scripts/taler-monitoring/android-test + +# Minimal hybrid (install + deep-link + light taps) +STACK=goa ./run-android-pay-smoke.sh + +# GUI multi-round taps +STACK=goa ./run-android-gui-smoke.sh +STACK=stage ./run-android-gui-smoke.sh + +# Explorer mint + multi withdraw/pay chain (from gui-workflows) +EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ + STACK=goa PKG=net.taler.wallet.fdroid.debug \ + ./run-goa-gui-chain.sh + +# Build fix-branch APK then smoke (GUI=1 for gui smoke) +./run-android-build-and-smoke.sh +GUI=1 ./run-android-build-and-smoke.sh +``` + +--- + +## Platform capability flags (planned + defaults) + +Automation can run on hosts that support **Android only**, **GUI tooling only** +(conceptual), or **both**. Flags keep that explicit for CI and laptops. + +### Proposed env flags + +| Variable | Values | Meaning | +|----------|--------|---------| +| `AUTO_ANDROID` | `0` / `1` | Run Android wallet automation (adb, APK, emulator/device) | +| `AUTO_GUI` | `0` / `1` | Drive **graphical** UI (uiautomator taps, multi-round). If `0`, deep-link/CLI-only install smoke. | +| `AUTO_PLATFORM` | `auto` / `linux` / `macos` / `ios` | Host family (optional override) | +| `EMULATOR_HEADLESS` | **`1` (default)** / `0` | No host window (`-no-window`). Server/CI-safe. | +| `WINDOWED` | `0` / `1` | Shortcut: `WINDOWED=1` ⇒ show emulator window (`EMULATOR_HEADLESS=0`) | +| `EMULATOR_GPU` | default **`swiftshader_indirect`** | Guest GLES still runs headless (layout/taps/screenshots). Override `host` only if windowed + real GPU. | +| `AUTO_START_EMULATOR` | **`1` (default)** / `0` | If no adb device, start AVD via `start-android-emulator.sh --wait` | + +--- + +## Headless mode (default) + +**Default for all GUI / pay smokes:** no window on the host. Opening an emulator +GUI on servers (or headless CI) is wrong; these scripts therefore start the AVD +with **`-no-window`** and still keep **graphics logic** in the guest: + +| Piece | Headless default | +|-------|------------------| +| Host window | **off** (`-no-window`, `QT_QPA_PLATFORM=offscreen`, no `DISPLAY`) | +| Guest GLES | **on** via **SwiftShader** (`-gpu swiftshader_indirect`) | +| UI automation | **on** — `uiautomator` dump/tap + `screencap` (no X11 needed) | +| Audio / boot anim | off (`-no-audio -no-boot-anim`) | + +```bash +# explicit (same as default) +./start-android-emulator.sh --wait + +# or let smoke auto-start headless AVD when no phone is plugged in +STACK=goa ./run-android-gui-smoke.sh + +# laptop: show the emulator window +WINDOWED=1 ./start-android-emulator.sh --wait +# or: EMULATOR_HEADLESS=0 EMULATOR_GPU=host ./start-android-emulator.sh --wait +``` + +Shared code: `lib_android_env.sh` + `start-android-emulator.sh`. + +**Semantics:** + +| `AUTO_ANDROID` | `AUTO_GUI` | Behaviour | +|----------------|------------|-----------| +| `1` | `0` | Android install + deep-link smoke only (no multi-round taps) | +| `1` | `1` | Android + graphical drive (vanilla GUI / chain) | +| `0` | `1` | Reserved (e.g. future desktop/web GUI); currently **no-op** with a clear message | +| `0` | `0` | Skip mobile automation | + +### Defaults by host (when flags unset) + +| Host (`uname -s`) | Default `AUTO_ANDROID` | Default `AUTO_GUI` | Rationale | +|-------------------|------------------------|--------------------|-----------| +| **Linux** | **`1`** | **`1`** if device/emulator present, else scripts exit 3 | This repo’s day-to-day path; **Android only** (no iOS here) | +| **Darwin (macOS)** | `1` | `1` | Can run Android emulators **and** (separately) taler-ios GUI helpers; both flags on for Android scripts; iOS is out of tree | +| **Other** | `0` | `0` | Fail closed | + +**Linux default = Android** (no second mobile platform in this suite). +**macOS** may enable both ecosystems in the wider monorepo sense; for *these* +scripts only Android is implemented — set `AUTO_ANDROID=1` (default) and keep +iOS under `taler-ios`. + +### Resolution helper (convention for future wrappers) + +```bash +# Example for a future run-all-mobile.sh +os=$(uname -s) +: "${AUTO_PLATFORM:=auto}" +case "$AUTO_PLATFORM" in + auto) case "$os" in Linux) AUTO_PLATFORM=linux ;; Darwin) AUTO_PLATFORM=macos ;; *) AUTO_PLATFORM=other ;; esac ;; +esac +case "$AUTO_PLATFORM" in + linux) + : "${AUTO_ANDROID:=1}" + : "${AUTO_GUI:=1}" + # no iOS + ;; + macos) + : "${AUTO_ANDROID:=1}" + : "${AUTO_GUI:=1}" + # optional later: AUTO_IOS=1 for taler-ios scripts + ;; + *) + : "${AUTO_ANDROID:=0}" + : "${AUTO_GUI:=0}" + ;; +esac +``` + +Scripts **today** implement Android only; they should honour: + +- `AUTO_ANDROID=0` → exit 0 with “skipped (AUTO_ANDROID=0)” +- `AUTO_GUI=0` → call deep-link smoke without multi-round GUI (or set `GUI_ROUNDS=0`) + +--- + +## Vanilla layers (detail) + +| Layer | What | Tooling | +|-------|------|---------| +| **A — Install** | F-Droid or from-source APK | `adb install` | +| **B — Entry (shortcut)** | `taler://withdraw/…` / `taler://pay…` | `adb am start -a VIEW` | +| **C — Graphical UI** | Confirm / ToS / Pay taps; ANR Wait | `uiautomator` + `lib_ui.py` / chain | +| **D — Evidence** | Screenshots, XML, logcat, URIs | `out*/` | + +### Legitimate shortcuts + +| Shortcut | Replaces | +|----------|----------| +| Deep-link withdraw/pay | Camera QR / opening paywall | +| Host `POST` template → pay URI | Browser shop checkout UI | +| Label-list taps | Human reading button text | +| Skip full ToS scroll | Tap Accept if shown | + +### Scripts + +| Script | Role | +|--------|------| +| `run-android-pay-smoke.sh` | Hybrid: install + deep-link + light taps | +| `run-android-gui-smoke.sh` | Multi-round GUI drive | +| `run-goa-gui-chain.sh` | Multi mint/withdraw/pay (gui-workflows port) | +| `run-android-build-and-smoke.sh` | Build git ref (branch/tag) + smoke (`GUI=1` optional) | +| `run-android-variant-matrix.sh` | **published + stable-self + master** (compare builds) | +| `start-android-emulator.sh` | Start AVD (**headless default**, SwiftShader) | +| `lib_android_env.sh` | Flags, PATH, `android_ensure_device` | +| `lib_ui.py` | Dump/tap/ANR helpers | + +### Build variants (anti–“weird local build”) + +| Variant | Source | Severity | Purpose | +|---------|--------|----------|---------| +| **stable-self** | **only** rebuild of current stable tag | WARN | Self-build vs published for **one** release | +| **published** | F-Droid current APK | **≤14d → BLOCKER**, else WARN | What users install | +| **older / milestones** | F-Droid APK ~**3 / 6 / 9 / 12 mo** | **always WARN** | No self-build of old tags | +| **master** | `origin/master` rebuild | WARN | Tip — never hard-blocks | +| **fix** (optional) | inference branch | soft | GOA automation fixes | + +**Self-build:** current stable only (plus master/fix if requested) — not older releases. +**Blocker window:** releases ≤ 14 days. +**Milestones:** nearest tags to 90/180/270/365d with a still-hosted F-Droid APK. + +--- + +## Stacks under test + +| Stack | Withdraw | Pay | +|-------|----------|-----| +| **GOA** | explorer mint / `demo-withdraw.json` @ bank.hacktivism.ch | goa-shop templates / Paivana pay-template | +| **stage** | explorer / demo-withdraw @ stage.bank… | fermes / jardin public templates | + +### Lageübersicht GOA + stage (as-is survey · 2026-07-17 ~22:05–22:18) + +**Device:** headless `emulator-5554` · host ~3.7 GiB RAM (often <300 MiB free) · timeout per leg ≤420 s. +**Legs:** Abheben + Bezahlen (`DO_WITHDRAW=1 DO_PAY=1`). Artefakte: `out-survey-20260717-220527/` (gitignored). + +#### Varianten + +| Variante | Quelle | Bemerkung | +|----------|--------|-----------| +| **stable (published)** | F-Droid `net.taler.wallet.fdroid` **1.6.1 / 854** | as-is | +| **master as-is** | rebuild `origin/master` @ `f65a976c` → `.fdroid.debug` | as-is, no patches | +| **ältere Releases** (3/6/9/12 mon) | F-Droid APKs für `1.5.0`/`1.3.0`/`1.1.0`/`1.0.8` | **nicht auffindbar** (HTTP miss) → **SKIP** (kein Selbstbau älterer Tags) | +| **master + inference** (extra) | local branch `local/master-with-inference` @ `3ebe8b09` | merges `fix-bank-withdraw-auto-exchange` + `fix-withdraw-spinner-fallback` · **not pushed** | + +#### Ergebnis-Matrix + +| Variant | Stack | Install / deep-link | Withdraw URI | Pay URI | wallet-core Abheben | wallet-core Pay | ANR | Exit | +|---------|-------|---------------------|--------------|---------|---------------------|-----------------|-----|------| +| stable published | **goa** (hacktivism) | OK | OK bank.hacktivism.ch | OK paivana template | **NO** | soft/YES once* | **YES** | **10** | +| stable published | **stage** (*.lefrancpaysan) | OK | OK stage.bank… | OK fermes order | **NO** | **NO** | **YES** | **10** | +| master as-is | **goa** | OK | OK | OK paivana | **NO** | **NO** | **YES** | **10** | +| master as-is | **stage** | OK | OK | OK fermes | **NO** | **NO** | **YES** | **10** | +| master+inference | **goa** | OK | OK | OK | **NO** | **NO** | **YES** | **10** | +| master+inference | **stage** | OK | OK | OK | **NO** | **NO** | **YES** | **10** | + +\*stable-goa console once reported pay wallet-core YES mid-run; final score still ANR-dominated / not reliable settlement. + +#### Was pro Stack **steht** (Infra) + +| Check | **goa / hacktivism** | **stage / \*.lefrancpaysan** | +|-------|----------------------|------------------------------| +| Bank `…/intro/demo-withdraw.json` → `taler://withdraw/…` | **OK** | **OK** | +| Pay entry | paivana `pay-template` **OK** | fermes `panier-legumes` POST → `taler://pay/…` **OK** | +| adb `VIEW` Intent delivery | **OK** | **OK** | +| Unattended Confirm/Abheben/Pay GUI | **FAIL (ANR)** | **FAIL (ANR)** | +| wallet-core prepare/accept withdraw in logcat | **not seen** | **not seen** | + +**Fazit as-is:** Beide Stacks liefern die **Eingänge** (Withdraw- + Pay-URIs). Die **GUI/wallet-core-Strecke** ist auf diesem Emulator für **alle** getesteten App-Varianten (stable, master, inference) **gleich tot** — System UI / Android System ANR, keine brauchbaren Taps. Das ist **kein Stack-Unterschied** GOA vs stage und **kein** belastbarer App-Versions-Vergleich. + +#### Inference-Patch-Versuch (lokal, **kein Push**) + +Weil **nichts** der as-is-Varianten wallet-core-grün war, wurde zusätzlich gebaut: + +```text +taler-android branch: local/master-with-inference # NOT pushed +base: origin/master +merged: origin/dev/hernani-inference/fix-bank-withdraw-auto-exchange + origin/dev/hernani-inference/fix-withdraw-spinner-fallback +commit: 3ebe8b091379 +APK: apks/wallet-master-inference-3ebe8b091379.apk (local) +``` + +| Stack | Inference smoke | Besser als master/stable? | +|-------|-----------------|---------------------------| +| goa | exit 10, ANR, no withdraw core | **Nein** | +| stage | exit 10, ANR, no withdraw core | **Nein** | + +**Kein grünes Inference-Signal** auf diesem Host → wir können **nicht** belegen, dass die Patches Withdraw/Pay „retten“. +Ebenso können wir **nicht** belegen, dass master ohne Patches „schuld“ ist: die Umgebung erreicht wallet-core gar nicht. + +> **Wenn später auf Gerät/mehr RAM die Inference-APK plötzlich grün und master/stable rot ist:** +> das ist der **klare Fix-Hinweis** — Patches aus `dev/hernani-inference/*` upstreamen / in master holen. +> **Heute:** noch **nicht** der Fall; primärer Blocker = **Emulator/RAM/ANR**. + +#### Nächste sinnvolle Schritte (später) + +1. Physisches Gerät oder Host ≥6–8 GiB free RAM +2. Nur **eine** Wallet-Package-ID installiert (kein Open-with) +3. Survey wiederholen: stable · master · `local/master-with-inference` auf **goa + stage** +4. Ältere F-Droid-APKs nur wenn wieder gehostet; kein Selbstbau alter Tags + +--- + +## Upstream branches (`taler-android` `dev/hernani-inference/*`) + +| Branch | Role | +|--------|------| +| `gui-workflows` | macOS emulator helpers + original `goa-chain-emu.sh` | +| `fix/bank-withdraw-auto-exchange` | **Minimal** GOA exchange auto-add (default build branch) | +| `fix/withdraw-spinner-fallback` | Spinner + OIM; use if spinner still hangs | +| `experimental-oim*` | Optional cash UI | + +--- + +## Checklist + +- [x] Minimal variant documented +- [x] Minimalvariante: was geht / was nicht (Beschränkungen) +- [x] Two stacks (goa / stage) +- [x] Dual-stack run documented (2026-07-17: both FAIL ANR, landings OK) +- [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults) +- [x] **Headless emulator default** (`EMULATOR_HEADLESS=1`, SwiftShader) +- [x] Linux default = Android suite only +- [x] Inference-branch policy +- [ ] Wrapper enforces flags in every entry script (incremental) +- [ ] Reliable unattended Confirm on low-RAM Linux emulator diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md new file mode 100644 index 0000000..d91d82c --- /dev/null +++ b/scripts/taler-monitoring/android-test/README.md @@ -0,0 +1,111 @@ +# Android wallet smoke (Abheben + Bezahlen) + +Smokes always cover **withdraw first**, then **pay** (not pay-only). + +| Leg | Default | What | +|-----|---------|------| +| **1 WITHDRAW** | on | `taler://withdraw/…` + Confirm/Abheben taps | +| **2 PAY** | on | `taler://pay…` / pay-template + Pay taps | + +```bash +# needs adb device/emulator +./run-android-pay-smoke.sh +STACK=goa ./run-android-pay-smoke.sh +DO_PAY=0 ./run-android-pay-smoke.sh # withdraw only +DO_WITHDRAW=0 DO_PAY=1 … # pay only (unusual) +``` + +APK: F-Droid **1.6.1** (`net.taler.wallet.fdroid`). + +## 2) Source build + same tests + +Builds `taler-android` at a git **ref** (`:wallet:assembleFdroidDebug`), then +runs the **same** smoke (install + withdraw/pay deep-links). + +```bash +# default: inference fix branch (GOA withdraw reliability) +./run-android-build-and-smoke.sh +# tip of master +BRANCH=master ./run-android-build-and-smoke.sh +# self-build of published stable **tag** (same release as F-Droid 1.6.1) +BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh +SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh +TALER_ANDROID_SRC=/path/to/taler-android ./run-android-build-and-smoke.sh +``` + +Built package id: `net.taler.wallet.fdroid.debug`. +Needs `ANDROID_HOME` (e.g. `$HOME/Android/Sdk`), Java 17+, network for Maven. + +## 2b) Variant matrix (published + stable self-build + master) + +Runs several smokes so a single odd developer build cannot be the only sample: + +| Variant | What | +|---------|------| +| Order | Variant | Severity | What | +|------|---------|----------|------| +| **1** | **stable-self** | WARN | **Only** self-build: current stable tag | +| **2** | **published** | age ≤14d → **BLOCKER**, else WARN | Current F-Droid APK | +| **3** | **older / milestones** | **always WARN** | F-Droid APKs nearest **3 / 6 / 9 / 12 months** (if still hosted) — **no self-build** | +| **4** | **master** | WARN | Rebuild `origin/master` | + +**Self-build policy:** only **current published stable** (`stable-self`) + optional **master**. +Historical versions: published F-Droid APK only; skip if APK gone. +**Milestones:** `MILESTONE_DAYS="90 180 270 365"` (±50d), warn only. +**Young releases** (≤ 14 days): hard blocker on fail. + +```bash +./run-android-variant-matrix.sh +STACK=goa ./run-android-variant-matrix.sh +INCLUDE_OLDER=0 VARIANTS=stable-self,published,master ./run-android-variant-matrix.sh +# custom milestones (days): +MILESTONE_DAYS="90 180 270 365" ./run-android-variant-matrix.sh +``` + +Artifacts + `SUMMARY.txt` under `out-matrix-/`. + +## 3) Graphical UI (vanilla) + +Deep-link entry (≡ QR) **plus** multi-round uiautomator taps: + +```bash +STACK=goa ./run-android-gui-smoke.sh +STACK=stage ./run-android-gui-smoke.sh +# after source build: +GUI=1 ./run-android-build-and-smoke.sh +``` + +## 4) Full GOA/stage chain (from `dev/hernani-inference/gui-workflows`) + +Bank API mint → withdraw deep-link → taps → template pay → taps: + +```bash +# needs explorer password + device +EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ + STACK=goa PKG=net.taler.wallet.fdroid.debug \ + ./run-goa-gui-chain.sh + +STACK=stage ./run-goa-gui-chain.sh +``` + +Canonical notes (minimal variant, **flags**, Linux defaults): +**[GUI-AUTOMATION-NOTES.md](./GUI-AUTOMATION-NOTES.md)**. + +| Flag | Linux default | Meaning | +|------|---------------|---------| +| `AUTO_ANDROID` | `1` | Run Android adb/APK automation | +| `AUTO_GUI` | `1` | Graphical uiautomator drive (else deep-link only) | +| `AUTO_PLATFORM` | `auto` → `linux` | Host family; macOS can later pair with iOS out of tree | +| **`EMULATOR_HEADLESS`** | **`1`** | **No host window** (`-no-window`). Server-safe. | +| `EMULATOR_GPU` | `swiftshader_indirect` | Guest GLES still active for UI/taps/screenshots | +| `AUTO_START_EMULATOR` | `1` | Auto-start headless AVD if no adb device | +| `WINDOWED` | `0` | `WINDOWED=1` → show emulator window | + +```bash +# headless AVD only (default path for servers) +./start-android-emulator.sh --wait +# windowed (laptops) +WINDOWED=1 ./start-android-emulator.sh --wait +``` + +See also parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/lib_android_env.sh b/scripts/taler-monitoring/android-test/lib_android_env.sh new file mode 100644 index 0000000..670d796 --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_android_env.sh @@ -0,0 +1,158 @@ +# shellcheck shell=bash +# Shared Android env for GUI / adb smokes. +# Source: . "$ROOT/lib_android_env.sh" +# +# Defaults favour **servers / CI**: no host window (headless emulator), while the +# guest still runs GLES via SwiftShader so UI layout / uiautomator work. +# +# See GUI-AUTOMATION-NOTES.md → Headless mode + +# --- SDK / PATH --------------------------------------------------------------- +export ANDROID_HOME="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-$HOME/Android/Sdk}}" +export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" +for d in \ + "$ANDROID_HOME/platform-tools" \ + "$ANDROID_HOME/emulator" \ + "$ANDROID_HOME/cmdline-tools/latest/bin" +do + [ -d "$d" ] && case ":$PATH:" in *":$d:"*) ;; *) PATH="$d:$PATH" ;; esac +done +export PATH + +# AVD path (some hosts keep AVDs under ~/.config/.android) +if [ -z "${ANDROID_AVD_HOME:-}" ]; then + if [ -d "$HOME/.android/avd" ]; then + export ANDROID_AVD_HOME="$HOME/.android/avd" + elif [ -d "$HOME/.config/.android/avd" ]; then + export ANDROID_AVD_HOME="$HOME/.config/.android/avd" + # emulator also looks at ~/.android/avd — soft-link if missing + if [ ! -e "$HOME/.android/avd" ]; then + mkdir -p "$HOME/.android" + ln -sfn "$HOME/.config/.android/avd" "$HOME/.android/avd" 2>/dev/null || true + fi + fi +fi + +# --- Platform / feature flags ------------------------------------------------- +os=$(uname -s) +case "${AUTO_PLATFORM:-auto}" in + auto) + case "$os" in + Linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; + esac + ;; + linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; +esac + +# Headless = default (no X11/Wayland window on host). GUI *logic* still runs in +# the guest via software GPU. Set EMULATOR_HEADLESS=0 or WINDOWED=1 for a window. +: "${EMULATOR_HEADLESS:=1}" +if [ "${WINDOWED:-0}" = "1" ]; then + EMULATOR_HEADLESS=0 +fi +export EMULATOR_HEADLESS + +# Soft GLES for headless; windowed may override to host/auto if set by user. +if [ "${EMULATOR_HEADLESS}" = "1" ]; then + : "${EMULATOR_GPU:=swiftshader_indirect}" +else + : "${EMULATOR_GPU:=swiftshader_indirect}" # still safe default; host GPU: EMULATOR_GPU=host +fi +export EMULATOR_GPU + +: "${EMULATOR_AVD:=TalerWallet34}" +: "${EMULATOR_MEMORY:=1536}" +: "${EMULATOR_CORES:=2}" +# If no adb device: start headless AVD (0 = require phone/manual emulator) +: "${AUTO_START_EMULATOR:=1}" +: "${EMULATOR_BOOT_TIMEOUT:=360}" +: "${EMULATOR_LOG:=/tmp/taler-android-emulator.log}" +: "${EMULATOR_NO_SNAPSHOT:=1}" + +android_adb_serial() { + adb devices 2>/dev/null | awk '/\tdevice$/{print $1; exit}' +} + +android_has_device() { + [ -n "$(android_adb_serial)" ] +} + +# Build emulator argv for current headless/windowed mode. +# Prints args; caller exec/nohup. +android_emulator_args() { + local avd="${1:-$EMULATOR_AVD}" + local args=( -avd "$avd" -gpu "$EMULATOR_GPU" -no-audio -no-boot-anim ) + if [ "${EMULATOR_HEADLESS}" = "1" ]; then + args+=( -no-window ) + # Avoid Qt trying to open a display on servers + export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}" + fi + # Broken default_boot snapshots leave adb offline forever — cold boot by default. + if [ "${EMULATOR_NO_SNAPSHOT:-1}" = "1" ]; then + args+=( -no-snapshot -no-snapshot-load -no-snapshot-save ) + fi + if [ -n "${EMULATOR_MEMORY:-}" ]; then + args+=( -memory "$EMULATOR_MEMORY" ) + fi + if [ -n "${EMULATOR_CORES:-}" ]; then + args+=( -cores "$EMULATOR_CORES" ) + fi + # extra user flags + # shellcheck disable=SC2206 + if [ -n "${EMULATOR_EXTRA_ARGS:-}" ]; then + # intentional word-split for extra flags + args+=( ${EMULATOR_EXTRA_ARGS} ) + fi + printf '%s\n' "${args[@]}" +} + +# Ensure an adb "device" is online. Optionally start headless emulator. +# Sets SERIAL / ANDROID_SERIAL on success. Returns 0 ok, 3 no device. +android_ensure_device() { + local serial + serial="${SERIAL:-${ANDROID_SERIAL:-}}" + if [ -z "$serial" ]; then + serial=$(android_adb_serial || true) + fi + if [ -n "$serial" ]; then + SERIAL="$serial" + export ANDROID_SERIAL="$serial" + return 0 + fi + + if [ "${AUTO_START_EMULATOR}" != "1" ]; then + echo "no adb device (AUTO_START_EMULATOR=0 — plug phone or start AVD manually)" >&2 + return 3 + fi + + if ! command -v emulator >/dev/null 2>&1; then + echo "no adb device and emulator binary missing under $ANDROID_HOME" >&2 + return 3 + fi + + echo "no adb device → starting AVD ${EMULATOR_AVD} (headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU})" + local starter + starter="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/start-android-emulator.sh" + if [ ! -x "$starter" ]; then + echo "missing $starter" >&2 + return 3 + fi + EMULATOR_HEADLESS="$EMULATOR_HEADLESS" EMULATOR_GPU="$EMULATOR_GPU" \ + EMULATOR_AVD="$EMULATOR_AVD" EMULATOR_MEMORY="$EMULATOR_MEMORY" \ + EMULATOR_CORES="$EMULATOR_CORES" EMULATOR_LOG="$EMULATOR_LOG" \ + EMULATOR_BOOT_TIMEOUT="$EMULATOR_BOOT_TIMEOUT" \ + "$starter" --wait || return 3 + + serial=$(android_adb_serial || true) + if [ -z "$serial" ]; then + echo "emulator started but no adb device yet — see $EMULATOR_LOG" >&2 + return 3 + fi + SERIAL="$serial" + export ANDROID_SERIAL="$serial" + echo "device: $SERIAL (headless=${EMULATOR_HEADLESS})" + return 0 +} diff --git a/scripts/taler-monitoring/android-test/lib_release_age.sh b/scripts/taler-monitoring/android-test/lib_release_age.sh new file mode 100644 index 0000000..7be935f --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_release_age.sh @@ -0,0 +1,133 @@ +# shellcheck shell=bash +# Release age helpers for Android wallet matrix. +# Source after ROOT/lib_android_env.sh (optional). +# +# BLOCKER only when a *released* app is within BLOCKER_MAX_AGE_DAYS (default 14). +# Older releases: try them, but failures are WARN only. + +: "${BLOCKER_MAX_AGE_DAYS:=14}" +: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}" + +# Days since git tag (tagger/committer date). Empty if unknown. +# Usage: release_age_days wallet-1.6.1 → e.g. 28 +release_age_days() { + local tag="$1" + local src="${2:-$TALER_ANDROID_SRC}" + [ -d "$src/.git" ] || return 1 + local epoch + epoch=$(git -C "$src" log -1 --format=%ct "refs/tags/${tag}" 2>/dev/null || true) + [ -n "$epoch" ] || epoch=$(git -C "$src" log -1 --format=%ct "$tag" 2>/dev/null || true) + [ -n "$epoch" ] || return 1 + local now + now=$(date +%s) + echo $(( (now - epoch) / 86400 )) +} + +# severity from age: blocker if age_days <= BLOCKER_MAX_AGE_DAYS, else warn +# Unknown age → warn (never accidental hard fail on mystery builds) +severity_from_age_days() { + local days="${1:-}" + if [ -z "$days" ] || ! [[ "$days" =~ ^[0-9]+$ ]]; then + echo warn + return + fi + if [ "$days" -le "${BLOCKER_MAX_AGE_DAYS}" ]; then + echo blocker + else + echo warn + fi +} + +severity_for_tag() { + local tag="$1" + local days + days=$(release_age_days "$tag" 2>/dev/null || true) + severity_from_age_days "$days" +} + +# List wallet-* release tags newest first (exclude rc/dev). +# Args: max count (default 5). Pass 0 or "all" for no limit. +list_wallet_release_tags() { + local n="${1:-5}" + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + [ -d "$src/.git" ] || return 1 + git -C "$src" fetch --tags origin 2>/dev/null || true + local cmd=(git -C "$src" for-each-ref --sort=-creatordate --format='%(refname:short)' refs/tags) + if [ "$n" = "all" ] || [ "$n" = "0" ]; then + "${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' + else + "${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' | head -n "$n" + fi +} + +# Pick one tag closest to each age milestone (days). Always intended as WARN samples. +# Default milestones: 3,6,9,12 months ≈ 90,180,270,365 days. +# Env: MILESTONE_DAYS="90 180 270 365" MILESTONE_TOLERANCE_DAYS=50 +# Prints lines: TAG AGE_DAYS TARGET_DAYS (only when a tag within tolerance exists) +select_milestone_tags() { + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + local milestones="${MILESTONE_DAYS:-90 180 270 365}" + local tol="${MILESTONE_TOLERANCE_DAYS:-50}" + local skip="${1:-}" # optional tag to exclude (e.g. current STABLE_TAG) + local -a tags=() + mapfile -t tags < <(list_wallet_release_tags all 2>/dev/null || true) + [ "${#tags[@]}" -gt 0 ] || return 0 + + local target best_tag best_age best_delta age t delta + local -A used=() + for target in $milestones; do + best_tag="" + best_age="" + best_delta=99999 + for t in "${tags[@]}"; do + [ -n "$t" ] || continue + [ "$t" = "$skip" ] && continue + [ -n "${used[$t]:-}" ] && continue + age=$(release_age_days "$t" "$src" 2>/dev/null || true) + [[ "$age" =~ ^[0-9]+$ ]] || continue + delta=$(( age > target ? age - target : target - age )) + if [ "$delta" -le "$tol" ] && [ "$delta" -lt "$best_delta" ]; then + best_delta=$delta + best_tag=$t + best_age=$age + fi + done + if [ -n "$best_tag" ]; then + used[$best_tag]=1 + echo "$best_tag $best_age $target" + fi + done +} + +# Guess F-Droid APK URL for a wallet version name (e.g. 1.6.1). +# Uses versionCode from tag's build.gradle if possible, else STABLE mapping. +# Prints: versionCode apk_url or returns 1 +fdroid_apk_for_version_name() { + local vn="$1" # 1.6.1 + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + local tag="wallet-${vn}" + local vc="" + if git -C "$src" cat-file -e "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null; then + vc=$(git -C "$src" show "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null \ + | sed -n 's/^[[:space:]]*versionCode[[:space:]]\+\([0-9]\+\).*/\1/p' | head -1) + fi + # ABI x86_64 → +4 in override scheme 10*vc+abi (common F-Droid multi-apk) + # Prefer universal-style code used by F-Droid: for 1.6.1 → 854 = 10*85+4 + local apk_vc="" + if [ -n "$vc" ]; then + apk_vc=$((10 * vc + 4)) + fi + # Known overrides / suggested + case "$vn" in + 1.6.1) apk_vc=854 ;; + esac + [ -n "$apk_vc" ] || return 1 + echo "${apk_vc}|https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk" +} + +# Check HTTP HEAD/GET that APK exists (0 = ok) +fdroid_apk_exists() { + local url="$1" + curl -sfS -m 15 -o /dev/null -r 0-0 "$url" 2>/dev/null \ + || curl -sfS -m 15 -o /dev/null -I "$url" 2>/dev/null +} diff --git a/scripts/taler-monitoring/android-test/lib_ui.py b/scripts/taler-monitoring/android-test/lib_ui.py new file mode 100755 index 0000000..a7cdad5 --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_ui.py @@ -0,0 +1,192 @@ +#!/usr/bin/env python3 +"""Minimal adb + uiautomator helpers for Taler wallet GUI smoke (no root). + +Does not replace Maestro/Appium; uses only adb shell + uiautomator dump. +""" +from __future__ import annotations + +import re +import subprocess +import time +from pathlib import Path +from typing import Iterable, List, Optional, Sequence, Tuple + +Bounds = Tuple[int, int, int, int] + + +def adb(serial: str, *args: str, check: bool = False, timeout: Optional[float] = 60) -> subprocess.CompletedProcess: + cmd = ["adb", "-s", serial, *args] + return subprocess.run(cmd, check=check, capture_output=True, text=True, timeout=timeout) + + +def dump_ui(serial: str, dest: Path) -> str: + adb(serial, "shell", "uiautomator", "dump", "/sdcard/ui.xml", timeout=90) + adb(serial, "pull", "/sdcard/ui.xml", str(dest), timeout=30) + return dest.read_text(errors="replace") if dest.is_file() else "" + + +def screencap(serial: str, dest: Path) -> None: + p = subprocess.run( + ["adb", "-s", serial, "exec-out", "screencap", "-p"], + capture_output=True, + timeout=60, + ) + if p.returncode == 0 and p.stdout: + dest.write_bytes(p.stdout) + + +def texts(xml: str) -> List[str]: + return [t for t in re.findall(r'text="([^"]*)"', xml) if t.strip()] + + +def find_bounds(xml: str, label: str) -> List[Bounds]: + """Return list of (x1,y1,x2,y2) for nodes whose text or content-desc matches label (exact).""" + out: List[Bounds] = [] + esc = re.escape(label) + patterns = [ + rf'(?:text|content-desc)="{esc}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*(?:text|content-desc)="{esc}"', + ] + for pat in patterns: + for m in re.finditer(pat, xml, flags=re.I): + out.append(tuple(map(int, m.groups()))) # type: ignore[arg-type] + return out + + +def find_bounds_re(xml: str, pattern: str) -> List[Tuple[str, Bounds]]: + out: List[Tuple[str, Bounds]] = [] + for m in re.finditer( + rf'(?:text|content-desc)="([^"]*{pattern}[^"]*)"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + xml, + flags=re.I, + ): + label = m.group(1) + b = tuple(map(int, m.group(2, 3, 4, 5))) + out.append((label, b)) # type: ignore[arg-type] + return out + + +def center(b: Bounds) -> Tuple[int, int]: + x1, y1, x2, y2 = b + return (x1 + x2) // 2, (y1 + y2) // 2 + + +def tap(serial: str, x: int, y: int) -> None: + adb(serial, "shell", "input", "tap", str(x), str(y)) + + +def tap_bounds(serial: str, b: Bounds) -> None: + x, y = center(b) + tap(serial, x, y) + + +def dismiss_anr(serial: str, xml: str) -> bool: + """If System UI isn't responding, prefer Wait over Close.""" + if "isn't responding" not in xml and "reagiert nicht" not in xml.lower(): + return False + for label in ("Wait", "Warten", "OK"): + bs = find_bounds(xml, label) + if bs: + tap_bounds(serial, bs[0]) + return True + # fallback: right-ish button often Wait + adb(serial, "shell", "input", "keyevent", "22") + adb(serial, "shell", "input", "keyevent", "66") + return True + + +# Labels seen on wallet UI (EN/DE/FR) — expand as needed +CONFIRM_LABELS = [ + "Confirm", + "Confirm withdrawal", + "Withdraw", + "Accept", + "I accept", + "Agree", + "Continue", + "Next", + "OK", + "Pay", + "Pay now", + "Bestätigen", + "Abheben", + "Akzeptieren", + "Zustimmen", + "Weiter", + "Bezahlen", + "Confirmer", + "Retirer", + "Accepter", + "Payer", +] + +TOS_LABELS = [ + "I accept", + "Accept", + "Agree", + "Akzeptieren", + "Zustimmen", + "Accepter", +] + + +def try_tap_any(serial: str, xml: str, labels: Sequence[str]) -> Optional[str]: + for label in labels: + bs = find_bounds(xml, label) + if bs: + tap_bounds(serial, bs[0]) + return label + return None + + +def gui_drive( + serial: str, + out_dir: Path, + *, + rounds: int = 8, + sleep_s: float = 3.0, + phase: str = "gui", +) -> dict: + """Poll UI, dismiss ANR, tap confirm/ToS-like buttons. Returns status dict.""" + out_dir.mkdir(parents=True, exist_ok=True) + seen_texts: List[str] = [] + taps: List[str] = [] + anr = 0 + for i in range(1, rounds + 1): + xml_path = out_dir / f"{phase}-{i:02d}-ui.xml" + png_path = out_dir / f"{phase}-{i:02d}.png" + try: + xml = dump_ui(serial, xml_path) + except Exception as e: + taps.append(f"dump-fail:{e}") + time.sleep(sleep_s) + continue + try: + screencap(serial, png_path) + except Exception: + pass + ts = texts(xml) + seen_texts.extend(ts[:20]) + if dismiss_anr(serial, xml): + anr += 1 + time.sleep(sleep_s) + continue + hit = try_tap_any(serial, xml, CONFIRM_LABELS) + if hit: + taps.append(hit) + time.sleep(sleep_s) + continue + # partial regex for amount confirm rows + for label, b in find_bounds_re(xml, r"Confirm|Withdraw|Pay|Accept|Bestätig|Abheben|Payer"): + tap_bounds(serial, b) + taps.append(f"re:{label}") + time.sleep(sleep_s) + break + else: + time.sleep(sleep_s) + return { + "taps": taps, + "anr_dismissals": anr, + "sample_texts": seen_texts[-40:], + "rounds": rounds, + } diff --git a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh new file mode 100755 index 0000000..659b446 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +# Build GNU Taler Android wallet from source at a git **ref**, then run the same +# adb deep-link (or GUI) smoke as run-android-pay-smoke.sh. +# +# Source (default): $HOME/taler/taler-android +# Build: ./gradlew :wallet:assembleFdroidDebug +# Package id: net.taler.wallet.fdroid.debug +# +# Typical refs: +# BRANCH=master # tip of master +# BRANCH=wallet-1.6.1 # published stable **tag** (self-build) +# BRANCH=dev/hernani-inference/fix-… # GOA automation fix branch +# +# For a multi-variant matrix (master + stable self-build + published APK): +# ./run-android-variant-matrix.sh +# +# Usage: +# ./run-android-build-and-smoke.sh +# BRANCH=master STACK=goa ./run-android-build-and-smoke.sh +# BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh +# SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +SMOKE="$ROOT/run-android-pay-smoke.sh" +SRC="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" +SKIP_PULL="${SKIP_PULL:-0}" +# Prefer existing inference fix branch for GOA withdraw (spinner/exchange resolve). +# Matrix uses BRANCH=master and BRANCH=$STABLE_TAG explicitly. +BRANCH="${BRANCH:-dev/hernani-inference/fix-bank-withdraw-auto-exchange}" +# Optional human label for logs / artifact names (default = sanitized BRANCH) +LABEL="${LABEL:-}" +VARIANT="${VARIANT:-fdroidDebug}" +export ANDROID_HOME="${ANDROID_HOME:-$HOME/Android/Sdk}" +export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" + +if [ ! -x "$SRC/gradlew" ]; then + echo "taler-android not found at $SRC (set TALER_ANDROID_SRC=…)" >&2 + exit 2 +fi +if [ ! -d "$ANDROID_HOME" ]; then + echo "ANDROID_HOME missing: $ANDROID_HOME" >&2 + exit 2 +fi + +printf 'sdk.dir=%s\n' "$ANDROID_HOME" >"$SRC/local.properties" + +# --- checkout REF (branch, origin/branch, or tag) -------------------------------- +cd "$SRC" +if [ -d .git ]; then + git fetch origin --tags 2>/dev/null || git fetch origin 2>/dev/null || true + REF="$BRANCH" + checked=0 + if [ -n "$REF" ]; then + if git rev-parse --verify "refs/tags/${REF}" >/dev/null 2>&1; then + echo "checkout tag ${REF} (stable / release self-build)" + git checkout -f "refs/tags/${REF}" + checked=1 + SKIP_PULL=1 + elif git rev-parse --verify "origin/${REF}" >/dev/null 2>&1; then + echo "checkout origin/${REF}" + git checkout -B "$REF" "origin/${REF}" 2>/dev/null || git checkout "$REF" 2>/dev/null || true + checked=1 + elif git rev-parse --verify "${REF}" >/dev/null 2>&1; then + echo "checkout ${REF}" + git checkout -f "$REF" + checked=1 + else + echo "WARN: ref '${REF}' not found locally after fetch — staying on current HEAD" >&2 + fi + fi + if [ "$SKIP_PULL" != "1" ] && [ "$checked" = "1" ]; then + # only pull real branches, never tags + if git symbolic-ref -q HEAD >/dev/null 2>&1; then + br=$(git rev-parse --abbrev-ref HEAD) + git pull --ff-only origin "$br" 2>/dev/null || true + fi + fi +fi + +COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown) +HEAD_DESC=$(git describe --tags --always 2>/dev/null || echo "$COMMIT") +BR_NOW=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo detached) +if [ -z "$LABEL" ]; then + LABEL=$(printf '%s' "$BRANCH" | tr '/:' '--' | tr -c 'A-Za-z0-9._-' '-' | cut -c1-48) +fi +echo "source: $SRC @ $COMMIT ($HEAD_DESC) ref=$BRANCH label=$LABEL head=$BR_NOW" +echo "variant: $VARIANT" + +# Map VARIANT → gradle task +case "$VARIANT" in + fdroidDebug|FdroidDebug) task=assembleFdroidDebug ;; + fdroidRelease|FdroidRelease) task=assembleFdroidRelease ;; + nightlyDebug|NightlyDebug) task=assembleNightlyDebug ;; + googleDebug|GoogleDebug) task=assembleGoogleDebug ;; + *) + task="assemble$(printf '%s' "$VARIANT" | python3 -c 'import sys; s=sys.stdin.read().strip(); print(s[:1].upper()+s[1:] if s else "FdroidDebug")')" + ;; +esac + +# Low-RAM hosts: cap JVM +export GRADLE_OPTS="${GRADLE_OPTS:--Xmx1280m -Dorg.gradle.daemon=false -Dorg.gradle.workers.max=1}" +echo "gradle :wallet:$task …" +./gradlew ":wallet:$task" --no-daemon --max-workers="${GRADLE_MAX_WORKERS:-1}" \ + -Dorg.gradle.jvmargs="${GRADLE_JVMARGS:--Xmx1280m}" + +# Prefer fdroid debug APK if multiple exist +APK=$(find "$SRC/wallet/build/outputs/apk" -path '*fdroid*debug*.apk' -type f 2>/dev/null | sort | tail -1) +if [ -z "$APK" ] || [ ! -f "$APK" ]; then + APK=$(find "$SRC/wallet/build/outputs/apk" -name '*.apk' -type f 2>/dev/null | sort | tail -1) +fi +if [ -z "$APK" ] || [ ! -f "$APK" ]; then + echo "no APK under wallet/build/outputs/apk" >&2 + exit 4 +fi +echo "built apk: $APK ($(wc -c <"$APK") bytes)" + +PKG="${PKG:-}" +if [ -z "$PKG" ] && command -v aapt >/dev/null 2>&1; then + PKG=$(aapt dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1) +fi +if [ -z "$PKG" ] && command -v aapt2 >/dev/null 2>&1; then + PKG=$(aapt2 dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1) +fi +: "${PKG:=net.taler.wallet.fdroid.debug}" + +mkdir -p "$ROOT/apks" "$ROOT/out" +STAMP_APK="$ROOT/apks/wallet-${LABEL}-${COMMIT}.apk" +cp -f "$APK" "$STAMP_APK" +echo "copied → $STAMP_APK" +{ + echo "label=$LABEL" + echo "ref=$BRANCH" + echo "commit=$COMMIT" + echo "describe=$HEAD_DESC" + echo "apk=$STAMP_APK" + echo "pkg=$PKG" + echo "task=$task" +} | tee "$ROOT/out/build-${LABEL}.txt" +echo "$COMMIT" >"$ROOT/out/build-commit.txt" +echo "$APK" >"$ROOT/out/build-apk-path.txt" + +export APK_PATH="$STAMP_APK" +export APK_NAME +APK_NAME="$(basename "$STAMP_APK")" +export PKG +export OUT_DIR="${OUT_DIR:-$ROOT/out-source-$LABEL}" +mkdir -p "$OUT_DIR" +cp -f "$ROOT/out/build-${LABEL}.txt" "$OUT_DIR/build-meta.txt" 2>/dev/null || true + +SMOKE_CMD="$SMOKE" +if [ "${GUI:-0}" = "1" ]; then + SMOKE_CMD="$ROOT/run-android-gui-smoke.sh" + case "$OUT_DIR" in + *-gui) ;; + *) OUT_DIR="${OUT_DIR}-gui"; export OUT_DIR; mkdir -p "$OUT_DIR" ;; + esac +fi +echo "=== smoke label=$LABEL PKG=$PKG GUI=${GUI:-0} OUT_DIR=$OUT_DIR ===" +# Do not exec: allow matrix wrapper to run multiple variants +set +e +env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" APK_PATH="$APK_PATH" PKG="$PKG" OUT_DIR="$OUT_DIR" \ + "$SMOKE_CMD" +rc=$? +set -e +echo "=== smoke label=$LABEL exit=$rc ===" +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh new file mode 100755 index 0000000..a22f1b8 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh @@ -0,0 +1,201 @@ +#!/usr/bin/env bash +# GUI-oriented Android smoke: **withdraw (Abheben)** then **pay**, each with +# deep-link entry + multi-round uiautomator taps. Not pay-only. +# +# 1) taler://withdraw/… + Confirm/Abheben/ToS taps +# 2) taler://pay… + Pay/Confirm taps +# +# See GUI-AUTOMATION-NOTES.md +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" + +[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; } +if [ "${AUTO_GUI}" != "1" ]; then + echo "AUTO_GUI=0 → deep-link smoke only" + exec env AUTO_ANDROID=1 "$ROOT/run-android-pay-smoke.sh" +fi + +export OUT_DIR="${OUT_DIR:-$ROOT/out-gui}" +export GUI_ROUNDS="${GUI_ROUNDS:-10}" +export GUI_SLEEP="${GUI_SLEEP:-3}" + +# Install + deliver intents via existing smoke, but skip its weak single-tap +# by setting GUI_MODE=1 which we implement here more fully. +# Simpler: do install/URI here by sourcing patterns from pay-smoke via env + python. + +# Delegate install + URI start to pay-smoke with GUI_AFTER=1 if we patch it; +# instead run a self-contained flow calling pay-smoke pieces. + +APK_DIR="${APK_DIR:-$ROOT/apks}" +APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" +APK_PATH="${APK_PATH:-}" +PKG="${PKG:-net.taler.wallet.fdroid}" +STACK="${STACK:-auto}" +SERIAL="${SERIAL:-}" +mkdir -p "$OUT_DIR" + +command -v adb >/dev/null || { echo "adb missing" >&2; exit 2; } + +# Headless default: no host window; guest GLES via SwiftShader (uiautomator). +android_ensure_device || { echo "no adb device (try ./start-android-emulator.sh --wait)" >&2; exit 3; } +ADB=(adb -s "$SERIAL") +echo "device: $SERIAL mode: GUI (uiautomator) headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU}" + +case "$STACK" in + auto) + if curl -sfS -m 5 -o /dev/null https://bank.hacktivism.ch/config; then STACK=goa; else STACK=stage; fi + ;; +esac +case "$STACK" in + goa) + BANK=https://bank.hacktivism.ch + MERCHANT=https://taler.hacktivism.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + ;; + stage) + BANK=https://stage.bank.lefrancpaysan.ch + MERCHANT=https://stage.monnaie.lefrancpaysan.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + ;; + *) echo "unknown STACK" >&2; exit 2 ;; +esac +echo "stack: $STACK" + +if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then + : +elif [ -f "$APK_DIR/$APK_NAME" ]; then + APK_PATH="$APK_DIR/$APK_NAME" +else + APK_PATH="$APK_DIR/$APK_NAME" + mkdir -p "$APK_DIR" + curl -fL --retry 3 -o "$APK_PATH" "$APK_URL" +fi +echo "apk: $APK_PATH pkg: $PKG" + +"${ADB[@]}" wait-for-device +"${ADB[@]}" install -r "$APK_PATH" +"${ADB[@]}" shell settings put global window_animation_scale 0 || true +"${ADB[@]}" shell settings put global transition_animation_scale 0 || true +"${ADB[@]}" shell settings put global animator_duration_scale 0 || true + +: "${DO_WITHDRAW:=1}" +: "${DO_PAY:=1}" +: "${SMOKE_STRICT:=0}" + +WURI="" +PAY_URI="" +WITHDRAW_OK=0 +PAY_OK=0 + +"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true +"${ADB[@]}" logcat -c 2>/dev/null || true + +if [ "$DO_WITHDRAW" = "1" ]; then + echo + echo "======== 1 WITHDRAW (Abheben) ========" + WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') + [ -n "$WURI" ] || { echo "no withdraw uri" >&2; exit 4; } + echo "withdraw: $WURI" + echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" | tee "$OUT_DIR/start-withdraw.txt" + sleep 5 + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd") +(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2)) +print("gui-withdraw taps:", st.get("taps")) +print("gui-withdraw anr:", st.get("anr_dismissals")) +PY +else + echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========" +fi + +if [ "$DO_PAY" = "1" ]; then + echo + echo "======== 2 PAY (Bezahlen) ========" + if [ "$STACK" = "stage" ]; then + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c 'import json,sys;d=json.load(sys.stdin);oid=d.get("order_id")or"";tok=d.get("token")or""; +print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok) if oid and tok else "")') + fi + else + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" + fi + if [ -n "$PAY_URI" ]; then + echo "pay: $PAY_URI" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" | tee "$OUT_DIR/start-pay.txt" + sleep 5 + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay") +(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2)) +print("gui-pay taps:", st.get("taps")) +print("gui-pay anr:", st.get("anr_dismissals")) +PY + else + echo "pay: (no URI)" + fi +else + echo "======== 2 PAY skipped (DO_PAY=0) ========" +fi + +"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|Error|success|withdraw|pay' \ + "$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true + +if [ "$DO_WITHDRAW" = "1" ] && grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + WITHDRAW_OK=1 +fi +if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ] && \ + grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + PAY_OK=1 +fi + +rc=0 +echo +echo "=== GUI smoke summary ($STACK) — withdraw + pay ===" +echo "package: $PKG DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY STRICT=$SMOKE_STRICT" +if [ "$DO_WITHDRAW" = "1" ]; then + echo "withdraw URI: delivered → $OUT_DIR/gui-withdraw-status.json" + echo "withdraw wallet-core: $([ "$WITHDRAW_OK" = 1 ] && echo YES || echo NO)" + if [ "$SMOKE_STRICT" = "1" ] && [ "$WITHDRAW_OK" != "1" ]; then + echo "FAIL: strict withdraw missing"; rc=11 + fi +fi +if [ "$DO_PAY" = "1" ]; then + echo "pay URI: ${PAY_URI:-none} → $OUT_DIR/gui-pay-status.json" + echo "pay wallet-core: $([ "$PAY_OK" = 1 ] && echo YES || echo NO)" + if [ "$SMOKE_STRICT" = "1" ] && [ "$PAY_OK" != "1" ]; then + echo "FAIL: strict pay missing"; [ "$rc" -eq 0 ] && rc=13 + fi +fi +if grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \ + || grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" "$OUT_DIR/gui-pay" 2>/dev/null; then + echo "FAIL: ANR"; rc=10 +fi +echo "artifacts: $OUT_DIR/" +ls -la "$OUT_DIR" | sed 's/^/ /' +echo "Documented shortcuts → GUI-AUTOMATION-NOTES.md" +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh new file mode 100755 index 0000000..48ec49f --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -0,0 +1,308 @@ +#!/usr/bin/env bash +# Android wallet smoke: install APK, then **withdraw (Abheben)** and **pay** via +# deep-link + best-effort UI taps. Both legs are first-class (not pay-only). +# +# Flow (default both on): +# 1) WITHDRAW taler://withdraw/… from bank demo-withdraw.json + confirm taps +# 2) PAY taler://pay… / pay-template + optional taps +# +# Env: +# DO_WITHDRAW=1 (default) DO_PAY=1 (default) +# SMOKE_STRICT=1 → require wallet-core withdraw (+ pay if DO_PAY) + no ANR +# WITHDRAW_GUI_ROUNDS multi-round Abheben taps (default 6) +# +# Usage: +# ./android-test/run-android-pay-smoke.sh +# STACK=goa ./android-test/run-android-pay-smoke.sh +# DO_PAY=0 ./android-test/run-android-pay-smoke.sh # withdraw-only +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" + +if [ "${AUTO_ANDROID}" != "1" ]; then + echo "skipped: AUTO_ANDROID=${AUTO_ANDROID} (platform flags — see GUI-AUTOMATION-NOTES.md)" + exit 0 +fi + +APK_DIR="${APK_DIR:-$ROOT/apks}" +APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" +# Optional override: full path to an already-built APK (from-source) +APK_PATH="${APK_PATH:-}" +PKG="${PKG:-net.taler.wallet.fdroid}" +STACK="${STACK:-auto}" # auto | goa | stage +SERIAL="${SERIAL:-}" +OUT_DIR="${OUT_DIR:-$ROOT/out}" +mkdir -p "$APK_DIR" "$OUT_DIR" + +if ! command -v adb >/dev/null 2>&1; then + echo "adb missing — install android-tools-adb or user SDK platform-tools" >&2 + exit 2 +fi + +# Default: headless AVD (no host window). WINDOWED=1 / EMULATOR_HEADLESS=0 for UI. +if ! android_ensure_device; then + cat >&2 </dev/null; then + STACK=goa + else + STACK=stage + fi + ;; +esac +case "$STACK" in + goa) + BANK=https://bank.hacktivism.ch + MERCHANT=https://taler.hacktivism.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + PAY_HINT="paivana template / goa-shop" + ;; + stage) + BANK=https://stage.bank.lefrancpaysan.ch + MERCHANT=https://stage.monnaie.lefrancpaysan.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + PAY_HINT="farmer shop templates" + ;; + *) echo "unknown STACK=$STACK" >&2; exit 2 ;; +esac +echo "stack: $STACK bank: $BANK" + +# APK: explicit path, or download published F-Droid into APK_DIR +if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then + : +elif [ -f "$APK_DIR/$APK_NAME" ]; then + APK_PATH="$APK_DIR/$APK_NAME" +else + APK_PATH="$APK_DIR/$APK_NAME" + echo "downloading published wallet APK…" + curl -fL --retry 3 -o "$APK_PATH" "$APK_URL" +fi +echo "apk: $APK_PATH ($(wc -c <"$APK_PATH") bytes)" +echo "pkg: $PKG" + +: "${DO_WITHDRAW:=1}" +: "${DO_PAY:=1}" +: "${WITHDRAW_GUI_ROUNDS:=6}" +: "${PAY_GUI_ROUNDS:=4}" +: "${SMOKE_STRICT:=0}" + +"${ADB[@]}" wait-for-device +"${ADB[@]}" install -r "$APK_PATH" +"${ADB[@]}" shell settings put global window_animation_scale 0 || true +"${ADB[@]}" shell settings put global transition_animation_scale 0 || true +"${ADB[@]}" shell settings put global animator_duration_scale 0 || true + +"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true +"${ADB[@]}" logcat -c 2>/dev/null || true + +WURI="" +PAY_URI="" +WITHDRAW_OK=0 +PAY_OK=0 + +# ═══════════════════════════════════════════════════════════════ +# 1) WITHDRAW / Abheben (required by default — not pay-only) +# ═══════════════════════════════════════════════════════════════ +if [ "$DO_WITHDRAW" = "1" ]; then + echo + echo "======== 1 WITHDRAW (Abheben) ========" + WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') + if [ -z "$WURI" ]; then + echo "FAIL: no taler_withdraw_uri from $WITHDRAW_JSON" >&2 + exit 4 + fi + echo "withdraw URI: $WURI" + echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" 2>&1 | tee "$OUT_DIR/start-withdraw.txt" + sleep 6 + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/01-after-withdraw-uri.png" || true + "${ADB[@]}" shell uiautomator dump /sdcard/ui.xml 2>/dev/null || true + "${ADB[@]}" pull /sdcard/ui.xml "$OUT_DIR/01-ui.xml" 2>/dev/null || true + + # Multi-round confirm / Abheben / ToS (same idea as GUI smoke) + if [ -f "$ROOT/lib_ui.py" ]; then + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$WITHDRAW_GUI_ROUNDS" 2 <<'PY' || true +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd") +(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2)) +print("withdraw GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals")) +PY + else + python3 - "$SERIAL" "$OUT_DIR/01-ui.xml" <<'PY' || true +import re, subprocess, sys +serial, path = sys.argv[1], sys.argv[2] +try: + xml = open(path, errors="replace").read() +except FileNotFoundError: + raise SystemExit(0) +labels = [ + "Confirm", "Confirm withdrawal", "Withdraw", "Accept", "Continue", "OK", + "I accept", "Agree", "Bestätigen", "Abheben", "Akzeptieren", "Retirer", +] +for label in labels: + for pat in ( + r'text="%s"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"' % re.escape(label), + r'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*text="%s"' % re.escape(label), + ): + for m in re.finditer(pat, xml, flags=re.I): + x1, y1, x2, y2 = map(int, m.groups()) + x, y = (x1 + x2) // 2, (y1 + y2) // 2 + print(f"tap {label} @ {x},{y}") + subprocess.run(["adb", "-s", serial, "shell", "input", "tap", str(x), str(y)], check=False) +PY + sleep 8 + fi + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/02-after-withdraw-taps.png" || true + # Mid-flow logcat: withdraw must register before pay + "${ADB[@]}" logcat -d -t 400 >"$OUT_DIR/logcat-after-withdraw.txt" || true + if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat-after-withdraw.txt" 2>/dev/null; then + WITHDRAW_OK=1 + echo "withdraw: wallet-core activity YES" + else + echo "withdraw: wallet-core activity NOT seen yet (onboarding/ANR/slow?)" + fi +else + echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========" +fi + +# ═══════════════════════════════════════════════════════════════ +# 2) PAY (after withdraw; still first-class, but second leg) +# ═══════════════════════════════════════════════════════════════ +if [ "$DO_PAY" = "1" ]; then + echo + echo "======== 2 PAY (Bezahlen) ========" + if [ "$STACK" = "stage" ]; then + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c ' +import json,sys +d=json.load(sys.stdin) +oid=d.get("order_id") or "" +tok=d.get("token") or "" +if oid and tok: + print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok)) +' 2>/dev/null || true) + fi + elif [ "$STACK" = "goa" ]; then + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" + fi + + if [ -n "$PAY_URI" ]; then + echo "pay URI: $PAY_URI ($PAY_HINT)" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" 2>&1 | tee "$OUT_DIR/start-pay.txt" + sleep 6 + if [ -f "$ROOT/lib_ui.py" ]; then + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$PAY_GUI_ROUNDS" 2 <<'PY' || true +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay") +(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2)) +print("pay GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals")) +PY + fi + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/03-after-pay-uri.png" || true + else + echo "pay: (no public template URI for stack=$STACK)" + fi +else + echo "======== 2 PAY skipped (DO_PAY=0) ========" +fi + +# Final evidence +"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|error|Error|success|withdraw|pay' \ + "$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true + +# Re-evaluate withdraw/pay from full logcat +if [ "$DO_WITHDRAW" = "1" ]; then + if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + WITHDRAW_OK=1 + fi +fi +if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ]; then + if grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + PAY_OK=1 + fi +fi + +rc=0 +echo +echo "=== summary (withdraw + pay) ===" +echo "package: $PKG stack: $STACK" +echo "DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY SMOKE_STRICT=$SMOKE_STRICT" +if [ "$DO_WITHDRAW" = "1" ]; then + echo "withdraw URI: ${WURI:-(none)}" + if [ "$WITHDRAW_OK" = "1" ]; then + echo "withdraw wallet-core: YES" + else + echo "withdraw wallet-core: NO" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires withdraw (prepareBank/accept/confirm) in logcat" + rc=11 + fi + fi +fi +if [ "$DO_PAY" = "1" ]; then + echo "pay URI: ${PAY_URI:-(none)}" + if [ -z "$PAY_URI" ]; then + echo "pay: no URI (template/API)" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires a pay URI when DO_PAY=1" + rc=12 + fi + elif [ "$PAY_OK" = "1" ]; then + echo "pay wallet-core: YES" + else + echo "pay wallet-core: NO (intent may still have been delivered)" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires pay path activity in logcat" + [ "$rc" -eq 0 ] && rc=13 + fi + fi +fi +if grep -qiE "isn't responding|isn.t responding|System UI isn" "$OUT_DIR/01-ui.xml" 2>/dev/null \ + || grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \ + || grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" 2>/dev/null; then + echo "FAIL: System UI / app ANR" + rc=10 +fi +echo "artifacts: $OUT_DIR/" +ls -la "$OUT_DIR" | sed 's/^/ /' +echo +echo "NOTE: Full unattended Abheben+Bezahlen needs responsive device (≥6–8 GiB host RAM)." +echo "Smoke legs: withdraw=$( [ "$DO_WITHDRAW" = 1 ] && echo on || echo off) pay=$( [ "$DO_PAY" = 1 ] && echo on || echo off) rc=$rc" +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh new file mode 100755 index 0000000..54fe5dc --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -0,0 +1,322 @@ +#!/usr/bin/env bash +# Run Android wallet smoke against several build / published variants. +# +# Default order: +# 1) stable-self — **only** self-build of current published stable (WARN) +# 2) published — current F-Droid APK +# 3) older — F-Droid APKs ~3/6/9/12 mo if still hosted (WARN, no self-build) +# 4) master — tip self-build (WARN) +# +# Self-build policy: current stable + master/fix only — never rebuild old releases. +# +# Severity: age ≤14d → BLOCKER; milestones/older/self-builds → WARN +# +# Env: +# STABLE_TAG, STABLE_APK_URL, STABLE_APK_NAME +# MASTER_BRANCH, FIX_BRANCH +# VARIANTS default: stable-self,published,older,master +# INCLUDE_OLDER=1 expand token "older" / "milestones" (default 1) +# MILESTONE_DAYS default "90 180 270 365" (≈ 3,6,9,12 months) +# MILESTONE_TOLERANCE_DAYS default 50 — max distance to accept a tag +# BLOCKER_MAX_AGE_DAYS default 14 +# PUBLISHED_FORCE_BLOCKER=1 always treat current published as blocker +# FAIL_FAST=1, GUI=0, STACK=… +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" +# shellcheck source=lib_release_age.sh +. "$ROOT/lib_release_age.sh" + +: "${STABLE_TAG:=wallet-1.6.1}" +: "${STABLE_APK_URL:=https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +: "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}" +: "${MASTER_BRANCH:=master}" +: "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" +: "${VARIANTS:=stable-self,published,older,master}" +: "${INCLUDE_OLDER:=1}" +: "${MILESTONE_DAYS:=90 180 270 365}" +: "${MILESTONE_TOLERANCE_DAYS:=50}" +: "${BLOCKER_MAX_AGE_DAYS:=14}" +: "${PUBLISHED_FORCE_BLOCKER:=0}" +: "${FAIL_FAST:=1}" +: "${GUI:=0}" +: "${STACK:=auto}" +: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}" +export TALER_ANDROID_SRC BLOCKER_MAX_AGE_DAYS MILESTONE_DAYS MILESTONE_TOLERANCE_DAYS + +TS=$(date +%Y%m%d-%H%M%S) +MATRIX_OUT="${MATRIX_OUT:-$ROOT/out-matrix-$TS}" +mkdir -p "$MATRIX_OUT" +SUMMARY="$MATRIX_OUT/SUMMARY.txt" +: >"$SUMMARY" + +echo "=== Android variant matrix ===" | tee -a "$SUMMARY" +echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" +echo "STABLE_TAG=$STABLE_TAG BLOCKER_MAX_AGE_DAYS=$BLOCKER_MAX_AGE_DAYS" | tee -a "$SUMMARY" +echo "milestones(days)=$MILESTONE_DAYS tolerance=$MILESTONE_TOLERANCE_DAYS (always WARN)" | tee -a "$SUMMARY" +echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" + +android_ensure_device || { + echo "no adb device — start ./start-android-emulator.sh --wait" >&2 + exit 3 +} +export SERIAL ANDROID_SERIAL +echo "device: $SERIAL" | tee -a "$SUMMARY" + +# --- expand VARIANTS (token "older" → concrete older-pub / older-self entries) --- +# Append one older release as WARN-only — **F-Droid APK only** (no self-build). +# Returns 0 if queued, 1 if skipped (no published APK). +append_older_warn() { + local t="$1" age="$2" target="${3:-}" + local vn meta apk_vc apk_url + vn="${t#wallet-}" + if [ -n "$target" ]; then + echo " milestone ~${target}d → $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY" + else + echo " older $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY" + fi + meta=$(fdroid_apk_for_version_name "$vn" 2>/dev/null || true) + if [ -n "$meta" ]; then + apk_vc="${meta%%|*}" + apk_url="${meta#*|}" + if fdroid_apk_exists "$apk_url"; then + out+=("older-pub:${vn}:${apk_vc}:warn") + return 0 + fi + echo " SKIP $t — no F-Droid APK at ${apk_url:-?} (self-build only for current stable)" | tee -a "$SUMMARY" + else + echo " SKIP $t — cannot map F-Droid versionCode (self-build only for current stable)" | tee -a "$SUMMARY" + fi + return 1 +} + +expand_variants() { + local raw="$1" + local -a out=() + local v tag vn age sev line t target + + IFS=',' read -r -a toks <<<"$raw" + for v in "${toks[@]}"; do + v=$(echo "$v" | tr -d '[:space:]') + [ -n "$v" ] || continue + if [ "$v" = "older" ] || [ "$v" = "older-releases" ] || [ "$v" = "milestones" ]; then + [ "$INCLUDE_OLDER" = "1" ] || continue + # 3/6/9/12-month samples: published APK only, always WARN. + local found=0 queued=0 + while read -r t age target; do + [ -n "$t" ] || continue + [ "$t" = "$STABLE_TAG" ] && continue + found=1 + if append_older_warn "$t" "$age" "$target"; then + queued=$((queued + 1)) + fi + done < <(select_milestone_tags "$STABLE_TAG" 2>/dev/null || true) + if [ "$found" = "0" ]; then + echo " (no milestone tags within ±${MILESTONE_TOLERANCE_DAYS}d of $MILESTONE_DAYS)" | tee -a "$SUMMARY" + elif [ "$queued" = "0" ]; then + echo " (milestone tags found but none still on F-Droid — no older APK smokes)" | tee -a "$SUMMARY" + fi + else + out+=("$v") + fi + done + printf '%s\n' "${out[@]}" +} + +mapfile -t VLIST < <(expand_variants "$VARIANTS") +echo "expanded variants: ${VLIST[*]}" | tee -a "$SUMMARY" + +declare -a RESULTS=() +declare -a WARNINGS=() +overall=0 + +severity_for_published_current() { + if [ "${PUBLISHED_FORCE_BLOCKER}" = "1" ]; then + echo blocker + return + fi + local days + days=$(release_age_days "$STABLE_TAG" 2>/dev/null || true) + severity_from_age_days "$days" +} + +run_published_apk() { + local label="$1" + local apk_url="$2" + local apk_name="$3" + local sev="$4" + local out="$MATRIX_OUT/$label" + mkdir -p "$out" + echo "" | tee -a "$SUMMARY" + echo "-------- variant: $label (F-Droid APK) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY" + echo " url=$apk_url" | tee -a "$SUMMARY" + local smoke="$ROOT/run-android-pay-smoke.sh" + [ "$GUI" = "1" ] && smoke="$ROOT/run-android-gui-smoke.sh" + local strict=1 + [ "$sev" = "warn" ] && strict="${SMOKE_STRICT_WARN:-0}" + [ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}" + set +e + env STACK="$STACK" \ + APK_URL="$apk_url" \ + APK_NAME="$apk_name" \ + APK_PATH="" \ + PKG=net.taler.wallet.fdroid \ + OUT_DIR="$out" \ + SERIAL="$SERIAL" \ + AUTO_START_EMULATOR=0 \ + SMOKE_STRICT="$strict" \ + "$smoke" + local rc=$? + set -e + echo "$label exit=$rc severity=$sev" | tee -a "$SUMMARY" + echo "$label $rc $sev $apk_url" >>"$MATRIX_OUT/results.tsv" + return "$rc" +} + +run_selfbuild() { + local label="$1" + local ref="$2" + local sev="${3:-warn}" + local out="$MATRIX_OUT/$label" + mkdir -p "$out" + echo "" | tee -a "$SUMMARY" + echo "-------- variant: $label (self-build ref=$ref) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY" + local strict="${SMOKE_STRICT:-1}" + [ "$sev" = "warn" ] && strict="${SMOKE_STRICT_SELF:-0}" + [ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}" + set +e + env STACK="$STACK" \ + BRANCH="$ref" \ + LABEL="$label" \ + GUI="$GUI" \ + OUT_DIR="$out" \ + SERIAL="$SERIAL" \ + ANDROID_SERIAL="$SERIAL" \ + AUTO_START_EMULATOR=0 \ + PKG=net.taler.wallet.fdroid.debug \ + SMOKE_STRICT="$strict" \ + TALER_ANDROID_SRC="$TALER_ANDROID_SRC" \ + "$ROOT/run-android-build-and-smoke.sh" + local rc=$? + set -e + echo "$label (ref=$ref) exit=$rc severity=$sev" | tee -a "$SUMMARY" + echo "$label $rc $ref $sev" >>"$MATRIX_OUT/results.tsv" + if [ -f "$out/build-meta.txt" ]; then + echo "build-meta:" | tee -a "$SUMMARY" + cat "$out/build-meta.txt" | tee -a "$SUMMARY" + fi + return "$rc" +} + +record_result() { + local v="$1" rc="$2" sev="$3" + if [ "$rc" -eq 0 ]; then + RESULTS+=("$v:OK($sev)") + return + fi + case "$sev" in + warn) + echo "WARN: $v failed rc=$rc — not a blocker; continuing" | tee -a "$SUMMARY" + WARNINGS+=("$v:$rc") + RESULTS+=("$v:WARN($rc)") + ;; + blocker) + echo "BLOCKER: $v failed rc=$rc (release within ${BLOCKER_MAX_AGE_DAYS}d window)" | tee -a "$SUMMARY" + RESULTS+=("$v:BLOCKER($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after blocker $v" | tee -a "$SUMMARY" + return 2 + fi + ;; + soft) + echo "FAIL: $v rc=$rc" | tee -a "$SUMMARY" + RESULTS+=("$v:FAIL($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after $v" | tee -a "$SUMMARY" + return 2 + fi + ;; + esac + return 0 +} + +for raw in "${VLIST[@]}"; do + v=$(echo "$raw" | tr -d '[:space:]') + [ -n "$v" ] || continue + rc=0 + sev=warn + stop=0 + + case "$v" in + published|fdroid|stable-published) + sev=$(severity_for_published_current) + age=$(release_age_days "$STABLE_TAG" 2>/dev/null || echo "?") + echo "published current: tag=$STABLE_TAG age_days=$age → severity=$sev" | tee -a "$SUMMARY" + run_published_apk "published" "$STABLE_APK_URL" "$STABLE_APK_NAME" "$sev" || rc=$? + ;; + stable-self|stable|self-stable|release-self) + # rebuild of stable: always warn (toolchain), age does not make self-build a blocker + sev=warn + run_selfbuild "stable-self" "$STABLE_TAG" warn || rc=$? + ;; + master|main) + sev=warn + run_selfbuild "master" "$MASTER_BRANCH" warn || rc=$? + ;; + fix|fix-branch|inference) + sev=soft + run_selfbuild "fix" "$FIX_BRANCH" soft || rc=$? + ;; + older-pub:*) + # older-pub:VERSION:VC:SEV + IFS=':' read -r _ vn apk_vc sev <<<"$v" + : "${sev:=warn}" + label="older-pub-${vn}" + url="https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk" + name="net.taler.wallet.fdroid_${apk_vc}.apk" + age=$(release_age_days "wallet-${vn}" 2>/dev/null || echo "?") + echo "older published $vn age_days=$age severity=$sev" | tee -a "$SUMMARY" + run_published_apk "$label" "$url" "$name" "$sev" || rc=$? + v="$label" + ;; + older-self:*) + # Explicit older-self is disabled by policy (self-build = current stable only). + echo "SKIP $v — self-build only for current stable ($STABLE_TAG); use older-pub or published" | tee -a "$SUMMARY" + RESULTS+=("$v:SKIP(no-selfbuild-older)") + continue + ;; + *) + # raw git ref: only master-like / fix via named tokens; bare wallet-* tags → no self-build + if [[ "$v" =~ ^wallet-[0-9] ]]; then + echo "SKIP self-build $v — only current stable self-build ($STABLE_TAG) is enabled" | tee -a "$SUMMARY" + RESULTS+=("$v:SKIP(no-selfbuild-older)") + continue + fi + sev=warn + run_selfbuild "ref-${v//\//-}" "$v" warn || rc=$? + ;; + esac + + record_result "$v" "$rc" "$sev" || stop=$? + [ "$stop" = "2" ] && break +done + +echo "" | tee -a "$SUMMARY" +echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY" +echo "rule: age ≤ ${BLOCKER_MAX_AGE_DAYS}d → BLOCKER; 3/6/9/12mo milestones + older → WARN only" | tee -a "$SUMMARY" +for r in "${RESULTS[@]}"; do + echo " $r" | tee -a "$SUMMARY" +done +if [ "${#WARNINGS[@]}" -gt 0 ]; then + echo "warnings (non-blocking):" | tee -a "$SUMMARY" + for w in "${WARNINGS[@]}"; do + echo " WARN $w" | tee -a "$SUMMARY" + done +fi +echo "" +echo "Artifacts: $MATRIX_OUT" +exit "$overall" diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh new file mode 100755 index 0000000..1c20a63 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -0,0 +1,366 @@ +#!/usr/bin/env bash +# GOA (and optionally stage) GUI chain on Android — adapted from +# taler-android branch **dev/hernani-inference/gui-workflows** +# (`scripts/goa-chain-emu.sh`, docs/gui-workflows.md). +# +# Linux + macOS: uses adb + uiautomator only (no Homebrew hard dependency). +# Upstream scripts are documented as macOS-only for SDK install paths. +# +# Flow: +# 1) API mint withdraw (explorer) → VIEW taler://withdraw → GUI taps +# 2) Merchant template POST → VIEW taler://pay → GUI taps +# +# Usage: +# ./run-goa-gui-chain.sh +# STACK=stage ./run-goa-gui-chain.sh +# WITHDRAW_AMOUNTS='GOA:10' PAY_LIMIT=2 ./run-goa-gui-chain.sh +# PKG=net.taler.wallet.fdroid.debug ./run-goa-gui-chain.sh --no-clear +# +# Env: ANDROID_SERIAL, PKG, EXP_PW_FILE, BANK, MERCHANT, INSTANCE, +# WITHDRAW_AMOUNTS, PRODUCTS, PAY_LIMIT, SHOTDIR, PAUSE +# +# See GUI-AUTOMATION-NOTES.md +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) + +CLEAR=1 +for a in "$@"; do + case "$a" in + --no-clear) CLEAR=0 ;; + -h|--help) sed -n '2,28p' "$0"; exit 0 ;; + esac +done + +die() { echo "ERROR: $*" >&2; exit 1; } +step() { echo; echo "======== $* ========"; } +info() { echo " $*"; } +pause() { sleep "${1:-${PAUSE:-0.8}}"; } + +# Platform flags + headless emulator defaults (see GUI-AUTOMATION-NOTES.md) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" +[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; } +[ "${AUTO_GUI}" = "1" ] || { echo "skipped: AUTO_GUI=${AUTO_GUI} (chain is GUI)"; exit 0; } + +command -v adb >/dev/null || die "adb not found" +command -v python3 >/dev/null || die "python3 required" + +STACK="${STACK:-goa}" +case "$STACK" in + goa|hacktivism) + PKG="${PKG:-net.taler.wallet.fdroid.debug}" + BANK="${BANK:-https://bank.hacktivism.ch}" + MERCHANT="${MERCHANT:-https://taler.hacktivism.ch}" + INSTANCE=$(printf '%s' "${INSTANCE:-goa-shop}" | tr '[:upper:]' '[:lower:]') + DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-goa-demo-cp4zqk}" | tr '[:upper:]' '[:lower:]') + DONATE_TEMPLATE="${DONATE_TEMPLATE:-goa-free}" + DEFAULT_AMOUNTS=("GOA:10" "GOA:20") + DEFAULT_PAYS=("product:orbit-sticker" "product:nebula-coffee" "product:voidwave-playlist" "donate:GOA:12") + CUR_HINT=GOA + ;; + stage|testpaysan) + PKG="${PKG:-net.taler.wallet.fdroid.debug}" + BANK="${BANK:-https://stage.bank.lefrancpaysan.ch}" + MERCHANT="${MERCHANT:-https://stage.monnaie.lefrancpaysan.ch}" + INSTANCE=$(printf '%s' "${INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]') + DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]') + DONATE_TEMPLATE="${DONATE_TEMPLATE:-don-panier-libre}" + DEFAULT_AMOUNTS=("TESTPAYSAN:10" "TESTPAYSAN:20") + DEFAULT_PAYS=("product:panier-legumes" "product:fromage-chevre" "product:oeufs-6") + CUR_HINT=TESTPAYSAN + ;; + *) die "STACK must be goa|stage (got $STACK)" ;; +esac + +ACT="${ACT:-$PKG/net.taler.wallet.main.MainActivity}" +EXP_USER="${EXP_USER:-explorer}" +PAY_LIMIT="${PAY_LIMIT:-4}" +PAUSE="${PAUSE:-0.8}" +SHOTDIR="${SHOTDIR:-$ROOT/out-gui-chain/$(date +%Y%m%d-%H%M%S)-$STACK}" +mkdir -p "$SHOTDIR" + +# Explorer password (same search as taler-monitoring ladder) +if [[ -z "${EXP_PW_FILE:-}" ]]; then + for cand in \ + "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/bank-explorer-password.txt" + do + [[ -f "$cand" ]] && EXP_PW_FILE="$cand" && break + done +fi +# stage: try stagepaysan secret via ssh if missing +if [[ ! -f "${EXP_PW_FILE:-}" && "$STACK" = "stage" ]]; then + for host in francpaysan-stage-user francpaysan-host; do + tmp=$(mktemp) + if ssh -o BatchMode=yes -o ConnectTimeout=10 "$host" \ + 'tr -d "\n\r" "$tmp" 2>/dev/null && [[ -s "$tmp" ]]; then + EXP_PW_FILE="$tmp" + info "explorer password via $host" + break + fi + rm -f "$tmp" + done +fi +[[ -n "${EXP_PW_FILE:-}" && -f "$EXP_PW_FILE" ]] || die "set EXP_PW_FILE (explorer password)" + +# Headless AVD by default (no host window). WINDOWED=1 for a visible emulator. +SERIAL="${SERIAL:-${ANDROID_SERIAL:-}}" +android_ensure_device || die "no adb device (./start-android-emulator.sh --wait)" +export ANDROID_SERIAL="$SERIAL" +info "ANDROID_SERIAL=$ANDROID_SERIAL STACK=$STACK PKG=$PKG SHOTDIR=$SHOTDIR headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU" +ADB=(adb -s "$ANDROID_SERIAL") + +adb_sh() { "${ADB[@]}" shell "$@"; } + +shot() { + local n="$1" + adb_sh screencap -p "/sdcard/demo-$n.png" 2>/dev/null || true + "${ADB[@]}" pull "/sdcard/demo-$n.png" "$SHOTDIR/$n.png" >/dev/null 2>&1 || true + info "[shot] $SHOTDIR/$n.png" +} + +tap_text() { + local label="$1" + adb_sh uiautomator dump /sdcard/ui.xml >/dev/null 2>&1 || return 1 + "${ADB[@]}" pull /sdcard/ui.xml "$SHOTDIR/ui.xml" >/dev/null 2>&1 || return 1 + SERIAL="$ANDROID_SERIAL" python3 - "$label" "$SHOTDIR/ui.xml" <<'PY' +import os, re, subprocess, sys +from pathlib import Path +label, path = sys.argv[1], sys.argv[2] +serial = os.environ.get("SERIAL") or os.environ.get("ANDROID_SERIAL") or "" +xml = Path(path).read_text(errors="replace") +# ANR dismiss +if "isn't responding" in xml or "reagiert nicht" in xml.lower(): + for lab in ("Wait", "Warten"): + for m in re.finditer(rf'text="{lab}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', xml): + x = (int(m.group(1))+int(m.group(3)))//2 + y = (int(m.group(2))+int(m.group(4)))//2 + subprocess.run(["adb","-s",serial,"shell","input","tap",str(x),str(y)], check=False) + print(f"anr-wait @{x},{y}") + sys.exit(0) +for pat in [ + rf'text="{re.escape(label)}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'text="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'content-desc="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', +]: + for m in re.finditer(pat, xml, re.I): + chunk = xml[max(0, m.start()-120):m.end()] + if 'enabled="false"' in chunk: + continue + x = (int(m.group(1))+int(m.group(3)))//2 + y = (int(m.group(2))+int(m.group(4)))//2 + print(f"tap {label!r} @{x},{y}") + cmd = ["adb","shell","input","tap",str(x),str(y)] + if serial: + cmd = ["adb","-s",serial,"shell","input","tap",str(x),str(y)] + subprocess.run(cmd, check=False) + sys.exit(0) +print("miss", label) +sys.exit(1) +PY +} + +mint_withdraw() { + local amount="$1" + EXP_USER="$EXP_USER" EXP_PW_FILE="$EXP_PW_FILE" BANK="$BANK" \ + python3 - "$amount" <<'PY' +import base64, json, os, ssl, sys, urllib.request +from pathlib import Path +amount = sys.argv[1] +bank = os.environ["BANK"].rstrip("/") +user = os.environ["EXP_USER"] +pw = Path(os.environ["EXP_PW_FILE"]).read_text().strip() +auth = "Basic " + base64.b64encode(f"{user}:{pw}".encode()).decode() +ctx = ssl.create_default_context() + +def req(method, url, data=None, headers=None): + h = dict(headers or {}) + body = None + if data is not None: + body = json.dumps(data).encode() + h["Content-Type"] = "application/json" + r = urllib.request.Request(url, data=body, headers=h, method=method) + with urllib.request.urlopen(r, context=ctx, timeout=25) as resp: + return json.load(resp) + +tok = req( + "POST", + f"{bank}/accounts/{user}/token", + {"scope": "readwrite", "duration": {"d_us": 3600_000_000}}, + {"Authorization": auth}, +)["access_token"] +wd = req( + "POST", + f"{bank}/accounts/{user}/withdrawals", + {"amount": amount}, + {"Authorization": f"Bearer {tok}"}, +) +print(wd.get("taler_withdraw_uri") or wd.get("talerWithdrawUri") or "") +PY +} + +mint_pay() { + local mode="$1" + MERCHANT="$MERCHANT" INSTANCE="$INSTANCE" \ + DONATE_INSTANCE="$DONATE_INSTANCE" DONATE_TEMPLATE="$DONATE_TEMPLATE" \ + python3 - "$mode" <<'PY' +import json, os, ssl, sys, urllib.error, urllib.request +mode = sys.argv[1] +merchant = os.environ["MERCHANT"].rstrip("/") +inst = os.environ["INSTANCE"] +d_inst = os.environ["DONATE_INSTANCE"] +d_tmpl = os.environ["DONATE_TEMPLATE"] +ctx = ssl.create_default_context() + +def post(url, data): + body = json.dumps(data).encode() + r = urllib.request.Request(url, data=body, method="POST", + headers={"Content-Type": "application/json"}) + try: + with urllib.request.urlopen(r, context=ctx, timeout=25) as resp: + return resp.status, json.load(resp) + except urllib.error.HTTPError as e: + raw = e.read() if hasattr(e, "read") else b"" + try: + return getattr(e, "code", None), json.loads(raw) + except Exception as err: + raise SystemExit(f"POST {url} failed: {e} body={raw[:200]!r}") from err + +if mode.startswith("product:"): + pid = mode.split(":", 1)[1] + st, d = post(f"{merchant}/instances/{inst}/templates/{pid}", {}) + use_inst = inst +elif mode.startswith("donate:"): + amt = mode.split(":", 1)[1] + st, d = post(f"{merchant}/instances/{d_inst}/templates/{d_tmpl}", {"amount": amt}) + use_inst = d_inst +else: + raise SystemExit(f"bad mode {mode!r}") + +if "taler_pay_uri" in d: + print(d["taler_pay_uri"]); raise SystemExit(0) +oid, tok = d.get("order_id"), d.get("token") +if not oid or not tok: + raise SystemExit(f"no order: {d!r}"[:200]) +host = merchant.replace("https://","").replace("http://","").split("/")[0] +print(f"taler://pay/{host}/instances/{use_inst}/{oid}/?c={tok}") +PY +} + +click_through() { + local labels=("$@") + local i lab misses=0 + # Prefer dismissing ANR before hunting Confirm (low-RAM hosts thrash System UI) + for i in 1 2 3 4 5 6; do + if tap_text "Wait" 2>/dev/null || tap_text "Warten" 2>/dev/null; then + info "dismissed ANR (Wait)" + pause 2.0 + misses=0 + continue + fi + local hit=0 + for lab in "${labels[@]}"; do + if tap_text "$lab" 2>/dev/null; then + hit=1 + misses=0 + pause 0.9 + break + fi + done + if [[ "$hit" -eq 0 ]]; then + misses=$((misses + 1)) + # stop early if hierarchy has nothing useful (avoid burning order deadlines) + [[ "$misses" -ge 3 ]] && break + fi + pause 0.5 + done +} + +if [[ -n "${WITHDRAW_AMOUNTS:-}" ]]; then + # shellcheck disable=SC2206 + AMOUNTS=($WITHDRAW_AMOUNTS) +else + AMOUNTS=("${DEFAULT_AMOUNTS[@]}") +fi +if [[ -n "${PRODUCTS:-}" ]]; then + PAYS=() + for p in $PRODUCTS; do PAYS+=("product:$p"); done +else + PAYS=("${DEFAULT_PAYS[@]}") +fi + +############################ +step "0 device + wallet ($STACK / $CUR_HINT)" +adb_sh input keyevent KEYCODE_WAKEUP 2>/dev/null || true +adb_sh wm dismiss-keyguard 2>/dev/null || true +if [[ "$CLEAR" -eq 1 ]]; then + info "pm clear $PKG" + adb_sh pm clear "$PKG" >/dev/null 2>&1 || true +fi +adb_sh am force-stop "$PKG" 2>/dev/null || true +pause 0.4 +# Prefer explicit MainActivity (from gui-workflows); fall back to VIEW launcher +if ! adb_sh am start -n "$ACT" >/dev/null 2>&1; then + info "MainActivity path failed — monkey launch" + adb_sh monkey -p "$PKG" -c android.intent.category.LAUNCHER 1 >/dev/null 2>&1 || true +fi +pause 2.0 +shot "00-start" + +############################ +step "2 withdraw chain" +idx=0 +for AMT in "${AMOUNTS[@]}"; do + idx=$((idx + 1)) + step "2.$idx withdraw $AMT" + URI="$(mint_withdraw "$AMT" | head -1)" + [[ -n "$URI" ]] || die "mint failed for $AMT" + # wallet / monitoring QR rules: strip default ports + URI=$(printf '%s' "$URI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g') + info "URI=$URI" + echo "$URI" >>"$SHOTDIR/withdraw-uris.txt" + adb_sh am start -a android.intent.action.VIEW -d "$URI" "$PKG" >/dev/null + pause 1.5 + shot "w${idx}-open" + click_through "Accept" "I accept" "Accept terms" "Confirm" "Continue" "Withdraw" "Next" "OK" "Agree" "Bestätigen" "Abheben" "Akzeptieren" + pause 3.0 + shot "w${idx}-after" + adb_sh am start -n "$ACT" >/dev/null 2>&1 || true + pause 0.6 + tap_text "Assets" 2>/dev/null || true + pause 0.4 + shot "w${idx}-assets" +done + +############################ +step "3 pay chain (limit=$PAY_LIMIT)" +pidx=0 +for PAY in "${PAYS[@]}"; do + pidx=$((pidx + 1)) + [[ "$pidx" -gt "$PAY_LIMIT" ]] && break + step "3.$pidx pay $PAY" + if ! PAYURI="$(mint_pay "$PAY" | head -1)"; then + info "mint_pay failed for $PAY — skip" + continue + fi + [[ -n "$PAYURI" ]] || { info "empty pay uri — skip"; continue; } + PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g') + info "PAYURI=$PAYURI" + echo "$PAYURI" >>"$SHOTDIR/pay-uris.txt" + adb_sh am start -a android.intent.action.VIEW -d "$PAYURI" "$PKG" >/dev/null + pause 1.5 + shot "p${pidx}-open" + click_through "Pay" "Confirm" "Accept" "Continue" "OK" "Next" "Bezahlen" "Bestätigen" "Payer" + pause 1.5 + shot "p${pidx}-after" + adb_sh am start -n "$ACT" >/dev/null 2>&1 || true + pause 0.5 +done + +step "4 done → $SHOTDIR" +ls -la "$SHOTDIR" | tail -40 +adb_sh dumpsys window 2>/dev/null | grep mCurrentFocus | head -1 || true +info "PKG=$PKG amounts=${AMOUNTS[*]} pays(limit)=$PAY_LIMIT" +echo "See GUI-AUTOMATION-NOTES.md (ported from taler-android dev/hernani-inference/gui-workflows)" diff --git a/scripts/taler-monitoring/android-test/start-android-emulator.sh b/scripts/taler-monitoring/android-test/start-android-emulator.sh new file mode 100755 index 0000000..e62b5f4 --- /dev/null +++ b/scripts/taler-monitoring/android-test/start-android-emulator.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# Start the Taler Android AVD. **Default = headless** (no host window). +# Guest GLES still runs via SwiftShader so uiautomator / screenshots work. +# +# Usage: +# ./start-android-emulator.sh # headless, background +# ./start-android-emulator.sh --wait # headless + wait for boot +# ./start-android-emulator.sh --windowed # show emulator window +# EMULATOR_HEADLESS=0 ./start-android-emulator.sh +# EMULATOR_GPU=host WINDOWED=1 ./start-android-emulator.sh +# +# Env: EMULATOR_AVD EMULATOR_MEMORY EMULATOR_CORES EMULATOR_GPU +# EMULATOR_HEADLESS EMULATOR_LOG EMULATOR_BOOT_TIMEOUT EMULATOR_EXTRA_ARGS +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" + +WAIT=0 +for a in "$@"; do + case "$a" in + --wait) WAIT=1 ;; + --windowed| --gui) EMULATOR_HEADLESS=0; export EMULATOR_HEADLESS ;; + --headless) EMULATOR_HEADLESS=1; export EMULATOR_HEADLESS ;; + -h|--help) + sed -n '2,18p' "$0" + exit 0 + ;; + esac +done + +command -v emulator >/dev/null || { + echo "emulator not found (ANDROID_HOME=$ANDROID_HOME)" >&2 + exit 2 +} +command -v adb >/dev/null || { + echo "adb not found" >&2 + exit 2 +} + +# Already booted? +if serial=$(android_adb_serial); [ -n "$serial" ]; then + boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true) + if [ "$boot" = "1" ]; then + echo "already running: $serial (boot_completed=1) — headless default still applies to *new* starts" + exit 0 + fi +fi + +# Avoid pkill patterns that match this wrapper's argv (self-kill). +mapfile -t EMU_ARGS < <(android_emulator_args "$EMULATOR_AVD") +echo "starting: emulator ${EMU_ARGS[*]}" +echo " headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU log=$EMULATOR_LOG" + +if [ "${EMULATOR_HEADLESS}" = "1" ]; then + export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}" + # Do not require DISPLAY + unset DISPLAY || true +fi + +mkdir -p "$(dirname "$EMULATOR_LOG")" +# Background; do not use pkill -f emulator from callers matching full cmdline +nohup emulator "${EMU_ARGS[@]}" >"$EMULATOR_LOG" 2>&1 & +echo "emulator pid=$! log=$EMULATOR_LOG" + +if [ "$WAIT" != "1" ]; then + echo "not waiting (pass --wait for boot). adb devices when ready." + exit 0 +fi + +echo "waiting for boot (timeout ${EMULATOR_BOOT_TIMEOUT}s)…" +deadline=$((SECONDS + EMULATOR_BOOT_TIMEOUT)) +serial="" +while (( SECONDS < deadline )); do + adb wait-for-device 2>/dev/null || true + serial=$(android_adb_serial || true) + if [ -n "$serial" ]; then + boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true) + if [ "$boot" = "1" ]; then + echo "BOOT_OK serial=$serial headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU" + exit 0 + fi + fi + sleep 2 +done + +echo "boot timeout — last log:" >&2 +tail -30 "$EMULATOR_LOG" >&2 || true +exit 3 diff --git a/scripts/taler-monitoring/check_auth401.sh b/scripts/taler-monitoring/check_auth401.sh new file mode 100755 index 0000000..dfca606 --- /dev/null +++ b/scripts/taler-monitoring/check_auth401.sh @@ -0,0 +1,704 @@ +#!/usr/bin/env bash +# auth401 — merchant HTTP 401 / case-sensitivity matrix (WebUI-shaped flows) +# +# Exercises the same APIs the merchant SPA uses for: +# • self-provision signup (POST /instances) — mixed-case id +# • login / createAccessToken (POST …/private/token) — Basic user case +# • password change (POST …/private/auth) +# • admin-create (POST /management/instances) when admin bearer available +# • access-token + private/ with Bearer +# +# Backend fact (#11340): Basic-auth *username* is case-sensitive; URL path is +# usually not. SPA must always lowercase the username (createAccessToken, signup +# id, admin-create, pw change re-login). These tests assert the *raw API* +# behaviour so a missing SPA lowercasing shows up as 401 where 200 is wanted +# only after lowercasing. +# +# ./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401 +# ./taler-monitoring.sh -d hacktivism.ch auth401 +# +# Continue through all groups / collect every ERROR (optional): +# AUTH401_CONTINUE=1 ./taler-monitoring.sh -d stage… auth401 +# CONTINUE_ON_ERROR=1 ./taler-monitoring.sh -d stage… auth401 # same +# +# Env: see secrets.env.example (AUTH401_*). +# +# NOTE: we intentionally do NOT use `set -e`. A single failed curl/grep under +# set -e would abort with no ERROR summary — the opposite of what monitoring +# needs. Optional CONTINUE_ON_ERROR still controls hard prereq early-exit. +set -uo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib.sh +source "$ROOT/lib.sh" +load_monitoring_secrets_env 2>/dev/null || true + +tmp=$(mktemp -d) +AUTH401_FINISHED=0 +_auth401_finish() { + [ "${AUTH401_FINISHED}" = "1" ] && return 0 + AUTH401_FINISHED=1 + # summary() returns non-zero when FAIL_N>0 — never let that skip the print + summary || true + rm -rf "${tmp:-}" + if [ "${FAIL_N:-0}" -gt 0 ]; then + exit 1 + fi + exit 0 +} +trap '_auth401_finish' EXIT + +# --------------------------------------------------------------------------- +# Parameters +# --------------------------------------------------------------------------- +: "${AUTH401_TIMEOUT:=${TIMEOUT:-20}}" +: "${AUTH401_CREATE:=1}" +: "${AUTH401_INSTANCE:=}" +: "${AUTH401_PASSWORD:=${MERCHANT_INSTANCE_PASSWORD:-${AUTH401_PASS:-}}}" +: "${AUTH401_ID_PREFIX:=mon401}" +: "${AUTH401_SKIP_IF_MFA:=1}" +: "${AUTH401_BEARER:=${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}}" +: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}" +: "${AUTH401_DURABLE_ID:=mon401}" +# Prefer durable alone (skip throwaway) — default 0 so full matrix always runs +: "${AUTH401_PREFER_DURABLE:=0}" +# Run expanded groups (signup/login/pwchange/admin/webui). Default 1. +: "${AUTH401_FULL:=1}" +# Admin bearer for POST /management/instances (optional) +: "${AUTH401_ADMIN_TOKEN:=${MERCHANT_ADMIN_TOKEN:-}}" +: "${AUTH401_ADMIN_PASSWORD:=}" +: "${AUTH401_ADMIN_USER:=admin}" +# 1 = do not abort on hard prereq fail; run remaining groups (default 0) +# Alias: CONTINUE_ON_ERROR=1 +: "${AUTH401_CONTINUE:=${CONTINUE_ON_ERROR:-0}}" +# Never rotate password on durable account (default 1 — keep mon401.password valid) +: "${AUTH401_PWCHANGE_THROWAY_ONLY:=1}" + +TIMEOUT="$AUTH401_TIMEOUT" +export TIMEOUT + +_auth401_abort_or_continue() { + local why="${1:-prereq failed}" + if [ "${AUTH401_CONTINUE}" = "1" ]; then + warn "continue" "${why} (AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1)" + return 0 + fi + # hard stop — EXIT trap prints summary + all ERRORS collected so far + exit 1 +} + +BASE="${MERCHANT_PUBLIC%/}" +if [ -z "$BASE" ]; then + echo "MERCHANT_PUBLIC empty — set -d domain or --merchant" >&2 + exit 2 +fi + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- +expect_http() { + local label="$1" want="$2" got="$3" detail="${4:-}" + case ",$want," in + *",$got,"*) ok "$label" "HTTP $got${detail:+ · $detail}" ;; + *) fail "$label" "HTTP $got (want $want)${detail:+ · $detail}" ;; + esac +} + +to_lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; } +to_upper() { printf '%s' "$1" | tr '[:lower:]' '[:upper:]'; } + +# First char upper, rest lower → synthetic MIX for Basic user (always ≠ low if len>0) +synth_mix() { + python3 -c "s='''$1'''; print((s[:1].upper()+s[1:].lower()) if s else s)" +} + +# First 3 upper (partial mix) +synth_partmix() { + python3 -c "s='''$1'''; print(s[:3].upper()+s[3:] if len(s)>=3 else s.upper())" +} + +# Normalize curl status: "200", "000", never "200000" / "000000" +_http_norm() { + local raw="$1" dig + dig=$(printf '%s' "$raw" | tr -cd '0-9') + if [ -z "$dig" ]; then + printf '000' + else + # first 3 digits = real code (curl 200 then || echo 000 → 200000) + printf '%s' "$dig" | head -c 3 + fi +} + +token_http() { + local path_id="$1" user="$2" password="$3" + local b64 raw + b64=$(printf '%s' "${user}:${password}" | base64 -w0 2>/dev/null \ + || printf '%s' "${user}:${password}" | base64) + raw=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/last_token.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${path_id}/private/token" \ + -H "Authorization: Basic ${b64}" \ + -H 'Content-Type: application/json' \ + -d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \ + 2>/dev/null || true) + _http_norm "${raw:-}" +} + +json_code_hint() { + python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true +import json, sys +try: + d = json.load(open(sys.argv[1])) +except Exception: + print("") + raise SystemExit +c = d.get("code", "") +h = (d.get("hint") or d.get("detail") or "")[:70] +print(f"code={c} {h}".strip()) +PY +} + +extract_token() { + python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true +import json, sys +try: + d = json.load(open(sys.argv[1])) +except Exception: + print("") + raise SystemExit(0) +print(d.get("access_token") or d.get("token") or "") +PY +} + +instance_body() { + local id="$1" pass="$2" name="${3:-auth401 probe}" + python3 - "$id" "$pass" "$name" <<'PY' +import json, sys +print(json.dumps({ + "address": {}, + "auth": {"method": "token", "password": sys.argv[2]}, + "default_pay_delay": {"d_us": 60000000}, + "default_wire_transfer_delay": {"d_us": 30000000}, + "default_refund_delay": {"d_us": 30000000}, + "id": sys.argv[1], + "jurisdiction": {}, + "name": sys.argv[3], + "use_stefan": True, +})) +PY +} + +auth_change_http() { + local path_id="$1" bearer="$2" new_pass="$3" + local tok="$bearer" body + case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac + body=$(NEW_PASS="$new_pass" python3 -c 'import json,os; print(json.dumps({"method":"token","password":os.environ["NEW_PASS"]}))') + curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/auth_change.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${path_id}/private/auth" \ + -H "Authorization: Bearer ${tok}" \ + -H 'Content-Type: application/json' \ + -d "$body" \ + 2>/dev/null || echo 000 +} + +# --------------------------------------------------------------------------- +# Stage secrets (quiet) +# --------------------------------------------------------------------------- +_auth401_stage_secret_roots() { + printf '%s\n' \ + "${FRANCPAYSAN_SECRETS:-}" \ + "${HOME}/francpaysan-secrets" \ + "${HOME}/src/francpaysan-secrets" \ + "${HOME}/taler/src/francpaysan-secrets" +} + +_auth401_resolve_secrets_quiet() { + local root inst_id pass + for root in $(_auth401_stage_secret_roots); do + [ -n "$root" ] && [ -d "$root" ] || continue + # durable ids: prefer mon401fix when mon401 was rotated by old tests + if [ -z "${DURABLE_PASS:-}" ]; then + for inst_id in "${AUTH401_DURABLE_ID}" mon401fix mon401 "${MERCHANT_INSTANCE:-}"; do + [ -n "$inst_id" ] || continue + pass="" + if [ -f "$root/stage/instances/${inst_id}.password" ]; then + pass=$(tr -d '\n\r' <"$root/stage/instances/${inst_id}.password") + fi + # skip empty password files + if [ -n "$pass" ]; then + # verify file is for this id (mon401.password may be stale copy of mon401fix) + DURABLE_PASS="$pass" + DURABLE_ID=$(to_lower "$inst_id") + AUTH401_SECRET_NOTE="durable ${inst_id} from ${root}/stage/instances/" + # only set AUTH401_PASSWORD if caller did not pin an instance + if [ -z "${AUTH401_PASSWORD}" ] && [ -z "${AUTH401_INSTANCE}" ]; then + AUTH401_PASSWORD="$pass" + fi + break + fi + done + fi + if [ -z "${AUTH401_BEARER}" ] && [ -f "$root/stage/default-instance-token.txt" ]; then + AUTH401_BEARER=$(tr -d '\n\r' <"$root/stage/default-instance-token.txt") + AUTH401_BEARER_NOTE="default-instance-token.txt" + fi + if [ -z "${AUTH401_ADMIN_TOKEN}" ] && [ -f "$root/stage/admin-instance-token.txt" ]; then + AUTH401_ADMIN_TOKEN=$(tr -d '\n\r' <"$root/stage/admin-instance-token.txt") + AUTH401_ADMIN_NOTE="admin-instance-token.txt" + fi + [ -n "${DURABLE_PASS:-}" ] && break + done + # also keep AUTH401_INSTANCE for durable-only mode + if [ "${AUTH401_PREFER_DURABLE}" = "1" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + AUTH401_INSTANCE="${AUTH401_INSTANCE:-$DURABLE_ID}" + AUTH401_PASSWORD="${AUTH401_PASSWORD:-$DURABLE_PASS}" + AUTH401_CREATE=0 + AUTH401_MODE_NOTE="PREFER_DURABLE=1 → only durable ${AUTH401_INSTANCE}" + fi +} + +_auth401_resolve_secrets_quiet +: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}" +export AUTH401_INSTANCE AUTH401_PASSWORD AUTH401_BEARER AUTH401_CREATE AUTH401_ADMIN_TOKEN + +# --------------------------------------------------------------------------- +set_area auth401 +section "auth401 · merchant 401 / case matrix (WebUI APIs) · ${BASE}" + +if [ -n "${AUTH401_SECRET_NOTE:-}" ]; then info "stage secret" "$AUTH401_SECRET_NOTE"; fi +if [ -n "${AUTH401_BEARER_NOTE:-}" ]; then info "stage secret" "AUTH401_BEARER from ${AUTH401_BEARER_NOTE}"; fi +if [ -n "${AUTH401_ADMIN_NOTE:-}" ]; then info "stage secret" "AUTH401_ADMIN_TOKEN from ${AUTH401_ADMIN_NOTE}"; fi +if [ -n "${AUTH401_MODE_NOTE:-}" ]; then info "auth401 mode" "$AUTH401_MODE_NOTE"; fi +info "auth401 scope" "FULL=${AUTH401_FULL} CREATE=${AUTH401_CREATE} PREFER_DURABLE=${AUTH401_PREFER_DURABLE} CONTINUE=${AUTH401_CONTINUE}" +if [ "${AUTH401_CONTINUE}" = "1" ]; then + info "continue mode" "CONTINUE_ON_ERROR/AUTH401_CONTINUE=1 — hard prereqs soft; always full ERROR list via EXIT trap" +fi + +# =========================================================================== +set_group config +# =========================================================================== +cfg_code=$(http_body "$BASE/config" "$tmp/mer-config.json") +expect_http "merchant /config" 200 "$cfg_code" "$BASE/config" + +HAVE_SP=0 +MANDATORY_TAN=0 +if [ "$cfg_code" = "200" ]; then + eval "$(python3 - "$tmp/mer-config.json" <<'PY' +import json, sys +d = json.load(open(sys.argv[1])) +sp = 1 if d.get("have_self_provisioning") else 0 +tans = d.get("mandatory_tan_channels") or [] +print(f"HAVE_SP={sp}") +print(f"MANDATORY_TAN={1 if tans else 0}") +print(f"TAN_CH={','.join(tans) if tans else ''}") +print(f"API_VER={d.get('version','')}") +PY +)" + info "self-provision" "have_self_provisioning=${HAVE_SP} mandatory_tan=${MANDATORY_TAN}${TAN_CH:+ ($TAN_CH)} api=${API_VER:-?}" +else + fail "merchant config required for auth401" "HTTP $cfg_code" + _auth401_abort_or_continue "no /config" +fi + +# =========================================================================== +set_group webui +# SPA surface: /webui/ loads + version stamp; optional toLowerCase in bundle +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + wcode=$(http_code "$BASE/webui/") + case "$wcode" in + 200|301|302) ok "webui /" "HTTP $wcode" ;; + *) warn "webui /" "HTTP $wcode" ;; + esac + vcode=$(http_body "$BASE/webui/version.txt" "$tmp/spa-ver.txt" 2>/dev/null || echo 000) + if [ "$vcode" = "200" ] && [ -s "$tmp/spa-ver.txt" ]; then + ok "webui version.txt" "$(tr -d '\n\r' <"$tmp/spa-ver.txt")" + else + warn "webui version.txt" "HTTP ${vcode:-000}" + fi + # Download a slice of index.js — look for lowercasing mitigation (#11340) + jcode=$(curl -skS -m 60 -o "$tmp/spa-index.js" -w '%{http_code}' "$BASE/webui/index.js" 2>/dev/null || echo 000) + jcode=$(printf '%s' "$jcode" | tr -cd '0-9' | head -c 3) + if [ "$jcode" = "200" ] && [ -s "$tmp/spa-index.js" ]; then + ok "webui index.js" "HTTP 200 · $(wc -c <"$tmp/spa-index.js" | tr -d ' ') bytes" + if grep -q 'toLowerCase' "$tmp/spa-index.js" 2>/dev/null; then + ok "webui bundle toLowerCase" "present (SPA may mitigate #11340)" + else + warn "webui bundle toLowerCase" "not found in index.js — login MIX may 401 without lowercasing" + fi + else + warn "webui index.js" "HTTP ${jcode:-000}" + fi +fi + +# =========================================================================== +set_group signup +# WebUI self-provision: POST /instances with mixed-case id (as typed) +# =========================================================================== +CREATED=0 +USE_ID="" +USE_PASS="" +ID_MIX="" +ID_LOW="" +CREATE_PASS="$AUTH401_CREATE_PASSWORD" +stamp=$(date +%H%M%S) +# Mixed-case id like the SPA would send before lowercasing (or if it forgot) +ID_MIX="$(printf '%s' "${AUTH401_ID_PREFIX}" | awk '{print toupper(substr($0,1,1)) substr($0,2)}')X${stamp}" +ID_LOW=$(to_lower "$ID_MIX") + +if [ "${AUTH401_FULL}" = "1" ] && [ "$AUTH401_CREATE" = "1" ] && [ "$HAVE_SP" = "1" ]; then + if [ "$MANDATORY_TAN" = "1" ] && [ "$AUTH401_SKIP_IF_MFA" = "1" ]; then + warn "signup skipped" "mandatory_tan_channels set — set AUTH401_INSTANCE+PASSWORD" + else + body=$(instance_body "$ID_MIX" "$CREATE_PASS" "auth401 signup MIX") + ccode=$(_http_norm "$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/create.json" -w '%{http_code}' \ + -X POST "${BASE}/instances" \ + -H 'Content-Type: application/json' \ + -d "$body" 2>/dev/null || true)") + case "$ccode" in + 200|204) + ok "signup POST /instances id=MIX" "HTTP $ccode id=$ID_MIX (SPA self-provision)" + CREATED=1 + USE_ID="$ID_MIX" + USE_PASS="$CREATE_PASS" + # Immediate login as SPA after signup: must use *lowercase* Basic user + code=$(token_http "$ID_LOW" "$ID_LOW" "$CREATE_PASS") + expect_http "signup→login SPA-sim path=low user=low → 200" 200 "$code" "$(json_code_hint)" + # Bug path: SPA forgot toLowerCase on login after signup with MIX id typed + code=$(token_http "$ID_LOW" "$ID_MIX" "$CREATE_PASS") + expect_http "signup→login RAW user=MIX (no lower) → 401" 401 "$code" "$(json_code_hint)" + ;; + 202) + warn "signup needs MFA" "HTTP 202 — use AUTH401_INSTANCE" + ;; + *) + fail "signup POST /instances" "HTTP $ccode $(head -c 120 "$tmp/create.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi +elif [ -n "${AUTH401_INSTANCE:-}" ] && [ -n "${AUTH401_PASSWORD:-}" ]; then + USE_ID=$(to_lower "$AUTH401_INSTANCE") + USE_PASS="$AUTH401_PASSWORD" + ID_LOW=$(to_lower "$USE_ID") + ID_MIX=$(synth_mix "$ID_LOW") + info "signup" "using existing AUTH401_INSTANCE=$USE_ID (no POST /instances)" + ok "using existing instance" "$USE_ID" +else + warn "signup skipped" "CREATE=${AUTH401_CREATE} HAVE_SP=${HAVE_SP} (need instance for matrix)" +fi + +# Fallback durable +if [ -z "$USE_ID" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + USE_ID="$DURABLE_ID" + USE_PASS="$DURABLE_PASS" + ID_LOW=$(to_lower "$USE_ID") + ID_MIX=$(synth_mix "$ID_LOW") + info "fallback durable" "$USE_ID" + ok "using durable instance" "$USE_ID" +fi + +if [ -z "$USE_ID" ]; then + fail "no instance under test" "need signup OK or AUTH401_INSTANCE+PASSWORD or mon401.password" + _auth401_abort_or_continue "no instance under test" + # CONTINUE: synthesize dummy ids so later groups still run (expect many 404/401) + ID_LOW_USE="auth401-missing-instance" + ID_USER_MIX=$(synth_mix "$ID_LOW_USE") + ID_PATH_MIX="$ID_USER_MIX" + ID_UPPER=$(to_upper "$ID_LOW_USE") + ID_PARTMIX=$(synth_partmix "$ID_LOW_USE") + PASS_OK="invalid" + PASS_BAD='auth401-wrong-password-NOT-VALID' + USE_ID="$ID_LOW_USE" + USE_PASS="$PASS_OK" +fi + +ID_LOW_USE=$(to_lower "$USE_ID") +# Always synthetic case variants (even for mon401) so MIX ≠ low +ID_USER_MIX=$(synth_mix "$ID_LOW_USE") +ID_PATH_MIX="$ID_USER_MIX" +ID_UPPER=$(to_upper "$ID_LOW_USE") +ID_PARTMIX=$(synth_partmix "$ID_LOW_USE") +PASS_OK="$USE_PASS" +PASS_BAD='auth401-wrong-password-NOT-VALID' + +info "probe ids" "low=${ID_LOW_USE} MIX_user=${ID_USER_MIX} UPPER=${ID_UPPER} created=${CREATED} pass_len=${#PASS_OK}" + +# =========================================================================== +set_group login +# WebUI login = POST …/private/token with Basic (username, password) +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + expect_http "login SPA-sim user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK") + expect_http "login RAW user=MIX pwd=OK → 401 (#11340)" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK") + expect_http "login RAW user=UPPER pwd=OK → 401" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK") + expect_http "login path=MIX user=low pwd=OK → 200 (path tolerant)" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK") + expect_http "login path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" +fi + +# =========================================================================== +set_group case +# Full Basic case matrix (always hard expects — synthetic MIX) +# =========================================================================== +code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_OK") +expect_http "path=MIX user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=MIX user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK") +expect_http "path=low user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=low user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK") +expect_http "path=low user=UPPER pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_PARTMIX" "$PASS_OK") +expect_http "path=low user=MiX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +# =========================================================================== +set_group password +# =========================================================================== +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_BAD") +expect_http "pwd=BAD → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "") +expect_http "pwd=EMPTY → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_BAD") +expect_http "path=MIX user=MIX pwd=BAD → 401" 401 "$code" "$(json_code_hint)" + +# =========================================================================== +set_group pwchange +# WebUI password change: POST private/auth then re-login +# Only on throwaway signup instances — never rotate durable mon401.password +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + if [ "${AUTH401_PWCHANGE_THROWAY_ONLY}" = "1" ] && [ "${CREATED:-0}" != "1" ]; then + info "pwchange skipped" "not a throwaway instance (CREATED=0) — refuse to rotate durable password" + else + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + if [ "$code" != "200" ]; then + fail "pwchange: login for token" "HTTP $code — skip pwchange" + else + TOK=$(extract_token) + if [ -z "$TOK" ]; then + fail "pwchange: parse token" "empty access_token" + else + ok "pwchange: got bearer" "len=${#TOK}" + NEW_PASS="Auth401-New-${stamp}!" + ac=$(_http_norm "$(auth_change_http "$ID_LOW_USE" "$TOK" "$NEW_PASS")") + case "$ac" in + 200|204) + ok "pwchange POST private/auth" "HTTP $ac (SPA password form)" + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + expect_http "pwchange: old pwd → 401" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$NEW_PASS") + expect_http "pwchange: new pwd user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$NEW_PASS") + expect_http "pwchange: new pwd user=MIX → 401" 401 "$code" "$(json_code_hint)" + PASS_OK="$NEW_PASS" + USE_PASS="$NEW_PASS" + ;; + 202) + warn "pwchange needs MFA" "HTTP 202" + ;; + *) + fail "pwchange POST private/auth" "HTTP $ac $(head -c 100 "$tmp/auth_change.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi + fi + fi +fi + +# =========================================================================== +set_group admin +# Admin UI: POST /management/instances (needs admin bearer) +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + ADMIN_TOK="${AUTH401_ADMIN_TOKEN:-}" + # Optional: login as admin user with password + if [ -z "$ADMIN_TOK" ] && [ -n "${AUTH401_ADMIN_PASSWORD}" ]; then + code=$(token_http "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_PASSWORD") + if [ "$code" = "200" ]; then + ADMIN_TOK=$(extract_token) + ok "admin login user=${AUTH401_ADMIN_USER}" "got token" + else + # try lower only + au=$(to_lower "$AUTH401_ADMIN_USER") + code=$(token_http "$au" "$au" "$AUTH401_ADMIN_PASSWORD") + if [ "$code" = "200" ]; then + ADMIN_TOK=$(extract_token) + ok "admin login user=${au}" "got token" + else + warn "admin login" "HTTP $code — set AUTH401_ADMIN_TOKEN" + fi + fi + fi + + if [ -z "$ADMIN_TOK" ]; then + info "admin-create skipped" "no AUTH401_ADMIN_TOKEN / AUTH401_ADMIN_PASSWORD (stage default is not admin)" + else + case "$ADMIN_TOK" in secret-token:*) ;; *) ADMIN_TOK="secret-token:${ADMIN_TOK}" ;; esac + AID_MIX="Adm401X${stamp}" + AID_LOW=$(to_lower "$AID_MIX") + APASS="Adm401-Pass-${stamp}!" + body=$(instance_body "$AID_MIX" "$APASS" "auth401 admin-create MIX") + acode=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/admin_create.json" -w '%{http_code}' \ + -X POST "${BASE}/management/instances" \ + -H "Authorization: Bearer ${ADMIN_TOK}" \ + -H 'Content-Type: application/json' \ + -d "$body" 2>/dev/null || echo 000) + case "$acode" in + 200|204) + ok "admin-create POST /management/instances id=MIX" "HTTP $acode id=$AID_MIX" + code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS") + expect_http "admin-create→login user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$AID_LOW" "$AID_MIX" "$APASS") + expect_http "admin-create→login user=MIX → 401" 401 "$code" "$(json_code_hint)" + # admin-set auth on new instance + code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS") + if [ "$code" = "200" ]; then + AT=$(extract_token) + NEW_A="Adm401-New-${stamp}!" + ch=$(auth_change_http "$AID_LOW" "$AT" "$NEW_A") + case "$ch" in + 200|204) + ok "admin-created instance pwchange" "HTTP $ch" + code=$(token_http "$AID_LOW" "$AID_LOW" "$NEW_A") + expect_http "admin-created re-login new pwd → 200" 200 "$code" "$(json_code_hint)" + ;; + *) warn "admin-created pwchange" "HTTP $ch" ;; + esac + fi + ;; + 401|403) + warn "admin-create unauthorized" "HTTP $acode — token not admin (expected on stage default)" + ;; + 202) + warn "admin-create MFA" "HTTP 202" + ;; + *) + fail "admin-create" "HTTP $acode $(head -c 120 "$tmp/admin_create.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi +fi + +# =========================================================================== +set_group missing +# =========================================================================== +code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/last_token.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${ID_LOW_USE}/private/token" \ + -H 'Content-Type: application/json' \ + -d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \ + 2>/dev/null || echo 000) +expect_http "private/token NO_AUTH → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "auth401-not-a-real-user" "$PASS_OK") +expect_http "user=OTHER pwd=OK → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "auth401-does-not-exist-${stamp}" "$ID_LOW_USE" "$PASS_OK") +# Must not look like auth failure (401). Prefer 404; 502 from proxy is soft. +case "$code" in + 404) ok "path=missing → 404 (not 401)" "HTTP 404 · $(json_code_hint)" ;; + 401) fail "path=missing must not be 401" "HTTP 401 · $(json_code_hint)" ;; + 502|503) warn "path=missing" "HTTP $code (want 404; proxy flake, not auth)" ;; + *) fail "path=missing → 404 (not 401)" "HTTP $code (want 404) · $(json_code_hint)" ;; +esac + +# =========================================================================== +set_group bearer +# =========================================================================== +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") +if [ "$code" = "200" ]; then + TOKEN=$(extract_token) + if [ -n "$TOKEN" ]; then + ok "got access_token" "len=${#TOKEN}" + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" \ + -H "Authorization: Bearer ${TOKEN}" 2>/dev/null || echo 000) + expect_http "GET private/ Bearer OK → 200" 200 "$code" + + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" \ + -H "Authorization: Bearer secret-token:AUTH401INVALID000" 2>/dev/null || echo 000) + expect_http "GET private/ Bearer BAD → 401" 401 "$code" + + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" 2>/dev/null || echo 000) + expect_http "GET private/ NO_AUTH → 401" 401 "$code" + else + fail "parse access_token" "HTTP $code but no token field" + fi +else + fail "token for bearer tests" "HTTP $code" +fi + +code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/mgmt.json" -w '%{http_code}' \ + "${BASE}/management/instances" 2>/dev/null || echo 000) +expect_http "GET /management/instances NO_AUTH → 401" 401 "$code" + +if [ -n "${AUTH401_BEARER:-}" ]; then + if [ -z "${AUTH401_BEARER_INSTANCE:-}" ]; then + case "${AUTH401_BEARER_NOTE:-}" in + *default-instance-token*) AUTH401_BEARER_INSTANCE=default ;; + *) AUTH401_BEARER_INSTANCE="${DURABLE_ID:-default}" ;; + esac + fi + inst=$(to_lower "$AUTH401_BEARER_INSTANCE") + tok="$AUTH401_BEARER" + case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv2.json" -w '%{http_code}' \ + "${BASE}/instances/${inst}/private/" \ + -H "Authorization: Bearer ${tok}" 2>/dev/null || echo 000) + case "$code" in + 200) ok "GET private/ secrets Bearer" "HTTP 200 instance=$inst" ;; + 401) warn "secrets Bearer rejected" "HTTP 401 on $inst" ;; + *) warn "secrets Bearer" "HTTP $code on $inst" ;; + esac +fi + +# =========================================================================== +set_group durable +# Extra: durable mon401 still works after throwaway tests (if present) +# =========================================================================== +if [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + dlow=$(to_lower "$DURABLE_ID") + dmix=$(synth_mix "$dlow") + code=$(token_http "$dlow" "$dlow" "$DURABLE_PASS") + expect_http "durable ${dlow} login user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$dlow" "$dmix" "$DURABLE_PASS") + expect_http "durable ${dlow} login user=MIX → 401" 401 "$code" "$(json_code_hint)" +else + info "durable" "no mon401.password — skipped" +fi + +# =========================================================================== +set_group report +# =========================================================================== +info "summary matrix" "401: Basic user case≠low | bad/empty pwd | no auth | bad bearer | wrong user · 200: user=low + path any case · 404: missing instance · SPA must toLowerCase username on login/signup/admin/pw-relogin" +if [ "$CREATED" = "1" ]; then + info "throwaway left" "id=${ID_MIX:-?} low=${ID_LOW_USE} (password rotated if pwchange ran)" +fi + +# fall through → EXIT trap runs _auth401_finish (summary + full ERROR list) +: diff --git a/scripts/taler-monitoring/check_goa_ladder.sh b/scripts/taler-monitoring/check_goa_ladder.sh index 8b311ce..d7c60ae 100755 --- a/scripts/taler-monitoring/check_goa_ladder.sh +++ b/scripts/taler-monitoring/check_goa_ladder.sh @@ -1,35 +1,35 @@ #!/usr/bin/env bash -# check_goa_ladder.sh — GOA withdraw ladder for taler-monitoring +# check_goa_ladder.sh — withdraw/pay amount ladder (GOA + stage TESTPAYSAN) # -# bank.hacktivism.ch flow (landing): -# 1) GET /intro/auto-account.json → personal goa-account-* (GOA:0) -# 2) Mint pool withdrawals as explorer (shared pool) + confirm when selected -# 3) wallet-cli accept-uri only (no run-until-done — hangs / developer ban) -# 4) bank confirm when selected; settle = poll balance + bank transfer_done +# Flow (bank landings): +# 1) GET /intro/auto-account.json → personal *account-* (balance 0) +# 2) Mint pool withdrawals as explorer + confirm when selected +# 3) wallet-cli accept-uri only (no run-until-done) +# 4) bank confirm when selected; settle = balance + transfer_done # -# Amounts: random strictly increasing; fixed pins 0, max-1, max (23 default). -# Phase A: withdraw ladder into ONE cumulative wallet (mids scaled up for pay budget). -# Phase B: pay ladder 0 → low → … → max-1 → max (same step count). -# On first hard failure: stop, print timing report, exit 1. -# Soft: GOA:0 / wallet 7006 / absolute max (CEILING_REJECT) → WARN and continue. +# Amounts: random strictly increasing; pins 0, max-1, max. +# Phase A: withdraw ladder (cumulative wallet). Phase B: pay ladder. # -# Env: -# LADDER_STEPS total rungs (default 23) = 0 + (N-3) random + max-1 + max -# LADDER_MAX_AMOUNT absolute last pin (libeufin ceiling 4503599627370496) -# LADDER_WITHDRAW_SCALE mid withdraw amounts × this vs pay mids (default 1.5) -# LADDER_PAY=0 skip payment phase -# LADDER_TIMEOUT_S default 3600 -# LADDER_LOAD=0 skip host load snapshots -# LADDER_SETTLE_ROUNDS / LADDER_SETTLE_SLEEP — balance poll only (no shepherd) -# EXP_PW_FILE, LADDER_REPORT_DIR, MERCHANT_INSTANCE, … +# Stacks: +# GOA — LADDER_MAX_AMOUNT = libeufin ceiling; explorer from koopa-admin-secrets +# TESTPAYSAN stage — auto max_wire from bank /config, min denom from /keys; +# explorer from stagepaysan secrets / SSH # -# Phase: ./taler-monitoring.sh ladder +# Usage: +# ./taler-monitoring.sh ladder +# ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder +# LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder +# +# Env (see body): LADDER_STEPS, LADDER_MAX_AMOUNT, LADDER_MIN_AMOUNT, LADDER_STACK_AUTO, +# LADDER_PAY, EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, … set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) # shellcheck source=lib.sh source "$ROOT/lib.sh" +# When invoked standalone (not via taler-monitoring.sh), still load secrets.env +load_monitoring_secrets_env 2>/dev/null || true -# Area ladder.* — GOA withdraw/pay ladder +# Area ladder.* — withdraw/pay amount ladder (GOA + TESTPAYSAN) # Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report set_area ladder set_group plan @@ -44,12 +44,16 @@ elapsed_ms() { : "${LADDER_SETTLE_ROUNDS:=18}" : "${LADDER_SETTLE_SLEEP:=2}" : "${EXP_USER:=explorer}" -: "${EXP_PW_FILE:=/Users/newkamek/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt}" -: "${CLI_JS:=/Users/newkamek/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs}" +# EXP_PW_FILE / EXP_PW optional overrides; otherwise read_secret + SECRETS_ROOT / stage SSH +: "${EXP_PW_FILE:=}" +: "${EXP_PW:=}" +: "${CLI_JS:=}" +# GOA libeufin amount ceiling (hacktivism). Stage auto-replaces via bank /config. : "${LADDER_MAX_AMOUNT:=4503599627370496}" : "${LADDER_STEPS:=23}" : "${LADDER_LOAD:=1}" : "${LADDER_PAY:=1}" +: "${LADDER_STACK_AUTO:=1}" # Withdraw mids = pay mids × scale (so wallet can afford the pay ladder) : "${LADDER_WITHDRAW_SCALE:=1.5}" # Floor for random mids (must be ≥ smallest exchange coin; GOA min denom ≈ 0.000001) @@ -57,12 +61,97 @@ elapsed_ms() { : "${LADDER_CONFIRM_POLLS:=40}" : "${LADDER_PAY_SETTLE_ROUNDS:=6}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" +# Public variable template for stage pays (optional; fixed templates used when amount maps) +: "${LADDER_PAY_TEMPLATE:=}" +: "${LADDER_PAY_TEMPLATE_INSTANCE:=fermes-des-collines}" + +GOA_LADDER_CEILING="4503599627370496" + +# Resolve wallet-cli .mjs (no hardcoded laptop path) +if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then + CLI_JS=$(find_wallet_cli 2>/dev/null || true) +fi +if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then + # allow PATH wrapper as last resort (wcli falls back to taler-wallet-cli) + CLI_JS="" +fi CUR="${EXPECT_CURRENCY:-GOA}" BANK="${BANK_PUBLIC%/}" EX="${EXCHANGE_PUBLIC%/}/" MER="${MERCHANT_PUBLIC%/}" INST="${MERCHANT_INSTANCE}" + +# --- Stack-specific ladder maxima (TESTPAYSAN stage) --- +# Uses bank max_wire_transfer_amount and exchange min denom when still on GOA defaults. +apply_ladder_stack_defaults() { + [ "${LADDER_STACK_AUTO}" = "1" ] || return 0 + if [ "${CUR}" != "TESTPAYSAN" ]; then + case "${TALER_DOMAIN:-}" in + stage.lefrancpaysan.ch|stage.bank.lefrancpaysan.ch|stage.exchange.lefrancpaysan.ch|stage.monnaie.lefrancpaysan.ch) + CUR="TESTPAYSAN" + ;; + *) return 0 ;; + esac + fi + + local bank_cfg max_wire min_denom + bank_cfg=$(curl -sS -m 12 "${BANK}/config" 2>/dev/null || true) + max_wire=$(printf '%s' "$bank_cfg" | python3 -c ' +import json,sys +try: + d=json.load(sys.stdin) +except Exception: + print("") + raise SystemExit(0) +a=d.get("max_wire_transfer_amount") or "" +print(a.split(":",1)[-1] if a else "") +' 2>/dev/null || true) + + min_denom=$(curl -sS -m 25 -H 'Accept: application/json' "${EX%/}/keys" 2>/dev/null | python3 -c ' +import json,sys +try: + d=json.load(sys.stdin) +except Exception: + print("") + raise SystemExit(0) +den=d.get("denoms") or d.get("denominations") or [] +vals=[] +for x in den if isinstance(den,list) else []: + v=x.get("value") or x.get("amount") or "" + if not v: continue + try: vals.append(float(str(v).split(":")[-1])) + except Exception: pass +print(min(vals) if vals else "") +' 2>/dev/null || true) + + # Fallback maxima if public config unreachable + [ -n "$max_wire" ] || max_wire="2000" + [ -n "$min_denom" ] || min_denom="0.01" + + # Only rewrite when caller left GOA defaults (explicit LADDER_MAX_AMOUNT=… kept) + if [ -n "$max_wire" ] && { [ "${LADDER_MAX_AMOUNT}" = "${GOA_LADDER_CEILING}" ] || [ "${LADDER_MAX_AMOUNT}" = "4503599627370496" ]; }; then + LADDER_MAX_AMOUNT="$max_wire" + fi + if [ -n "$min_denom" ] && { [ "${LADDER_MIN_AMOUNT}" = "0.000001" ] || [ "${LADDER_MIN_AMOUNT}" = "0.00000001" ]; }; then + LADDER_MIN_AMOUNT="$min_denom" + fi + # Slightly fewer rungs on stage (full GOA 23 still ok if user set LADDER_STEPS) + if [ "${LADDER_STEPS}" = "23" ]; then + LADDER_STEPS=15 + fi + # Stage farmer shops: private goa-demo instance does not exist + if [ "${MERCHANT_INSTANCE}" = "goa-demo-cp4zqk" ]; then + MERCHANT_INSTANCE="${LADDER_PAY_TEMPLATE_INSTANCE:-fermes-des-collines}" + INST="$MERCHANT_INSTANCE" + fi + # Prefer public templates for pays when no merchant token (set later) + : "${E2E_USE_TEMPLATES:=1}" + export E2E_USE_TEMPLATES + export LADDER_MAX_AMOUNT LADDER_MIN_AMOUNT LADDER_STEPS MERCHANT_INSTANCE + info "ladder stack" "TESTPAYSAN · max=${CUR}:${LADDER_MAX_AMOUNT} min=${CUR}:${LADDER_MIN_AMOUNT} steps=${LADDER_STEPS} (from bank max_wire / keys denoms)" +} +apply_ladder_stack_defaults SCRATCH=$(mktemp -d) WDB="$SCRATCH/wallet.sqlite3" REPORT_DIR="${LADDER_REPORT_DIR:-$SCRATCH}" @@ -82,13 +171,75 @@ ladder_over() { } wcli() { - if [ -f "$CLI_JS" ]; then + if [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then node "$CLI_JS" --wallet-db="$WDB" --no-throttle "$@" else taler-wallet-cli --wallet-db="$WDB" --no-throttle "$@" fi } +# Explorer pool password: env → file override → stack secrets → SSH +resolve_explorer_pw() { + local pw="" f="" host="" + if [ -n "${EXP_PW:-}" ]; then + printf '%s' "$EXP_PW" + return 0 + fi + if [ -n "${EXP_PW_FILE:-}" ] && [ -f "$EXP_PW_FILE" ]; then + tr -d '\n\r' <"$EXP_PW_FILE" + return 0 + fi + + # Stage TESTPAYSAN — never use GOA koopa-admin-secrets explorer pw first + if [ "${CUR}" = "TESTPAYSAN" ] \ + || [[ "${TALER_DOMAIN:-}" == stage.*lefrancpaysan* ]] \ + || [[ "${TALER_DOMAIN:-}" == *stage.lefrancpaysan* ]]; then + for f in \ + "${FRANCPAYSAN_SECRETS:-}/stage/bank-explorer-password.txt" \ + "${HOME}/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/src/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/taler/src/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/stage-bank-explorer-password.txt" + do + [ -n "$f" ] && [ -f "$f" ] || continue + tr -d '\n\r' <"$f" + return 0 + done + for host in \ + "${INSIDE_SSH:-}" \ + francpaysan-stage-user \ + francpaysan-host + do + [ -n "$host" ] || continue + pw=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \ + 'tr -d "\n\r" /dev/null \ + || sudo tr -d "\n\r" /dev/null' \ + 2>/dev/null || true) + if [ -n "$pw" ]; then + printf '%s' "$pw" + return 0 + fi + done + return 1 + fi + + # GOA / default: SECRETS_ROOT / ~/.config / koopa SSH + if pw=$(read_secret "taler-bank/bank-explorer-password.txt" 2>/dev/null) && [ -n "$pw" ]; then + printf '%s' "$pw" + return 0 + fi + for f in \ + "${SECRETS_ROOT:+${SECRETS_ROOT}/taler-bank/bank-explorer-password.txt}" \ + "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/bank-explorer-password.txt" + do + [ -n "$f" ] && [ -f "$f" ] || continue + tr -d '\n\r' <"$f" + return 0 + done + return 1 +} + wallet_avail() { wcli balance 2>/dev/null | python3 -c ' import json,sys @@ -248,7 +399,7 @@ if [ "${LADDER_LOAD:-1}" = "0" ]; then export METRICS_LOAD fi -section "ladder · GOA withdraw (0 → random → max · ${LADDER_STEPS} steps)" +section "ladder · ${CUR} withdraw (0 → random → max · ${LADDER_STEPS} steps)" info "bank" "$BANK" info "exchange" "$EX" info "currency" "$CUR" @@ -259,10 +410,11 @@ else warn "alt_unit_names" "using built-in SI fallback ($ALT_UNITS_FILE)" fi info "budget" "${LADDER_TIMEOUT_S}s" -_max_m1=$(python3 -c 'import sys; print(int(sys.argv[1])-1)' "${LADDER_MAX_AMOUNT}" 2>/dev/null || echo "${LADDER_MAX_AMOUNT}-1") +_max_m1=$(python3 -c 'import sys; from decimal import Decimal; m=Decimal(sys.argv[1]); print(m-1 if m>1 else m)' "${LADDER_MAX_AMOUNT}" 2>/dev/null || echo "${LADDER_MAX_AMOUNT}-1") info "steps" "${LADDER_STEPS} (0 + random≥${LADDER_MIN_AMOUNT} + max-1=${CUR}:${_max_m1} + max=${CUR}:${LADDER_MAX_AMOUNT})" info "withdraw_scale" "${LADDER_WITHDRAW_SCALE}× pay mids (fund pay ladder)" info "pay_phase" "$([ "${LADDER_PAY}" = "1" ] && echo enabled || echo disabled)" +info "currency/domain" "${CUR} · ${TALER_DOMAIN:-?} · bank=${BANK}" set_group load section "ladder · load snapshot (before withdraws)" @@ -270,11 +422,22 @@ metrics_report_load "$LOAD_BEFORE" "ladder-start" || true # Fresh wallet per rung — baseline empty (or last-rung DB if re-used later) metrics_report_coins "ladder-start" || true -if [ ! -f "$EXP_PW_FILE" ]; then - err bank "explorer password missing" "$EXP_PW_FILE" +if ! EXP_PW=$(resolve_explorer_pw); then + err bank "explorer password missing" \ + "set EXP_PW / EXP_PW_FILE or SECRETS_ROOT=…/koopa/host-root (taler-bank/bank-explorer-password.txt)${SECRETS_ROOT:+ · SECRETS_ROOT=${SECRETS_ROOT}}" + secrets_hint 2>/dev/null || true exit 1 fi -EXP_PW=$(tr -d '\n' <"$EXP_PW_FILE") +if [ -n "${SECRETS_ROOT:-}" ]; then + info "explorer secret" "resolved (SECRETS_ROOT=${SECRETS_ROOT} · stage uses stagepaysan secret first when CUR=TESTPAYSAN)" +else + info "explorer secret" "resolved via EXP_PW / EXP_PW_FILE / stage SSH / koopa" +fi +if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then + info "wallet-cli" "$CLI_JS" +else + info "wallet-cli" "PATH taler-wallet-cli ($(command -v taler-wallet-cli 2>/dev/null || echo missing))" +fi # --- auto-account --- t0=$(now_ms) @@ -288,7 +451,7 @@ if ! python3 -c 'import json; d=json.load(open("'"$SCRATCH"'/auto-account.json") exit 1 fi ACCT_USER=$(python3 -c 'import json; print(json.load(open("'"$SCRATCH"'/auto-account.json"))["username"])') -ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal GOA:0; pool=explorer" +ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal ${CUR}:0; pool=explorer" info "auto-account password" "(see $SCRATCH/auto-account.json — not logged)" # --- explorer token --- @@ -790,7 +953,7 @@ if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; set_group pay section "ladder · phase B · pay" metrics_report_coins "before-pay-ladder" || true - # Merchant secret (same as e2e) + # Merchant secret (same as e2e) — stage often has only public shop templates MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}" if [ -z "$MPW" ]; then MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true) @@ -798,11 +961,41 @@ if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; if [ -z "$MPW" ]; then MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true) fi + if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then + for f in \ + "${FRANCPAYSAN_SECRETS:-}/stage/default-instance-token.txt" \ + "${HOME}/francpaysan-secrets/stage/default-instance-token.txt" \ + "${HOME}/src/francpaysan-secrets/stage/default-instance-token.txt" + do + [ -f "$f" ] || continue + MPW=$(tr -d '\n\r' <"$f") + break + done + if [ -z "$MPW" ]; then + for host in "${INSIDE_SSH:-}" francpaysan-stage-user francpaysan-host; do + [ -n "$host" ] || continue + MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \ + 'tr -d "\n\r" /dev/null || true' \ + 2>/dev/null || true) + [ -n "$MPW" ] && break + done + fi + fi if [ -z "$MPW" ]; then - warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)" + if [ "${CUR}" = "TESTPAYSAN" ]; then + # Stage: withdraw ladder is the main probe; pays need instance token or + # public templates for *exact* face values (not continuous ladder amounts). + info pay "no merchant token — skip pay ladder on TESTPAYSAN (withdraw maxima still covered)" + info pay "hint: set E2E_MERCHANT_TOKEN or use ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e for shop templates" + else + warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)" + fi else ok "merchant token" "instance ${INST}" - AUTH="Authorization: Bearer secret-token:${MPW}" + case "$MPW" in + secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;; + *) AUTH="Authorization: Bearer secret-token:${MPW}" ;; + esac wallet_prepare "main" # shellcheck disable=SC2086 set -- $PAY_LIST diff --git a/scripts/taler-monitoring/check_urls.sh b/scripts/taler-monitoring/check_urls.sh index 387d4e2..36520d4 100755 --- a/scripts/taler-monitoring/check_urls.sh +++ b/scripts/taler-monitoring/check_urls.sh @@ -841,6 +841,97 @@ PY fi check_url_soft "merchant /webui/" 200 "$MERCHANT_PUBLIC/webui/" check_url_soft "merchant /" 302,301,200 "$MERCHANT_PUBLIC/" + + # --- merchant SPA fingerprints (selfbuild / package overlay) --- + # version.txt = SPA package version (e.g. 1.6.11) + # version-overlay.txt = optional deploy stamp (selfbuild-v1.6.11-0e4807845) + # Env: + # EXPECT_WEBUI_VERSION=1.6.11 → fail if version.txt ≠ + # EXPECT_WEBUI_OVERLAY=substr → fail if overlay missing that substr + # WEBUI_OVERLAY_DENY=regex → fail if overlay matches (e.g. master-11340) + # CHECK_WEBUI_SPA=0 → skip this block + if [ "${CHECK_WEBUI_SPA:-1}" = "1" ]; then + set_group webui + section "www · merchant SPA /webui/ fingerprints" + vcode=$(http_body "$MERCHANT_PUBLIC/webui/version.txt" "$tmp/webui-version.txt") + if [ "$vcode" = "200" ] && [ -s "$tmp/webui-version.txt" ]; then + got_ver=$(tr -d ' \n\r\t' <"$tmp/webui-version.txt") + ok "merchant /webui/version.txt" "HTTP 200 · ${got_ver}" + if [ -n "${EXPECT_WEBUI_VERSION:-}" ]; then + if [ "$got_ver" = "${EXPECT_WEBUI_VERSION}" ]; then + ok "merchant webui version pin" "EXPECT_WEBUI_VERSION=${EXPECT_WEBUI_VERSION}" + else + fail "merchant webui version pin" "got ${got_ver} want ${EXPECT_WEBUI_VERSION}" + fi + else + info "merchant webui version" "${got_ver} (set EXPECT_WEBUI_VERSION= to pin)" + fi + else + if [ "${LOCAL_STACK:-1}" = "1" ]; then + fail "merchant /webui/version.txt" "HTTP ${vcode:-000} (SPA missing version stamp)" + else + warn "merchant /webui/version.txt" "HTTP ${vcode:-000}" + fi + fi + + ocode=$(http_body "$MERCHANT_PUBLIC/webui/version-overlay.txt" "$tmp/webui-overlay.txt") + if [ "$ocode" = "200" ] && [ -s "$tmp/webui-overlay.txt" ]; then + got_ov=$(tr -d '\n\r' <"$tmp/webui-overlay.txt" | head -c 200) + ok "merchant /webui/version-overlay.txt" "HTTP 200 · ${got_ov}" + if [ -n "${EXPECT_WEBUI_OVERLAY:-}" ]; then + case "$got_ov" in + *"${EXPECT_WEBUI_OVERLAY}"*) + ok "merchant webui overlay pin" "contains ${EXPECT_WEBUI_OVERLAY}" + ;; + *) + fail "merchant webui overlay pin" "got «${got_ov}» want substr ${EXPECT_WEBUI_OVERLAY}" + ;; + esac + fi + if [ -n "${WEBUI_OVERLAY_DENY:-}" ]; then + if printf '%s' "$got_ov" | grep -qE -- "${WEBUI_OVERLAY_DENY}"; then + fail "merchant webui overlay deny" "«${got_ov}» matches WEBUI_OVERLAY_DENY=${WEBUI_OVERLAY_DENY}" + else + ok "merchant webui overlay deny" "no match for ${WEBUI_OVERLAY_DENY}" + fi + fi + else + # Overlay is optional for pure package installs; warn on local selfbuild stacks + if [ "${LOCAL_STACK:-1}" = "1" ]; then + warn "merchant /webui/version-overlay.txt" "HTTP ${ocode:-000} (optional; selfbuild writes it)" + else + info "merchant /webui/version-overlay.txt" "HTTP ${ocode:-000} (optional)" + fi + fi + + # Core SPA assets (paths used by selfbuild dist/prod). + # index.js is large — longer timeout; normalize curl "200000" (200 + ||000). + _webui_asset_code() { + local url="$1" tmo="${2:-${TIMEOUT}}" + local raw + raw=$(curl -skS --max-redirs 0 -m "$tmo" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || true) + # take first 3 digits only (avoid 200000 glue when curl exits non-zero after writing code) + printf '%s' "$raw" | tr -cd '0-9' | head -c 3 + [ -n "$(printf '%s' "$raw" | tr -cd '0-9')" ] || printf '000' + } + for asset in index.html index.js; do + tmo="${TIMEOUT}" + [ "$asset" = "index.js" ] && tmo="${WEBUI_INDEX_JS_TIMEOUT:-60}" + acode=$(_webui_asset_code "$MERCHANT_PUBLIC/webui/${asset}" "$tmo") + case "$acode" in + 200) ok "merchant /webui/${asset}" "HTTP 200" ;; + *) + if [ "${LOCAL_STACK:-1}" = "1" ]; then + fail "merchant /webui/${asset}" "HTTP ${acode:-000}" + else + warn "merchant /webui/${asset}" "HTTP ${acode:-000}" + fi + ;; + esac + done + # return group for any later merchant checks in this block + set_group merchant + fi fi # Merchant legal docs @@ -1236,7 +1327,8 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then fi fi -# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js + shops.css +# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js + +# shops.css + QR encoder (shop-ui modal needs /intro/qrcode.min.js or /qrcode.min.js). _ma=0 _ma_need=2 _ma_label="shop-pay.js + .css" @@ -1244,9 +1336,16 @@ if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.js" && _ma=$((_ma + 1)) _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.css" && _ma=$((_ma + 1)) else - _ma_label="shop-ui.js + shops.css" + # stage farmer shops: UI + stylesheet + qrcode (pay modal QR_Taler) + _ma_need=3 + _ma_label="shop-ui.js + shops.css + qrcode.min.js" _landing_probe "$MERCHANT_PUBLIC/assets/shop-ui.js" && _ma=$((_ma + 1)) _landing_probe "$MERCHANT_PUBLIC/assets/shops.css" && _ma=$((_ma + 1)) + # Prefer /intro/ (always on landings tree); root alias is Caddy-only convenience + if _landing_probe "$MERCHANT_PUBLIC/intro/qrcode.min.js" \ + || _landing_probe "$MERCHANT_PUBLIC/qrcode.min.js"; then + _ma=$((_ma + 1)) + fi fi if [ "$_ma" -eq "$_ma_need" ]; then ok "landing merchant shop assets" "${_ma_label}" @@ -1254,9 +1353,11 @@ else if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then warn "landing merchant shop assets" "${_ma}/${_ma_need} present (soft) · want ${_ma_label}" else - # stage: soft INFO if missing; WARN only if zero - if [ "$_ma" -eq 0 ]; then - warn "landing merchant shop assets" "0/${_ma_need} · want ${_ma_label}" + # stage: hard-ish — missing qrcode broke pay QR («bibliothèque manquante») + if [ "$_ma" -lt 2 ]; then + fail "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}" + elif [ "$_ma" -lt "$_ma_need" ]; then + warn "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}" else info "landing merchant shop assets" "${_ma}/${_ma_need} · ${_ma_label}" fi diff --git a/scripts/taler-monitoring/lib.sh b/scripts/taler-monitoring/lib.sh index 7bde111..841d6e8 100755 --- a/scripts/taler-monitoring/lib.sh +++ b/scripts/taler-monitoring/lib.sh @@ -14,6 +14,9 @@ # When LAN Host "koopa" is unreachable, try WAN DNAT (see ~/.ssh/config Host koopa-external). : "${KOOPA_SSH_FALLBACKS:=koopa-external}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" +# Merchant portal /private API: instance ids must be lowercase or you get odd 401s. +# (Display names may use capitals; path segment /instances/{id}/ must not.) +MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower:]') : "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses ATM notes : "${PAY_AMT:=GOA:0.01}" : "${CREDIT_AMT:=GOA:4700}" # covers ATM ladder 20+50+100+200+4200 (paivana) @@ -201,6 +204,7 @@ apply_taler_domain() { MERCHANT_INSTANCE=fermes-des-collines ;; esac + MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower:]') WITHDRAW_AMT="${WITHDRAW_AMT:-TESTPAYSAN:50}" PAY_AMT="${PAY_AMT:-TESTPAYSAN:5}" # Cover oeufs(5)+fromage(8.5)+jus(6)+shop pick(~5–12) + fees @@ -1115,8 +1119,11 @@ find_wallet_cli() { local c cand for c in \ "${HOME}/.local/bin/taler-wallet-cli.mjs" \ + "${HOME}/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ + "${HOME}/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ "${HOME}/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ /home/hernani/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \ + /home/hernani/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs diff --git a/scripts/taler-monitoring/secrets.env.example b/scripts/taler-monitoring/secrets.env.example index 064b14d..87ec88c 100644 --- a/scripts/taler-monitoring/secrets.env.example +++ b/scripts/taler-monitoring/secrets.env.example @@ -43,6 +43,40 @@ # PROGRESS_SHOW_EVERY=8 # PROGRESS_OFF=1 +# --- Merchant SPA /webui/ fingerprints (urls phase, group www.webui) --- +# EXPECT_WEBUI_VERSION=1.6.11 +# EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 +# WEBUI_OVERLAY_DENY=c778af04|master-11340 +# CHECK_WEBUI_SPA=0 + +# --- auth401 phase (WebUI signup/login/pwchange/admin + case matrix) --- +# Default FULL=1 creates throwaway MIX instance + runs all groups. +# Stage also loads durable mon401 for extra durable.* checks: +# AUTH401_FULL=1 +# AUTH401_CONTINUE=1 # optional: hard prereqs soft; full ERROR list always via EXIT trap +# CONTINUE_ON_ERROR=1 # alias for AUTH401_CONTINUE +# (auth401 no longer uses set -e — silent mid-run abort fixed) +# AUTH401_CREATE=1 +# AUTH401_PREFER_DURABLE=0 # 1 = only mon401, skip throwaway signup +# AUTH401_INSTANCE=mon401 +# AUTH401_PASSWORD= +# AUTH401_DURABLE_ID=mon401 +# AUTH401_CREATE_PASSWORD=Mon401-Test-Pass! +# AUTH401_SKIP_IF_MFA=1 +# AUTH401_BEARER= # default-instance-token (not admin) +# AUTH401_ADMIN_TOKEN= # required for admin-create group +# AUTH401_ADMIN_PASSWORD= # or login as AUTH401_ADMIN_USER=admin +# AUTH401_ADMIN_USER=admin +# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets + +# --- Ladder (GOA withdraw ladder · explorer pool) --- +# Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt). +# Overrides only if needed: +# EXP_USER=explorer +# EXP_PW= +# EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt +# CLI_JS=$HOME/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs + # Values that typically appear on a full stack (for orientation only): # bank-admin-password.txt # bank-explorer-password.txt diff --git a/scripts/taler-monitoring/taler-monitoring.sh b/scripts/taler-monitoring/taler-monitoring.sh index d8f6197..8b20e72 100755 --- a/scripts/taler-monitoring/taler-monitoring.sh +++ b/scripts/taler-monitoring/taler-monitoring.sh @@ -21,13 +21,16 @@ Usage: Phases: urls public HTTPS (no SSH) ← default with --domain + (includes merchant /webui/ SPA fingerprints: version.txt, overlay, assets) inside container status via SSH (local stack only) versions taler packages vs deb.taler.net trixie + repo availability sanity public + optional server server server-side only (SSH) e2e withdraw + pay (small amounts; remote aborts on login/KYC) - ladder GOA withdraw ladder (auto-account + explorer pool + timings) + ladder withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire) + auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance) all urls + inside + versions + sanity + e2e (SSH phases only on koopa) + full all + server + ladder + auth401 (maximum; long-running, needs secrets) Options: -d, --domain DOMAIN load profile from domains.conf (bank/exchange/merchant) @@ -63,6 +66,23 @@ Env (same meaning): QR_ECC=M qrencode ECC level (default M) KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external) METRICS_LOAD=0 skip host/container RAM/CPU probes (e2e/ladder/inside) + EXPECT_WEBUI_VERSION=1.6.11 pin merchant /webui/version.txt (urls) + EXPECT_WEBUI_OVERLAY=selfbuild require substr in version-overlay.txt + WEBUI_OVERLAY_DENY='master-11340' fail if overlay matches regex + CHECK_WEBUI_SPA=0 skip SPA fingerprint block in urls + AUTH401_* see check_auth401.sh / secrets.env.example + AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1 + auth401: run all groups, collect every ERROR (no mid-run halt) + +Full load (GOA / hacktivism): + ./taler-monitoring.sh -d hacktivism.ch full + # or explicit: + ./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401 + +SPA pin after selfbuild: + EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \\ + WEBUI_OVERLAY_DENY='c778af04|master-11340' \\ + ./taler-monitoring.sh -d hacktivism.ch urls EOF } @@ -110,7 +130,7 @@ while [ $# -gt 0 ]; do CURRENCY_OVERRIDE="$2"; shift 2 ;; --no-probe) NO_PROBE=1; shift ;; - urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|all) PHASES+=("$1"); shift ;; + urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|auth401|all|full) PHASES+=("$1"); shift ;; *) # bare domain shorthand: ./taler-monitoring.sh taler.net if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then @@ -183,6 +203,9 @@ export LADDER_MAX_AMOUNT LADDER_INCLUDE_ZERO LADDER_INCLUDE_MAX export LADDER_HIGH_FROM LADDER_HIGH_RUNGS LADDER_LOAD export LADDER_PAY LADDER_WITHDRAW_SCALE LADDER_PAY_SETTLE_ROUNDS export TALER_DOMAIN_APPLIED=1 +# Optional: auth401 and other phases may honor CONTINUE_ON_ERROR +export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-}" +export AUTH401_CONTINUE="${AUTH401_CONTINUE:-${CONTINUE_ON_ERROR:-}}" # Default phases # - local GOA (koopa): urls + inside + versions + e2e @@ -212,6 +235,15 @@ for p in "${PHASES[@]}"; do # remote without inside SSH: public + optional e2e OUT_PHASES+=(urls versions e2e) fi + elif [ "$p" = "full" ]; then + # Maximum coverage. Long. Needs secrets for e2e/ladder/auth401. + if [ "${LOCAL_STACK}" = "1" ]; then + OUT_PHASES+=(urls inside versions sanity server e2e ladder auth401) + elif [ -n "${INSIDE_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "stage-lfp" ]; then + OUT_PHASES+=(urls inside versions e2e ladder auth401) + else + OUT_PHASES+=(urls versions e2e auth401) + fi else OUT_PHASES+=("$p") fi @@ -247,7 +279,7 @@ if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then _pt=0 for p in "${PHASES[@]}"; do case "$p" in - urls) _pt=$((_pt + 85)) ;; # +qr group + urls) _pt=$((_pt + 95)) ;; # +qr + webui SPA fingerprints inside) _pt=$((_pt + 30)) ;; versions) _pt=$((_pt + 25)) ;; sanity) _pt=$((_pt + 35)) ;; @@ -255,6 +287,7 @@ if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then # e2e: ATM ladder emits many INFO lines (coins before/after each note) e2e) _pt=$((_pt + 240)) ;; ladder|goa-ladder) _pt=$((_pt + 120)) ;; + auth401) _pt=$((_pt + 70)) ;; esac done set_progress_total "$_pt" @@ -266,6 +299,7 @@ printf ' bank %s\n' "$BANK_PUBLIC" printf ' exchange %s\n' "$EXCHANGE_PUBLIC" printf ' merchant %s\n' "$MERCHANT_PUBLIC" printf ' currency expect=%s skip_ssh=%s\n' "${EXPECT_CURRENCY:-any}" "$SKIP_SSH" +printf ' phases %s\n' "${PHASES[*]}" printf ' progress total≈%s (set PROGRESS_TOTAL= to override; PROGRESS_OFF=1 to hide)\n' "${PROGRESS_TOTAL:-0}" chmod +x "$ROOT"/check_*.sh 2>/dev/null || true @@ -280,6 +314,7 @@ for p in "${PHASES[@]}"; do server) "$ROOT/check_server.sh" || ec=1 ;; e2e) "$ROOT/check_e2e.sh" || ec=1 ;; ladder|goa-ladder) "$ROOT/check_goa_ladder.sh" || ec=1 ;; + auth401) "$ROOT/check_auth401.sh" || ec=1 ;; esac done exit "$ec"