From c887617bc74d237922e9810a8ca03a9d2e1892a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 19:58:10 +0200 Subject: [PATCH 01/38] monitoring: resolve ladder explorer secret without laptop-hardcoded path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ladder used /Users/newkamek/… for bank-explorer password and wallet-cli. Use SECRETS_ROOT/read_secret, EXP_PW/EXP_PW_FILE overrides, and find_wallet_cli. Document Android automation prereqs; stage shop assets also require qrcode.min.js. --- .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 87 +++++++++++++++++++ scripts/taler-monitoring/README.md | 3 +- scripts/taler-monitoring/TESTS.md | 2 +- scripts/taler-monitoring/check_goa_ladder.sh | 65 ++++++++++++-- scripts/taler-monitoring/check_urls.sh | 20 +++-- scripts/taler-monitoring/lib.sh | 3 + scripts/taler-monitoring/secrets.env.example | 8 ++ 7 files changed, 174 insertions(+), 14 deletions(-) create mode 100644 scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md new file mode 100644 index 0000000..3253701 --- /dev/null +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -0,0 +1,87 @@ +# Git / automation notes (taler-monitoring + mobile) + +Notes for **automated checks** around the Taler stacks (GOA, stage +TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet +UI automation is separate tooling** and is **not** installed on the +default ops laptop yet. + +## What runs without Android + +| Phase | Needs | +|-------|--------| +| `urls` / `sanity` | public HTTPS only | +| `inside` / `versions` | SSH (`koopa` or `francpaysan-stage-user`) | +| `e2e` / `ladder` | `taler-wallet-cli` + bank/merchant secrets | + +```bash +./taler-monitoring.sh -d stage.lefrancpaysan.ch urls +./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env +``` + +Mobile app checks (manual withdraw/pay, then automation) are **out of +band** until the software below is present. + +--- + +## Android automation software — **required** (not installed yet) + +To automate **GNU Taler Wallet** on Android (stable + master builds; +manual then auto withdraw; fixed + variable pay) against stage +TESTPAYSAN, the following must be available on the runner host. + +### Status on this host (2026-07-17) + +| Component | Status | +|-----------|--------| +| Java (OpenJDK 21) | present (`/usr/bin/java`) | +| `aapt` / `aapt2` | present | +| **`adb`** (platform-tools) | **missing** (package was removed: `android-tools-adb` rc) | +| **Android SDK** (`ANDROID_HOME` / `ANDROID_SDK_ROOT`) | **missing** / empty | +| **Emulator + AVD** | **missing** | +| Appium / Maestro / `uiautomator2` | **missing** | +| scrcpy | missing (optional) | + +### Must install before mobile automation + +1. **Android platform-tools** — at least `adb` + - Debian: `android-tools-adb` (and preferably full SDK platform-tools) +2. **Android SDK command-line tools** — `sdkmanager`, platforms, build-tools + - set `ANDROID_HOME` (or `ANDROID_SDK_ROOT`) permanently +3. **System image + AVD** — for emulator runs (x86_64 or host-matching ABI) + - `emulator`, `avdmanager`, KVM/acceleration if available +4. **UI automation stack** (pick one, prefer the leaner path for wallet flows): + - **Maestro**, or + - **Appium** + **UiAutomator2** (+ Node/Python bindings as needed) +5. **Optional:** `scrcpy` for live view during manual debugging + +### Explicitly out of scope until freigabe + +- Installing production wallet APKs and driving real stage withdraw/pay + without a human go-ahead +- Committing secrets, device serials, or APK binaries into git + +### Operator still needs to supply + +- Stable / master **APK** (or store/build URLs) for the wallet +- USB debugging consent on a physical device, **or** agreement to use AVD only +- Which flows are in-scope first (withdraw communal QR → shop pay, …) + +### Suggested freigabe prompt (for an agent) + +See session notes: full copy-paste block under “Android-Automatisierung +vollständig einrichten” — install tooling, verify `adb devices` + emulator, +document paths; **no** real wallet payments until approved. + +--- + +## Related docs + +| Doc | Role | +|-----|------| +| [README.md](./README.md) | how to run `taler-monitoring.sh` | +| [TESTS.md](./TESTS.md) | grouped check IDs | +| [secrets.env.example](./secrets.env.example) | e2e secrets layout | +| `francpaysan-admin-log/TODO.md` | stage wallet mobile test backlog | + +When Android tooling is installed, append a short “installed on …” +section here (date, `adb version`, AVD name, `ANDROID_HOME`). diff --git a/scripts/taler-monitoring/README.md b/scripts/taler-monitoring/README.md index e627a35..c04508e 100644 --- a/scripts/taler-monitoring/README.md +++ b/scripts/taler-monitoring/README.md @@ -13,7 +13,8 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs** IDs reset per **group** inside an area — so “too many www tests” become `www.exchange-*`, `www.bank-*`, `www.landing-*`, etc. -Catalog: **[TESTS.md](./TESTS.md)**. +Catalog: **[TESTS.md](./TESTS.md)**. +Android UI automation prerequisites: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)** (tools **not** installed by default). ```text ┌ OK ┐ www.exchange-01 exchange /config · HTTP 200 diff --git a/scripts/taler-monitoring/TESTS.md b/scripts/taler-monitoring/TESTS.md index 7bd51c1..96e4512 100644 --- a/scripts/taler-monitoring/TESTS.md +++ b/scripts/taler-monitoring/TESTS.md @@ -53,7 +53,7 @@ Numbering follows **executed** checks (early skip may shift later NN inside the | **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` | | **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** | | **www.paivana-** | local GOA paywall front (redirect to template) | -| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; cross-links local | +| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local | | **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` | **QR rule:** `taler://withdraw/HOST/taler-integration/UUID` (no default `:443`/`:80`); `taler://pay/` / `pay-template/`; `payto://…` shape OK; decoded PNG must equal payload. Form errors on withdraw/pay/payto are **ERROR** on local/GOA. diff --git a/scripts/taler-monitoring/check_goa_ladder.sh b/scripts/taler-monitoring/check_goa_ladder.sh index 8b311ce..90a6f1b 100755 --- a/scripts/taler-monitoring/check_goa_ladder.sh +++ b/scripts/taler-monitoring/check_goa_ladder.sh @@ -21,13 +21,17 @@ # LADDER_TIMEOUT_S default 3600 # LADDER_LOAD=0 skip host load snapshots # LADDER_SETTLE_ROUNDS / LADDER_SETTLE_SLEEP — balance poll only (no shepherd) -# EXP_PW_FILE, LADDER_REPORT_DIR, MERCHANT_INSTANCE, … +# EXP_PW / EXP_PW_FILE / EXP_USER — explorer pool (default: secrets tree) +# CLI_JS / WALLET_CLI — wallet-cli .mjs (default: find_wallet_cli) +# LADDER_REPORT_DIR, MERCHANT_INSTANCE, … # # Phase: ./taler-monitoring.sh ladder set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) # shellcheck source=lib.sh source "$ROOT/lib.sh" +# When invoked standalone (not via taler-monitoring.sh), still load secrets.env +load_monitoring_secrets_env 2>/dev/null || true # Area ladder.* — GOA withdraw/pay ladder # Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report @@ -44,8 +48,10 @@ elapsed_ms() { : "${LADDER_SETTLE_ROUNDS:=18}" : "${LADDER_SETTLE_SLEEP:=2}" : "${EXP_USER:=explorer}" -: "${EXP_PW_FILE:=/Users/newkamek/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt}" -: "${CLI_JS:=/Users/newkamek/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs}" +# EXP_PW_FILE / EXP_PW optional overrides; otherwise read_secret + SECRETS_ROOT +: "${EXP_PW_FILE:=}" +: "${EXP_PW:=}" +: "${CLI_JS:=}" : "${LADDER_MAX_AMOUNT:=4503599627370496}" : "${LADDER_STEPS:=23}" : "${LADDER_LOAD:=1}" @@ -58,6 +64,15 @@ elapsed_ms() { : "${LADDER_PAY_SETTLE_ROUNDS:=6}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" +# Resolve wallet-cli .mjs (no hardcoded laptop path) +if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then + CLI_JS=$(find_wallet_cli 2>/dev/null || true) +fi +if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then + # allow PATH wrapper as last resort (wcli falls back to taler-wallet-cli) + CLI_JS="" +fi + CUR="${EXPECT_CURRENCY:-GOA}" BANK="${BANK_PUBLIC%/}" EX="${EXCHANGE_PUBLIC%/}/" @@ -82,13 +97,42 @@ ladder_over() { } wcli() { - if [ -f "$CLI_JS" ]; then + if [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then node "$CLI_JS" --wallet-db="$WDB" --no-throttle "$@" else taler-wallet-cli --wallet-db="$WDB" --no-throttle "$@" fi } +# Explorer pool password: env → file override → secrets tree → SSH koopa +resolve_explorer_pw() { + local pw="" f="" + if [ -n "${EXP_PW:-}" ]; then + printf '%s' "$EXP_PW" + return 0 + fi + if [ -n "${EXP_PW_FILE:-}" ] && [ -f "$EXP_PW_FILE" ]; then + tr -d '\n\r' <"$EXP_PW_FILE" + return 0 + fi + # Prefer shared read_secret (SECRETS_ROOT / ~/.config / koopa SSH) + if pw=$(read_secret "taler-bank/bank-explorer-password.txt" 2>/dev/null) && [ -n "$pw" ]; then + printf '%s' "$pw" + return 0 + fi + # Extra local candidates (sibling of admin-log, taler-landing config) + for f in \ + "${SECRETS_ROOT:+${SECRETS_ROOT}/taler-bank/bank-explorer-password.txt}" \ + "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/bank-explorer-password.txt" + do + [ -n "$f" ] && [ -f "$f" ] || continue + tr -d '\n\r' <"$f" + return 0 + done + return 1 +} + wallet_avail() { wcli balance 2>/dev/null | python3 -c ' import json,sys @@ -270,11 +314,18 @@ metrics_report_load "$LOAD_BEFORE" "ladder-start" || true # Fresh wallet per rung — baseline empty (or last-rung DB if re-used later) metrics_report_coins "ladder-start" || true -if [ ! -f "$EXP_PW_FILE" ]; then - err bank "explorer password missing" "$EXP_PW_FILE" +if ! EXP_PW=$(resolve_explorer_pw); then + err bank "explorer password missing" \ + "set EXP_PW / EXP_PW_FILE or SECRETS_ROOT=…/koopa/host-root (taler-bank/bank-explorer-password.txt)${SECRETS_ROOT:+ · SECRETS_ROOT=${SECRETS_ROOT}}" + secrets_hint 2>/dev/null || true exit 1 fi -EXP_PW=$(tr -d '\n' <"$EXP_PW_FILE") +info "explorer secret" "${SECRETS_ROOT:+SECRETS_ROOT=${SECRETS_ROOT}}${SECRETS_ROOT:-via EXP_PW/EXP_PW_FILE/SSH}" +if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then + info "wallet-cli" "$CLI_JS" +else + info "wallet-cli" "PATH taler-wallet-cli ($(command -v taler-wallet-cli 2>/dev/null || echo missing))" +fi # --- auto-account --- t0=$(now_ms) diff --git a/scripts/taler-monitoring/check_urls.sh b/scripts/taler-monitoring/check_urls.sh index 387d4e2..7191609 100755 --- a/scripts/taler-monitoring/check_urls.sh +++ b/scripts/taler-monitoring/check_urls.sh @@ -1236,7 +1236,8 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then fi fi -# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js + shops.css +# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js + +# shops.css + QR encoder (shop-ui modal needs /intro/qrcode.min.js or /qrcode.min.js). _ma=0 _ma_need=2 _ma_label="shop-pay.js + .css" @@ -1244,9 +1245,16 @@ if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.js" && _ma=$((_ma + 1)) _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.css" && _ma=$((_ma + 1)) else - _ma_label="shop-ui.js + shops.css" + # stage farmer shops: UI + stylesheet + qrcode (pay modal QR_Taler) + _ma_need=3 + _ma_label="shop-ui.js + shops.css + qrcode.min.js" _landing_probe "$MERCHANT_PUBLIC/assets/shop-ui.js" && _ma=$((_ma + 1)) _landing_probe "$MERCHANT_PUBLIC/assets/shops.css" && _ma=$((_ma + 1)) + # Prefer /intro/ (always on landings tree); root alias is Caddy-only convenience + if _landing_probe "$MERCHANT_PUBLIC/intro/qrcode.min.js" \ + || _landing_probe "$MERCHANT_PUBLIC/qrcode.min.js"; then + _ma=$((_ma + 1)) + fi fi if [ "$_ma" -eq "$_ma_need" ]; then ok "landing merchant shop assets" "${_ma_label}" @@ -1254,9 +1262,11 @@ else if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then warn "landing merchant shop assets" "${_ma}/${_ma_need} present (soft) · want ${_ma_label}" else - # stage: soft INFO if missing; WARN only if zero - if [ "$_ma" -eq 0 ]; then - warn "landing merchant shop assets" "0/${_ma_need} · want ${_ma_label}" + # stage: hard-ish — missing qrcode broke pay QR («bibliothèque manquante») + if [ "$_ma" -lt 2 ]; then + fail "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}" + elif [ "$_ma" -lt "$_ma_need" ]; then + warn "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}" else info "landing merchant shop assets" "${_ma}/${_ma_need} · ${_ma_label}" fi diff --git a/scripts/taler-monitoring/lib.sh b/scripts/taler-monitoring/lib.sh index 7bde111..a113212 100755 --- a/scripts/taler-monitoring/lib.sh +++ b/scripts/taler-monitoring/lib.sh @@ -1115,8 +1115,11 @@ find_wallet_cli() { local c cand for c in \ "${HOME}/.local/bin/taler-wallet-cli.mjs" \ + "${HOME}/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ + "${HOME}/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ "${HOME}/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ /home/hernani/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \ + /home/hernani/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \ /usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs diff --git a/scripts/taler-monitoring/secrets.env.example b/scripts/taler-monitoring/secrets.env.example index 064b14d..212a7f2 100644 --- a/scripts/taler-monitoring/secrets.env.example +++ b/scripts/taler-monitoring/secrets.env.example @@ -43,6 +43,14 @@ # PROGRESS_SHOW_EVERY=8 # PROGRESS_OFF=1 +# --- Ladder (GOA withdraw ladder · explorer pool) --- +# Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt). +# Overrides only if needed: +# EXP_USER=explorer +# EXP_PW= +# EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt +# CLI_JS=$HOME/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs + # Values that typically appear on a full stack (for orientation only): # bank-admin-password.txt # bank-explorer-password.txt From c711f14a92055b870515bc6e58375686007b1b7c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:02:34 +0200 Subject: [PATCH 02/38] monitoring: stage TESTPAYSAN amount ladder (bank max_wire maxima) Reuse check_goa_ladder for stage.*lefrancpaysan.ch: auto max from max_wire_transfer_amount (2000), min denom from /keys, explorer secret via stagepaysan SSH, fewer default steps. Document usage in README. --- scripts/taler-monitoring/README.md | 8 +- scripts/taler-monitoring/TESTS.md | 2 +- scripts/taler-monitoring/check_goa_ladder.sh | 214 +++++++++++++++---- scripts/taler-monitoring/taler-monitoring.sh | 2 +- 4 files changed, 187 insertions(+), 39 deletions(-) diff --git a/scripts/taler-monitoring/README.md b/scripts/taler-monitoring/README.md index c04508e..35dba91 100644 --- a/scripts/taler-monitoring/README.md +++ b/scripts/taler-monitoring/README.md @@ -198,7 +198,13 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC. ```bash ./taler-monitoring.sh e2e ./taler-monitoring.sh -d taler.net urls e2e -# customize: +# Amount ladder (withdraw 0 → … → max, then pay): same script for GOA + stage +./taler-monitoring.sh ladder # GOA / current domain +./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima +# Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01). +# Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=). +# Override: LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 … +# customize e2e: E2E_WITHDRAW_VALUES="20 50 100" E2E_PAY_VALUES="0.05 1 5" ./taler-monitoring.sh e2e E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e # single fixed # GOA shop catalog (local hacktivism): full list in E2E_SHOP_PRODUCTS; each run diff --git a/scripts/taler-monitoring/TESTS.md b/scripts/taler-monitoring/TESTS.md index 96e4512..4ad45e9 100644 --- a/scripts/taler-monitoring/TESTS.md +++ b/scripts/taler-monitoring/TESTS.md @@ -114,7 +114,7 @@ When filing an issue, quote the full id + label, e.g. | **versions.inside-** | packages in containers | | **versions.compare-** | installed vs suite | | **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component | -| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | GOA amount ladder | +| **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder (GOA ceiling or stage TESTPAYSAN `max_wire`) | | **server-** | SSH host ports / processes (flat unless grouped later) | --- diff --git a/scripts/taler-monitoring/check_goa_ladder.sh b/scripts/taler-monitoring/check_goa_ladder.sh index 90a6f1b..d7c60ae 100755 --- a/scripts/taler-monitoring/check_goa_ladder.sh +++ b/scripts/taler-monitoring/check_goa_ladder.sh @@ -1,31 +1,27 @@ #!/usr/bin/env bash -# check_goa_ladder.sh — GOA withdraw ladder for taler-monitoring +# check_goa_ladder.sh — withdraw/pay amount ladder (GOA + stage TESTPAYSAN) # -# bank.hacktivism.ch flow (landing): -# 1) GET /intro/auto-account.json → personal goa-account-* (GOA:0) -# 2) Mint pool withdrawals as explorer (shared pool) + confirm when selected -# 3) wallet-cli accept-uri only (no run-until-done — hangs / developer ban) -# 4) bank confirm when selected; settle = poll balance + bank transfer_done +# Flow (bank landings): +# 1) GET /intro/auto-account.json → personal *account-* (balance 0) +# 2) Mint pool withdrawals as explorer + confirm when selected +# 3) wallet-cli accept-uri only (no run-until-done) +# 4) bank confirm when selected; settle = balance + transfer_done # -# Amounts: random strictly increasing; fixed pins 0, max-1, max (23 default). -# Phase A: withdraw ladder into ONE cumulative wallet (mids scaled up for pay budget). -# Phase B: pay ladder 0 → low → … → max-1 → max (same step count). -# On first hard failure: stop, print timing report, exit 1. -# Soft: GOA:0 / wallet 7006 / absolute max (CEILING_REJECT) → WARN and continue. +# Amounts: random strictly increasing; pins 0, max-1, max. +# Phase A: withdraw ladder (cumulative wallet). Phase B: pay ladder. # -# Env: -# LADDER_STEPS total rungs (default 23) = 0 + (N-3) random + max-1 + max -# LADDER_MAX_AMOUNT absolute last pin (libeufin ceiling 4503599627370496) -# LADDER_WITHDRAW_SCALE mid withdraw amounts × this vs pay mids (default 1.5) -# LADDER_PAY=0 skip payment phase -# LADDER_TIMEOUT_S default 3600 -# LADDER_LOAD=0 skip host load snapshots -# LADDER_SETTLE_ROUNDS / LADDER_SETTLE_SLEEP — balance poll only (no shepherd) -# EXP_PW / EXP_PW_FILE / EXP_USER — explorer pool (default: secrets tree) -# CLI_JS / WALLET_CLI — wallet-cli .mjs (default: find_wallet_cli) -# LADDER_REPORT_DIR, MERCHANT_INSTANCE, … +# Stacks: +# GOA — LADDER_MAX_AMOUNT = libeufin ceiling; explorer from koopa-admin-secrets +# TESTPAYSAN stage — auto max_wire from bank /config, min denom from /keys; +# explorer from stagepaysan secrets / SSH # -# Phase: ./taler-monitoring.sh ladder +# Usage: +# ./taler-monitoring.sh ladder +# ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder +# LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder +# +# Env (see body): LADDER_STEPS, LADDER_MAX_AMOUNT, LADDER_MIN_AMOUNT, LADDER_STACK_AUTO, +# LADDER_PAY, EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, … set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) # shellcheck source=lib.sh @@ -33,7 +29,7 @@ source "$ROOT/lib.sh" # When invoked standalone (not via taler-monitoring.sh), still load secrets.env load_monitoring_secrets_env 2>/dev/null || true -# Area ladder.* — GOA withdraw/pay ladder +# Area ladder.* — withdraw/pay amount ladder (GOA + TESTPAYSAN) # Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report set_area ladder set_group plan @@ -48,14 +44,16 @@ elapsed_ms() { : "${LADDER_SETTLE_ROUNDS:=18}" : "${LADDER_SETTLE_SLEEP:=2}" : "${EXP_USER:=explorer}" -# EXP_PW_FILE / EXP_PW optional overrides; otherwise read_secret + SECRETS_ROOT +# EXP_PW_FILE / EXP_PW optional overrides; otherwise read_secret + SECRETS_ROOT / stage SSH : "${EXP_PW_FILE:=}" : "${EXP_PW:=}" : "${CLI_JS:=}" +# GOA libeufin amount ceiling (hacktivism). Stage auto-replaces via bank /config. : "${LADDER_MAX_AMOUNT:=4503599627370496}" : "${LADDER_STEPS:=23}" : "${LADDER_LOAD:=1}" : "${LADDER_PAY:=1}" +: "${LADDER_STACK_AUTO:=1}" # Withdraw mids = pay mids × scale (so wallet can afford the pay ladder) : "${LADDER_WITHDRAW_SCALE:=1.5}" # Floor for random mids (must be ≥ smallest exchange coin; GOA min denom ≈ 0.000001) @@ -63,6 +61,11 @@ elapsed_ms() { : "${LADDER_CONFIRM_POLLS:=40}" : "${LADDER_PAY_SETTLE_ROUNDS:=6}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" +# Public variable template for stage pays (optional; fixed templates used when amount maps) +: "${LADDER_PAY_TEMPLATE:=}" +: "${LADDER_PAY_TEMPLATE_INSTANCE:=fermes-des-collines}" + +GOA_LADDER_CEILING="4503599627370496" # Resolve wallet-cli .mjs (no hardcoded laptop path) if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then @@ -78,6 +81,77 @@ BANK="${BANK_PUBLIC%/}" EX="${EXCHANGE_PUBLIC%/}/" MER="${MERCHANT_PUBLIC%/}" INST="${MERCHANT_INSTANCE}" + +# --- Stack-specific ladder maxima (TESTPAYSAN stage) --- +# Uses bank max_wire_transfer_amount and exchange min denom when still on GOA defaults. +apply_ladder_stack_defaults() { + [ "${LADDER_STACK_AUTO}" = "1" ] || return 0 + if [ "${CUR}" != "TESTPAYSAN" ]; then + case "${TALER_DOMAIN:-}" in + stage.lefrancpaysan.ch|stage.bank.lefrancpaysan.ch|stage.exchange.lefrancpaysan.ch|stage.monnaie.lefrancpaysan.ch) + CUR="TESTPAYSAN" + ;; + *) return 0 ;; + esac + fi + + local bank_cfg max_wire min_denom + bank_cfg=$(curl -sS -m 12 "${BANK}/config" 2>/dev/null || true) + max_wire=$(printf '%s' "$bank_cfg" | python3 -c ' +import json,sys +try: + d=json.load(sys.stdin) +except Exception: + print("") + raise SystemExit(0) +a=d.get("max_wire_transfer_amount") or "" +print(a.split(":",1)[-1] if a else "") +' 2>/dev/null || true) + + min_denom=$(curl -sS -m 25 -H 'Accept: application/json' "${EX%/}/keys" 2>/dev/null | python3 -c ' +import json,sys +try: + d=json.load(sys.stdin) +except Exception: + print("") + raise SystemExit(0) +den=d.get("denoms") or d.get("denominations") or [] +vals=[] +for x in den if isinstance(den,list) else []: + v=x.get("value") or x.get("amount") or "" + if not v: continue + try: vals.append(float(str(v).split(":")[-1])) + except Exception: pass +print(min(vals) if vals else "") +' 2>/dev/null || true) + + # Fallback maxima if public config unreachable + [ -n "$max_wire" ] || max_wire="2000" + [ -n "$min_denom" ] || min_denom="0.01" + + # Only rewrite when caller left GOA defaults (explicit LADDER_MAX_AMOUNT=… kept) + if [ -n "$max_wire" ] && { [ "${LADDER_MAX_AMOUNT}" = "${GOA_LADDER_CEILING}" ] || [ "${LADDER_MAX_AMOUNT}" = "4503599627370496" ]; }; then + LADDER_MAX_AMOUNT="$max_wire" + fi + if [ -n "$min_denom" ] && { [ "${LADDER_MIN_AMOUNT}" = "0.000001" ] || [ "${LADDER_MIN_AMOUNT}" = "0.00000001" ]; }; then + LADDER_MIN_AMOUNT="$min_denom" + fi + # Slightly fewer rungs on stage (full GOA 23 still ok if user set LADDER_STEPS) + if [ "${LADDER_STEPS}" = "23" ]; then + LADDER_STEPS=15 + fi + # Stage farmer shops: private goa-demo instance does not exist + if [ "${MERCHANT_INSTANCE}" = "goa-demo-cp4zqk" ]; then + MERCHANT_INSTANCE="${LADDER_PAY_TEMPLATE_INSTANCE:-fermes-des-collines}" + INST="$MERCHANT_INSTANCE" + fi + # Prefer public templates for pays when no merchant token (set later) + : "${E2E_USE_TEMPLATES:=1}" + export E2E_USE_TEMPLATES + export LADDER_MAX_AMOUNT LADDER_MIN_AMOUNT LADDER_STEPS MERCHANT_INSTANCE + info "ladder stack" "TESTPAYSAN · max=${CUR}:${LADDER_MAX_AMOUNT} min=${CUR}:${LADDER_MIN_AMOUNT} steps=${LADDER_STEPS} (from bank max_wire / keys denoms)" +} +apply_ladder_stack_defaults SCRATCH=$(mktemp -d) WDB="$SCRATCH/wallet.sqlite3" REPORT_DIR="${LADDER_REPORT_DIR:-$SCRATCH}" @@ -104,9 +178,9 @@ wcli() { fi } -# Explorer pool password: env → file override → secrets tree → SSH koopa +# Explorer pool password: env → file override → stack secrets → SSH resolve_explorer_pw() { - local pw="" f="" + local pw="" f="" host="" if [ -n "${EXP_PW:-}" ]; then printf '%s' "$EXP_PW" return 0 @@ -115,12 +189,45 @@ resolve_explorer_pw() { tr -d '\n\r' <"$EXP_PW_FILE" return 0 fi - # Prefer shared read_secret (SECRETS_ROOT / ~/.config / koopa SSH) + + # Stage TESTPAYSAN — never use GOA koopa-admin-secrets explorer pw first + if [ "${CUR}" = "TESTPAYSAN" ] \ + || [[ "${TALER_DOMAIN:-}" == stage.*lefrancpaysan* ]] \ + || [[ "${TALER_DOMAIN:-}" == *stage.lefrancpaysan* ]]; then + for f in \ + "${FRANCPAYSAN_SECRETS:-}/stage/bank-explorer-password.txt" \ + "${HOME}/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/src/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/taler/src/francpaysan-secrets/stage/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/stage-bank-explorer-password.txt" + do + [ -n "$f" ] && [ -f "$f" ] || continue + tr -d '\n\r' <"$f" + return 0 + done + for host in \ + "${INSIDE_SSH:-}" \ + francpaysan-stage-user \ + francpaysan-host + do + [ -n "$host" ] || continue + pw=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \ + 'tr -d "\n\r" /dev/null \ + || sudo tr -d "\n\r" /dev/null' \ + 2>/dev/null || true) + if [ -n "$pw" ]; then + printf '%s' "$pw" + return 0 + fi + done + return 1 + fi + + # GOA / default: SECRETS_ROOT / ~/.config / koopa SSH if pw=$(read_secret "taler-bank/bank-explorer-password.txt" 2>/dev/null) && [ -n "$pw" ]; then printf '%s' "$pw" return 0 fi - # Extra local candidates (sibling of admin-log, taler-landing config) for f in \ "${SECRETS_ROOT:+${SECRETS_ROOT}/taler-bank/bank-explorer-password.txt}" \ "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \ @@ -292,7 +399,7 @@ if [ "${LADDER_LOAD:-1}" = "0" ]; then export METRICS_LOAD fi -section "ladder · GOA withdraw (0 → random → max · ${LADDER_STEPS} steps)" +section "ladder · ${CUR} withdraw (0 → random → max · ${LADDER_STEPS} steps)" info "bank" "$BANK" info "exchange" "$EX" info "currency" "$CUR" @@ -303,10 +410,11 @@ else warn "alt_unit_names" "using built-in SI fallback ($ALT_UNITS_FILE)" fi info "budget" "${LADDER_TIMEOUT_S}s" -_max_m1=$(python3 -c 'import sys; print(int(sys.argv[1])-1)' "${LADDER_MAX_AMOUNT}" 2>/dev/null || echo "${LADDER_MAX_AMOUNT}-1") +_max_m1=$(python3 -c 'import sys; from decimal import Decimal; m=Decimal(sys.argv[1]); print(m-1 if m>1 else m)' "${LADDER_MAX_AMOUNT}" 2>/dev/null || echo "${LADDER_MAX_AMOUNT}-1") info "steps" "${LADDER_STEPS} (0 + random≥${LADDER_MIN_AMOUNT} + max-1=${CUR}:${_max_m1} + max=${CUR}:${LADDER_MAX_AMOUNT})" info "withdraw_scale" "${LADDER_WITHDRAW_SCALE}× pay mids (fund pay ladder)" info "pay_phase" "$([ "${LADDER_PAY}" = "1" ] && echo enabled || echo disabled)" +info "currency/domain" "${CUR} · ${TALER_DOMAIN:-?} · bank=${BANK}" set_group load section "ladder · load snapshot (before withdraws)" @@ -320,7 +428,11 @@ if ! EXP_PW=$(resolve_explorer_pw); then secrets_hint 2>/dev/null || true exit 1 fi -info "explorer secret" "${SECRETS_ROOT:+SECRETS_ROOT=${SECRETS_ROOT}}${SECRETS_ROOT:-via EXP_PW/EXP_PW_FILE/SSH}" +if [ -n "${SECRETS_ROOT:-}" ]; then + info "explorer secret" "resolved (SECRETS_ROOT=${SECRETS_ROOT} · stage uses stagepaysan secret first when CUR=TESTPAYSAN)" +else + info "explorer secret" "resolved via EXP_PW / EXP_PW_FILE / stage SSH / koopa" +fi if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then info "wallet-cli" "$CLI_JS" else @@ -339,7 +451,7 @@ if ! python3 -c 'import json; d=json.load(open("'"$SCRATCH"'/auto-account.json") exit 1 fi ACCT_USER=$(python3 -c 'import json; print(json.load(open("'"$SCRATCH"'/auto-account.json"))["username"])') -ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal GOA:0; pool=explorer" +ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal ${CUR}:0; pool=explorer" info "auto-account password" "(see $SCRATCH/auto-account.json — not logged)" # --- explorer token --- @@ -841,7 +953,7 @@ if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; set_group pay section "ladder · phase B · pay" metrics_report_coins "before-pay-ladder" || true - # Merchant secret (same as e2e) + # Merchant secret (same as e2e) — stage often has only public shop templates MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}" if [ -z "$MPW" ]; then MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true) @@ -849,11 +961,41 @@ if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; if [ -z "$MPW" ]; then MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true) fi + if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then + for f in \ + "${FRANCPAYSAN_SECRETS:-}/stage/default-instance-token.txt" \ + "${HOME}/francpaysan-secrets/stage/default-instance-token.txt" \ + "${HOME}/src/francpaysan-secrets/stage/default-instance-token.txt" + do + [ -f "$f" ] || continue + MPW=$(tr -d '\n\r' <"$f") + break + done + if [ -z "$MPW" ]; then + for host in "${INSIDE_SSH:-}" francpaysan-stage-user francpaysan-host; do + [ -n "$host" ] || continue + MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \ + 'tr -d "\n\r" /dev/null || true' \ + 2>/dev/null || true) + [ -n "$MPW" ] && break + done + fi + fi if [ -z "$MPW" ]; then - warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)" + if [ "${CUR}" = "TESTPAYSAN" ]; then + # Stage: withdraw ladder is the main probe; pays need instance token or + # public templates for *exact* face values (not continuous ladder amounts). + info pay "no merchant token — skip pay ladder on TESTPAYSAN (withdraw maxima still covered)" + info pay "hint: set E2E_MERCHANT_TOKEN or use ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e for shop templates" + else + warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)" + fi else ok "merchant token" "instance ${INST}" - AUTH="Authorization: Bearer secret-token:${MPW}" + case "$MPW" in + secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;; + *) AUTH="Authorization: Bearer secret-token:${MPW}" ;; + esac wallet_prepare "main" # shellcheck disable=SC2086 set -- $PAY_LIST diff --git a/scripts/taler-monitoring/taler-monitoring.sh b/scripts/taler-monitoring/taler-monitoring.sh index d8f6197..e63c922 100755 --- a/scripts/taler-monitoring/taler-monitoring.sh +++ b/scripts/taler-monitoring/taler-monitoring.sh @@ -26,7 +26,7 @@ Phases: sanity public + optional server server server-side only (SSH) e2e withdraw + pay (small amounts; remote aborts on login/KYC) - ladder GOA withdraw ladder (auto-account + explorer pool + timings) + ladder withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire) all urls + inside + versions + sanity + e2e (SSH phases only on koopa) Options: From 9a07eb1dfec093cf855d744731470c00cbed3e34 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:03:46 +0200 Subject: [PATCH 03/38] =?UTF-8?q?docs:=20git=20commit/push=20cadence=207?= =?UTF-8?q?=E2=80=9311=20min=20+=20helper=20script?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Match historical area: message style; optional cadence commit/push helper. --- scripts/git-cadence-commit-push.sh | 116 ++++++++++++++++++ .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 48 ++++++++ 2 files changed, 164 insertions(+) create mode 100755 scripts/git-cadence-commit-push.sh diff --git a/scripts/git-cadence-commit-push.sh b/scripts/git-cadence-commit-push.sh new file mode 100755 index 0000000..1276fc1 --- /dev/null +++ b/scripts/git-cadence-commit-push.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# git-cadence-commit-push.sh — optional WIP commit + push on the historical cadence +# +# Model (from koopa-admin-log / francpaysan-admin-log histories, 2026-07): +# - message: area: short description (monitoring: … / landings: … / docs: …) +# - size: small logical steps, not giant dumps +# - interval: about **7–11 minutes** while work is in progress (sometimes denser) +# - push: after each commit when the remote accepts credentials +# +# Usage: +# ./scripts/git-cadence-commit-push.sh # this repo +# ./scripts/git-cadence-commit-push.sh /path/to/repo +# CADENCE_MSG='monitoring: wip ladder' ./scripts/git-cadence-commit-push.sh +# CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh # show only +# +# Cron / agent every ~9 min: +# */9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh +# */9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh \ +# $HOME/taler/src/francpaysan-admin-log +# +set -euo pipefail + +REPO="${1:-}" +if [ -z "$REPO" ]; then + REPO="$(cd "$(dirname "$0")/.." && pwd)" +fi +REPO="$(cd "$REPO" && pwd)" +cd "$REPO" + +CADENCE_DRY="${CADENCE_DRY:-0}" +# Never auto-add secrets +SKIP_GLOBS=( + 'secrets.env' + '**/secrets.env' + '**/*password*' + '**/*secret*' + '**/*.pem' + '**/*token*' +) + +if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "not a git repo: $REPO" >&2 + exit 1 +fi + +branch=$(git rev-parse --abbrev-ref HEAD) +remote=$(git remote 2>/dev/null | head -1 || true) +: "${remote:=origin}" + +# Dirty? (tracked + untracked, ignoring unreadable) +if git diff --quiet && git diff --cached --quiet \ + && [ -z "$(git ls-files --others --exclude-standard)" ]; then + echo "clean · $REPO · $branch (nothing to commit)" + # still try push if ahead + ahead=$(git rev-list --count "${remote}/${branch}..HEAD" 2>/dev/null || echo 0) + if [ "${ahead:-0}" -gt 0 ]; then + echo "ahead of ${remote}/${branch} by $ahead — push" + if [ "$CADENCE_DRY" = "1" ]; then + echo "DRY: git push $remote $branch" + else + git push "$remote" "$branch" || echo "WARN: push failed (auth?)" >&2 + fi + fi + exit 0 +fi + +# Stage safe paths only +git add -A +for g in "${SKIP_GLOBS[@]}"; do + git reset -q -- "$g" 2>/dev/null || true +done +# unstage anything still matching secret names +while IFS= read -r f; do + case "$f" in + *secret*|*password*|*token*|secrets.env) + git reset -q -- "$f" 2>/dev/null || true + ;; + esac +done < <(git diff --cached --name-only 2>/dev/null || true) + +if git diff --cached --quiet; then + echo "only secret/skip paths dirty — not committing · $REPO" + exit 0 +fi + +# Message from paths (historical area: style) +msg="${CADENCE_MSG:-}" +if [ -z "$msg" ]; then + files=$(git diff --cached --name-only | head -20) + area="wip" + case "$files" in + *taler-monitoring*|*scripts/taler-monitoring*) area="monitoring" ;; + *landing*|*landings*|*configs/*landing*) area="landing" ;; + *docs*|*README*|*TODO*|*NOTES*) area="docs" ;; + *caddy*) area="caddy" ;; + *scripts*) area="scripts" ;; + esac + n=$(git diff --cached --name-only | wc -l | tr -d ' ') + sample=$(git diff --cached --name-only | head -1 | xargs -r basename) + msg="${area}: cadence ${n} file(s) · ${sample}" +fi + +if [ "$CADENCE_DRY" = "1" ]; then + echo "DRY commit: $msg" + git diff --cached --stat + echo "DRY push: $remote $branch" + exit 0 +fi + +git commit -m "$msg" +echo "committed · $msg" +if ! git push "$remote" "$branch"; then + echo "WARN: push failed for $REPO ($remote $branch) — commit is local" >&2 + exit 0 +fi +echo "pushed · $remote $branch" diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index 3253701..fe6fd1c 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -5,6 +5,54 @@ TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet UI automation is separate tooling** and is **not** installed on the default ops laptop yet. +--- + +## Git commit / push cadence (existing model) + +Observed in `koopa-admin-log` and `francpaysan-admin-log` histories +(active work on 2026-07-17): + +| Rule | Practice | +|------|----------| +| **Interval** | About **7–11 minutes** between publishable steps while a topic is open (bursts of 0–2 min are fine for tiny follow-ups) | +| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …`, `scripts: …`, `todo: …` | +| **Size** | One logical change (or a tight bundle), not end-of-day megacommits | +| **Push** | After each commit when the remote accepts auth (`git push origin …`) | + +### Agent / human workflow + +1. Finish a small slice of work. +2. `git status` / diff → commit with the area-prefix style. +3. `git push` (or run the helper below). +4. Continue; next commit window ~**7–11 min** later (or sooner if the next slice is ready). + +### Helper script + +From the admin-log root: + +```bash +# dry-run +CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh + +# commit + push this repo if dirty (skips secrets.env / *password* / *token*) +./scripts/git-cadence-commit-push.sh + +# other checkout +./scripts/git-cadence-commit-push.sh ~/taler/src/francpaysan-admin-log +``` + +Optional timer (every 9 min ≈ mid of 7–11): + +```bash +# user crontab example +*/9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1 +*/9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh $HOME/taler/src/francpaysan-admin-log >>$HOME/.cache/git-cadence-lfp.log 2>&1 +``` + +**Auth note (this host):** `francpaysan-admin-log` → `ssh://git@git.gnunet.org/…` works. +`koopa-admin-log` → `https://git.hacktivism.ch/…` needs credentials / SSH key for +`git.hacktivism.ch` (push may stay local until that is fixed). + ## What runs without Android | Phase | Needs | From 93fe5f34f7ac0acd0525474f9435c2ed05984e9f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:04:35 +0200 Subject: [PATCH 04/38] =?UTF-8?q?docs:=20cadence=207=E2=80=9313=20min=20ji?= =?UTF-8?q?tter;=20commit=20often,=20push=20only=20at=20end?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Soft range instead of fixed 9m; helper defaults to no push. --- scripts/git-cadence-commit-push.sh | 107 ++++++++++++------ .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 42 ++++--- 2 files changed, 94 insertions(+), 55 deletions(-) diff --git a/scripts/git-cadence-commit-push.sh b/scripts/git-cadence-commit-push.sh index 1276fc1..9775bef 100755 --- a/scripts/git-cadence-commit-push.sh +++ b/scripts/git-cadence-commit-push.sh @@ -1,22 +1,24 @@ #!/usr/bin/env bash -# git-cadence-commit-push.sh — optional WIP commit + push on the historical cadence +# git-cadence-commit.sh — optional local WIP commit on a soft cadence # -# Model (from koopa-admin-log / francpaysan-admin-log histories, 2026-07): -# - message: area: short description (monitoring: … / landings: … / docs: …) -# - size: small logical steps, not giant dumps -# - interval: about **7–11 minutes** while work is in progress (sometimes denser) -# - push: after each commit when the remote accepts credentials +# Model (koopa-admin-log / francpaysan-admin-log histories + ops preference): +# - message: area: short description +# - size: small logical steps; **prefer more commits over few** +# - interval: soft range **7–13 minutes** while work is open (not a strict timer; +# denser bursts of 0–2 min are fine for tiny follow-ups) +# - push: **only at the end of a work session** (CADENCE_PUSH=1), never by default # # Usage: -# ./scripts/git-cadence-commit-push.sh # this repo +# ./scripts/git-cadence-commit-push.sh # commit only if dirty # ./scripts/git-cadence-commit-push.sh /path/to/repo # CADENCE_MSG='monitoring: wip ladder' ./scripts/git-cadence-commit-push.sh -# CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh # show only +# CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh +# CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh # explicit push (end of work) +# CADENCE_MIN_AGE_S=420 # skip if last commit younger than this (default 0) # -# Cron / agent every ~9 min: -# */9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh -# */9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh \ -# $HOME/taler/src/francpaysan-admin-log +# Soft timer (7–13 min, not fixed): fire often, skip until age ∈ [7,13] min target. +# # agent/cron every 7m — script enforces random target in range when CADENCE_JITTER=1 +# CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh # set -euo pipefail @@ -28,7 +30,14 @@ REPO="$(cd "$REPO" && pwd)" cd "$REPO" CADENCE_DRY="${CADENCE_DRY:-0}" -# Never auto-add secrets +# Push only when explicitly requested (end of session) — default OFF +CADENCE_PUSH="${CADENCE_PUSH:-0}" +# Soft range 7–13 minutes (seconds) +CADENCE_RANGE_MIN_S="${CADENCE_RANGE_MIN_S:-420}" # 7 min +CADENCE_RANGE_MAX_S="${CADENCE_RANGE_MAX_S:-780}" # 13 min +CADENCE_JITTER="${CADENCE_JITTER:-0}" +CADENCE_MIN_AGE_S="${CADENCE_MIN_AGE_S:-0}" + SKIP_GLOBS=( 'secrets.env' '**/secrets.env' @@ -47,29 +56,66 @@ branch=$(git rev-parse --abbrev-ref HEAD) remote=$(git remote 2>/dev/null | head -1 || true) : "${remote:=origin}" -# Dirty? (tracked + untracked, ignoring unreadable) +# Jitter: only proceed if last commit is at least a random target in [7,13] min +if [ "$CADENCE_JITTER" = "1" ]; then + last=$(git log -1 --format=%ct 2>/dev/null || echo 0) + now=$(date +%s) + age=$((now - last)) + # stable-ish target per hour so repeated 7m fires don't all commit + target=$(python3 -c " +import random, time +random.seed(int(time.time()) // 60) # changes every minute +print(random.randint(${CADENCE_RANGE_MIN_S}, ${CADENCE_RANGE_MAX_S})) +" 2>/dev/null || echo 600) + if [ "$age" -lt "$target" ]; then + echo "jitter wait · age=${age}s < target=${target}s (range ${CADENCE_RANGE_MIN_S}-${CADENCE_RANGE_MAX_S}s) · $REPO" + exit 0 + fi +elif [ "${CADENCE_MIN_AGE_S}" -gt 0 ] 2>/dev/null; then + last=$(git log -1 --format=%ct 2>/dev/null || echo 0) + now=$(date +%s) + age=$((now - last)) + if [ "$age" -lt "$CADENCE_MIN_AGE_S" ]; then + echo "min-age wait · age=${age}s < ${CADENCE_MIN_AGE_S}s · $REPO" + exit 0 + fi +fi + +do_push() { + local ahead + ahead=$(git rev-list --count "${remote}/${branch}..HEAD" 2>/dev/null || echo 0) + if [ "${ahead:-0}" -le 0 ]; then + echo "nothing to push · $REPO · $branch" + return 0 + fi + echo "ahead of ${remote}/${branch} by $ahead" + if [ "$CADENCE_DRY" = "1" ]; then + echo "DRY: git push $remote $branch" + return 0 + fi + if [ "$CADENCE_PUSH" != "1" ]; then + echo "skip push (CADENCE_PUSH!=1 — push only at end of work)" + return 0 + fi + git push "$remote" "$branch" || { + echo "WARN: push failed for $REPO ($remote $branch)" >&2 + return 0 + } + echo "pushed · $remote $branch" +} + +# Clean working tree → maybe push only if git diff --quiet && git diff --cached --quiet \ && [ -z "$(git ls-files --others --exclude-standard)" ]; then echo "clean · $REPO · $branch (nothing to commit)" - # still try push if ahead - ahead=$(git rev-list --count "${remote}/${branch}..HEAD" 2>/dev/null || echo 0) - if [ "${ahead:-0}" -gt 0 ]; then - echo "ahead of ${remote}/${branch} by $ahead — push" - if [ "$CADENCE_DRY" = "1" ]; then - echo "DRY: git push $remote $branch" - else - git push "$remote" "$branch" || echo "WARN: push failed (auth?)" >&2 - fi - fi + do_push exit 0 fi -# Stage safe paths only git add -A for g in "${SKIP_GLOBS[@]}"; do git reset -q -- "$g" 2>/dev/null || true done -# unstage anything still matching secret names while IFS= read -r f; do case "$f" in *secret*|*password*|*token*|secrets.env) @@ -83,7 +129,6 @@ if git diff --cached --quiet; then exit 0 fi -# Message from paths (historical area: style) msg="${CADENCE_MSG:-}" if [ -z "$msg" ]; then files=$(git diff --cached --name-only | head -20) @@ -103,14 +148,10 @@ fi if [ "$CADENCE_DRY" = "1" ]; then echo "DRY commit: $msg" git diff --cached --stat - echo "DRY push: $remote $branch" + [ "$CADENCE_PUSH" = "1" ] && echo "DRY push: $remote $branch" || echo "DRY: no push" exit 0 fi git commit -m "$msg" echo "committed · $msg" -if ! git push "$remote" "$branch"; then - echo "WARN: push failed for $REPO ($remote $branch) — commit is local" >&2 - exit 0 -fi -echo "pushed · $remote $branch" +do_push diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index fe6fd1c..c4f11db 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -9,49 +9,47 @@ default ops laptop yet. ## Git commit / push cadence (existing model) -Observed in `koopa-admin-log` and `francpaysan-admin-log` histories -(active work on 2026-07-17): +Observed in `koopa-admin-log` / `francpaysan-admin-log` histories, plus ops preference: | Rule | Practice | |------|----------| -| **Interval** | About **7–11 minutes** between publishable steps while a topic is open (bursts of 0–2 min are fine for tiny follow-ups) | -| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …`, `scripts: …`, `todo: …` | -| **Size** | One logical change (or a tight bundle), not end-of-day megacommits | -| **Push** | After each commit when the remote accepts auth (`git push origin …`) | +| **Commits** | **Prefer frequent** small commits over rare large ones | +| **Interval** | Soft range **7–13 minutes** while a topic is open — **not** a fixed 9‑minute metronome; denser 0–2 min bursts for tiny follow-ups are fine | +| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …` | +| **Push** | **Only at the end of a work session** (`CADENCE_PUSH=1` or manual `git push`) — not after every commit | ### Agent / human workflow -1. Finish a small slice of work. -2. `git status` / diff → commit with the area-prefix style. -3. `git push` (or run the helper below). -4. Continue; next commit window ~**7–11 min** later (or sooner if the next slice is ready). +1. Finish a small slice → commit (`area: …`). Repeat often. +2. Do **not** push after each commit. +3. When the topic/session is done → one `git push` (or `CADENCE_PUSH=1`). +4. Soft next-commit window: about **7–13 min** (jitter), sooner if the next slice is ready. ### Helper script -From the admin-log root: - ```bash # dry-run CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh -# commit + push this repo if dirty (skips secrets.env / *password* / *token*) +# commit only if dirty (default: no push; skips secrets) ./scripts/git-cadence-commit-push.sh - -# other checkout ./scripts/git-cadence-commit-push.sh ~/taler/src/francpaysan-admin-log + +# end of session — push commits that are already local +CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh + +# soft timer: only commit if last commit age ≥ random target in 7–13 min +CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh ``` -Optional timer (every 9 min ≈ mid of 7–11): +Optional user cron (check every 7 min; **jitter** keeps effective gaps in **7–13 min**): ```bash -# user crontab example -*/9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1 -*/9 * * * * $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh $HOME/taler/src/francpaysan-admin-log >>$HOME/.cache/git-cadence-lfp.log 2>&1 +*/7 * * * * CADENCE_JITTER=1 $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1 ``` -**Auth note (this host):** `francpaysan-admin-log` → `ssh://git@git.gnunet.org/…` works. -`koopa-admin-log` → `https://git.hacktivism.ch/…` needs credentials / SSH key for -`git.hacktivism.ch` (push may stay local until that is fixed). +**Auth note:** `francpaysan-admin-log` (gnunet SSH) pushes OK. +`koopa-admin-log` (hacktivism HTTPS) needs credentials/SSH key before end-of-session push works from this host. ## What runs without Android From 13614f537fc4fbee5fb0712ab4c040f1d963a9cc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:06:16 +0200 Subject: [PATCH 05/38] docs(paivana): disable racey upstream unit; smoke expects 302/402 Compose unit starts both containers; standalone upstream podman-start raced at boot (exit 125). Live stack already healthy on hacktivism. --- configs/paivana/README.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/configs/paivana/README.md b/configs/paivana/README.md index e813a82..3b8700a 100644 --- a/configs/paivana/README.md +++ b/configs/paivana/README.md @@ -48,15 +48,31 @@ podman logs -f koopa-paivana curl -si http://127.0.0.1:9025/ | head # 302 → /.well-known/paivana/templates/paivana#… ``` -Autostart: +Autostart (**only** the compose unit — starts **both** `koopa-paivana` and +`koopa-paivana-upstream`): ```bash mkdir -p ~/.config/systemd/user cp ~/koopa-paivana/container-koopa-paivana.service ~/.config/systemd/user/ +# optional: After= merchant (configs/systemd/user/container-koopa-paivana.service.d/) systemctl --user daemon-reload systemctl --user enable --now container-koopa-paivana.service ``` +**Do not** enable a separate `container-koopa-paivana-upstream.service` that only +runs `podman start koopa-paivana-upstream`. At boot that unit races compose +(container not created yet → exit 125 → start-limit-hit). Compose already +starts the upstream. If such a unit exists: +`systemctl --user disable --now container-koopa-paivana-upstream.service`. + +Healthy smoke: + +| Check | Expect | +|-------|--------| +| `curl -si http://127.0.0.1:9025/` | **302** → `/.well-known/paivana/templates/paivana#…` | +| template URL / public `https://paivana.hacktivism.ch/` | **402** paywall HTML + `paivana: taler://pay-template/…` | +| monitoring | `./scripts/taler-monitoring/taler-monitoring.sh -d hacktivism.ch urls` → `www.paivana` OK | + ## Network notes Rootless pasta cannot hairpin public DNS for the merchant. Compose sets: From 6fae59b20c97911c1c59cc25619f43b307c4eb1c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:31:52 +0200 Subject: [PATCH 06/38] monitoring: Android pay smoke with published F-Droid wallet APK User-local emulator/adb path; install APK, deep-link withdraw and pay-template without root. Document host RAM limits for full UI. --- .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 47 +++++ .../taler-monitoring/android-test/.gitignore | 3 + .../taler-monitoring/android-test/README.md | 13 ++ .../android-test/run-android-pay-smoke.sh | 194 ++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 scripts/taler-monitoring/android-test/.gitignore create mode 100644 scripts/taler-monitoring/android-test/README.md create mode 100755 scripts/taler-monitoring/android-test/run-android-pay-smoke.sh diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index c4f11db..57be728 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -131,3 +131,50 @@ document paths; **no** real wallet payments until approved. When Android tooling is installed, append a short “installed on …” section here (date, `adb version`, AVD name, `ANDROID_HOME`). + +--- + +## Android pay smoke (published APK, no root preferred) + +Script: [`android-test/run-android-pay-smoke.sh`](./android-test/run-android-pay-smoke.sh) + +| Step | Detail | +|------|--------| +| APK | F-Droid **Taler Wallet 1.6.1** (`net.taler.wallet.fdroid_854.apk`) | +| Install | `adb install -r` (no root) | +| Withdraw | `taler://withdraw/…` from `…/intro/demo-withdraw.json` (GOA or stage) | +| Pay | stage: shop template order → `taler://pay/…`; GOA: `taler://pay-template/…/goa-shop/paivana` | +| Evidence | screenshots + logcat under `android-test/out/` | + +```bash +# device or emulator already booted +./android-test/run-android-pay-smoke.sh +STACK=goa ./android-test/run-android-pay-smoke.sh +STACK=stage ./android-test/run-android-pay-smoke.sh +``` + +**User-local emulator (no apt root if SDK already under `$HOME/Android/Sdk`):** + +```bash +export ANDROID_HOME=$HOME/Android/Sdk +export PATH=$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH +# AVD e.g. TalerWallet34 (google_apis x86_64) +emulator -avd TalerWallet34 -no-window -gpu swiftshader_indirect -memory 1536 & +adb wait-for-device +./android-test/run-android-pay-smoke.sh +``` + +**Limits (clementine 2026-07-17):** host RAM ~3.7 GiB → emulator often ANRs System UI; +wallet-core still receives withdraw intents (`prepareBankIntegratedWithdrawal` in logcat). +For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 GiB** free RAM. + +### Status snapshot (2026-07-17) + +| Item | Status | +|------|--------| +| `adb` | present (`android-sdk-platform-tools`) | +| F-Droid APK | downloaded under `android-test/apks/` | +| User SDK + emulator + AVD `TalerWallet34` | installed under `$HOME/Android/Sdk` (~5 GiB) | +| `adb install` wallet | **Success** on emulator-5554 | +| Deep-link withdraw | **delivered**; wallet-core started `prepareBankIntegratedWithdrawal` for GOA | +| Unattended UI confirm | flaky (ANR) on this host’s RAM | diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore new file mode 100644 index 0000000..2b9c0a5 --- /dev/null +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -0,0 +1,3 @@ +apks/*.apk +out/ +*.png diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md new file mode 100644 index 0000000..96b760f --- /dev/null +++ b/scripts/taler-monitoring/android-test/README.md @@ -0,0 +1,13 @@ +# Android pay smoke (published wallet) + +Installs the **F-Droid** GNU Taler Wallet APK and deep-links +`taler://withdraw/…` + optional `taler://pay…` against GOA or stage. + +```bash +# needs adb device/emulator +./run-android-pay-smoke.sh +STACK=goa ./run-android-pay-smoke.sh +STACK=stage ./run-android-pay-smoke.sh +``` + +See parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh new file mode 100755 index 0000000..da328b7 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -0,0 +1,194 @@ +#!/usr/bin/env bash +# Android wallet smoke: install published F-Droid APK, open withdraw/pay URIs via adb. +# +# No root required if: +# - adb is available (package android-tools-adb / platform-tools) +# - a device or emulator is connected (`adb devices` shows "device") +# - optional user-local SDK under $HOME/Android/Sdk (emulator) +# +# Usage: +# ./android-test/run-android-pay-smoke.sh # auto stack (GOA if local, else stage) +# STACK=goa ./android-test/run-android-pay-smoke.sh +# STACK=stage ./android-test/run-android-pay-smoke.sh +# SERIAL=emulator-5554 ./android-test/run-android-pay-smoke.sh +# +# Published APK (default): F-Droid Taler Wallet 1.6.1 (854) +# https://f-droid.org/packages/net.taler.wallet.fdroid/ +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +APK_DIR="${APK_DIR:-$ROOT/apks}" +APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" +PKG="${PKG:-net.taler.wallet.fdroid}" +STACK="${STACK:-auto}" # auto | goa | stage +SERIAL="${SERIAL:-}" +OUT_DIR="${OUT_DIR:-$ROOT/out}" +mkdir -p "$APK_DIR" "$OUT_DIR" + +if ! command -v adb >/dev/null 2>&1; then + if [ -x "$HOME/Android/Sdk/platform-tools/adb" ]; then + export PATH="$HOME/Android/Sdk/platform-tools:$PATH" + else + echo "adb missing — install android-tools-adb or user SDK platform-tools" >&2 + exit 2 + fi +fi + +if [ -z "$SERIAL" ]; then + SERIAL=$(adb devices | awk '/\tdevice$/{print $1; exit}') +fi +if [ -z "$SERIAL" ]; then + cat >&2 <<'EOF' +No adb device. Options (no root for user-local emulator if /dev/kvm is usable): + 1) Plug phone with USB debugging + 2) Start AVD: $HOME/Android/Sdk/emulator/emulator -avd TalerWallet34 -no-window + 3) Then re-run this script +EOF + exit 3 +fi +ADB=(adb -s "$SERIAL") +echo "device: $SERIAL" + +# Resolve stack endpoints +case "$STACK" in + auto) + if curl -sfS -m 5 -o /dev/null https://bank.hacktivism.ch/config 2>/dev/null; then + STACK=goa + else + STACK=stage + fi + ;; +esac +case "$STACK" in + goa) + BANK=https://bank.hacktivism.ch + MERCHANT=https://taler.hacktivism.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + PAY_HINT="paivana template / goa-shop" + ;; + stage) + BANK=https://stage.bank.lefrancpaysan.ch + MERCHANT=https://stage.monnaie.lefrancpaysan.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + PAY_HINT="farmer shop templates" + ;; + *) echo "unknown STACK=$STACK" >&2; exit 2 ;; +esac +echo "stack: $STACK bank: $BANK" + +# APK +APK_PATH="$APK_DIR/$APK_NAME" +if [ ! -f "$APK_PATH" ]; then + echo "downloading published wallet APK…" + curl -fL --retry 3 -o "$APK_PATH" "$APK_URL" +fi +echo "apk: $APK_PATH ($(wc -c <"$APK_PATH") bytes)" + +"${ADB[@]}" wait-for-device +"${ADB[@]}" install -r "$APK_PATH" +"${ADB[@]}" shell settings put global window_animation_scale 0 || true +"${ADB[@]}" shell settings put global transition_animation_scale 0 || true +"${ADB[@]}" shell settings put global animator_duration_scale 0 || true + +# Withdraw URI from live mint +WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') +if [ -z "$WURI" ]; then + echo "FAIL: no taler_withdraw_uri from $WITHDRAW_JSON" >&2 + exit 4 +fi +echo "withdraw: $WURI" +echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + +"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true +"${ADB[@]}" logcat -c 2>/dev/null || true +"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" 2>&1 | tee "$OUT_DIR/start-withdraw.txt" +sleep 12 + +"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/01-after-withdraw-uri.png" || true +"${ADB[@]}" shell uiautomator dump /sdcard/ui.xml 2>/dev/null || true +"${ADB[@]}" pull /sdcard/ui.xml "$OUT_DIR/01-ui.xml" 2>/dev/null || true + +# Best-effort: tap common confirm labels +python3 - "$SERIAL" "$OUT_DIR/01-ui.xml" <<'PY' || true +import re, subprocess, sys +serial, path = sys.argv[1], sys.argv[2] +try: + xml = open(path, errors="replace").read() +except FileNotFoundError: + raise SystemExit(0) +labels = [ + "Confirm", "Withdraw", "Accept", "Continue", "OK", + "I accept", "Agree", "Bestätigen", "Abheben", "Akzeptieren", +] +for label in labels: + for pat in ( + r'text="%s"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"' % re.escape(label), + r'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*text="%s"' % re.escape(label), + ): + for m in re.finditer(pat, xml): + x1, y1, x2, y2 = map(int, m.groups()) + x, y = (x1 + x2) // 2, (y1 + y2) // 2 + print(f"tap {label} @ {x},{y}") + subprocess.run(["adb", "-s", serial, "shell", "input", "tap", str(x), str(y)], check=False) +PY +sleep 8 +"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/02-after-confirm-tap.png" || true + +# Optional public pay template (stage fixed product / GOA shop if amount known) +# Stage: open monnaies shop is browser-side; deep link pay after template POST if available. +PAY_URI="" +if [ "$STACK" = "stage" ]; then + # fixed template panier-legumes → taler://pay/… + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c ' +import json,sys +d=json.load(sys.stdin) +oid=d.get("order_id") or "" +tok=d.get("token") or "" +if oid and tok: + print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok)) +' 2>/dev/null || true) + fi +elif [ "$STACK" = "goa" ]; then + # Paivana pay-template deep link (wallet prepares order) + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" +fi + +if [ -n "$PAY_URI" ]; then + echo "pay: $PAY_URI ($PAY_HINT)" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" 2>&1 | tee "$OUT_DIR/start-pay.txt" + sleep 12 + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/03-after-pay-uri.png" || true +else + echo "pay: (skipped — no public template order)" +fi + +# Evidence from logcat +"${ADB[@]}" logcat -d -t 300 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|preparePay|confirm|error|Error|success' "$OUT_DIR/logcat.txt" \ + | tail -80 >"$OUT_DIR/logcat-taler.txt" || true + +echo +echo "=== summary ===" +echo "package installed: $PKG" +echo "withdraw intent delivered: $WURI" +if grep -q 'prepareBankIntegratedWithdrawal' "$OUT_DIR/logcat.txt" 2>/dev/null; then + echo "wallet-core: prepareBankIntegratedWithdrawal seen (wallet accepted withdraw URI)" +else + echo "wallet-core: prepareBankIntegratedWithdrawal NOT seen (UI may still show onboarding/ANR on small RAM)" +fi +if [ -n "$PAY_URI" ]; then + if grep -qiE 'preparePay|confirmPay|pay-template|preparePurchase' "$OUT_DIR/logcat.txt" 2>/dev/null; then + echo "wallet-core: pay path activity seen" + else + echo "wallet-core: pay path not confirmed in logcat (check $OUT_DIR/03-after-pay-uri.png)" + fi +fi +echo "artifacts: $OUT_DIR/" +ls -la "$OUT_DIR" | sed 's/^/ /' +echo +echo "NOTE: Full unattended UI payment needs a responsive device/emulator (≥6–8 GiB host RAM recommended)." +echo "This smoke proves: published APK install + deep-link into wallet-core on stack=$STACK." From 4598aca0830233bc5e81ae49fc83b697b0ac2440 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 20:59:31 +0200 Subject: [PATCH 07/38] monitoring: build latest taler-android then same adb pay smoke Add run-android-build-and-smoke.sh (assembleFdroidDebug from taler-android master) and wire APK_PATH/PKG into the existing smoke. --- .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 13 +++ .../taler-monitoring/android-test/.gitignore | 1 + .../taler-monitoring/android-test/README.md | 23 ++++- .../run-android-build-and-smoke.sh | 93 +++++++++++++++++++ .../android-test/run-android-pay-smoke.sh | 22 +++-- 5 files changed, 143 insertions(+), 9 deletions(-) create mode 100755 scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index 57be728..73e654a 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -178,3 +178,16 @@ For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 | `adb install` wallet | **Success** on emulator-5554 | | Deep-link withdraw | **delivered**; wallet-core started `prepareBankIntegratedWithdrawal` for GOA | | Unattended UI confirm | flaky (ANR) on this host’s RAM | +| Source build `assembleFdroidDebug` | **OK** (2026-07-17, `taler-android@f65a976c0d4a`, ~12 min, APK ~59 MiB) | + +### From-source smoke (same tests as F-Droid) + +```bash +# build latest master + adb smoke +./android-test/run-android-build-and-smoke.sh +STACK=goa ./android-test/run-android-build-and-smoke.sh +# already built: +APK_PATH=$HOME/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ + PKG=net.taler.wallet.fdroid.debug \ + ./android-test/run-android-pay-smoke.sh +``` diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index 2b9c0a5..3119fff 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -1,3 +1,4 @@ apks/*.apk out/ *.png +out-source/ diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 96b760f..b943083 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -1,7 +1,6 @@ -# Android pay smoke (published wallet) +# Android pay smoke -Installs the **F-Droid** GNU Taler Wallet APK and deep-links -`taler://withdraw/…` + optional `taler://pay…` against GOA or stage. +## 1) Published F-Droid wallet ```bash # needs adb device/emulator @@ -10,4 +9,22 @@ STACK=goa ./run-android-pay-smoke.sh STACK=stage ./run-android-pay-smoke.sh ``` +APK: F-Droid **1.6.1** (`net.taler.wallet.fdroid`). + +## 2) Latest source build + same tests + +Pulls/builds `taler-android` master (`:wallet:assembleFdroidDebug`), then +runs the **same** smoke (install + withdraw/pay deep-links). + +```bash +# default source: $HOME/taler/taler-android +./run-android-build-and-smoke.sh +STACK=goa ./run-android-build-and-smoke.sh +SKIP_PULL=1 ./run-android-build-and-smoke.sh # use tree as-is +TALER_ANDROID_SRC=/path/to/taler-android ./run-android-build-and-smoke.sh +``` + +Built package id: `net.taler.wallet.fdroid.debug`. +Needs `ANDROID_HOME` (e.g. `$HOME/Android/Sdk`), Java 17+, network for Maven. + See parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh new file mode 100755 index 0000000..958ca84 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# Build the **latest** GNU Taler Android wallet from source, then run the same +# adb deep-link smoke as run-android-pay-smoke.sh. +# +# Source (default): $HOME/taler/taler-android (git@git.taler.net:taler-android.git) +# Build: ./gradlew :wallet:assembleFdroidDebug +# Package id: net.taler.wallet.fdroid.debug +# +# No root required if ANDROID_HOME points at a user SDK ($HOME/Android/Sdk). +# +# Usage: +# ./android-test/run-android-build-and-smoke.sh +# STACK=goa ./android-test/run-android-build-and-smoke.sh +# STACK=stage SKIP_PULL=1 ./android-test/run-android-build-and-smoke.sh +# TALER_ANDROID_SRC=/path/to/taler-android ./android-test/run-android-build-and-smoke.sh +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +SMOKE="$ROOT/run-android-pay-smoke.sh" +SRC="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" +SKIP_PULL="${SKIP_PULL:-0}" +VARIANT="${VARIANT:-fdroidDebug}" # assemble task suffix: FdroidDebug / NightlyDebug / … +export ANDROID_HOME="${ANDROID_HOME:-$HOME/Android/Sdk}" +export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" + +if [ ! -x "$SRC/gradlew" ]; then + echo "taler-android not found at $SRC (set TALER_ANDROID_SRC=…)" >&2 + exit 2 +fi +if [ ! -d "$ANDROID_HOME" ]; then + echo "ANDROID_HOME missing: $ANDROID_HOME" >&2 + exit 2 +fi + +printf 'sdk.dir=%s\n' "$ANDROID_HOME" >"$SRC/local.properties" + +cd "$SRC" +if [ "$SKIP_PULL" != "1" ] && [ -d .git ]; then + echo "git pull --ff-only origin $(git rev-parse --abbrev-ref HEAD)" + git pull --ff-only origin "$(git rev-parse --abbrev-ref HEAD)" || \ + git fetch origin && git merge --ff-only "origin/$(git rev-parse --abbrev-ref HEAD)" || true +fi +COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown) +echo "source: $SRC @ $COMMIT" +echo "variant: assemble${VARIANT^}" 2>/dev/null || true + +# Map VARIANT → gradle task (fdroidDebug → assembleFdroidDebug) +task="assemble$(printf '%s' "$VARIANT" | python3 -c 'import sys; s=sys.stdin.read().strip(); print(s[:1].upper()+s[1:] if s else "FdroidDebug")')" +# Prefer explicit known tasks +case "$VARIANT" in + fdroidDebug|FdroidDebug) task=assembleFdroidDebug ;; + fdroidRelease|FdroidRelease) task=assembleFdroidRelease ;; + nightlyDebug|NightlyDebug) task=assembleNightlyDebug ;; + googleDebug|GoogleDebug) task=assembleGoogleDebug ;; +esac + +# Low-RAM hosts: cap JVM (override GRADLE_OPTS to raise) +export GRADLE_OPTS="${GRADLE_OPTS:--Xmx1280m -Dorg.gradle.daemon=false -Dorg.gradle.workers.max=1}" +echo "gradle :wallet:$task …" +./gradlew ":wallet:$task" --no-daemon --max-workers="${GRADLE_MAX_WORKERS:-1}" \ + -Dorg.gradle.jvmargs="${GRADLE_JVMARGS:--Xmx1280m}" + +# Locate APK +APK=$(find "$SRC/wallet/build/outputs/apk" -name '*.apk' -type f 2>/dev/null | sort | tail -1) +if [ -z "$APK" ] || [ ! -f "$APK" ]; then + echo "no APK under wallet/build/outputs/apk" >&2 + exit 4 +fi +echo "built apk: $APK ($(wc -c <"$APK") bytes)" + +# Package id from aapt if available +PKG="${PKG:-}" +if [ -z "$PKG" ] && command -v aapt >/dev/null 2>&1; then + PKG=$(aapt dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1) +fi +: "${PKG:=net.taler.wallet.fdroid.debug}" + +mkdir -p "$ROOT/apks" +STAMP_APK="$ROOT/apks/wallet-from-source-${COMMIT}.apk" +cp -f "$APK" "$STAMP_APK" +echo "copied → $STAMP_APK" +echo "$COMMIT" >"$ROOT/out/build-commit.txt" 2>/dev/null || mkdir -p "$ROOT/out" && echo "$COMMIT" >"$ROOT/out/build-commit.txt" +echo "$APK" >"$ROOT/out/build-apk-path.txt" + +# Same smoke as published F-Droid path +export APK_PATH="$STAMP_APK" +export APK_NAME="$(basename "$STAMP_APK")" +export PKG +export OUT_DIR="${OUT_DIR:-$ROOT/out-source}" +mkdir -p "$OUT_DIR" +echo "=== smoke with built APK (PKG=$PKG) ===" +exec env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" PKG="$PKG" OUT_DIR="$OUT_DIR" \ + "$SMOKE" diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh index da328b7..907cb07 100755 --- a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# Android wallet smoke: install published F-Droid APK, open withdraw/pay URIs via adb. +# Android wallet smoke: install APK (published F-Droid **or** from-source), open +# withdraw/pay URIs via adb. # # No root required if: # - adb is available (package android-tools-adb / platform-tools) @@ -7,10 +8,12 @@ # - optional user-local SDK under $HOME/Android/Sdk (emulator) # # Usage: -# ./android-test/run-android-pay-smoke.sh # auto stack (GOA if local, else stage) +# ./android-test/run-android-pay-smoke.sh # published F-Droid APK # STACK=goa ./android-test/run-android-pay-smoke.sh # STACK=stage ./android-test/run-android-pay-smoke.sh -# SERIAL=emulator-5554 ./android-test/run-android-pay-smoke.sh +# # from-source (built first): +# ./android-test/run-android-build-and-smoke.sh +# APK_PATH=/path/to.apk PKG=net.taler.wallet.fdroid.debug ./run-android-pay-smoke.sh # # Published APK (default): F-Droid Taler Wallet 1.6.1 (854) # https://f-droid.org/packages/net.taler.wallet.fdroid/ @@ -20,6 +23,8 @@ ROOT=$(cd "$(dirname "$0")" && pwd) APK_DIR="${APK_DIR:-$ROOT/apks}" APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" +# Optional override: full path to an already-built APK (from-source) +APK_PATH="${APK_PATH:-}" PKG="${PKG:-net.taler.wallet.fdroid}" STACK="${STACK:-auto}" # auto | goa | stage SERIAL="${SERIAL:-}" @@ -77,13 +82,18 @@ case "$STACK" in esac echo "stack: $STACK bank: $BANK" -# APK -APK_PATH="$APK_DIR/$APK_NAME" -if [ ! -f "$APK_PATH" ]; then +# APK: explicit path, or download published F-Droid into APK_DIR +if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then + : +elif [ -f "$APK_DIR/$APK_NAME" ]; then + APK_PATH="$APK_DIR/$APK_NAME" +else + APK_PATH="$APK_DIR/$APK_NAME" echo "downloading published wallet APK…" curl -fL --retry 3 -o "$APK_PATH" "$APK_URL" fi echo "apk: $APK_PATH ($(wc -c <"$APK_PATH") bytes)" +echo "pkg: $PKG" "${ADB[@]}" wait-for-device "${ADB[@]}" install -r "$APK_PATH" From 2b89526c60bca09927fa078d37e68289b20a779b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:00:56 +0200 Subject: [PATCH 08/38] monitoring: vanilla Android GUI smoke + AUTOMATED-GUI-NOTES uiautomator multi-round taps for goa/stage; document legitimate deep-link shortcuts vs pure CLI install tests. --- .../taler-monitoring/android-test/.gitignore | 2 + .../android-test/AUTOMATED-GUI-NOTES.md | 119 +++++++++++ .../taler-monitoring/android-test/README.md | 15 +- .../taler-monitoring/android-test/lib_ui.py | 192 ++++++++++++++++++ .../run-android-build-and-smoke.sh | 14 +- .../android-test/run-android-gui-smoke.sh | 172 ++++++++++++++++ 6 files changed, 509 insertions(+), 5 deletions(-) create mode 100644 scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md create mode 100755 scripts/taler-monitoring/android-test/lib_ui.py create mode 100755 scripts/taler-monitoring/android-test/run-android-gui-smoke.sh diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index 3119fff..de62278 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -2,3 +2,5 @@ apks/*.apk out/ *.png out-source/ +out-gui/ +out-source-gui/ diff --git a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md new file mode 100644 index 0000000..4013505 --- /dev/null +++ b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md @@ -0,0 +1,119 @@ +# Automated GUI notes — Android wallet (vanilla level) + +Goal: tests for **two systems** (GOA / hacktivism and stage TESTPAYSAN) that +exercise the wallet **graphically in the UI** as far as practical, not only +CLI/`adb` deep-links into wallet-core. + +## Vanilla level (this tree) + +| Layer | What we do | Tooling | +|-------|------------|---------| +| **A — Install** | Install published F-Droid APK **or** APK built from `taler-android` master | `adb install` | +| **B — Entry (shortcut)** | Deliver withdraw / pay as `taler://…` intents | `adb shell am start -a VIEW -d …` | +| **C — Graphical UI** | Dump hierarchy, tap Confirm / ToS / Pay-like buttons, dismiss ANR | `uiautomator dump` + `input tap` (`lib_ui.py`) | +| **D — Evidence** | Screenshots + UI XML + logcat + JSON status | `android-test/out-gui/` | + +Scripts: + +| Script | Role | +|--------|------| +| `run-android-pay-smoke.sh` | Install + deep-link + light taps (hybrid / CLI-heavy) | +| `run-android-gui-smoke.sh` | **Vanilla GUI path**: deep-link entry + multi-round UI driving | +| `run-android-build-and-smoke.sh` | Build latest source APK → same smoke (set `GUI=1` to call GUI script) | +| `lib_ui.py` | Shared dump/tap/ANR helpers | + +```bash +cd scripts/taler-monitoring/android-test + +# Published APK, graphical drive, both stacks +STACK=goa ./run-android-gui-smoke.sh +STACK=stage ./run-android-gui-smoke.sh + +# From-source APK + GUI +APK_PATH=~/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ + PKG=net.taler.wallet.fdroid.debug \ + STACK=goa ./run-android-gui-smoke.sh +``` + +## Legitimate shortcuts (document, do not hide) + +These are **not** full human UX, but they are the same contracts as QR scan / +paywall open on the landings: + +| Shortcut | Replaces | Why legitimate | +|----------|----------|----------------| +| `adb` `VIEW` `taler://withdraw/…` | Camera QR scan of communal / demo withdraw | Identical URI payload as landings (`demo-withdraw.json`) | +| `adb` `VIEW` `taler://pay…` / `pay-template/…` | Scan shop/paywall QR or open “pay with Taler” | Same URI merchant/backend would show | +| Host-side `POST …/templates/{id}` then pay URI | Browser shop UI to create order | Stage farmer templates are public; order create is API, pay is still wallet UI | +| Multi-language label list for taps | Human reading the button | Same visible buttons; brittle if copy changes | +| Skip full ToS reading | User scrolling long TOS | Tap “Accept” if shown; no legal review automation | + +**Not automated yet (out of vanilla scope):** + +- Typing personal bank passwords into the wallet webview +- NFC +- Real camera / vision-based QR +- Biometric unlock +- Full onboarding wizard every time (we force-stop + re-intent) + +## Two systems under test + +| Stack | Withdraw source | Pay source | +|-------|-----------------|------------| +| **GOA** (`STACK=goa`) | `https://bank.hacktivism.ch/intro/demo-withdraw.json` | `taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana` (Paivana) | +| **stage** (`STACK=stage`) | `https://stage.bank.lefrancpaysan.ch/intro/demo-withdraw.json` | Public template e.g. `fermes-des-collines` / `panier-legumes` → `taler://pay/…` | + +Both use the **same** GUI driver (`lib_ui.py`); only URLs and package under test differ. + +## GUI driver behaviour (`lib_ui.gui_drive`) + +1. Every round: `uiautomator dump` + optional screencap. +2. If **“System UI isn’t responding”** → tap **Wait** (prefer over Close). +3. Else try exact taps on Confirm / Withdraw / Accept / Pay / DE/FR synonyms. +4. Else regex partial match on those words. +5. Repeat `GUI_ROUNDS` times (default 10), `GUI_SLEEP` seconds apart. + +Success signals (layered): + +- **Intent delivered** — `am start` OK +- **wallet-core** — logcat `prepareBankIntegratedWithdrawal` / pay-related ops +- **GUI taps** — non-empty `taps` in `gui-*-status.json` +- **Screenshots** — human-readable progress under `gui-withdraw/` / `gui-pay/` + +A run can be “vanilla-partial” if intents + some taps succeed but final +balance/settlement is not asserted (settlement still better covered by +`taler-monitoring.sh e2e` CLI wallet). + +## Host limits (clementine, 2026-07) + +| Constraint | Effect | +|------------|--------| +| ~3.7 GiB RAM | Emulator ANRs; GUI rounds often only dismiss ANR | +| Prefer physical phone or ≥6–8 GiB host | Real Confirm/Pay taps complete more often | +| No root required | User SDK + adb + F-Droid/source APK | + +## Relation to CLI e2e + +| Suite | Strength | +|-------|----------| +| `taler-monitoring.sh e2e` / `ladder` | Full withdraw+pay **settlement** via `taler-wallet-cli` | +| `run-android-gui-smoke.sh` | **Android app UI** surface for the same stacks | +| Combined | CLI = money path green; GUI = app UX green as far as taps go | + +## Future upgrades (not required for vanilla) + +- Maestro / Appium flows with stable resource-ids +- Explicit wait for text “Withdrawal complete” / balance +- Separate jobs: F-Droid vs nightly source vs Play flavor +- Record video (`adb shell screenrecord`) on failure + +## Checklist — “vanille erreicht?” + +- [x] Two stacks parameterized (goa / stage) +- [x] Published APK path +- [x] Source-build path (same smoke after build) +- [x] Deep-link shortcuts documented +- [x] Multi-round **graphical** taps (uiautomator) +- [x] Screenshots + UI XML + logcat artifacts +- [ ] Fully reliable unattended Confirm→settled on low-RAM emulator (host-limited) +- [ ] Assert final balance in app UI (optional next step) diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index b943083..19df5ff 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -27,4 +27,17 @@ TALER_ANDROID_SRC=/path/to/taler-android ./run-android-build-and-smoke.sh Built package id: `net.taler.wallet.fdroid.debug`. Needs `ANDROID_HOME` (e.g. `$HOME/Android/Sdk`), Java 17+, network for Maven. -See parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). +## 3) Graphical UI (vanilla) + +Deep-link entry (≡ QR) **plus** multi-round uiautomator taps: + +```bash +STACK=goa ./run-android-gui-smoke.sh +STACK=stage ./run-android-gui-smoke.sh +# after source build: +GUI=1 ./run-android-build-and-smoke.sh +``` + +Documented shortcuts and limits: **[AUTOMATED-GUI-NOTES.md](./AUTOMATED-GUI-NOTES.md)**. + +See also parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/lib_ui.py b/scripts/taler-monitoring/android-test/lib_ui.py new file mode 100755 index 0000000..a7cdad5 --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_ui.py @@ -0,0 +1,192 @@ +#!/usr/bin/env python3 +"""Minimal adb + uiautomator helpers for Taler wallet GUI smoke (no root). + +Does not replace Maestro/Appium; uses only adb shell + uiautomator dump. +""" +from __future__ import annotations + +import re +import subprocess +import time +from pathlib import Path +from typing import Iterable, List, Optional, Sequence, Tuple + +Bounds = Tuple[int, int, int, int] + + +def adb(serial: str, *args: str, check: bool = False, timeout: Optional[float] = 60) -> subprocess.CompletedProcess: + cmd = ["adb", "-s", serial, *args] + return subprocess.run(cmd, check=check, capture_output=True, text=True, timeout=timeout) + + +def dump_ui(serial: str, dest: Path) -> str: + adb(serial, "shell", "uiautomator", "dump", "/sdcard/ui.xml", timeout=90) + adb(serial, "pull", "/sdcard/ui.xml", str(dest), timeout=30) + return dest.read_text(errors="replace") if dest.is_file() else "" + + +def screencap(serial: str, dest: Path) -> None: + p = subprocess.run( + ["adb", "-s", serial, "exec-out", "screencap", "-p"], + capture_output=True, + timeout=60, + ) + if p.returncode == 0 and p.stdout: + dest.write_bytes(p.stdout) + + +def texts(xml: str) -> List[str]: + return [t for t in re.findall(r'text="([^"]*)"', xml) if t.strip()] + + +def find_bounds(xml: str, label: str) -> List[Bounds]: + """Return list of (x1,y1,x2,y2) for nodes whose text or content-desc matches label (exact).""" + out: List[Bounds] = [] + esc = re.escape(label) + patterns = [ + rf'(?:text|content-desc)="{esc}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*(?:text|content-desc)="{esc}"', + ] + for pat in patterns: + for m in re.finditer(pat, xml, flags=re.I): + out.append(tuple(map(int, m.groups()))) # type: ignore[arg-type] + return out + + +def find_bounds_re(xml: str, pattern: str) -> List[Tuple[str, Bounds]]: + out: List[Tuple[str, Bounds]] = [] + for m in re.finditer( + rf'(?:text|content-desc)="([^"]*{pattern}[^"]*)"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + xml, + flags=re.I, + ): + label = m.group(1) + b = tuple(map(int, m.group(2, 3, 4, 5))) + out.append((label, b)) # type: ignore[arg-type] + return out + + +def center(b: Bounds) -> Tuple[int, int]: + x1, y1, x2, y2 = b + return (x1 + x2) // 2, (y1 + y2) // 2 + + +def tap(serial: str, x: int, y: int) -> None: + adb(serial, "shell", "input", "tap", str(x), str(y)) + + +def tap_bounds(serial: str, b: Bounds) -> None: + x, y = center(b) + tap(serial, x, y) + + +def dismiss_anr(serial: str, xml: str) -> bool: + """If System UI isn't responding, prefer Wait over Close.""" + if "isn't responding" not in xml and "reagiert nicht" not in xml.lower(): + return False + for label in ("Wait", "Warten", "OK"): + bs = find_bounds(xml, label) + if bs: + tap_bounds(serial, bs[0]) + return True + # fallback: right-ish button often Wait + adb(serial, "shell", "input", "keyevent", "22") + adb(serial, "shell", "input", "keyevent", "66") + return True + + +# Labels seen on wallet UI (EN/DE/FR) — expand as needed +CONFIRM_LABELS = [ + "Confirm", + "Confirm withdrawal", + "Withdraw", + "Accept", + "I accept", + "Agree", + "Continue", + "Next", + "OK", + "Pay", + "Pay now", + "Bestätigen", + "Abheben", + "Akzeptieren", + "Zustimmen", + "Weiter", + "Bezahlen", + "Confirmer", + "Retirer", + "Accepter", + "Payer", +] + +TOS_LABELS = [ + "I accept", + "Accept", + "Agree", + "Akzeptieren", + "Zustimmen", + "Accepter", +] + + +def try_tap_any(serial: str, xml: str, labels: Sequence[str]) -> Optional[str]: + for label in labels: + bs = find_bounds(xml, label) + if bs: + tap_bounds(serial, bs[0]) + return label + return None + + +def gui_drive( + serial: str, + out_dir: Path, + *, + rounds: int = 8, + sleep_s: float = 3.0, + phase: str = "gui", +) -> dict: + """Poll UI, dismiss ANR, tap confirm/ToS-like buttons. Returns status dict.""" + out_dir.mkdir(parents=True, exist_ok=True) + seen_texts: List[str] = [] + taps: List[str] = [] + anr = 0 + for i in range(1, rounds + 1): + xml_path = out_dir / f"{phase}-{i:02d}-ui.xml" + png_path = out_dir / f"{phase}-{i:02d}.png" + try: + xml = dump_ui(serial, xml_path) + except Exception as e: + taps.append(f"dump-fail:{e}") + time.sleep(sleep_s) + continue + try: + screencap(serial, png_path) + except Exception: + pass + ts = texts(xml) + seen_texts.extend(ts[:20]) + if dismiss_anr(serial, xml): + anr += 1 + time.sleep(sleep_s) + continue + hit = try_tap_any(serial, xml, CONFIRM_LABELS) + if hit: + taps.append(hit) + time.sleep(sleep_s) + continue + # partial regex for amount confirm rows + for label, b in find_bounds_re(xml, r"Confirm|Withdraw|Pay|Accept|Bestätig|Abheben|Payer"): + tap_bounds(serial, b) + taps.append(f"re:{label}") + time.sleep(sleep_s) + break + else: + time.sleep(sleep_s) + return { + "taps": taps, + "anr_dismissals": anr, + "sample_texts": seen_texts[-40:], + "rounds": rounds, + } diff --git a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh index 958ca84..277ad07 100755 --- a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh @@ -82,12 +82,18 @@ echo "copied → $STAMP_APK" echo "$COMMIT" >"$ROOT/out/build-commit.txt" 2>/dev/null || mkdir -p "$ROOT/out" && echo "$COMMIT" >"$ROOT/out/build-commit.txt" echo "$APK" >"$ROOT/out/build-apk-path.txt" -# Same smoke as published F-Droid path +# Same smoke as published F-Droid path (CLI hybrid or GUI) export APK_PATH="$STAMP_APK" export APK_NAME="$(basename "$STAMP_APK")" export PKG export OUT_DIR="${OUT_DIR:-$ROOT/out-source}" mkdir -p "$OUT_DIR" -echo "=== smoke with built APK (PKG=$PKG) ===" -exec env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" PKG="$PKG" OUT_DIR="$OUT_DIR" \ - "$SMOKE" +SMOKE_CMD="$SMOKE" +if [ "${GUI:-0}" = "1" ]; then + SMOKE_CMD="$ROOT/run-android-gui-smoke.sh" + OUT_DIR="${OUT_DIR%-source}-source-gui" + export OUT_DIR +fi +echo "=== smoke with built APK (PKG=$PKG · GUI=${GUI:-0}) ===" +exec env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" APK_PATH="$APK_PATH" PKG="$PKG" OUT_DIR="$OUT_DIR" \ + "$SMOKE_CMD" diff --git a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh new file mode 100755 index 0000000..241ab2c --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh @@ -0,0 +1,172 @@ +#!/usr/bin/env bash +# GUI-oriented Android smoke (vanilla level): deep-link entry + UI taps. +# +# Entry shortcut (legitimate — same as scanning a QR on the landing): +# adb VIEW taler://withdraw/… and taler://pay… +# Then drives the **graphical UI** via uiautomator dump + taps (Confirm / ToS / Pay). +# +# Systems: STACK=goa | stage | auto +# APK: published F-Droid or APK_PATH=… (from-source) +# +# See AUTOMATED-GUI-NOTES.md +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=run-android-pay-smoke.sh +# We re-use install/URI resolution by calling common pieces inline. + +export OUT_DIR="${OUT_DIR:-$ROOT/out-gui}" +export GUI_ROUNDS="${GUI_ROUNDS:-10}" +export GUI_SLEEP="${GUI_SLEEP:-3}" + +# Install + deliver intents via existing smoke, but skip its weak single-tap +# by setting GUI_MODE=1 which we implement here more fully. +# Simpler: do install/URI here by sourcing patterns from pay-smoke via env + python. + +# Delegate install + URI start to pay-smoke with GUI_AFTER=1 if we patch it; +# instead run a self-contained flow calling pay-smoke pieces. + +APK_DIR="${APK_DIR:-$ROOT/apks}" +APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" +APK_PATH="${APK_PATH:-}" +PKG="${PKG:-net.taler.wallet.fdroid}" +STACK="${STACK:-auto}" +SERIAL="${SERIAL:-}" +mkdir -p "$OUT_DIR" + +if ! command -v adb >/dev/null 2>&1; then + [ -x "$HOME/Android/Sdk/platform-tools/adb" ] && export PATH="$HOME/Android/Sdk/platform-tools:$PATH" +fi +command -v adb >/dev/null || { echo "adb missing" >&2; exit 2; } + +if [ -z "$SERIAL" ]; then + SERIAL=$(adb devices | awk '/\tdevice$/{print $1; exit}') +fi +[ -n "$SERIAL" ] || { echo "no adb device" >&2; exit 3; } +ADB=(adb -s "$SERIAL") +echo "device: $SERIAL mode: GUI (uiautomator)" + +case "$STACK" in + auto) + if curl -sfS -m 5 -o /dev/null https://bank.hacktivism.ch/config; then STACK=goa; else STACK=stage; fi + ;; +esac +case "$STACK" in + goa) + BANK=https://bank.hacktivism.ch + MERCHANT=https://taler.hacktivism.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + ;; + stage) + BANK=https://stage.bank.lefrancpaysan.ch + MERCHANT=https://stage.monnaie.lefrancpaysan.ch + WITHDRAW_JSON="$BANK/intro/demo-withdraw.json" + ;; + *) echo "unknown STACK" >&2; exit 2 ;; +esac +echo "stack: $STACK" + +if [ -n "$APK_PATH" ] && [ -f "$APK_PATH" ]; then + : +elif [ -f "$APK_DIR/$APK_NAME" ]; then + APK_PATH="$APK_DIR/$APK_NAME" +else + APK_PATH="$APK_DIR/$APK_NAME" + mkdir -p "$APK_DIR" + curl -fL --retry 3 -o "$APK_PATH" "$APK_URL" +fi +echo "apk: $APK_PATH pkg: $PKG" + +"${ADB[@]}" wait-for-device +"${ADB[@]}" install -r "$APK_PATH" +"${ADB[@]}" shell settings put global window_animation_scale 0 || true +"${ADB[@]}" shell settings put global transition_animation_scale 0 || true +"${ADB[@]}" shell settings put global animator_duration_scale 0 || true + +WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') +[ -n "$WURI" ] || { echo "no withdraw uri" >&2; exit 4; } +echo "withdraw: $WURI" +echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + +"${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true +"${ADB[@]}" logcat -c 2>/dev/null || true + +# --- Shortcut A: deep-link withdraw (≡ QR scan on landing) --- +"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" | tee "$OUT_DIR/start-withdraw.txt" +sleep 5 + +# --- Graphical: drive UI --- +python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd") +(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2)) +print("gui-withdraw taps:", st.get("taps")) +print("gui-withdraw anr:", st.get("anr_dismissals")) +PY + +# Pay URI +PAY_URI="" +if [ "$STACK" = "stage" ]; then + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c 'import json,sys;d=json.load(sys.stdin);oid=d.get("order_id")or"";tok=d.get("token")or""; +print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok) if oid and tok else "")') + fi +else + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" +fi + +if [ -n "$PAY_URI" ]; then + echo "pay: $PAY_URI" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + # Shortcut B: deep-link pay (≡ scan pay QR / open paywall link) + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" | tee "$OUT_DIR/start-pay.txt" + sleep 5 + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay") +(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2)) +print("gui-pay taps:", st.get("taps")) +print("gui-pay anr:", st.get("anr_dismissals")) +PY +fi + +"${ADB[@]}" logcat -d -t 400 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|preparePay|confirm|Error|success' "$OUT_DIR/logcat.txt" \ + | tail -100 >"$OUT_DIR/logcat-taler.txt" || true + +echo +echo "=== GUI smoke summary ($STACK) ===" +echo "package: $PKG" +echo "shortcut withdraw URI: delivered" +echo "GUI withdraw: see $OUT_DIR/gui-withdraw-status.json + screenshots" +[ -n "$PAY_URI" ] && echo "shortcut pay URI: delivered" +[ -n "$PAY_URI" ] && echo "GUI pay: see $OUT_DIR/gui-pay-status.json + screenshots" +if grep -q prepareBankIntegratedWithdrawal "$OUT_DIR/logcat.txt" 2>/dev/null; then + echo "wallet-core: prepareBankIntegratedWithdrawal YES" +else + echo "wallet-core: prepareBankIntegratedWithdrawal no (ANR/slow UI?)" +fi +if grep -qiE 'preparePay|confirmPay|pay-template|preparePurchase' "$OUT_DIR/logcat.txt" 2>/dev/null; then + echo "wallet-core: pay path YES" +else + echo "wallet-core: pay path not confirmed in logcat" +fi +echo "artifacts: $OUT_DIR/" +ls -la "$OUT_DIR" | sed 's/^/ /' +echo "Documented shortcuts → AUTOMATED-GUI-NOTES.md" From ed6fca0958bf538943ff75adb18f090c19017309 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:02:41 +0200 Subject: [PATCH 09/38] monitoring: port GOA GUI chain from taler-android inference branches Document dev/hernani-inference/gui-workflows and related fix branches; add Linux-capable run-goa-gui-chain.sh (mint + deep-link + UI taps). --- .../android-test/AUTOMATED-GUI-NOTES.md | 55 ++- .../taler-monitoring/android-test/README.md | 15 +- .../android-test/run-goa-gui-chain.sh | 345 ++++++++++++++++++ 3 files changed, 413 insertions(+), 2 deletions(-) create mode 100755 scripts/taler-monitoring/android-test/run-goa-gui-chain.sh diff --git a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md index 4013505..2e3993a 100644 --- a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md +++ b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md @@ -100,11 +100,61 @@ balance/settlement is not asserted (settlement still better covered by | `run-android-gui-smoke.sh` | **Android app UI** surface for the same stacks | | Combined | CLI = money path green; GUI = app UX green as far as taps go | +## Upstream: `taler-android` `dev/hernani-inference/*` + +Branch name is **`hernani-inference`** (not “inferenced”). Remote: +`git@git.taler.net:taler-android.git`. + +| Branch | Relevance for automation | +|--------|---------------------------| +| **`dev/hernani-inference/gui-workflows`** | **Primary:** macOS emulator helpers + **GOA GUI chain** (`scripts/goa-chain-emu.sh`, `docs/gui-workflows.md`, `setup-avds.sh`, `start-emulators.sh`). Deep-link + `uiautomator` text taps + bank API mint + shop templates. | +| **`fix/bank-withdraw-auto-exchange`** | App fix: resolve exchange when bank omits it (needed for GOA withdraw UI). | +| **`fix/withdraw-spinner-fallback`** | App fix: stop endless withdraw spinner (+ OIM UI work). | +| **`experimental-oim` / `experimental-oim-pos`** | Experimental cash denomination UI; optional for demos, not required for chain. | + +### Port into this tree (Linux-capable) + +| Upstream (gui-workflows) | Here (`android-test/`) | +|--------------------------|-------------------------| +| `goa-chain-emu.sh` | **`run-goa-gui-chain.sh`** (goa + stage, Linux paths, explorer secrets as monitoring ladder) | +| `tap_text` / `click_through` | same idea in chain + `lib_ui.py` | +| `mint_withdraw` explorer API | same (Basic → token → POST withdrawals) | +| `mint_pay` public templates | same (goa-shop products / stage fermes templates) | +| `android-env.sh` Homebrew | use `$HOME/Android/Sdk` + system `adb` (see GIT-AUTOMATION-NOTES) | +| macOS-only docs | **Linux supported** for chain/smoke; AVD setup still DIY on Linux | + +Open issues mirrored from upstream `ISSUES.md` (gui-workflows): + +- **I-and-gui-001** — taps still best-effort; ToS/confirm may need human +- **I-and-gui-003** — `EXP_PW_FILE` required for mint +- **I-and-gui-004** — emulator RAM cost (this host ~3.7 GiB) +- **I-and-gui-007** — for reliable withdraw GUI, build/install app from **fix/bank-withdraw-auto-exchange** or **fix/withdraw-spinner-fallback**, not plain master alone + +Recommended demo stack: + +```text +1) Checkout taler-android: fix/withdraw-spinner-fallback (or merge fixes) +2) ./gradlew :wallet:installFdroidDebug +3) Boot emulator / phone +4) android-test/run-goa-gui-chain.sh # STACK=goa|stage +``` + +```bash +# GOA chain (explorer mint + GUI taps) +EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ + STACK=goa PKG=net.taler.wallet.fdroid.debug \ + ./run-goa-gui-chain.sh + +# Stage TESTPAYSAN (explorer via stagepaysan SSH if EXP_PW_FILE unset) +STACK=stage ./run-goa-gui-chain.sh +``` + ## Future upgrades (not required for vanilla) - Maestro / Appium flows with stable resource-ids - Explicit wait for text “Withdrawal complete” / balance -- Separate jobs: F-Droid vs nightly source vs Play flavor +- Port Paivana unlock scripts (upstream I-and-gui-002) +- Dual AVD wallet+POS (upstream I-and-gui-008) - Record video (`adb shell screenrecord`) on failure ## Checklist — “vanille erreicht?” @@ -114,6 +164,9 @@ balance/settlement is not asserted (settlement still better covered by - [x] Source-build path (same smoke after build) - [x] Deep-link shortcuts documented - [x] Multi-round **graphical** taps (uiautomator) +- [x] GOA-style **chain** (multi withdraw + multi pay) from inference branch - [x] Screenshots + UI XML + logcat artifacts +- [x] Inference branch mapping documented - [ ] Fully reliable unattended Confirm→settled on low-RAM emulator (host-limited) +- [ ] App built from withdraw-fix branch by default in build script (optional) - [ ] Assert final balance in app UI (optional next step) diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 19df5ff..dab2365 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -38,6 +38,19 @@ STACK=stage ./run-android-gui-smoke.sh GUI=1 ./run-android-build-and-smoke.sh ``` -Documented shortcuts and limits: **[AUTOMATED-GUI-NOTES.md](./AUTOMATED-GUI-NOTES.md)**. +## 4) Full GOA/stage chain (from `dev/hernani-inference/gui-workflows`) + +Bank API mint → withdraw deep-link → taps → template pay → taps: + +```bash +# needs explorer password + device +EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ + STACK=goa PKG=net.taler.wallet.fdroid.debug \ + ./run-goa-gui-chain.sh + +STACK=stage ./run-goa-gui-chain.sh +``` + +Documented shortcuts, inference branches, limits: **[AUTOMATED-GUI-NOTES.md](./AUTOMATED-GUI-NOTES.md)**. See also parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh new file mode 100755 index 0000000..a1818d4 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -0,0 +1,345 @@ +#!/usr/bin/env bash +# GOA (and optionally stage) GUI chain on Android — adapted from +# taler-android branch **dev/hernani-inference/gui-workflows** +# (`scripts/goa-chain-emu.sh`, docs/gui-workflows.md). +# +# Linux + macOS: uses adb + uiautomator only (no Homebrew hard dependency). +# Upstream scripts are documented as macOS-only for SDK install paths. +# +# Flow: +# 1) API mint withdraw (explorer) → VIEW taler://withdraw → GUI taps +# 2) Merchant template POST → VIEW taler://pay → GUI taps +# +# Usage: +# ./run-goa-gui-chain.sh +# STACK=stage ./run-goa-gui-chain.sh +# WITHDRAW_AMOUNTS='GOA:10' PAY_LIMIT=2 ./run-goa-gui-chain.sh +# PKG=net.taler.wallet.fdroid.debug ./run-goa-gui-chain.sh --no-clear +# +# Env: ANDROID_SERIAL, PKG, EXP_PW_FILE, BANK, MERCHANT, INSTANCE, +# WITHDRAW_AMOUNTS, PRODUCTS, PAY_LIMIT, SHOTDIR, PAUSE +# +# See AUTOMATED-GUI-NOTES.md § "hernani-inference branches" +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) + +CLEAR=1 +for a in "$@"; do + case "$a" in + --no-clear) CLEAR=0 ;; + -h|--help) sed -n '2,28p' "$0"; exit 0 ;; + esac +done + +die() { echo "ERROR: $*" >&2; exit 1; } +step() { echo; echo "======== $* ========"; } +info() { echo " $*"; } +pause() { sleep "${1:-${PAUSE:-0.8}}"; } + +if ! command -v adb >/dev/null 2>&1; then + [ -x "$HOME/Android/Sdk/platform-tools/adb" ] && export PATH="$HOME/Android/Sdk/platform-tools:$PATH" +fi +command -v adb >/dev/null || die "adb not found" +command -v python3 >/dev/null || die "python3 required" + +STACK="${STACK:-goa}" +case "$STACK" in + goa|hacktivism) + PKG="${PKG:-net.taler.wallet.fdroid.debug}" + BANK="${BANK:-https://bank.hacktivism.ch}" + MERCHANT="${MERCHANT:-https://taler.hacktivism.ch}" + INSTANCE="${INSTANCE:-goa-shop}" + DONATE_INSTANCE="${DONATE_INSTANCE:-goa-demo-cp4zqk}" + DONATE_TEMPLATE="${DONATE_TEMPLATE:-goa-free}" + DEFAULT_AMOUNTS=("GOA:10" "GOA:20") + DEFAULT_PAYS=("product:orbit-sticker" "product:nebula-coffee" "product:voidwave-playlist" "donate:GOA:12") + CUR_HINT=GOA + ;; + stage|testpaysan) + PKG="${PKG:-net.taler.wallet.fdroid.debug}" + BANK="${BANK:-https://stage.bank.lefrancpaysan.ch}" + MERCHANT="${MERCHANT:-https://stage.monnaie.lefrancpaysan.ch}" + INSTANCE="${INSTANCE:-fermes-des-collines}" + DONATE_INSTANCE="${DONATE_INSTANCE:-fermes-des-collines}" + DONATE_TEMPLATE="${DONATE_TEMPLATE:-don-panier-libre}" + DEFAULT_AMOUNTS=("TESTPAYSAN:10" "TESTPAYSAN:20") + DEFAULT_PAYS=("product:panier-legumes" "product:fromage-chevre" "product:oeufs-6") + CUR_HINT=TESTPAYSAN + ;; + *) die "STACK must be goa|stage (got $STACK)" ;; +esac + +ACT="${ACT:-$PKG/net.taler.wallet.main.MainActivity}" +EXP_USER="${EXP_USER:-explorer}" +PAY_LIMIT="${PAY_LIMIT:-4}" +PAUSE="${PAUSE:-0.8}" +SHOTDIR="${SHOTDIR:-$ROOT/out-gui-chain/$(date +%Y%m%d-%H%M%S)-$STACK}" +mkdir -p "$SHOTDIR" + +# Explorer password (same search as taler-monitoring ladder) +if [[ -z "${EXP_PW_FILE:-}" ]]; then + for cand in \ + "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \ + "${HOME}/.config/taler-landing/bank-explorer-password.txt" + do + [[ -f "$cand" ]] && EXP_PW_FILE="$cand" && break + done +fi +# stage: try stagepaysan secret via ssh if missing +if [[ ! -f "${EXP_PW_FILE:-}" && "$STACK" = "stage" ]]; then + for host in francpaysan-stage-user francpaysan-host; do + tmp=$(mktemp) + if ssh -o BatchMode=yes -o ConnectTimeout=10 "$host" \ + 'tr -d "\n\r" "$tmp" 2>/dev/null && [[ -s "$tmp" ]]; then + EXP_PW_FILE="$tmp" + info "explorer password via $host" + break + fi + rm -f "$tmp" + done +fi +[[ -n "${EXP_PW_FILE:-}" && -f "$EXP_PW_FILE" ]] || die "set EXP_PW_FILE (explorer password)" + +if [[ -z "${ANDROID_SERIAL:-}" ]]; then + n=$(adb devices | awk 'NR>1 && $2=="device"{c++} END{print c+0}') + [[ "$n" -ge 1 ]] || die "no adb device (start emulator or plug phone)" + export ANDROID_SERIAL=$(adb devices | awk 'NR>1 && $2=="device"{print $1; exit}') +fi +info "ANDROID_SERIAL=$ANDROID_SERIAL STACK=$STACK PKG=$PKG SHOTDIR=$SHOTDIR" +ADB=(adb -s "$ANDROID_SERIAL") + +adb_sh() { "${ADB[@]}" shell "$@"; } + +shot() { + local n="$1" + adb_sh screencap -p "/sdcard/demo-$n.png" 2>/dev/null || true + "${ADB[@]}" pull "/sdcard/demo-$n.png" "$SHOTDIR/$n.png" >/dev/null 2>&1 || true + info "[shot] $SHOTDIR/$n.png" +} + +tap_text() { + local label="$1" + adb_sh uiautomator dump /sdcard/ui.xml >/dev/null 2>&1 || return 1 + "${ADB[@]}" pull /sdcard/ui.xml "$SHOTDIR/ui.xml" >/dev/null 2>&1 || return 1 + SERIAL="$ANDROID_SERIAL" python3 - "$label" "$SHOTDIR/ui.xml" <<'PY' +import os, re, subprocess, sys +from pathlib import Path +label, path = sys.argv[1], sys.argv[2] +serial = os.environ.get("SERIAL") or os.environ.get("ANDROID_SERIAL") or "" +xml = Path(path).read_text(errors="replace") +# ANR dismiss +if "isn't responding" in xml or "reagiert nicht" in xml.lower(): + for lab in ("Wait", "Warten"): + for m in re.finditer(rf'text="{lab}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', xml): + x = (int(m.group(1))+int(m.group(3)))//2 + y = (int(m.group(2))+int(m.group(4)))//2 + subprocess.run(["adb","-s",serial,"shell","input","tap",str(x),str(y)], check=False) + print(f"anr-wait @{x},{y}") + sys.exit(0) +for pat in [ + rf'text="{re.escape(label)}"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'text="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', + rf'content-desc="[^"]*{re.escape(label)}[^"]*"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', +]: + for m in re.finditer(pat, xml, re.I): + chunk = xml[max(0, m.start()-120):m.end()] + if 'enabled="false"' in chunk: + continue + x = (int(m.group(1))+int(m.group(3)))//2 + y = (int(m.group(2))+int(m.group(4)))//2 + print(f"tap {label!r} @{x},{y}") + cmd = ["adb","shell","input","tap",str(x),str(y)] + if serial: + cmd = ["adb","-s",serial,"shell","input","tap",str(x),str(y)] + subprocess.run(cmd, check=False) + sys.exit(0) +print("miss", label) +sys.exit(1) +PY +} + +mint_withdraw() { + local amount="$1" + EXP_USER="$EXP_USER" EXP_PW_FILE="$EXP_PW_FILE" BANK="$BANK" \ + python3 - "$amount" <<'PY' +import base64, json, os, ssl, sys, urllib.request +from pathlib import Path +amount = sys.argv[1] +bank = os.environ["BANK"].rstrip("/") +user = os.environ["EXP_USER"] +pw = Path(os.environ["EXP_PW_FILE"]).read_text().strip() +auth = "Basic " + base64.b64encode(f"{user}:{pw}".encode()).decode() +ctx = ssl.create_default_context() + +def req(method, url, data=None, headers=None): + h = dict(headers or {}) + body = None + if data is not None: + body = json.dumps(data).encode() + h["Content-Type"] = "application/json" + r = urllib.request.Request(url, data=body, headers=h, method=method) + with urllib.request.urlopen(r, context=ctx, timeout=25) as resp: + return json.load(resp) + +tok = req( + "POST", + f"{bank}/accounts/{user}/token", + {"scope": "readwrite", "duration": {"d_us": 3600_000_000}}, + {"Authorization": auth}, +)["access_token"] +wd = req( + "POST", + f"{bank}/accounts/{user}/withdrawals", + {"amount": amount}, + {"Authorization": f"Bearer {tok}"}, +) +print(wd.get("taler_withdraw_uri") or wd.get("talerWithdrawUri") or "") +PY +} + +mint_pay() { + local mode="$1" + MERCHANT="$MERCHANT" INSTANCE="$INSTANCE" \ + DONATE_INSTANCE="$DONATE_INSTANCE" DONATE_TEMPLATE="$DONATE_TEMPLATE" \ + python3 - "$mode" <<'PY' +import json, os, ssl, sys, urllib.error, urllib.request +mode = sys.argv[1] +merchant = os.environ["MERCHANT"].rstrip("/") +inst = os.environ["INSTANCE"] +d_inst = os.environ["DONATE_INSTANCE"] +d_tmpl = os.environ["DONATE_TEMPLATE"] +ctx = ssl.create_default_context() + +def post(url, data): + body = json.dumps(data).encode() + r = urllib.request.Request(url, data=body, method="POST", + headers={"Content-Type": "application/json"}) + try: + with urllib.request.urlopen(r, context=ctx, timeout=25) as resp: + return resp.status, json.load(resp) + except urllib.error.HTTPError as e: + raw = e.read() if hasattr(e, "read") else b"" + try: + return getattr(e, "code", None), json.loads(raw) + except Exception as err: + raise SystemExit(f"POST {url} failed: {e} body={raw[:200]!r}") from err + +if mode.startswith("product:"): + pid = mode.split(":", 1)[1] + st, d = post(f"{merchant}/instances/{inst}/templates/{pid}", {}) + use_inst = inst +elif mode.startswith("donate:"): + amt = mode.split(":", 1)[1] + st, d = post(f"{merchant}/instances/{d_inst}/templates/{d_tmpl}", {"amount": amt}) + use_inst = d_inst +else: + raise SystemExit(f"bad mode {mode!r}") + +if "taler_pay_uri" in d: + print(d["taler_pay_uri"]); raise SystemExit(0) +oid, tok = d.get("order_id"), d.get("token") +if not oid or not tok: + raise SystemExit(f"no order: {d!r}"[:200]) +host = merchant.replace("https://","").replace("http://","").split("/")[0] +print(f"taler://pay/{host}/instances/{use_inst}/{oid}/?c={tok}") +PY +} + +click_through() { + local labels=("$@") + local i lab + for i in 1 2 3 4 5 6 7 8; do + for lab in "${labels[@]}"; do + if tap_text "$lab" 2>/dev/null; then + pause 0.9 + fi + done + pause 0.4 + done +} + +if [[ -n "${WITHDRAW_AMOUNTS:-}" ]]; then + # shellcheck disable=SC2206 + AMOUNTS=($WITHDRAW_AMOUNTS) +else + AMOUNTS=("${DEFAULT_AMOUNTS[@]}") +fi +if [[ -n "${PRODUCTS:-}" ]]; then + PAYS=() + for p in $PRODUCTS; do PAYS+=("product:$p"); done +else + PAYS=("${DEFAULT_PAYS[@]}") +fi + +############################ +step "0 device + wallet ($STACK / $CUR_HINT)" +adb_sh input keyevent KEYCODE_WAKEUP 2>/dev/null || true +adb_sh wm dismiss-keyguard 2>/dev/null || true +if [[ "$CLEAR" -eq 1 ]]; then + info "pm clear $PKG" + adb_sh pm clear "$PKG" >/dev/null 2>&1 || true +fi +adb_sh am force-stop "$PKG" 2>/dev/null || true +pause 0.4 +# Prefer explicit MainActivity (from gui-workflows); fall back to VIEW launcher +if ! adb_sh am start -n "$ACT" >/dev/null 2>&1; then + info "MainActivity path failed — monkey launch" + adb_sh monkey -p "$PKG" -c android.intent.category.LAUNCHER 1 >/dev/null 2>&1 || true +fi +pause 2.0 +shot "00-start" + +############################ +step "2 withdraw chain" +idx=0 +for AMT in "${AMOUNTS[@]}"; do + idx=$((idx + 1)) + step "2.$idx withdraw $AMT" + URI="$(mint_withdraw "$AMT" | head -1)" + [[ -n "$URI" ]] || die "mint failed for $AMT" + info "URI=$URI" + echo "$URI" >>"$SHOTDIR/withdraw-uris.txt" + adb_sh am start -a android.intent.action.VIEW -d "$URI" "$PKG" >/dev/null + pause 1.5 + shot "w${idx}-open" + click_through "Accept" "I accept" "Accept terms" "Confirm" "Continue" "Withdraw" "Next" "OK" "Agree" "Bestätigen" "Abheben" "Akzeptieren" + pause 3.0 + shot "w${idx}-after" + adb_sh am start -n "$ACT" >/dev/null 2>&1 || true + pause 0.6 + tap_text "Assets" 2>/dev/null || true + pause 0.4 + shot "w${idx}-assets" +done + +############################ +step "3 pay chain (limit=$PAY_LIMIT)" +pidx=0 +for PAY in "${PAYS[@]}"; do + pidx=$((pidx + 1)) + [[ "$pidx" -gt "$PAY_LIMIT" ]] && break + step "3.$pidx pay $PAY" + if ! PAYURI="$(mint_pay "$PAY" | head -1)"; then + info "mint_pay failed for $PAY — skip" + continue + fi + [[ -n "$PAYURI" ]] || { info "empty pay uri — skip"; continue; } + info "PAYURI=$PAYURI" + echo "$PAYURI" >>"$SHOTDIR/pay-uris.txt" + adb_sh am start -a android.intent.action.VIEW -d "$PAYURI" "$PKG" >/dev/null + pause 1.5 + shot "p${pidx}-open" + click_through "Pay" "Confirm" "Accept" "Continue" "OK" "Next" "Bezahlen" "Bestätigen" "Payer" + pause 1.5 + shot "p${pidx}-after" + adb_sh am start -n "$ACT" >/dev/null 2>&1 || true + pause 0.5 +done + +step "4 done → $SHOTDIR" +ls -la "$SHOTDIR" | tail -40 +adb_sh dumpsys window 2>/dev/null | grep mCurrentFocus | head -1 || true +info "PKG=$PKG amounts=${AMOUNTS[*]} pays(limit)=$PAY_LIMIT" +echo "See AUTOMATED-GUI-NOTES.md (ported from taler-android dev/hernani-inference/gui-workflows)" From db343068c90c0bcc6a8a11de5e8d4e70e4125d75 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:02:50 +0200 Subject: [PATCH 10/38] monitoring: strip :443 from chain deep-link URIs --- scripts/taler-monitoring/android-test/run-goa-gui-chain.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh index a1818d4..af48da6 100755 --- a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -299,6 +299,8 @@ for AMT in "${AMOUNTS[@]}"; do step "2.$idx withdraw $AMT" URI="$(mint_withdraw "$AMT" | head -1)" [[ -n "$URI" ]] || die "mint failed for $AMT" + # wallet / monitoring QR rules: strip default ports + URI=$(printf '%s' "$URI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g') info "URI=$URI" echo "$URI" >>"$SHOTDIR/withdraw-uris.txt" adb_sh am start -a android.intent.action.VIEW -d "$URI" "$PKG" >/dev/null @@ -326,6 +328,7 @@ for PAY in "${PAYS[@]}"; do continue fi [[ -n "$PAYURI" ]] || { info "empty pay uri — skip"; continue; } + PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g; s/:80\//\//g') info "PAYURI=$PAYURI" echo "$PAYURI" >>"$SHOTDIR/pay-uris.txt" adb_sh am start -a android.intent.action.VIEW -d "$PAYURI" "$PKG" >/dev/null From cf5edc90e7dc40bf7ea03b58212ce199566aee22 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:22:45 +0200 Subject: [PATCH 11/38] monitoring: use inference fix branch by default; harden GUI chain ANR wait Default BUILD branch fix-bank-withdraw-auto-exchange; dismiss System UI ANR before confirm taps so low-RAM emulators fail less uselessly. --- .../android-test/AUTOMATED-GUI-NOTES.md | 23 +++++++++++++++---- .../out-gui-chain/run-211324/pay-uris.txt | 1 + .../out-gui-chain/run-211324/ui.xml | 1 + .../run-211324/withdraw-uris.txt | 1 + .../run-android-build-and-smoke.sh | 19 +++++++++++---- .../android-test/run-goa-gui-chain.sh | 22 +++++++++++++++--- 6 files changed, 55 insertions(+), 12 deletions(-) create mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt create mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml create mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt diff --git a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md index 2e3993a..7a9447a 100644 --- a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md +++ b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md @@ -130,15 +130,30 @@ Open issues mirrored from upstream `ISSUES.md` (gui-workflows): - **I-and-gui-004** — emulator RAM cost (this host ~3.7 GiB) - **I-and-gui-007** — for reliable withdraw GUI, build/install app from **fix/bank-withdraw-auto-exchange** or **fix/withdraw-spinner-fallback**, not plain master alone -Recommended demo stack: +### Ground rule (ops) + +1. **Bring it to run** first (scripts + live stack). +2. **App code changes only if needed**, and **minimal**. +3. Prefer **existing** `dev/hernani-inference/*` branches over new ones; extend them rather than forking random trees. + +### Recommended demo stack (2026-07) ```text -1) Checkout taler-android: fix/withdraw-spinner-fallback (or merge fixes) -2) ./gradlew :wallet:installFdroidDebug -3) Boot emulator / phone +1) taler-android checkout: + dev/hernani-inference/fix-bank-withdraw-auto-exchange + (or fix/withdraw-spinner-fallback for spinner + OIM) +2) ./gradlew :wallet:assembleFdroidDebug # package net.taler.wallet.fdroid.debug +3) Boot emulator / phone (prefer ≥6–8 GiB host RAM; 4 GiB → System UI ANR) 4) android-test/run-goa-gui-chain.sh # STACK=goa|stage ``` +`run-android-build-and-smoke.sh` defaults `BRANCH=dev/hernani-inference/fix-bank-withdraw-auto-exchange` +so source smokes pick up the GOA exchange auto-add fix without manual checkout. + +### Runtime note (clementine) + +On ~3.7 GiB hosts the emulator often shows **System UI isn’t responding** before Confirm is tappable. Chain script now prioritizes **Wait** over hunting Confirm. For a green end-to-end GUI path use a physical device or more RAM — the **mint + deep-link + fix-branch APK** path is otherwise ready. + ```bash # GOA chain (explorer mint + GUI taps) EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt new file mode 100644 index 0000000..6492537 --- /dev/null +++ b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt @@ -0,0 +1 @@ +taler://pay/taler.hacktivism.ch/instances/goa-shop/2026.198-93H450R000000/?c=1G3C43V1H89J8CQQ42TGDGVGMM diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml new file mode 100644 index 0000000..afdddb9 --- /dev/null +++ b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt new file mode 100644 index 0000000..e8e1589 --- /dev/null +++ b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt @@ -0,0 +1 @@ +taler://withdraw/bank.hacktivism.ch/taler-integration/39ba4ffc-f0e6-4e12-9e5b-d11fb8a2bcdd diff --git a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh index 277ad07..5779b88 100755 --- a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh @@ -19,6 +19,9 @@ ROOT=$(cd "$(dirname "$0")" && pwd) SMOKE="$ROOT/run-android-pay-smoke.sh" SRC="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" SKIP_PULL="${SKIP_PULL:-0}" +# Prefer existing inference fix branch for GOA withdraw (spinner/exchange resolve). +# Override: BRANCH=master or BRANCH=dev/hernani-inference/fix-withdraw-spinner-fallback +BRANCH="${BRANCH:-dev/hernani-inference/fix-bank-withdraw-auto-exchange}" VARIANT="${VARIANT:-fdroidDebug}" # assemble task suffix: FdroidDebug / NightlyDebug / … export ANDROID_HOME="${ANDROID_HOME:-$HOME/Android/Sdk}" export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" @@ -35,13 +38,19 @@ fi printf 'sdk.dir=%s\n' "$ANDROID_HOME" >"$SRC/local.properties" cd "$SRC" -if [ "$SKIP_PULL" != "1" ] && [ -d .git ]; then - echo "git pull --ff-only origin $(git rev-parse --abbrev-ref HEAD)" - git pull --ff-only origin "$(git rev-parse --abbrev-ref HEAD)" || \ - git fetch origin && git merge --ff-only "origin/$(git rev-parse --abbrev-ref HEAD)" || true +if [ -d .git ]; then + git fetch origin 2>/dev/null || true + if [ -n "$BRANCH" ] && git rev-parse --verify "origin/$BRANCH" >/dev/null 2>&1; then + echo "checkout origin/$BRANCH (minimal GOA/automation fixes; override BRANCH=)" + git checkout -B "$BRANCH" "origin/$BRANCH" 2>/dev/null || git checkout "$BRANCH" 2>/dev/null || true + fi + if [ "$SKIP_PULL" != "1" ]; then + br=$(git rev-parse --abbrev-ref HEAD) + git pull --ff-only origin "$br" 2>/dev/null || true + fi fi COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown) -echo "source: $SRC @ $COMMIT" +echo "source: $SRC @ $COMMIT (branch $(git rev-parse --abbrev-ref HEAD 2>/dev/null || ?))" echo "variant: assemble${VARIANT^}" 2>/dev/null || true # Map VARIANT → gradle task (fdroidDebug → assembleFdroidDebug) diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh index af48da6..571e825 100755 --- a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -249,14 +249,30 @@ PY click_through() { local labels=("$@") - local i lab - for i in 1 2 3 4 5 6 7 8; do + local i lab misses=0 + # Prefer dismissing ANR before hunting Confirm (low-RAM hosts thrash System UI) + for i in 1 2 3 4 5 6; do + if tap_text "Wait" 2>/dev/null || tap_text "Warten" 2>/dev/null; then + info "dismissed ANR (Wait)" + pause 2.0 + misses=0 + continue + fi + local hit=0 for lab in "${labels[@]}"; do if tap_text "$lab" 2>/dev/null; then + hit=1 + misses=0 pause 0.9 + break fi done - pause 0.4 + if [[ "$hit" -eq 0 ]]; then + misses=$((misses + 1)) + # stop early if hierarchy has nothing useful (avoid burning order deadlines) + [[ "$misses" -ge 3 ]] && break + fi + pause 0.5 done } From e51b3208c2a9176ffa0eb7c42a9a66f01ab48403 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:22:53 +0200 Subject: [PATCH 12/38] monitoring: gitignore gui-chain artifacts (do not track run dumps) --- scripts/taler-monitoring/android-test/.gitignore | 1 + .../android-test/out-gui-chain/run-211324/pay-uris.txt | 1 - .../android-test/out-gui-chain/run-211324/ui.xml | 1 - .../android-test/out-gui-chain/run-211324/withdraw-uris.txt | 1 - 4 files changed, 1 insertion(+), 3 deletions(-) delete mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt delete mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml delete mode 100644 scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index de62278..0867ff3 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -4,3 +4,4 @@ out/ out-source/ out-gui/ out-source-gui/ +out-gui-chain/ diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt deleted file mode 100644 index 6492537..0000000 --- a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/pay-uris.txt +++ /dev/null @@ -1 +0,0 @@ -taler://pay/taler.hacktivism.ch/instances/goa-shop/2026.198-93H450R000000/?c=1G3C43V1H89J8CQQ42TGDGVGMM diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml deleted file mode 100644 index afdddb9..0000000 --- a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/ui.xml +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt b/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt deleted file mode 100644 index e8e1589..0000000 --- a/scripts/taler-monitoring/android-test/out-gui-chain/run-211324/withdraw-uris.txt +++ /dev/null @@ -1 +0,0 @@ -taler://withdraw/bank.hacktivism.ch/taler-integration/39ba4ffc-f0e6-4e12-9e5b-d11fb8a2bcdd From c57f08b9d692bdbc9674baaaad43ce6f1c84b8c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:24:05 +0200 Subject: [PATCH 13/38] monitoring: GUI-AUTOMATION-NOTES + AUTO_ANDROID/AUTO_GUI flags Document minimal variant; Linux defaults to Android-only automation; macOS can enable both ecosystems later via flags. --- .../android-test/AUTOMATED-GUI-NOTES.md | 189 +----------------- .../android-test/GUI-AUTOMATION-NOTES.md | 179 +++++++++++++++++ .../taler-monitoring/android-test/README.md | 9 +- .../android-test/run-android-gui-smoke.sh | 19 +- .../android-test/run-android-pay-smoke.sh | 14 ++ .../android-test/run-goa-gui-chain.sh | 14 +- 6 files changed, 231 insertions(+), 193 deletions(-) create mode 100644 scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md diff --git a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md index 7a9447a..bfa5196 100644 --- a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md +++ b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md @@ -1,187 +1,4 @@ -# Automated GUI notes — Android wallet (vanilla level) +# Moved -Goal: tests for **two systems** (GOA / hacktivism and stage TESTPAYSAN) that -exercise the wallet **graphically in the UI** as far as practical, not only -CLI/`adb` deep-links into wallet-core. - -## Vanilla level (this tree) - -| Layer | What we do | Tooling | -|-------|------------|---------| -| **A — Install** | Install published F-Droid APK **or** APK built from `taler-android` master | `adb install` | -| **B — Entry (shortcut)** | Deliver withdraw / pay as `taler://…` intents | `adb shell am start -a VIEW -d …` | -| **C — Graphical UI** | Dump hierarchy, tap Confirm / ToS / Pay-like buttons, dismiss ANR | `uiautomator dump` + `input tap` (`lib_ui.py`) | -| **D — Evidence** | Screenshots + UI XML + logcat + JSON status | `android-test/out-gui/` | - -Scripts: - -| Script | Role | -|--------|------| -| `run-android-pay-smoke.sh` | Install + deep-link + light taps (hybrid / CLI-heavy) | -| `run-android-gui-smoke.sh` | **Vanilla GUI path**: deep-link entry + multi-round UI driving | -| `run-android-build-and-smoke.sh` | Build latest source APK → same smoke (set `GUI=1` to call GUI script) | -| `lib_ui.py` | Shared dump/tap/ANR helpers | - -```bash -cd scripts/taler-monitoring/android-test - -# Published APK, graphical drive, both stacks -STACK=goa ./run-android-gui-smoke.sh -STACK=stage ./run-android-gui-smoke.sh - -# From-source APK + GUI -APK_PATH=~/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ - PKG=net.taler.wallet.fdroid.debug \ - STACK=goa ./run-android-gui-smoke.sh -``` - -## Legitimate shortcuts (document, do not hide) - -These are **not** full human UX, but they are the same contracts as QR scan / -paywall open on the landings: - -| Shortcut | Replaces | Why legitimate | -|----------|----------|----------------| -| `adb` `VIEW` `taler://withdraw/…` | Camera QR scan of communal / demo withdraw | Identical URI payload as landings (`demo-withdraw.json`) | -| `adb` `VIEW` `taler://pay…` / `pay-template/…` | Scan shop/paywall QR or open “pay with Taler” | Same URI merchant/backend would show | -| Host-side `POST …/templates/{id}` then pay URI | Browser shop UI to create order | Stage farmer templates are public; order create is API, pay is still wallet UI | -| Multi-language label list for taps | Human reading the button | Same visible buttons; brittle if copy changes | -| Skip full ToS reading | User scrolling long TOS | Tap “Accept” if shown; no legal review automation | - -**Not automated yet (out of vanilla scope):** - -- Typing personal bank passwords into the wallet webview -- NFC -- Real camera / vision-based QR -- Biometric unlock -- Full onboarding wizard every time (we force-stop + re-intent) - -## Two systems under test - -| Stack | Withdraw source | Pay source | -|-------|-----------------|------------| -| **GOA** (`STACK=goa`) | `https://bank.hacktivism.ch/intro/demo-withdraw.json` | `taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana` (Paivana) | -| **stage** (`STACK=stage`) | `https://stage.bank.lefrancpaysan.ch/intro/demo-withdraw.json` | Public template e.g. `fermes-des-collines` / `panier-legumes` → `taler://pay/…` | - -Both use the **same** GUI driver (`lib_ui.py`); only URLs and package under test differ. - -## GUI driver behaviour (`lib_ui.gui_drive`) - -1. Every round: `uiautomator dump` + optional screencap. -2. If **“System UI isn’t responding”** → tap **Wait** (prefer over Close). -3. Else try exact taps on Confirm / Withdraw / Accept / Pay / DE/FR synonyms. -4. Else regex partial match on those words. -5. Repeat `GUI_ROUNDS` times (default 10), `GUI_SLEEP` seconds apart. - -Success signals (layered): - -- **Intent delivered** — `am start` OK -- **wallet-core** — logcat `prepareBankIntegratedWithdrawal` / pay-related ops -- **GUI taps** — non-empty `taps` in `gui-*-status.json` -- **Screenshots** — human-readable progress under `gui-withdraw/` / `gui-pay/` - -A run can be “vanilla-partial” if intents + some taps succeed but final -balance/settlement is not asserted (settlement still better covered by -`taler-monitoring.sh e2e` CLI wallet). - -## Host limits (clementine, 2026-07) - -| Constraint | Effect | -|------------|--------| -| ~3.7 GiB RAM | Emulator ANRs; GUI rounds often only dismiss ANR | -| Prefer physical phone or ≥6–8 GiB host | Real Confirm/Pay taps complete more often | -| No root required | User SDK + adb + F-Droid/source APK | - -## Relation to CLI e2e - -| Suite | Strength | -|-------|----------| -| `taler-monitoring.sh e2e` / `ladder` | Full withdraw+pay **settlement** via `taler-wallet-cli` | -| `run-android-gui-smoke.sh` | **Android app UI** surface for the same stacks | -| Combined | CLI = money path green; GUI = app UX green as far as taps go | - -## Upstream: `taler-android` `dev/hernani-inference/*` - -Branch name is **`hernani-inference`** (not “inferenced”). Remote: -`git@git.taler.net:taler-android.git`. - -| Branch | Relevance for automation | -|--------|---------------------------| -| **`dev/hernani-inference/gui-workflows`** | **Primary:** macOS emulator helpers + **GOA GUI chain** (`scripts/goa-chain-emu.sh`, `docs/gui-workflows.md`, `setup-avds.sh`, `start-emulators.sh`). Deep-link + `uiautomator` text taps + bank API mint + shop templates. | -| **`fix/bank-withdraw-auto-exchange`** | App fix: resolve exchange when bank omits it (needed for GOA withdraw UI). | -| **`fix/withdraw-spinner-fallback`** | App fix: stop endless withdraw spinner (+ OIM UI work). | -| **`experimental-oim` / `experimental-oim-pos`** | Experimental cash denomination UI; optional for demos, not required for chain. | - -### Port into this tree (Linux-capable) - -| Upstream (gui-workflows) | Here (`android-test/`) | -|--------------------------|-------------------------| -| `goa-chain-emu.sh` | **`run-goa-gui-chain.sh`** (goa + stage, Linux paths, explorer secrets as monitoring ladder) | -| `tap_text` / `click_through` | same idea in chain + `lib_ui.py` | -| `mint_withdraw` explorer API | same (Basic → token → POST withdrawals) | -| `mint_pay` public templates | same (goa-shop products / stage fermes templates) | -| `android-env.sh` Homebrew | use `$HOME/Android/Sdk` + system `adb` (see GIT-AUTOMATION-NOTES) | -| macOS-only docs | **Linux supported** for chain/smoke; AVD setup still DIY on Linux | - -Open issues mirrored from upstream `ISSUES.md` (gui-workflows): - -- **I-and-gui-001** — taps still best-effort; ToS/confirm may need human -- **I-and-gui-003** — `EXP_PW_FILE` required for mint -- **I-and-gui-004** — emulator RAM cost (this host ~3.7 GiB) -- **I-and-gui-007** — for reliable withdraw GUI, build/install app from **fix/bank-withdraw-auto-exchange** or **fix/withdraw-spinner-fallback**, not plain master alone - -### Ground rule (ops) - -1. **Bring it to run** first (scripts + live stack). -2. **App code changes only if needed**, and **minimal**. -3. Prefer **existing** `dev/hernani-inference/*` branches over new ones; extend them rather than forking random trees. - -### Recommended demo stack (2026-07) - -```text -1) taler-android checkout: - dev/hernani-inference/fix-bank-withdraw-auto-exchange - (or fix/withdraw-spinner-fallback for spinner + OIM) -2) ./gradlew :wallet:assembleFdroidDebug # package net.taler.wallet.fdroid.debug -3) Boot emulator / phone (prefer ≥6–8 GiB host RAM; 4 GiB → System UI ANR) -4) android-test/run-goa-gui-chain.sh # STACK=goa|stage -``` - -`run-android-build-and-smoke.sh` defaults `BRANCH=dev/hernani-inference/fix-bank-withdraw-auto-exchange` -so source smokes pick up the GOA exchange auto-add fix without manual checkout. - -### Runtime note (clementine) - -On ~3.7 GiB hosts the emulator often shows **System UI isn’t responding** before Confirm is tappable. Chain script now prioritizes **Wait** over hunting Confirm. For a green end-to-end GUI path use a physical device or more RAM — the **mint + deep-link + fix-branch APK** path is otherwise ready. - -```bash -# GOA chain (explorer mint + GUI taps) -EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ - STACK=goa PKG=net.taler.wallet.fdroid.debug \ - ./run-goa-gui-chain.sh - -# Stage TESTPAYSAN (explorer via stagepaysan SSH if EXP_PW_FILE unset) -STACK=stage ./run-goa-gui-chain.sh -``` - -## Future upgrades (not required for vanilla) - -- Maestro / Appium flows with stable resource-ids -- Explicit wait for text “Withdrawal complete” / balance -- Port Paivana unlock scripts (upstream I-and-gui-002) -- Dual AVD wallet+POS (upstream I-and-gui-008) -- Record video (`adb shell screenrecord`) on failure - -## Checklist — “vanille erreicht?” - -- [x] Two stacks parameterized (goa / stage) -- [x] Published APK path -- [x] Source-build path (same smoke after build) -- [x] Deep-link shortcuts documented -- [x] Multi-round **graphical** taps (uiautomator) -- [x] GOA-style **chain** (multi withdraw + multi pay) from inference branch -- [x] Screenshots + UI XML + logcat artifacts -- [x] Inference branch mapping documented -- [ ] Fully reliable unattended Confirm→settled on low-RAM emulator (host-limited) -- [ ] App built from withdraw-fix branch by default in build script (optional) -- [ ] Assert final balance in app UI (optional next step) +Canonical document: **[GUI-AUTOMATION-NOTES.md](./GUI-AUTOMATION-NOTES.md)** +(minimal variant, platform flags `AUTO_ANDROID` / `AUTO_GUI`, Linux defaults). diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md new file mode 100644 index 0000000..2fc90af --- /dev/null +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -0,0 +1,179 @@ +# GUI automation notes (Android wallet) + +Canonical notes for **graphical** Android wallet tests against GOA / stage. +Older name `AUTOMATED-GUI-NOTES.md` redirects here. + +--- + +## Minimal variant (implemented first) — status 2026-07 + +This is the **minimal vanilla** path we actually shipped; full dual-platform +GUI is planned via flags (below), not required to use these scripts today. + +| Piece | Status | Notes | +|-------|--------|--------| +| F-Droid published APK install | **done** | `net.taler.wallet.fdroid` 1.6.1 / 854 | +| Source build + same smoke | **done** | `:wallet:assembleFdroidDebug` → `net.taler.wallet.fdroid.debug` | +| Default source branch | **done** | `dev/hernani-inference/fix-bank-withdraw-auto-exchange` (minimal GOA exchange auto-add) | +| Deep-link entry (≡ QR) | **done** | `adb` `VIEW` `taler://withdraw/…` / `taler://pay…` | +| Explorer API mint (chain) | **done** | `run-goa-gui-chain.sh` (GOA + stage) | +| UI taps via uiautomator | **done** | best-effort Confirm/ToS/Pay; ANR → prefer **Wait** | +| Screenshots / logcat artifacts | **done** | under `out*`, `out-gui-chain/` (gitignored) | +| Host CLI e2e settlement | **separate** | `taler-monitoring.sh e2e` / `ladder` (not Android UI) | +| Reliable unattended Confirm on 4 GiB Linux host | **limited** | System UI ANR; use phone or more RAM | +| iOS GUI | **not in this tree** | see taler-ios `dev/hernani-inference/gui-workflows` | +| macOS full dual AVD helpers | **upstream** | taler-android `gui-workflows` (Homebrew); ported ideas here for Linux | + +**Ground rule:** bring it to run; app diffs only if needed and **minimal**; prefer +extending existing `dev/hernani-inference/*` branches over new trees. + +```bash +cd scripts/taler-monitoring/android-test + +# Minimal hybrid (install + deep-link + light taps) +STACK=goa ./run-android-pay-smoke.sh + +# GUI multi-round taps +STACK=goa ./run-android-gui-smoke.sh +STACK=stage ./run-android-gui-smoke.sh + +# Explorer mint + multi withdraw/pay chain (from gui-workflows) +EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt \ + STACK=goa PKG=net.taler.wallet.fdroid.debug \ + ./run-goa-gui-chain.sh + +# Build fix-branch APK then smoke (GUI=1 for gui smoke) +./run-android-build-and-smoke.sh +GUI=1 ./run-android-build-and-smoke.sh +``` + +--- + +## Platform capability flags (planned + defaults) + +Automation can run on hosts that support **Android only**, **GUI tooling only** +(conceptual), or **both**. Flags keep that explicit for CI and laptops. + +### Proposed env flags + +| Variable | Values | Meaning | +|----------|--------|---------| +| `AUTO_ANDROID` | `0` / `1` | Run Android wallet automation (adb, APK, emulator/device) | +| `AUTO_GUI` | `0` / `1` | Drive **graphical** UI (uiautomator taps, multi-round). If `0`, deep-link/CLI-only install smoke. | +| `AUTO_PLATFORM` | `auto` / `linux` / `macos` / `ios` | Host family (optional override) | + +**Semantics:** + +| `AUTO_ANDROID` | `AUTO_GUI` | Behaviour | +|----------------|------------|-----------| +| `1` | `0` | Android install + deep-link smoke only (no multi-round taps) | +| `1` | `1` | Android + graphical drive (vanilla GUI / chain) | +| `0` | `1` | Reserved (e.g. future desktop/web GUI); currently **no-op** with a clear message | +| `0` | `0` | Skip mobile automation | + +### Defaults by host (when flags unset) + +| Host (`uname -s`) | Default `AUTO_ANDROID` | Default `AUTO_GUI` | Rationale | +|-------------------|------------------------|--------------------|-----------| +| **Linux** | **`1`** | **`1`** if device/emulator present, else scripts exit 3 | This repo’s day-to-day path; **Android only** (no iOS here) | +| **Darwin (macOS)** | `1` | `1` | Can run Android emulators **and** (separately) taler-ios GUI helpers; both flags on for Android scripts; iOS is out of tree | +| **Other** | `0` | `0` | Fail closed | + +**Linux default = Android** (no second mobile platform in this suite). +**macOS** may enable both ecosystems in the wider monorepo sense; for *these* +scripts only Android is implemented — set `AUTO_ANDROID=1` (default) and keep +iOS under `taler-ios`. + +### Resolution helper (convention for future wrappers) + +```bash +# Example for a future run-all-mobile.sh +os=$(uname -s) +: "${AUTO_PLATFORM:=auto}" +case "$AUTO_PLATFORM" in + auto) case "$os" in Linux) AUTO_PLATFORM=linux ;; Darwin) AUTO_PLATFORM=macos ;; *) AUTO_PLATFORM=other ;; esac ;; +esac +case "$AUTO_PLATFORM" in + linux) + : "${AUTO_ANDROID:=1}" + : "${AUTO_GUI:=1}" + # no iOS + ;; + macos) + : "${AUTO_ANDROID:=1}" + : "${AUTO_GUI:=1}" + # optional later: AUTO_IOS=1 for taler-ios scripts + ;; + *) + : "${AUTO_ANDROID:=0}" + : "${AUTO_GUI:=0}" + ;; +esac +``` + +Scripts **today** implement Android only; they should honour: + +- `AUTO_ANDROID=0` → exit 0 with “skipped (AUTO_ANDROID=0)” +- `AUTO_GUI=0` → call deep-link smoke without multi-round GUI (or set `GUI_ROUNDS=0`) + +--- + +## Vanilla layers (detail) + +| Layer | What | Tooling | +|-------|------|---------| +| **A — Install** | F-Droid or from-source APK | `adb install` | +| **B — Entry (shortcut)** | `taler://withdraw/…` / `taler://pay…` | `adb am start -a VIEW` | +| **C — Graphical UI** | Confirm / ToS / Pay taps; ANR Wait | `uiautomator` + `lib_ui.py` / chain | +| **D — Evidence** | Screenshots, XML, logcat, URIs | `out*/` | + +### Legitimate shortcuts + +| Shortcut | Replaces | +|----------|----------| +| Deep-link withdraw/pay | Camera QR / opening paywall | +| Host `POST` template → pay URI | Browser shop checkout UI | +| Label-list taps | Human reading button text | +| Skip full ToS scroll | Tap Accept if shown | + +### Scripts + +| Script | Role | +|--------|------| +| `run-android-pay-smoke.sh` | Hybrid: install + deep-link + light taps | +| `run-android-gui-smoke.sh` | Multi-round GUI drive | +| `run-goa-gui-chain.sh` | Multi mint/withdraw/pay (gui-workflows port) | +| `run-android-build-and-smoke.sh` | Build inference fix branch + smoke (`GUI=1` optional) | +| `lib_ui.py` | Dump/tap/ANR helpers | + +--- + +## Stacks under test + +| Stack | Withdraw | Pay | +|-------|----------|-----| +| **GOA** | explorer mint / `demo-withdraw.json` @ bank.hacktivism.ch | goa-shop templates / Paivana pay-template | +| **stage** | explorer / demo-withdraw @ stage.bank… | fermes / jardin public templates | + +--- + +## Upstream branches (`taler-android` `dev/hernani-inference/*`) + +| Branch | Role | +|--------|------| +| `gui-workflows` | macOS emulator helpers + original `goa-chain-emu.sh` | +| `fix/bank-withdraw-auto-exchange` | **Minimal** GOA exchange auto-add (default build branch) | +| `fix/withdraw-spinner-fallback` | Spinner + OIM; use if spinner still hangs | +| `experimental-oim*` | Optional cash UI | + +--- + +## Checklist + +- [x] Minimal variant documented +- [x] Two stacks (goa / stage) +- [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults) +- [x] Linux default = Android suite only +- [x] Inference-branch policy +- [ ] Wrapper enforces flags in every entry script (incremental) +- [ ] Reliable unattended Confirm on low-RAM Linux emulator diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index dab2365..e1ab975 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -51,6 +51,13 @@ EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank- STACK=stage ./run-goa-gui-chain.sh ``` -Documented shortcuts, inference branches, limits: **[AUTOMATED-GUI-NOTES.md](./AUTOMATED-GUI-NOTES.md)**. +Canonical notes (minimal variant, **flags**, Linux defaults): +**[GUI-AUTOMATION-NOTES.md](./GUI-AUTOMATION-NOTES.md)**. + +| Flag | Linux default | Meaning | +|------|---------------|---------| +| `AUTO_ANDROID` | `1` | Run Android adb/APK automation | +| `AUTO_GUI` | `1` | Graphical uiautomator drive (else deep-link only) | +| `AUTO_PLATFORM` | `auto` → `linux` | Host family; macOS can later pair with iOS out of tree | See also parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh index 241ab2c..de280f5 100755 --- a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh @@ -8,12 +8,23 @@ # Systems: STACK=goa | stage | auto # APK: published F-Droid or APK_PATH=… (from-source) # -# See AUTOMATED-GUI-NOTES.md +# See GUI-AUTOMATION-NOTES.md # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) -# shellcheck source=run-android-pay-smoke.sh -# We re-use install/URI resolution by calling common pieces inline. + +# Platform flags (GUI-AUTOMATION-NOTES.md). Linux → Android only. +os=$(uname -s) +case "${AUTO_PLATFORM:-auto}" in + auto) case "$os" in Linux|Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; esac ;; + linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; +esac +[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; } +if [ "${AUTO_GUI}" != "1" ]; then + echo "AUTO_GUI=0 → deep-link smoke only" + exec env AUTO_ANDROID=1 "$ROOT/run-android-pay-smoke.sh" +fi export OUT_DIR="${OUT_DIR:-$ROOT/out-gui}" export GUI_ROUNDS="${GUI_ROUNDS:-10}" @@ -169,4 +180,4 @@ else fi echo "artifacts: $OUT_DIR/" ls -la "$OUT_DIR" | sed 's/^/ /' -echo "Documented shortcuts → AUTOMATED-GUI-NOTES.md" +echo "Documented shortcuts → GUI-AUTOMATION-NOTES.md" diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh index 907cb07..690c222 100755 --- a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -20,6 +20,20 @@ # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) + +# Platform flags (see GUI-AUTOMATION-NOTES.md). Linux default: Android only. +os=$(uname -s) +case "${AUTO_PLATFORM:-auto}" in + auto) case "$os" in Linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; esac ;; + linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; +esac +if [ "${AUTO_ANDROID}" != "1" ]; then + echo "skipped: AUTO_ANDROID=${AUTO_ANDROID} (platform flags — see GUI-AUTOMATION-NOTES.md)" + exit 0 +fi + APK_DIR="${APK_DIR:-$ROOT/apks}" APK_URL="${APK_URL:-https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" APK_NAME="${APK_NAME:-net.taler.wallet.fdroid_854.apk}" diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh index 571e825..52e4328 100755 --- a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -19,7 +19,7 @@ # Env: ANDROID_SERIAL, PKG, EXP_PW_FILE, BANK, MERCHANT, INSTANCE, # WITHDRAW_AMOUNTS, PRODUCTS, PAY_LIMIT, SHOTDIR, PAUSE # -# See AUTOMATED-GUI-NOTES.md § "hernani-inference branches" +# See GUI-AUTOMATION-NOTES.md # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) @@ -37,6 +37,16 @@ step() { echo; echo "======== $* ========"; } info() { echo " $*"; } pause() { sleep "${1:-${PAUSE:-0.8}}"; } +# Platform flags — Linux default Android+GUI (see GUI-AUTOMATION-NOTES.md) +os=$(uname -s) +case "${AUTO_PLATFORM:-auto}" in + auto) case "$os" in Linux|Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; esac ;; + linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; +esac +[ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; } +[ "${AUTO_GUI}" = "1" ] || { echo "skipped: AUTO_GUI=${AUTO_GUI} (chain is GUI)"; exit 0; } + if ! command -v adb >/dev/null 2>&1; then [ -x "$HOME/Android/Sdk/platform-tools/adb" ] && export PATH="$HOME/Android/Sdk/platform-tools:$PATH" fi @@ -361,4 +371,4 @@ step "4 done → $SHOTDIR" ls -la "$SHOTDIR" | tail -40 adb_sh dumpsys window 2>/dev/null | grep mCurrentFocus | head -1 || true info "PKG=$PKG amounts=${AMOUNTS[*]} pays(limit)=$PAY_LIMIT" -echo "See AUTOMATED-GUI-NOTES.md (ported from taler-android dev/hernani-inference/gui-workflows)" +echo "See GUI-AUTOMATION-NOTES.md (ported from taler-android dev/hernani-inference/gui-workflows)" From c8a7eef8d45a2f1aaa815d04b28c20fd9dba76fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:24:35 +0200 Subject: [PATCH 14/38] monitoring: rename notes to GUI-AUTOMATION-NOTES.md only Wrong filename AUTOMATED-GUI-NOTES was a mistake; drop redirect stub. --- scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md | 4 ---- scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md | 1 - 2 files changed, 5 deletions(-) delete mode 100644 scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md diff --git a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md b/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md deleted file mode 100644 index bfa5196..0000000 --- a/scripts/taler-monitoring/android-test/AUTOMATED-GUI-NOTES.md +++ /dev/null @@ -1,4 +0,0 @@ -# Moved - -Canonical document: **[GUI-AUTOMATION-NOTES.md](./GUI-AUTOMATION-NOTES.md)** -(minimal variant, platform flags `AUTO_ANDROID` / `AUTO_GUI`, Linux defaults). diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 2fc90af..95179b5 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -1,7 +1,6 @@ # GUI automation notes (Android wallet) Canonical notes for **graphical** Android wallet tests against GOA / stage. -Older name `AUTOMATED-GUI-NOTES.md` redirects here. --- From 59ee193f9890379db1e50a05b153e445347a970a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:25:37 +0200 Subject: [PATCH 15/38] monitoring: CLI-AUTOMATION-NOTES for recurring wallet-cli issues Prioritised list from e2e/ladder automation (run-until-done, settle, reserve_pub, exchange auto-add, URI ports, JSON) to guide CLI/core fixes. --- .../taler-monitoring/CLI-AUTOMATION-NOTES.md | 228 ++++++++++++++++++ scripts/taler-monitoring/README.md | 4 +- 2 files changed, 231 insertions(+), 1 deletion(-) create mode 100644 scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md diff --git a/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md new file mode 100644 index 0000000..3cd311a --- /dev/null +++ b/scripts/taler-monitoring/CLI-AUTOMATION-NOTES.md @@ -0,0 +1,228 @@ +# CLI automation notes (`taler-wallet-cli` + monitoring) + +Living list of **recurring issues** seen while driving GOA (hacktivism) and +stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring +e2e/ladder). Ordered roughly by **how often they hurt automation** and how +useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be. + +Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md). + +Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`. + +Legend: + +| Tag | Meaning | +|-----|---------| +| **cli** | wallet-cli UX / flags / hang behaviour | +| **core** | wallet-core protocol / state machine | +| **bank** | libeufin-bank / integration API | +| **ops** | secrets, paths, multi-host | +| **doc** | missing or wrong documentation | + +--- + +## 1. `run-until-done` is unusable for automation + +| | | +|--|--| +| **Seen** | e2e, ladder, Android notes, macOS comments: *hangs / banned in monitoring* | +| **Area** | **cli** / **core** | +| **Problem** | `taler-wallet-cli run-until-done` (or long shepherd runs) often **never returns** or blocks CI-length timeouts. Monitoring **forbids** it and polls balance + bank `transfer_done` instead. | +| **Workaround** | `advanced serve` + socket (`WSOCK`) or pure poll loops (`LADDER_SETTLE_*`, e2e settle wait). | +| **Wanted for wallet-cli** | Bounded wait: e.g. `run-until-done --timeout=Ns --exit-on=withdrawn|paid|idle`; stable non-zero exit on timeout; progress on stdout/JSON. | + +--- + +## 2. Withdraw completion is not observable from CLI alone + +| | | +|--|--| +| **Seen** | e2e “coins missing after withdraw”; ladder `OK_BANK` vs wallet avail | +| **Area** | **cli** / **core** / **bank** | +| **Problem** | After `accept-uri` + bank confirm, coins may lag (wirewatch). CLI has no single “withdraw settled” command with clear success/failure; automation reimplements status from bank integration JSON + balance polls. | +| **Workaround** | Poll `balance` + `…/withdrawal-operation/{id}` (`status`, `transfer_done`, `selected_reserve_pub`). | +| **Wanted** | `wallet-cli withdrawals wait --id=… --timeout=` or JSON event stream: `selected → confirmed → coins-available`. | + +--- + +## 3. `reserve_pub` / force-select is fragile (5114) + +| | | +|--|--| +| **Seen** | ladder force-select, e2e confirm path, Android spinner issues | +| **Area** | **cli** / **core** / **bank** | +| **Problem** | Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). Automation scrapes accept/tx dumps for candidates. | +| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. | +| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. | + +--- + +## 4. Exchange not auto-known for bank deep links / withdraw URIs + +| | | +|--|--| +| **Seen** | Android endless spinner; GOA `bank.hacktivism.ch` withdraw; fixed in app branch `fix/bank-withdraw-auto-exchange` | +| **Area** | **core** / **cli** | +| **Problem** | Withdraw URI names an exchange the wallet has never added → stuck loading / empty exchange list. CLI requires explicit `exchanges add` + `update` + `accept-tos` every fresh DB. | +| **Workaround** | e2e always adds exchange before ATM ladder; Android needs ensureExchange-style fix. | +| **Wanted** | CLI/core: on `accept-uri` withdraw, **auto-add** exchange from URI/bank details (`allowCompletion`), then surface ToS if needed in one command. | + +--- + +## 5. Default ports in URIs (`:443` / `:80`) break round-trips + +| | | +|--|--| +| **Seen** | ladder/e2e strip ports; QR checks; mint URIs with `:443` | +| **Area** | **cli** / **bank** / **doc** | +| **Problem** | Some bank/wallet outputs include `taler://…host:443/…`. QR validators and some clients reject or double-normalize inconsistently. | +| **Workaround** | `sed 's/:443\//\//g'` everywhere in automation. | +| **Wanted** | Canonical form without default ports in **all** wallet-cli and bank integration outputs; document as invariant. | + +--- + +## 6. No first-class “demo withdraw” / explorer mint in wallet-cli + +| | | +|--|--| +| **Seen** | e2e uses bank admin + ATM create; ladder/gui-chain reimplement explorer mint in Python | +| **Area** | **cli** / **ops** | +| **Problem** | Automation always hand-rolls Basic auth → token → `POST …/withdrawals` → `taler_withdraw_uri`. Easy to get wrong (auth header, amount currency). | +| **Workaround** | Shared Python in `goa-chain` / ladder / e2e; secrets files. | +| **Wanted** | Optional helper: `wallet-cli testing mint-withdraw --bank=URL --user=explorer --password-file=… --amount=GOA:10` (or bank-side only tool in libeufin) emitting a clean URI. | + +--- + +## 7. Long-lived wallet process required for pay/withdraw progress + +| | | +|--|--| +| **Seen** | e2e `advanced serve` / shepherd socket | +| **Area** | **cli** | +| **Problem** | One-shot CLI invocations do not keep background work alive; without serve/shepherd, withdraw/pay may stall mid-flight. | +| **Workaround** | Start serve once per e2e run; route `wcli` through the socket. | +| **Wanted** | Documented, supported “session mode”: start/stop serve; or each mutating command optionally `--background-until=…` with timeout. | + +--- + +## 8. ToS accept is a separate step that fails silently or blocks + +| | | +|--|--| +| **Seen** | e2e `exchanges accept-tos`; ladder after add; Android GUI ToS screens | +| **Area** | **cli** / **core** | +| **Problem** | Fresh exchange → operations need ToS; CLI must call `accept-tos` explicitly. Failures are easy to miss in multi-step scripts. | +| **Workaround** | Always `accept-tos` after `update` in e2e/ladder. | +| **Wanted** | `accept-uri --accept-tos` or auto-prompt with non-interactive `--yes` that covers exchange ToS for that URI’s exchange. | + +--- + +## 9. Pay template / public order path is merchant-shaped, not CLI-shaped + +| | | +|--|--| +| **Seen** | e2e shop templates, stage farmer shops, ladder private orders | +| **Area** | **cli** / **doc** | +| **Problem** | Wallet-cli pays via `handle-uri` on `taler://pay/…`; creating the order is always custom curl (template POST or private order + token). No unified “pay this amount to instance” for public templates. | +| **Workaround** | Monitoring builds URI externally then `handle-uri --yes`. | +| **Wanted** | Documented recipe only, or `wallet-cli testing pay-template --base=… --instance=… --id=…` for demos. | + +--- + +## 10. Amount / currency parsing edge cases + +| | | +|--|--| +| **Seen** | ladder min denom 0.01 vs GOA 1e-6; zero withdraw rejected (HTTP 409 / amount too low); max wire ceilings | +| **Area** | **cli** / **bank** | +| **Problem** | `CURRENCY:0` and sub-min amounts fail differently per stack; CLI error strings are not machine-stable. | +| **Workaround** | Soft-skip zero rung; clamp ladder min to exchange denoms / bank max_wire. | +| **Wanted** | Structured errors (`AMOUNT_too_small`, `currency_unknown`) in JSON mode; `wallet-cli amount validate --exchange=`. | + +--- + +## 11. Finding the right `taler-wallet-cli` binary + +| | | +|--|--| +| **Seen** | `find_wallet_cli` in `lib.sh`; hardcoded laptop paths; wrapper vs `.mjs` | +| **Area** | **ops** / **cli** | +| **Problem** | Debian package is a shell wrapper; some tools need `node …/taler-wallet-cli.mjs`. Hardcoded `/Users/…` paths break other hosts. | +| **Workaround** | `find_wallet_cli` search list; `WALLET_CLI=` override. | +| **Wanted** | Single install story: `wallet-cli --version` JSON with path + libversion; no need to pass `.mjs` to node by hand. | + +--- + +## 12. Secrets and multi-stack confusion + +| | | +|--|--| +| **Seen** | explorer vs admin password; GOA secrets used on stage; ladder EXP_PW | +| **Area** | **ops** | +| **Problem** | CLI does not know “which stack”; wrong password → 401 mid-ladder. Not a wallet-cli bug, but every CLI automation hits it. | +| **Workaround** | `SECRETS_ROOT`, stage SSH, `STACK=` profiles. | +| **Wanted** | Optional `~/.config/taler/stacks.d/goa.env` convention documented next to wallet-cli; still no secrets in repo. | + +--- + +## 13. No stable machine-readable “step result” for scripts + +| | | +|--|--| +| **Seen** | e2e greps accept output; ladder scrapes JSON from mixed stdout | +| **Area** | **cli** | +| **Problem** | Human logs + occasional JSON blobs; hard to parse reliably. | +| **Workaround** | Python scrapers, temp files, `tee`. | +| **Wanted** | Global `--json` / `--ndjson` for all commands; one object per completed operation with `ok`, `op`, `ids`, `amounts`. | + +--- + +## 14. Pay settlement wait is symmetric to withdraw pain + +| | | +|--|--| +| **Seen** | e2e pay settle loops; ladder pay settle rounds | +| **Area** | **cli** / **core** | +| **Problem** | After `handle-uri` pay, success is “order paid” / balance drop / merchant order status — not one CLI wait. | +| **Workaround** | Short poll loops; never run-until-done. | +| **Wanted** | Same as §1–2: bounded wait on transaction id / order id. | + +--- + +## Priority shortlist for `taler-wallet-cli` / core + +If only a few changes land, these unlock the most automation: + +1. **Bounded `run-until-done` / wait-for-state** (§1, §2, §14) +2. **Structured JSON on every command** (§13) +3. **Auto-add exchange + ToS on accept-uri** (§4, §8) +4. **Canonical URIs without :443** (§5) +5. **Emit reserve_pub for last withdraw** (§3) + +--- + +## How we work around today (monitoring) + +| Issue | Monitoring behaviour | +|-------|----------------------| +| run-until-done | Disabled; balance + bank poll | +| serve | Optional long-lived socket in e2e | +| exchange | Explicit add/update/accept-tos | +| ladder explorer | `read_secret` / EXP_PW_FILE | +| stage maxima | bank `max_wire` + keys min denom | +| force-select | multi-rpub try; soft-skip confirm | + +--- + +## How to add an issue + +```markdown +### N. short title +| | | +|--|--| +| **Seen** | where / which stack | +| **Area** | cli / core / bank / ops / doc | +| **Problem** | … | +| **Workaround** | … | +| **Wanted** | concrete CLI/core behaviour | +``` diff --git a/scripts/taler-monitoring/README.md b/scripts/taler-monitoring/README.md index 35dba91..eea5e12 100644 --- a/scripts/taler-monitoring/README.md +++ b/scripts/taler-monitoring/README.md @@ -14,7 +14,9 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs** IDs reset per **group** inside an area — so “too many www tests” become `www.exchange-*`, `www.bank-*`, `www.landing-*`, etc. Catalog: **[TESTS.md](./TESTS.md)**. -Android UI automation prerequisites: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)** (tools **not** installed by default). +Android / tooling: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**. +GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**. +CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**. ```text ┌ OK ┐ www.exchange-01 exchange /config · HTTP 200 From 10054b947b67f4bce86e99fb59f955f60add612f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:26:37 +0200 Subject: [PATCH 16/38] =?UTF-8?q?docs(android-test):=20Minimalvariante=20l?= =?UTF-8?q?imits=20=E2=80=94=20was=20geht=20/=20was=20nicht?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Document GUI smoke boundaries: deep-link + best-effort taps work; unattended settlement and full UI E2E on low-RAM emulator do not. --- .../android-test/GUI-AUTOMATION-NOTES.md | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 95179b5..fce915e 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -26,6 +26,66 @@ GUI is planned via flags (below), not required to use these scripts today. **Ground rule:** bring it to run; app diffs only if needed and **minimal**; prefer extending existing `dev/hernani-inference/*` branches over new trees. +### Minimalvariante: was geht / was nicht + +Einschränkungen der **aktuell implementierten** Minimalvariante (deep-link + +uiautomator best-effort). Kein voller Dual-Platform-GUI-Stack, kein Ersatz für +Host-CLI-Settlement. + +#### Geht (grün) + +| Fähigkeit | Wie / Hinweis | +|-----------|----------------| +| APK installieren | F-Droid `net.taler.wallet.fdroid` **oder** Source `assembleFdroidDebug` (`.debug`) | +| GOA-Exchange ohne manuelles Add | Source-Build von `fix-bank-withdraw-auto-exchange` (Default in `run-android-build-and-smoke.sh`) | +| Withdraw/Pay **starten** | `adb` deep-link `taler://withdraw/…` / `taler://pay…` (Ersatz für QR-Scan) | +| Light UI-Drive | Confirm / Accept / Pay-Taps über Label-Suche (`lib_ui.py`, gui-smoke, chain) | +| Multi-Runden-Taps | `run-android-gui-smoke.sh` / `run-goa-gui-chain.sh` (best-effort) | +| Explorer-Mint (Kette) | `run-goa-gui-chain.sh` + `EXP_PW_FILE` (GOA + stage) | +| Stack-Wahl | `STACK=goa` / `stage` (Bank/Merchant-URLs) | +| Artefakte | Screenshots, UI-XML, logcat unter `out*` / `out-gui-chain/` (gitignored) | +| Hybrid-Smoke | Install + Intent + leichte Taps: `run-android-pay-smoke.sh` | +| Settlement **nachweisen** | **Host-seitig**: `taler-monitoring.sh e2e` / `ladder` (nicht Android-UI) | +| Flags abschalten | `AUTO_ANDROID=0` skip; `AUTO_GUI=0` → nur deep-link/pay-smoke | + +#### Geht **nicht** oder nur eingeschränkt (rot / gelb) + +| Beschränkung | Folge / Workaround | +|--------------|--------------------| +| **Unattended Confirm → settled im Emulator (~4 GiB Host)** | Häufig **System UI ANR**; Taps bleiben hängen. Prefer **Wait** auf ANR-Dialog, echtes Gerät, oder Host mit mehr RAM. Kein zuverlässiges grünes E2E nur über GUI auf dem schwachen Emulator. | +| **Vollständige Wallet-Settlement-Assertion in der GUI** | Minimalvariante prüft **nicht** „Balance final / transfer_done“ in der App-Oberfläche. Proof bleibt CLI/e2e. | +| **Kamera / physischer QR-Scan** | Absichtlich nicht automatisiert; deep-link ist der Shortcut. Landing-QR-Pfade sind **HTTP**-Thema, nicht Android-GUI. | +| **Browser-Shop-Checkout-UI** | Kein Chromium/WebView-Drive; Pay-URI kommt per Template-`POST` + Intent. | +| **Volles ToS-Scroll / alle Dialoge** | Best-effort Accept-Tap; lange ToS oder unerwartete Sheets können hängen bleiben. | +| **Published F-Droid allein auf GOA** | Kann Exchange-Add / Spinner-Probleme zeigen → **fix-branch-APK** bauen; bei Spinner-Hang optional `fix-withdraw-spinner-fallback`. | +| **iOS-GUI** | Nicht in diesem Tree; siehe taler-ios `gui-workflows`. | +| **macOS dual AVD / Homebrew-Helpers** | Upstream `gui-workflows`; hier nur Linux-taugliche Port-Ideen. | +| **`AUTO_*` in jedem Script erzwungen** | Design + teilweise verdrahtet; Wrapper-Checkliste noch offen. | +| **`run-until-done` / CLI-Wallet in GUI-Skripten** | Gehört zu CLI-Automation, nicht GUI; siehe `CLI-AUTOMATION-NOTES.md`. | +| **CI grün = „User hat bezahlt“** | Smoke = Intent + best-effort UI. Wirtschaftlicher Erfolg = Host-e2e / ladder / Explorer. | + +#### Erwartetes Ergebnis pro Schicht + +| Schicht | Erfolgskriterium Minimalvariante | +|---------|----------------------------------| +| A Install | APK installed, App startet | +| B Deep-link | Intent delivered, Wallet öffnet Withdraw/Pay-Flow | +| C GUI taps | Dump+Tap ohne Crash; ANR ggf. mit Wait; **kein** Garant für „Paid“ | +| D Evidence | Artefakte geschrieben | +| Settlement | **Außerhalb** GUI: Host e2e/ladder | + +#### Wann Minimalvariante reicht + +- Smoke nach APK-/Branch-Build (Intent kommt an, App crasht nicht). +- Manuelle Nacharbeit am Gerät nach vorbereiteten URIs. +- GOA/stage **Erreichbarkeit** von Landing/Template + Wallet-Einstieg. + +#### Wann nicht reicht → nächster Schritt + +- Unattended grünes Pay-E2E auf dem Emulator → mehr RAM / physisches Gerät, ggf. `fix-withdraw-spinner-fallback`. +- Beweis „Münzen da / Merchant paid“ → CLI e2e/ladder, nicht GUI-smoke. +- iOS / Dual-Platform → andere Repos/Flags, nicht diese Skripte. + ```bash cd scripts/taler-monitoring/android-test @@ -170,6 +230,7 @@ Scripts **today** implement Android only; they should honour: ## Checklist - [x] Minimal variant documented +- [x] Minimalvariante: was geht / was nicht (Beschränkungen) - [x] Two stacks (goa / stage) - [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults) - [x] Linux default = Android suite only From a935f69582abae7ab45a605ef53f6116f3d9a9a0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:28:14 +0200 Subject: [PATCH 17/38] =?UTF-8?q?docs(paivana):=20clear=20postmortem=20?= =?UTF-8?q?=E2=80=94=20three=20failures=20that=20broke=20paywall?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Document boot race (upstream unit exit 125), start-before-merchant 502, and pasta AAAA/timeout needing 172.17.0.1 pin. Healthy smoke stays 302/402. --- 2026/2026-07-13--paivana-goa.md | 11 ++++- 2026/2026-07-17.md | 5 ++ configs/paivana/README.md | 86 +++++++++++++++++++++++++-------- 3 files changed, 81 insertions(+), 21 deletions(-) diff --git a/2026/2026-07-13--paivana-goa.md b/2026/2026-07-13--paivana-goa.md index 9606641..0095b40 100644 --- a/2026/2026-07-13--paivana-goa.md +++ b/2026/2026-07-13--paivana-goa.md @@ -7,13 +7,22 @@ - Image `localhost/koopa-paivana:latest` (built from `git.taler.net/paivana` + deb.taler.net libs; package not on apt yet) - Merchant: instance **`goa-shop`**, template **`paivana`** (`template_type: paivana`, **GOA:4200**) - After payment: site-wide cookie (`-g`) → example upstream free to browse -- `extra_hosts: taler.hacktivism.ch:host-gateway` so pasta can reach merchant - Flags: `-g` (site-wide), `-f` (X-Forwarded-For) - Smoke: `curl -si http://127.0.0.1:9025/` → **302** to `/.well-known/paivana/templates/paivana#…` - Config mirror: `configs/paivana/` - Caddy: `paivana.hacktivism.ch` → `:9025` (`configs/caddy/Caddyfile`) - User unit: `container-koopa-paivana.service` enabled +## Was später kaputt / nachgezogen (klar) + +Siehe **`configs/paivana/README.md` → „Was kaputt war“**. Kurz: + +1. **Extra Upstream-systemd-Unit** race mit Compose → exit **125** / start-limit — **nur** Compose-Unit behalten. +2. **Paivana vor Merchant-Apps** → öffentliches **502** — `order.conf` `After=taler-merchant-apps`. +3. **Pasta DNS/AAAA**: `host-gateway` reichte nicht → `extra_hosts` auf **`172.17.0.1`** pin (`f26bf6a`). + +Gesund: **302** lokal / **402** public paywall + monitoring `www.paivana`. + ## Refs - https://docs.taler.net/taler-paivana-manual.html diff --git a/2026/2026-07-17.md b/2026/2026-07-17.md index de35634..33ee314 100644 --- a/2026/2026-07-17.md +++ b/2026/2026-07-17.md @@ -3,3 +3,8 @@ - Bank: `taler://withdraw` **ohne `:443`** (Wallet-Apps); live deployed - Merchant/bank in-container apps after reboot via `ensure-taler-apps` + user units - Admin-log: 13 Commits (19:04–01:07) + follow-up; `main` force-pushed → Forgejo +- **Paivana — was kaputt war** klar dokumentiert (`configs/paivana/README.md`): + 1. Upstream-only systemd race (exit 125) + 2. Start vor Merchant-Apps → 502 + 3. Pasta AAAA/timeout → pin `172.17.0.1` + Live-Stack war gesund (302/402); Ursache/Fix nicht wiederholen. diff --git a/configs/paivana/README.md b/configs/paivana/README.md index 3b8700a..643fdb4 100644 --- a/configs/paivana/README.md +++ b/configs/paivana/README.md @@ -37,6 +37,65 @@ GNU Taler **paivana-httpd** reverse-proxy paywall (DD 95 / DD 76 style), GOA pay Mirror notes: `koopa-admin-secrets` → `koopa/home-hernani/koopa-paivana/secrets/` (paths only / examples). +## Was kaputt war (klar · 2026-07) + +Drei **unabhängige** Defekte; Symptom oft gleich (Paywall 502 / tot / kein Template). +Stand: behoben auf koopa; nicht wiederholen. + +### 1) Boot-Race: extra systemd-Unit nur für Upstream + +| | | +|--|--| +| **Symptom** | Nach Reboot: `container-koopa-paivana-upstream.service` **failed** / start-limit-hit; Upstream fehlt oder Compose und Unit kämpfen. | +| **Ursache** | Separate User-Unit tat nur `podman start koopa-paivana-upstream` **ohne** dass Compose die Container schon angelegt hatte → **exit 125**. Compose-Unit startet **beide** Services (`paivana` + `upstream`) ohnehin. | +| **Fix** | **Nur** `container-koopa-paivana.service` (compose up). Separate Upstream-Unit: `disable --now`. Nicht neu anlegen. | +| **Nicht verwechseln mit** | Problem 2 (Merchant noch down → 502 trotz laufender Container). | + +### 2) Start-Reihenfolge: Paivana vor Merchant-Apps → **502** + +| | | +|--|--| +| **Symptom** | `paivana.hacktivism.ch` / `:9025` **502** nach Boot; Container laufen, Paywall/Template-Flow tot. | +| **Ursache** | Merchant-Container ist hoch, aber **In-App** (`taler-merchant-httpd` etc.) startet erst über `taler-merchant-apps.service` / `ensure-taler-apps`. Paivana braucht die **private Merchant-API** (Template `paivana` / goa-shop). Zu früh → 502. | +| **Fix** | Drop-in `configs/systemd/user/container-koopa-paivana.service.d/order.conf`: `After=` / `Wants=` `taler-merchant-apps.service` (+ ggf. merchant-Container). Siehe `2026/2026-07-16--reboot-recovery.md`. | +| **Check** | Merchant `https://taler.hacktivism.ch/config` **200**, dann Paivana smoke (unten). | + +### 3) Pasta / DNS: Container erreicht Merchant nicht (Timeout / AAAA) + +| | | +|--|--| +| **Symptom** | Paivana läuft, aber Template-Laden / Merchant-Calls **hängen oder scheitern**; Logs: Timeouts Richtung `taler.hacktivism.ch`. | +| **Ursache** | Rootless **pasta** hairpint öffentliche DNS nicht sauber. `extra_hosts: …:host-gateway` allein reichte nicht: oft gewinnt noch die **öffentliche AAAA**, Connection timeout. Merchant sitzt am Host (**~:9010**), erreichbar über Pasta-IPv4 **`172.17.0.1`**. | +| **Fix** | In `compose.yml` feste IPv4-Hosts (nicht nur host-gateway): | + +```yaml +extra_hosts: + - "taler.hacktivism.ch:172.17.0.1" + - "bank.hacktivism.ch:172.17.0.1" + - "exchange.hacktivism.ch:172.17.0.1" +``` + +| | | +|--|--| +| **Commit** | `f26bf6a` (pin Taler hosts to pasta IPv4) | + +### Kurz: was *nicht* das Problem war + +- Caddy-Vhost an sich (leitet auf **9025**). +- Fehlen des Templates im Merchant (Template `paivana` / GOA:4200 war angelegt; ohne Merchant-Apps oder ohne Routing wirkt es so, als fehlte es). +- „Nur nginx-Upstream“ — Upstream ist unprivileged nginx **:8080**; ohne Problem 1/2/3 ist das stabil. + +### Erwartet gesund + +| Check | Expect | +|-------|--------| +| `curl -si http://127.0.0.1:9025/` | **302** → `/.well-known/paivana/templates/paivana#…` | +| `https://paivana.hacktivism.ch/` | **402** paywall HTML + `paivana: taler://pay-template/…` | +| monitoring `urls` | `www.paivana` OK | +| e2e (GOA, lokal) | `e2e.paivana` Template-Pay **GOA:4200** (wenn `E2E_PAIVANA=1`) | + +--- + ## Ops ```bash @@ -54,35 +113,22 @@ Autostart (**only** the compose unit — starts **both** `koopa-paivana` and ```bash mkdir -p ~/.config/systemd/user cp ~/koopa-paivana/container-koopa-paivana.service ~/.config/systemd/user/ -# optional: After= merchant (configs/systemd/user/container-koopa-paivana.service.d/) +# required on koopa: After= merchant apps +# configs/systemd/user/container-koopa-paivana.service.d/order.conf systemctl --user daemon-reload systemctl --user enable --now container-koopa-paivana.service ``` **Do not** enable a separate `container-koopa-paivana-upstream.service` that only -runs `podman start koopa-paivana-upstream`. At boot that unit races compose -(container not created yet → exit 125 → start-limit-hit). Compose already -starts the upstream. If such a unit exists: +runs `podman start koopa-paivana-upstream` — see failure **#1** above. +If such a unit exists: `systemctl --user disable --now container-koopa-paivana-upstream.service`. -Healthy smoke: - -| Check | Expect | -|-------|--------| -| `curl -si http://127.0.0.1:9025/` | **302** → `/.well-known/paivana/templates/paivana#…` | -| template URL / public `https://paivana.hacktivism.ch/` | **402** paywall HTML + `paivana: taler://pay-template/…` | -| monitoring | `./scripts/taler-monitoring/taler-monitoring.sh -d hacktivism.ch urls` → `www.paivana` OK | - ## Network notes -Rootless pasta cannot hairpin public DNS for the merchant. Compose sets: - -```yaml -extra_hosts: - - "taler.hacktivism.ch:host-gateway" -``` - -so `paivana-httpd` can load templates from the merchant API. +See failure **#3**. Live compose pins Taler hostnames to pasta IPv4 +`172.17.0.1` so `paivana-httpd` reaches the host merchant private API without +public AAAA timeouts. ## Caddy From 91b79ab80744ffc16509d33cbda1b8e3caa45c61 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:29:52 +0200 Subject: [PATCH 18/38] android-test: headless emulator default for GUI smokes Default EMULATOR_HEADLESS=1 (-no-window, QT offscreen) while guest GLES still runs via SwiftShader. Auto-start AVD when no adb device; WINDOWED=1 for a host window on laptops. --- .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 10 +- .../android-test/GUI-AUTOMATION-NOTES.md | 36 +++++ .../taler-monitoring/android-test/README.md | 11 ++ .../android-test/lib_android_env.sh | 153 ++++++++++++++++++ .../android-test/run-android-gui-smoke.sh | 20 +-- .../android-test/run-android-pay-smoke.sh | 35 ++-- .../android-test/run-goa-gui-chain.sh | 24 +-- .../android-test/start-android-emulator.sh | 90 +++++++++++ 8 files changed, 320 insertions(+), 59 deletions(-) create mode 100644 scripts/taler-monitoring/android-test/lib_android_env.sh create mode 100755 scripts/taler-monitoring/android-test/start-android-emulator.sh diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index 73e654a..f954cfd 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -156,12 +156,12 @@ STACK=stage ./android-test/run-android-pay-smoke.sh **User-local emulator (no apt root if SDK already under `$HOME/Android/Sdk`):** ```bash -export ANDROID_HOME=$HOME/Android/Sdk -export PATH=$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH -# AVD e.g. TalerWallet34 (google_apis x86_64) -emulator -avd TalerWallet34 -no-window -gpu swiftshader_indirect -memory 1536 & -adb wait-for-device +# Default = headless (no host window); guest GLES via SwiftShader. +# Prefer the helper — same flags smokes use: +./android-test/start-android-emulator.sh --wait ./android-test/run-android-pay-smoke.sh +# Or let smoke auto-start the AVD (AUTO_START_EMULATOR=1). +# Windowed laptop: WINDOWED=1 ./android-test/start-android-emulator.sh --wait ``` **Limits (clementine 2026-07-17):** host RAM ~3.7 GiB → emulator often ANRs System UI; diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index fce915e..27330ca 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -120,6 +120,39 @@ Automation can run on hosts that support **Android only**, **GUI tooling only** | `AUTO_ANDROID` | `0` / `1` | Run Android wallet automation (adb, APK, emulator/device) | | `AUTO_GUI` | `0` / `1` | Drive **graphical** UI (uiautomator taps, multi-round). If `0`, deep-link/CLI-only install smoke. | | `AUTO_PLATFORM` | `auto` / `linux` / `macos` / `ios` | Host family (optional override) | +| `EMULATOR_HEADLESS` | **`1` (default)** / `0` | No host window (`-no-window`). Server/CI-safe. | +| `WINDOWED` | `0` / `1` | Shortcut: `WINDOWED=1` ⇒ show emulator window (`EMULATOR_HEADLESS=0`) | +| `EMULATOR_GPU` | default **`swiftshader_indirect`** | Guest GLES still runs headless (layout/taps/screenshots). Override `host` only if windowed + real GPU. | +| `AUTO_START_EMULATOR` | **`1` (default)** / `0` | If no adb device, start AVD via `start-android-emulator.sh --wait` | + +--- + +## Headless mode (default) + +**Default for all GUI / pay smokes:** no window on the host. Opening an emulator +GUI on servers (or headless CI) is wrong; these scripts therefore start the AVD +with **`-no-window`** and still keep **graphics logic** in the guest: + +| Piece | Headless default | +|-------|------------------| +| Host window | **off** (`-no-window`, `QT_QPA_PLATFORM=offscreen`, no `DISPLAY`) | +| Guest GLES | **on** via **SwiftShader** (`-gpu swiftshader_indirect`) | +| UI automation | **on** — `uiautomator` dump/tap + `screencap` (no X11 needed) | +| Audio / boot anim | off (`-no-audio -no-boot-anim`) | + +```bash +# explicit (same as default) +./start-android-emulator.sh --wait + +# or let smoke auto-start headless AVD when no phone is plugged in +STACK=goa ./run-android-gui-smoke.sh + +# laptop: show the emulator window +WINDOWED=1 ./start-android-emulator.sh --wait +# or: EMULATOR_HEADLESS=0 EMULATOR_GPU=host ./start-android-emulator.sh --wait +``` + +Shared code: `lib_android_env.sh` + `start-android-emulator.sh`. **Semantics:** @@ -203,6 +236,8 @@ Scripts **today** implement Android only; they should honour: | `run-android-gui-smoke.sh` | Multi-round GUI drive | | `run-goa-gui-chain.sh` | Multi mint/withdraw/pay (gui-workflows port) | | `run-android-build-and-smoke.sh` | Build inference fix branch + smoke (`GUI=1` optional) | +| `start-android-emulator.sh` | Start AVD (**headless default**, SwiftShader) | +| `lib_android_env.sh` | Flags, PATH, `android_ensure_device` | | `lib_ui.py` | Dump/tap/ANR helpers | --- @@ -233,6 +268,7 @@ Scripts **today** implement Android only; they should honour: - [x] Minimalvariante: was geht / was nicht (Beschränkungen) - [x] Two stacks (goa / stage) - [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults) +- [x] **Headless emulator default** (`EMULATOR_HEADLESS=1`, SwiftShader) - [x] Linux default = Android suite only - [x] Inference-branch policy - [ ] Wrapper enforces flags in every entry script (incremental) diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index e1ab975..4c2e5f2 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -59,5 +59,16 @@ Canonical notes (minimal variant, **flags**, Linux defaults): | `AUTO_ANDROID` | `1` | Run Android adb/APK automation | | `AUTO_GUI` | `1` | Graphical uiautomator drive (else deep-link only) | | `AUTO_PLATFORM` | `auto` → `linux` | Host family; macOS can later pair with iOS out of tree | +| **`EMULATOR_HEADLESS`** | **`1`** | **No host window** (`-no-window`). Server-safe. | +| `EMULATOR_GPU` | `swiftshader_indirect` | Guest GLES still active for UI/taps/screenshots | +| `AUTO_START_EMULATOR` | `1` | Auto-start headless AVD if no adb device | +| `WINDOWED` | `0` | `WINDOWED=1` → show emulator window | + +```bash +# headless AVD only (default path for servers) +./start-android-emulator.sh --wait +# windowed (laptops) +WINDOWED=1 ./start-android-emulator.sh --wait +``` See also parent [GIT-AUTOMATION-NOTES.md](../GIT-AUTOMATION-NOTES.md). diff --git a/scripts/taler-monitoring/android-test/lib_android_env.sh b/scripts/taler-monitoring/android-test/lib_android_env.sh new file mode 100644 index 0000000..ad209de --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_android_env.sh @@ -0,0 +1,153 @@ +# shellcheck shell=bash +# Shared Android env for GUI / adb smokes. +# Source: . "$ROOT/lib_android_env.sh" +# +# Defaults favour **servers / CI**: no host window (headless emulator), while the +# guest still runs GLES via SwiftShader so UI layout / uiautomator work. +# +# See GUI-AUTOMATION-NOTES.md → Headless mode + +# --- SDK / PATH --------------------------------------------------------------- +export ANDROID_HOME="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-$HOME/Android/Sdk}}" +export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" +for d in \ + "$ANDROID_HOME/platform-tools" \ + "$ANDROID_HOME/emulator" \ + "$ANDROID_HOME/cmdline-tools/latest/bin" +do + [ -d "$d" ] && case ":$PATH:" in *":$d:"*) ;; *) PATH="$d:$PATH" ;; esac +done +export PATH + +# AVD path (some hosts keep AVDs under ~/.config/.android) +if [ -z "${ANDROID_AVD_HOME:-}" ]; then + if [ -d "$HOME/.android/avd" ]; then + export ANDROID_AVD_HOME="$HOME/.android/avd" + elif [ -d "$HOME/.config/.android/avd" ]; then + export ANDROID_AVD_HOME="$HOME/.config/.android/avd" + # emulator also looks at ~/.android/avd — soft-link if missing + if [ ! -e "$HOME/.android/avd" ]; then + mkdir -p "$HOME/.android" + ln -sfn "$HOME/.config/.android/avd" "$HOME/.android/avd" 2>/dev/null || true + fi + fi +fi + +# --- Platform / feature flags ------------------------------------------------- +os=$(uname -s) +case "${AUTO_PLATFORM:-auto}" in + auto) + case "$os" in + Linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; + esac + ;; + linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; + *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; +esac + +# Headless = default (no X11/Wayland window on host). GUI *logic* still runs in +# the guest via software GPU. Set EMULATOR_HEADLESS=0 or WINDOWED=1 for a window. +: "${EMULATOR_HEADLESS:=1}" +if [ "${WINDOWED:-0}" = "1" ]; then + EMULATOR_HEADLESS=0 +fi +export EMULATOR_HEADLESS + +# Soft GLES for headless; windowed may override to host/auto if set by user. +if [ "${EMULATOR_HEADLESS}" = "1" ]; then + : "${EMULATOR_GPU:=swiftshader_indirect}" +else + : "${EMULATOR_GPU:=swiftshader_indirect}" # still safe default; host GPU: EMULATOR_GPU=host +fi +export EMULATOR_GPU + +: "${EMULATOR_AVD:=TalerWallet34}" +: "${EMULATOR_MEMORY:=1536}" +: "${EMULATOR_CORES:=2}" +# If no adb device: start headless AVD (0 = require phone/manual emulator) +: "${AUTO_START_EMULATOR:=1}" +: "${EMULATOR_BOOT_TIMEOUT:=180}" +: "${EMULATOR_LOG:=/tmp/taler-android-emulator.log}" + +android_adb_serial() { + adb devices 2>/dev/null | awk '/\tdevice$/{print $1; exit}' +} + +android_has_device() { + [ -n "$(android_adb_serial)" ] +} + +# Build emulator argv for current headless/windowed mode. +# Prints args; caller exec/nohup. +android_emulator_args() { + local avd="${1:-$EMULATOR_AVD}" + local args=( -avd "$avd" -gpu "$EMULATOR_GPU" -no-audio -no-boot-anim ) + if [ "${EMULATOR_HEADLESS}" = "1" ]; then + args+=( -no-window ) + # Avoid Qt trying to open a display on servers + export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}" + fi + if [ -n "${EMULATOR_MEMORY:-}" ]; then + args+=( -memory "$EMULATOR_MEMORY" ) + fi + if [ -n "${EMULATOR_CORES:-}" ]; then + args+=( -cores "$EMULATOR_CORES" ) + fi + # extra user flags + # shellcheck disable=SC2206 + if [ -n "${EMULATOR_EXTRA_ARGS:-}" ]; then + # intentional word-split for extra flags + args+=( ${EMULATOR_EXTRA_ARGS} ) + fi + printf '%s\n' "${args[@]}" +} + +# Ensure an adb "device" is online. Optionally start headless emulator. +# Sets SERIAL / ANDROID_SERIAL on success. Returns 0 ok, 3 no device. +android_ensure_device() { + local serial + serial="${SERIAL:-${ANDROID_SERIAL:-}}" + if [ -z "$serial" ]; then + serial=$(android_adb_serial || true) + fi + if [ -n "$serial" ]; then + SERIAL="$serial" + export ANDROID_SERIAL="$serial" + return 0 + fi + + if [ "${AUTO_START_EMULATOR}" != "1" ]; then + echo "no adb device (AUTO_START_EMULATOR=0 — plug phone or start AVD manually)" >&2 + return 3 + fi + + if ! command -v emulator >/dev/null 2>&1; then + echo "no adb device and emulator binary missing under $ANDROID_HOME" >&2 + return 3 + fi + + echo "no adb device → starting AVD ${EMULATOR_AVD} (headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU})" + local starter + starter="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/start-android-emulator.sh" + if [ ! -x "$starter" ]; then + echo "missing $starter" >&2 + return 3 + fi + EMULATOR_HEADLESS="$EMULATOR_HEADLESS" EMULATOR_GPU="$EMULATOR_GPU" \ + EMULATOR_AVD="$EMULATOR_AVD" EMULATOR_MEMORY="$EMULATOR_MEMORY" \ + EMULATOR_CORES="$EMULATOR_CORES" EMULATOR_LOG="$EMULATOR_LOG" \ + EMULATOR_BOOT_TIMEOUT="$EMULATOR_BOOT_TIMEOUT" \ + "$starter" --wait || return 3 + + serial=$(android_adb_serial || true) + if [ -z "$serial" ]; then + echo "emulator started but no adb device yet — see $EMULATOR_LOG" >&2 + return 3 + fi + SERIAL="$serial" + export ANDROID_SERIAL="$serial" + echo "device: $SERIAL (headless=${EMULATOR_HEADLESS})" + return 0 +} diff --git a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh index de280f5..f52c783 100755 --- a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh @@ -12,14 +12,9 @@ # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" -# Platform flags (GUI-AUTOMATION-NOTES.md). Linux → Android only. -os=$(uname -s) -case "${AUTO_PLATFORM:-auto}" in - auto) case "$os" in Linux|Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; esac ;; - linux|macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; - *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; -esac [ "${AUTO_ANDROID}" = "1" ] || { echo "skipped: AUTO_ANDROID=${AUTO_ANDROID}"; exit 0; } if [ "${AUTO_GUI}" != "1" ]; then echo "AUTO_GUI=0 → deep-link smoke only" @@ -46,17 +41,12 @@ STACK="${STACK:-auto}" SERIAL="${SERIAL:-}" mkdir -p "$OUT_DIR" -if ! command -v adb >/dev/null 2>&1; then - [ -x "$HOME/Android/Sdk/platform-tools/adb" ] && export PATH="$HOME/Android/Sdk/platform-tools:$PATH" -fi command -v adb >/dev/null || { echo "adb missing" >&2; exit 2; } -if [ -z "$SERIAL" ]; then - SERIAL=$(adb devices | awk '/\tdevice$/{print $1; exit}') -fi -[ -n "$SERIAL" ] || { echo "no adb device" >&2; exit 3; } +# Headless default: no host window; guest GLES via SwiftShader (uiautomator). +android_ensure_device || { echo "no adb device (try ./start-android-emulator.sh --wait)" >&2; exit 3; } ADB=(adb -s "$SERIAL") -echo "device: $SERIAL mode: GUI (uiautomator)" +echo "device: $SERIAL mode: GUI (uiautomator) headless=${EMULATOR_HEADLESS} gpu=${EMULATOR_GPU}" case "$STACK" in auto) diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh index 690c222..ef2ca8e 100755 --- a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -20,15 +20,9 @@ # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" -# Platform flags (see GUI-AUTOMATION-NOTES.md). Linux default: Android only. -os=$(uname -s) -case "${AUTO_PLATFORM:-auto}" in - auto) case "$os" in Linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; Darwin) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; esac ;; - linux) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; - macos) : "${AUTO_ANDROID:=1}"; : "${AUTO_GUI:=1}" ;; - *) : "${AUTO_ANDROID:=0}"; : "${AUTO_GUI:=0}" ;; -esac if [ "${AUTO_ANDROID}" != "1" ]; then echo "skipped: AUTO_ANDROID=${AUTO_ANDROID} (platform flags — see GUI-AUTOMATION-NOTES.md)" exit 0 @@ -46,28 +40,23 @@ OUT_DIR="${OUT_DIR:-$ROOT/out}" mkdir -p "$APK_DIR" "$OUT_DIR" if ! command -v adb >/dev/null 2>&1; then - if [ -x "$HOME/Android/Sdk/platform-tools/adb" ]; then - export PATH="$HOME/Android/Sdk/platform-tools:$PATH" - else - echo "adb missing — install android-tools-adb or user SDK platform-tools" >&2 - exit 2 - fi + echo "adb missing — install android-tools-adb or user SDK platform-tools" >&2 + exit 2 fi -if [ -z "$SERIAL" ]; then - SERIAL=$(adb devices | awk '/\tdevice$/{print $1; exit}') -fi -if [ -z "$SERIAL" ]; then - cat >&2 <<'EOF' -No adb device. Options (no root for user-local emulator if /dev/kvm is usable): +# Default: headless AVD (no host window). WINDOWED=1 / EMULATOR_HEADLESS=0 for UI. +if ! android_ensure_device; then + cat >&2 </dev/null 2>&1; then - [ -x "$HOME/Android/Sdk/platform-tools/adb" ] && export PATH="$HOME/Android/Sdk/platform-tools:$PATH" -fi command -v adb >/dev/null || die "adb not found" command -v python3 >/dev/null || die "python3 required" @@ -112,12 +105,11 @@ if [[ ! -f "${EXP_PW_FILE:-}" && "$STACK" = "stage" ]]; then fi [[ -n "${EXP_PW_FILE:-}" && -f "$EXP_PW_FILE" ]] || die "set EXP_PW_FILE (explorer password)" -if [[ -z "${ANDROID_SERIAL:-}" ]]; then - n=$(adb devices | awk 'NR>1 && $2=="device"{c++} END{print c+0}') - [[ "$n" -ge 1 ]] || die "no adb device (start emulator or plug phone)" - export ANDROID_SERIAL=$(adb devices | awk 'NR>1 && $2=="device"{print $1; exit}') -fi -info "ANDROID_SERIAL=$ANDROID_SERIAL STACK=$STACK PKG=$PKG SHOTDIR=$SHOTDIR" +# Headless AVD by default (no host window). WINDOWED=1 for a visible emulator. +SERIAL="${SERIAL:-${ANDROID_SERIAL:-}}" +android_ensure_device || die "no adb device (./start-android-emulator.sh --wait)" +export ANDROID_SERIAL="$SERIAL" +info "ANDROID_SERIAL=$ANDROID_SERIAL STACK=$STACK PKG=$PKG SHOTDIR=$SHOTDIR headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU" ADB=(adb -s "$ANDROID_SERIAL") adb_sh() { "${ADB[@]}" shell "$@"; } diff --git a/scripts/taler-monitoring/android-test/start-android-emulator.sh b/scripts/taler-monitoring/android-test/start-android-emulator.sh new file mode 100755 index 0000000..e62b5f4 --- /dev/null +++ b/scripts/taler-monitoring/android-test/start-android-emulator.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# Start the Taler Android AVD. **Default = headless** (no host window). +# Guest GLES still runs via SwiftShader so uiautomator / screenshots work. +# +# Usage: +# ./start-android-emulator.sh # headless, background +# ./start-android-emulator.sh --wait # headless + wait for boot +# ./start-android-emulator.sh --windowed # show emulator window +# EMULATOR_HEADLESS=0 ./start-android-emulator.sh +# EMULATOR_GPU=host WINDOWED=1 ./start-android-emulator.sh +# +# Env: EMULATOR_AVD EMULATOR_MEMORY EMULATOR_CORES EMULATOR_GPU +# EMULATOR_HEADLESS EMULATOR_LOG EMULATOR_BOOT_TIMEOUT EMULATOR_EXTRA_ARGS +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" + +WAIT=0 +for a in "$@"; do + case "$a" in + --wait) WAIT=1 ;; + --windowed| --gui) EMULATOR_HEADLESS=0; export EMULATOR_HEADLESS ;; + --headless) EMULATOR_HEADLESS=1; export EMULATOR_HEADLESS ;; + -h|--help) + sed -n '2,18p' "$0" + exit 0 + ;; + esac +done + +command -v emulator >/dev/null || { + echo "emulator not found (ANDROID_HOME=$ANDROID_HOME)" >&2 + exit 2 +} +command -v adb >/dev/null || { + echo "adb not found" >&2 + exit 2 +} + +# Already booted? +if serial=$(android_adb_serial); [ -n "$serial" ]; then + boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true) + if [ "$boot" = "1" ]; then + echo "already running: $serial (boot_completed=1) — headless default still applies to *new* starts" + exit 0 + fi +fi + +# Avoid pkill patterns that match this wrapper's argv (self-kill). +mapfile -t EMU_ARGS < <(android_emulator_args "$EMULATOR_AVD") +echo "starting: emulator ${EMU_ARGS[*]}" +echo " headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU log=$EMULATOR_LOG" + +if [ "${EMULATOR_HEADLESS}" = "1" ]; then + export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}" + # Do not require DISPLAY + unset DISPLAY || true +fi + +mkdir -p "$(dirname "$EMULATOR_LOG")" +# Background; do not use pkill -f emulator from callers matching full cmdline +nohup emulator "${EMU_ARGS[@]}" >"$EMULATOR_LOG" 2>&1 & +echo "emulator pid=$! log=$EMULATOR_LOG" + +if [ "$WAIT" != "1" ]; then + echo "not waiting (pass --wait for boot). adb devices when ready." + exit 0 +fi + +echo "waiting for boot (timeout ${EMULATOR_BOOT_TIMEOUT}s)…" +deadline=$((SECONDS + EMULATOR_BOOT_TIMEOUT)) +serial="" +while (( SECONDS < deadline )); do + adb wait-for-device 2>/dev/null || true + serial=$(android_adb_serial || true) + if [ -n "$serial" ]; then + boot=$(adb -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r' || true) + if [ "$boot" = "1" ]; then + echo "BOOT_OK serial=$serial headless=$EMULATOR_HEADLESS gpu=$EMULATOR_GPU" + exit 0 + fi + fi + sleep 2 +done + +echo "boot timeout — last log:" >&2 +tail -30 "$EMULATOR_LOG" >&2 || true +exit 3 From c37cc88363976f7c5cd8ef66872de8d0d3854146 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:31:30 +0200 Subject: [PATCH 19/38] android-test: matrix published + stable-self + master builds Add run-android-variant-matrix.sh so smoke covers F-Droid stable APK, a self-build of the same release tag (wallet-1.6.1), and origin/master. Build helper accepts tags; avoids relying on a single developer tree. --- .../taler-monitoring/GIT-AUTOMATION-NOTES.md | 7 +- .../taler-monitoring/android-test/.gitignore | 2 + .../android-test/GUI-AUTOMATION-NOTES.md | 15 +- .../taler-monitoring/android-test/README.md | 32 +++- .../run-android-build-and-smoke.sh | 133 ++++++++++---- .../run-android-variant-matrix.sh | 167 ++++++++++++++++++ 6 files changed, 312 insertions(+), 44 deletions(-) create mode 100755 scripts/taler-monitoring/android-test/run-android-variant-matrix.sh diff --git a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md index f954cfd..4ad80b3 100644 --- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md @@ -183,8 +183,13 @@ For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 ### From-source smoke (same tests as F-Droid) ```bash -# build latest master + adb smoke +# single ref (default: inference fix branch) ./android-test/run-android-build-and-smoke.sh +BRANCH=master ./android-test/run-android-build-and-smoke.sh +# self-build published stable tag (parity with F-Droid 1.6.1 source) +BRANCH=wallet-1.6.1 LABEL=stable-self ./android-test/run-android-build-and-smoke.sh +# matrix: published APK + stable self-build + master (anti–weird-build) +./android-test/run-android-variant-matrix.sh STACK=goa ./android-test/run-android-build-and-smoke.sh # already built: APK_PATH=$HOME/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index 0867ff3..6d4e62e 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -2,6 +2,8 @@ apks/*.apk out/ *.png out-source/ +out-source-*/ out-gui/ out-source-gui/ out-gui-chain/ +out-matrix-*/ diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 27330ca..16023eb 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -235,11 +235,24 @@ Scripts **today** implement Android only; they should honour: | `run-android-pay-smoke.sh` | Hybrid: install + deep-link + light taps | | `run-android-gui-smoke.sh` | Multi-round GUI drive | | `run-goa-gui-chain.sh` | Multi mint/withdraw/pay (gui-workflows port) | -| `run-android-build-and-smoke.sh` | Build inference fix branch + smoke (`GUI=1` optional) | +| `run-android-build-and-smoke.sh` | Build git ref (branch/tag) + smoke (`GUI=1` optional) | +| `run-android-variant-matrix.sh` | **published + stable-self + master** (compare builds) | | `start-android-emulator.sh` | Start AVD (**headless default**, SwiftShader) | | `lib_android_env.sh` | Flags, PATH, `android_ensure_device` | | `lib_ui.py` | Dump/tap/ANR helpers | +### Build variants (anti–“weird local build”) + +| Variant | Source | Package | Purpose | +|---------|--------|---------|---------| +| **published** | F-Droid APK URL | `net.taler.wallet.fdroid` | What users install | +| **stable-self** | tag `wallet-1.6.1` (override `STABLE_TAG`) rebuilt here | `…fdroid.debug` | Same release tree as stable, **our** Gradle — excludes odd publisher-only or dirty-tree builds | +| **master** | `origin/master` rebuilt here | `…fdroid.debug` | Tip of mainline | +| **fix** (optional) | inference fix branch | `…fdroid.debug` | GOA automation fixes | + +Default matrix: `published,stable-self,master`. +`./run-android-variant-matrix.sh` — see README. + --- ## Stacks under test diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 4c2e5f2..16be18c 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -11,22 +11,44 @@ STACK=stage ./run-android-pay-smoke.sh APK: F-Droid **1.6.1** (`net.taler.wallet.fdroid`). -## 2) Latest source build + same tests +## 2) Source build + same tests -Pulls/builds `taler-android` master (`:wallet:assembleFdroidDebug`), then +Builds `taler-android` at a git **ref** (`:wallet:assembleFdroidDebug`), then runs the **same** smoke (install + withdraw/pay deep-links). ```bash -# default source: $HOME/taler/taler-android +# default: inference fix branch (GOA withdraw reliability) ./run-android-build-and-smoke.sh -STACK=goa ./run-android-build-and-smoke.sh -SKIP_PULL=1 ./run-android-build-and-smoke.sh # use tree as-is +# tip of master +BRANCH=master ./run-android-build-and-smoke.sh +# self-build of published stable **tag** (same release as F-Droid 1.6.1) +BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh +SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh TALER_ANDROID_SRC=/path/to/taler-android ./run-android-build-and-smoke.sh ``` Built package id: `net.taler.wallet.fdroid.debug`. Needs `ANDROID_HOME` (e.g. `$HOME/Android/Sdk`), Java 17+, network for Maven. +## 2b) Variant matrix (published + stable self-build + master) + +Runs several smokes so a single odd developer build cannot be the only sample: + +| Variant | What | +|---------|------| +| **published** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | +| **stable-self** | Rebuild tag `wallet-1.6.1` on this host (same source as stable) | +| **master** | Rebuild `origin/master` | + +```bash +./run-android-variant-matrix.sh +STACK=goa ./run-android-variant-matrix.sh +VARIANTS=master,stable-self ./run-android-variant-matrix.sh +VARIANTS=published,stable-self,master,fix GUI=0 ./run-android-variant-matrix.sh +``` + +Artifacts + `SUMMARY.txt` under `out-matrix-/`. + ## 3) Graphical UI (vanilla) Deep-link entry (≡ QR) **plus** multi-round uiautomator taps: diff --git a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh index 5779b88..659b446 100755 --- a/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-build-and-smoke.sh @@ -1,18 +1,24 @@ #!/usr/bin/env bash -# Build the **latest** GNU Taler Android wallet from source, then run the same -# adb deep-link smoke as run-android-pay-smoke.sh. +# Build GNU Taler Android wallet from source at a git **ref**, then run the same +# adb deep-link (or GUI) smoke as run-android-pay-smoke.sh. # -# Source (default): $HOME/taler/taler-android (git@git.taler.net:taler-android.git) +# Source (default): $HOME/taler/taler-android # Build: ./gradlew :wallet:assembleFdroidDebug # Package id: net.taler.wallet.fdroid.debug # -# No root required if ANDROID_HOME points at a user SDK ($HOME/Android/Sdk). +# Typical refs: +# BRANCH=master # tip of master +# BRANCH=wallet-1.6.1 # published stable **tag** (self-build) +# BRANCH=dev/hernani-inference/fix-… # GOA automation fix branch +# +# For a multi-variant matrix (master + stable self-build + published APK): +# ./run-android-variant-matrix.sh # # Usage: -# ./android-test/run-android-build-and-smoke.sh -# STACK=goa ./android-test/run-android-build-and-smoke.sh -# STACK=stage SKIP_PULL=1 ./android-test/run-android-build-and-smoke.sh -# TALER_ANDROID_SRC=/path/to/taler-android ./android-test/run-android-build-and-smoke.sh +# ./run-android-build-and-smoke.sh +# BRANCH=master STACK=goa ./run-android-build-and-smoke.sh +# BRANCH=wallet-1.6.1 LABEL=stable-self ./run-android-build-and-smoke.sh +# SKIP_PULL=1 BRANCH=master ./run-android-build-and-smoke.sh # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) @@ -20,9 +26,11 @@ SMOKE="$ROOT/run-android-pay-smoke.sh" SRC="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" SKIP_PULL="${SKIP_PULL:-0}" # Prefer existing inference fix branch for GOA withdraw (spinner/exchange resolve). -# Override: BRANCH=master or BRANCH=dev/hernani-inference/fix-withdraw-spinner-fallback +# Matrix uses BRANCH=master and BRANCH=$STABLE_TAG explicitly. BRANCH="${BRANCH:-dev/hernani-inference/fix-bank-withdraw-auto-exchange}" -VARIANT="${VARIANT:-fdroidDebug}" # assemble task suffix: FdroidDebug / NightlyDebug / … +# Optional human label for logs / artifact names (default = sanitized BRANCH) +LABEL="${LABEL:-}" +VARIANT="${VARIANT:-fdroidDebug}" export ANDROID_HOME="${ANDROID_HOME:-$HOME/Android/Sdk}" export ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" @@ -37,72 +45,123 @@ fi printf 'sdk.dir=%s\n' "$ANDROID_HOME" >"$SRC/local.properties" +# --- checkout REF (branch, origin/branch, or tag) -------------------------------- cd "$SRC" if [ -d .git ]; then - git fetch origin 2>/dev/null || true - if [ -n "$BRANCH" ] && git rev-parse --verify "origin/$BRANCH" >/dev/null 2>&1; then - echo "checkout origin/$BRANCH (minimal GOA/automation fixes; override BRANCH=)" - git checkout -B "$BRANCH" "origin/$BRANCH" 2>/dev/null || git checkout "$BRANCH" 2>/dev/null || true + git fetch origin --tags 2>/dev/null || git fetch origin 2>/dev/null || true + REF="$BRANCH" + checked=0 + if [ -n "$REF" ]; then + if git rev-parse --verify "refs/tags/${REF}" >/dev/null 2>&1; then + echo "checkout tag ${REF} (stable / release self-build)" + git checkout -f "refs/tags/${REF}" + checked=1 + SKIP_PULL=1 + elif git rev-parse --verify "origin/${REF}" >/dev/null 2>&1; then + echo "checkout origin/${REF}" + git checkout -B "$REF" "origin/${REF}" 2>/dev/null || git checkout "$REF" 2>/dev/null || true + checked=1 + elif git rev-parse --verify "${REF}" >/dev/null 2>&1; then + echo "checkout ${REF}" + git checkout -f "$REF" + checked=1 + else + echo "WARN: ref '${REF}' not found locally after fetch — staying on current HEAD" >&2 + fi fi - if [ "$SKIP_PULL" != "1" ]; then - br=$(git rev-parse --abbrev-ref HEAD) - git pull --ff-only origin "$br" 2>/dev/null || true + if [ "$SKIP_PULL" != "1" ] && [ "$checked" = "1" ]; then + # only pull real branches, never tags + if git symbolic-ref -q HEAD >/dev/null 2>&1; then + br=$(git rev-parse --abbrev-ref HEAD) + git pull --ff-only origin "$br" 2>/dev/null || true + fi fi fi -COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown) -echo "source: $SRC @ $COMMIT (branch $(git rev-parse --abbrev-ref HEAD 2>/dev/null || ?))" -echo "variant: assemble${VARIANT^}" 2>/dev/null || true -# Map VARIANT → gradle task (fdroidDebug → assembleFdroidDebug) -task="assemble$(printf '%s' "$VARIANT" | python3 -c 'import sys; s=sys.stdin.read().strip(); print(s[:1].upper()+s[1:] if s else "FdroidDebug")')" -# Prefer explicit known tasks +COMMIT=$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown) +HEAD_DESC=$(git describe --tags --always 2>/dev/null || echo "$COMMIT") +BR_NOW=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo detached) +if [ -z "$LABEL" ]; then + LABEL=$(printf '%s' "$BRANCH" | tr '/:' '--' | tr -c 'A-Za-z0-9._-' '-' | cut -c1-48) +fi +echo "source: $SRC @ $COMMIT ($HEAD_DESC) ref=$BRANCH label=$LABEL head=$BR_NOW" +echo "variant: $VARIANT" + +# Map VARIANT → gradle task case "$VARIANT" in fdroidDebug|FdroidDebug) task=assembleFdroidDebug ;; fdroidRelease|FdroidRelease) task=assembleFdroidRelease ;; nightlyDebug|NightlyDebug) task=assembleNightlyDebug ;; googleDebug|GoogleDebug) task=assembleGoogleDebug ;; + *) + task="assemble$(printf '%s' "$VARIANT" | python3 -c 'import sys; s=sys.stdin.read().strip(); print(s[:1].upper()+s[1:] if s else "FdroidDebug")')" + ;; esac -# Low-RAM hosts: cap JVM (override GRADLE_OPTS to raise) +# Low-RAM hosts: cap JVM export GRADLE_OPTS="${GRADLE_OPTS:--Xmx1280m -Dorg.gradle.daemon=false -Dorg.gradle.workers.max=1}" echo "gradle :wallet:$task …" ./gradlew ":wallet:$task" --no-daemon --max-workers="${GRADLE_MAX_WORKERS:-1}" \ -Dorg.gradle.jvmargs="${GRADLE_JVMARGS:--Xmx1280m}" -# Locate APK -APK=$(find "$SRC/wallet/build/outputs/apk" -name '*.apk' -type f 2>/dev/null | sort | tail -1) +# Prefer fdroid debug APK if multiple exist +APK=$(find "$SRC/wallet/build/outputs/apk" -path '*fdroid*debug*.apk' -type f 2>/dev/null | sort | tail -1) +if [ -z "$APK" ] || [ ! -f "$APK" ]; then + APK=$(find "$SRC/wallet/build/outputs/apk" -name '*.apk' -type f 2>/dev/null | sort | tail -1) +fi if [ -z "$APK" ] || [ ! -f "$APK" ]; then echo "no APK under wallet/build/outputs/apk" >&2 exit 4 fi echo "built apk: $APK ($(wc -c <"$APK") bytes)" -# Package id from aapt if available PKG="${PKG:-}" if [ -z "$PKG" ] && command -v aapt >/dev/null 2>&1; then PKG=$(aapt dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1) fi +if [ -z "$PKG" ] && command -v aapt2 >/dev/null 2>&1; then + PKG=$(aapt2 dump badging "$APK" 2>/dev/null | sed -n "s/^package: name='\([^']*\)'.*/\1/p" | head -1) +fi : "${PKG:=net.taler.wallet.fdroid.debug}" -mkdir -p "$ROOT/apks" -STAMP_APK="$ROOT/apks/wallet-from-source-${COMMIT}.apk" +mkdir -p "$ROOT/apks" "$ROOT/out" +STAMP_APK="$ROOT/apks/wallet-${LABEL}-${COMMIT}.apk" cp -f "$APK" "$STAMP_APK" echo "copied → $STAMP_APK" -echo "$COMMIT" >"$ROOT/out/build-commit.txt" 2>/dev/null || mkdir -p "$ROOT/out" && echo "$COMMIT" >"$ROOT/out/build-commit.txt" +{ + echo "label=$LABEL" + echo "ref=$BRANCH" + echo "commit=$COMMIT" + echo "describe=$HEAD_DESC" + echo "apk=$STAMP_APK" + echo "pkg=$PKG" + echo "task=$task" +} | tee "$ROOT/out/build-${LABEL}.txt" +echo "$COMMIT" >"$ROOT/out/build-commit.txt" echo "$APK" >"$ROOT/out/build-apk-path.txt" -# Same smoke as published F-Droid path (CLI hybrid or GUI) export APK_PATH="$STAMP_APK" -export APK_NAME="$(basename "$STAMP_APK")" +export APK_NAME +APK_NAME="$(basename "$STAMP_APK")" export PKG -export OUT_DIR="${OUT_DIR:-$ROOT/out-source}" +export OUT_DIR="${OUT_DIR:-$ROOT/out-source-$LABEL}" mkdir -p "$OUT_DIR" +cp -f "$ROOT/out/build-${LABEL}.txt" "$OUT_DIR/build-meta.txt" 2>/dev/null || true + SMOKE_CMD="$SMOKE" if [ "${GUI:-0}" = "1" ]; then SMOKE_CMD="$ROOT/run-android-gui-smoke.sh" - OUT_DIR="${OUT_DIR%-source}-source-gui" - export OUT_DIR + case "$OUT_DIR" in + *-gui) ;; + *) OUT_DIR="${OUT_DIR}-gui"; export OUT_DIR; mkdir -p "$OUT_DIR" ;; + esac fi -echo "=== smoke with built APK (PKG=$PKG · GUI=${GUI:-0}) ===" -exec env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" APK_PATH="$APK_PATH" PKG="$PKG" OUT_DIR="$OUT_DIR" \ +echo "=== smoke label=$LABEL PKG=$PKG GUI=${GUI:-0} OUT_DIR=$OUT_DIR ===" +# Do not exec: allow matrix wrapper to run multiple variants +set +e +env APK_DIR="$ROOT/apks" APK_NAME="$APK_NAME" APK_PATH="$APK_PATH" PKG="$PKG" OUT_DIR="$OUT_DIR" \ "$SMOKE_CMD" +rc=$? +set -e +echo "=== smoke label=$LABEL exit=$rc ===" +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh new file mode 100755 index 0000000..50d31a8 --- /dev/null +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +# Run Android wallet smoke against **several build variants** so a single +# "weird" developer tree or a bad local build cannot be the only sample. +# +# Default matrix: +# 1) **published** — F-Droid stable APK (bit-for-bit what users install) +# 2) **stable-self** — rebuild the same release **tag** as published (self-build) +# 3) **master** — rebuild origin/master (tip / CI-style) +# +# Why stable-self in addition to published? +# Rare, but a published APK *or* a one-off developer build can be odd +# (wrong flags, dirty tree, wrong ABI). Rebuilding the release tag on *this* +# machine with the same script path as master isolates "our toolchain" vs +# "F-Droid binary" vs "current master source". +# +# Usage: +# ./run-android-variant-matrix.sh +# STACK=goa ./run-android-variant-matrix.sh +# VARIANTS=master,stable-self ./run-android-variant-matrix.sh +# VARIANTS=published,stable-self,master,fix GUI=1 ./run-android-variant-matrix.sh +# STABLE_TAG=wallet-1.6.1 MASTER_BRANCH=master ./run-android-variant-matrix.sh +# +# Env: +# STABLE_TAG default wallet-1.6.1 (must match published F-Droid release) +# STABLE_APK_URL published APK (same as pay-smoke default) +# MASTER_BRANCH default master +# FIX_BRANCH inference fix branch (variant "fix") +# VARIANTS comma list: published,stable-self,master,fix +# GUI=1 use gui-smoke instead of pay-smoke after each build +# FAIL_FAST=1 stop on first non-zero smoke (default 0 = run all) +# MATRIX_OUT summary dir (default out-matrix-) +# +set -euo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib_android_env.sh +. "$ROOT/lib_android_env.sh" + +: "${STABLE_TAG:=wallet-1.6.1}" +: "${STABLE_APK_URL:=https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" +: "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}" +: "${MASTER_BRANCH:=master}" +: "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" +: "${VARIANTS:=published,stable-self,master}" +: "${FAIL_FAST:=0}" +: "${GUI:=0}" +: "${STACK:=auto}" + +TS=$(date +%Y%m%d-%H%M%S) +MATRIX_OUT="${MATRIX_OUT:-$ROOT/out-matrix-$TS}" +mkdir -p "$MATRIX_OUT" +SUMMARY="$MATRIX_OUT/SUMMARY.txt" +: >"$SUMMARY" + +echo "=== Android variant matrix ===" | tee -a "$SUMMARY" +echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" +echo "STABLE_TAG=$STABLE_TAG MASTER_BRANCH=$MASTER_BRANCH" | tee -a "$SUMMARY" +echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" + +# Ensure device once (headless AVD by default) +android_ensure_device || { + echo "no adb device — start ./start-android-emulator.sh --wait" >&2 + exit 3 +} +export SERIAL ANDROID_SERIAL +echo "device: $SERIAL" | tee -a "$SUMMARY" + +IFS=',' read -r -a VLIST <<<"$VARIANTS" +declare -a RESULTS=() +overall=0 + +run_published() { + local out="$MATRIX_OUT/published" + mkdir -p "$out" + echo "" | tee -a "$SUMMARY" + echo "-------- variant: published (F-Droid stable binary) --------" | tee -a "$SUMMARY" + local smoke="$ROOT/run-android-pay-smoke.sh" + [ "$GUI" = "1" ] && smoke="$ROOT/run-android-gui-smoke.sh" + set +e + env STACK="$STACK" \ + APK_URL="$STABLE_APK_URL" \ + APK_NAME="$STABLE_APK_NAME" \ + APK_PATH="" \ + PKG=net.taler.wallet.fdroid \ + OUT_DIR="$out" \ + SERIAL="$SERIAL" \ + AUTO_START_EMULATOR=0 \ + "$smoke" + local rc=$? + set -e + echo "published exit=$rc" | tee -a "$SUMMARY" + echo "published $rc" >>"$MATRIX_OUT/results.tsv" + return "$rc" +} + +run_selfbuild() { + local label="$1" + local ref="$2" + local out="$MATRIX_OUT/$label" + mkdir -p "$out" + echo "" | tee -a "$SUMMARY" + echo "-------- variant: $label (self-build ref=$ref) --------" | tee -a "$SUMMARY" + set +e + env STACK="$STACK" \ + BRANCH="$ref" \ + LABEL="$label" \ + GUI="$GUI" \ + OUT_DIR="$out" \ + SERIAL="$SERIAL" \ + ANDROID_SERIAL="$SERIAL" \ + AUTO_START_EMULATOR=0 \ + PKG=net.taler.wallet.fdroid.debug \ + "$ROOT/run-android-build-and-smoke.sh" + local rc=$? + set -e + echo "$label (ref=$ref) exit=$rc" | tee -a "$SUMMARY" + echo "$label $rc $ref" >>"$MATRIX_OUT/results.tsv" + if [ -f "$out/build-meta.txt" ]; then + echo "build-meta:" | tee -a "$SUMMARY" + cat "$out/build-meta.txt" | tee -a "$SUMMARY" + fi + return "$rc" +} + +for raw in "${VLIST[@]}"; do + v=$(echo "$raw" | tr -d '[:space:]') + [ -n "$v" ] || continue + rc=0 + case "$v" in + published|fdroid|stable-published) + run_published || rc=$? + ;; + stable-self|stable|self-stable|release-self) + run_selfbuild "stable-self" "$STABLE_TAG" || rc=$? + ;; + master|main) + run_selfbuild "master" "$MASTER_BRANCH" || rc=$? + ;; + fix|fix-branch|inference) + run_selfbuild "fix" "$FIX_BRANCH" || rc=$? + ;; + *) + # treat as raw git ref / label + run_selfbuild "ref-${v//\//-}" "$v" || rc=$? + ;; + esac + RESULTS+=("$v:$rc") + if [ "$rc" -ne 0 ]; then + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after $v rc=$rc" | tee -a "$SUMMARY" + break + fi + fi +done + +echo "" | tee -a "$SUMMARY" +echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY" +for r in "${RESULTS[@]}"; do + echo " $r" | tee -a "$SUMMARY" +done +echo "" +echo "Compare OUT_DIR trees under $MATRIX_OUT" +echo "Intent: published vs stable-self should behave similarly;" +echo " master may diverge (new features / fixes)." +echo "A odd stable-self vs published suggests toolchain or unsigned-debug differences;" +echo "a odd master-only failure suggests tip breakage, not F-Droid stable." +exit "$overall" From 1a8aad08cdb077a1946668e53bbb6cc5822fb44c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:52:40 +0200 Subject: [PATCH 20/38] android-test: gate order stable-self then published; FAIL_FAST default Matrix runs release self-build before F-Droid APK, then master. First failure stops. Emulator cold-boots without snapshots by default. --- scripts/taler-monitoring/android-test/README.md | 14 +++++++++----- .../android-test/lib_android_env.sh | 7 ++++++- .../android-test/run-android-variant-matrix.sh | 13 +++++++------ 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 16be18c..fc02faf 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -36,15 +36,19 @@ Runs several smokes so a single odd developer build cannot be the only sample: | Variant | What | |---------|------| -| **published** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | -| **stable-self** | Rebuild tag `wallet-1.6.1` on this host (same source as stable) | -| **master** | Rebuild `origin/master` | +| Order | Variant | What | +|------|---------|------| +| **1 (gate)** | **stable-self** | Rebuild tag `wallet-1.6.1` on this host | +| **2 (gate)** | **published** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | +| **3** | **master** | Rebuild `origin/master` (only if gates pass when `FAIL_FAST=1`) | + +**Default:** `FAIL_FAST=1` — first non-zero gate stops the matrix (no master if stable fails). ```bash ./run-android-variant-matrix.sh STACK=goa ./run-android-variant-matrix.sh -VARIANTS=master,stable-self ./run-android-variant-matrix.sh -VARIANTS=published,stable-self,master,fix GUI=0 ./run-android-variant-matrix.sh +# only the hard gates: +VARIANTS=stable-self,published FAIL_FAST=1 ./run-android-variant-matrix.sh ``` Artifacts + `SUMMARY.txt` under `out-matrix-/`. diff --git a/scripts/taler-monitoring/android-test/lib_android_env.sh b/scripts/taler-monitoring/android-test/lib_android_env.sh index ad209de..670d796 100644 --- a/scripts/taler-monitoring/android-test/lib_android_env.sh +++ b/scripts/taler-monitoring/android-test/lib_android_env.sh @@ -68,8 +68,9 @@ export EMULATOR_GPU : "${EMULATOR_CORES:=2}" # If no adb device: start headless AVD (0 = require phone/manual emulator) : "${AUTO_START_EMULATOR:=1}" -: "${EMULATOR_BOOT_TIMEOUT:=180}" +: "${EMULATOR_BOOT_TIMEOUT:=360}" : "${EMULATOR_LOG:=/tmp/taler-android-emulator.log}" +: "${EMULATOR_NO_SNAPSHOT:=1}" android_adb_serial() { adb devices 2>/dev/null | awk '/\tdevice$/{print $1; exit}' @@ -89,6 +90,10 @@ android_emulator_args() { # Avoid Qt trying to open a display on servers export QT_QPA_PLATFORM="${QT_QPA_PLATFORM:-offscreen}" fi + # Broken default_boot snapshots leave adb offline forever — cold boot by default. + if [ "${EMULATOR_NO_SNAPSHOT:-1}" = "1" ]; then + args+=( -no-snapshot -no-snapshot-load -no-snapshot-save ) + fi if [ -n "${EMULATOR_MEMORY:-}" ]; then args+=( -memory "$EMULATOR_MEMORY" ) fi diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index 50d31a8..a92d739 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -2,10 +2,10 @@ # Run Android wallet smoke against **several build variants** so a single # "weird" developer tree or a bad local build cannot be the only sample. # -# Default matrix: -# 1) **published** — F-Droid stable APK (bit-for-bit what users install) -# 2) **stable-self** — rebuild the same release **tag** as published (self-build) -# 3) **master** — rebuild origin/master (tip / CI-style) +# Default matrix (gate order — fail → hard stop when FAIL_FAST=1): +# 1) **stable-self** — rebuild release **tag** (same source as published) +# 2) **published** — F-Droid stable APK (what users install) +# 3) **master** — rebuild origin/master (only if 1–2 pass with FAIL_FAST) # # Why stable-self in addition to published? # Rare, but a published APK *or* a one-off developer build can be odd @@ -40,8 +40,9 @@ ROOT=$(cd "$(dirname "$0")" && pwd) : "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}" : "${MASTER_BRANCH:=master}" : "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" -: "${VARIANTS:=published,stable-self,master}" -: "${FAIL_FAST:=0}" +# Gate: stable self-build → published F-Droid; then master. FAIL_FAST default on. +: "${VARIANTS:=stable-self,published,master}" +: "${FAIL_FAST:=1}" : "${GUI:=0}" : "${STACK:=auto}" From 64b7ba2b2a4f196ebd67d00569b24cdd1c81b138 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:53:04 +0200 Subject: [PATCH 21/38] android-test: SMOKE_STRICT hard-fail on ANR / missing withdraw core Matrix gates set SMOKE_STRICT=1 so stable-self and published cannot pass on install-only when System UI ANRs or wallet-core never sees the URI. --- .../android-test/run-android-pay-smoke.sh | 15 ++++++++++++++- .../android-test/run-android-variant-matrix.sh | 2 ++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh index ef2ca8e..a5a85b5 100755 --- a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -184,14 +184,26 @@ fi grep -iE 'taler-wallet|prepareBank|preparePay|confirm|error|Error|success' "$OUT_DIR/logcat.txt" \ | tail -80 >"$OUT_DIR/logcat-taler.txt" || true +# SMOKE_STRICT=1 (matrix gate default): ANR or missing wallet-core withdraw = hard fail +: "${SMOKE_STRICT:=0}" +rc=0 echo echo "=== summary ===" echo "package installed: $PKG" echo "withdraw intent delivered: $WURI" +if grep -qiE "isn't responding|isn.t responding|System UI isn" "$OUT_DIR/01-ui.xml" 2>/dev/null \ + || grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null; then + echo "FAIL: System UI / app ANR (host too small or emulator hung)" + rc=10 +fi if grep -q 'prepareBankIntegratedWithdrawal' "$OUT_DIR/logcat.txt" 2>/dev/null; then echo "wallet-core: prepareBankIntegratedWithdrawal seen (wallet accepted withdraw URI)" else echo "wallet-core: prepareBankIntegratedWithdrawal NOT seen (UI may still show onboarding/ANR on small RAM)" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT=1 requires prepareBankIntegratedWithdrawal in logcat" + rc=11 + fi fi if [ -n "$PAY_URI" ]; then if grep -qiE 'preparePay|confirmPay|pay-template|preparePurchase' "$OUT_DIR/logcat.txt" 2>/dev/null; then @@ -204,4 +216,5 @@ echo "artifacts: $OUT_DIR/" ls -la "$OUT_DIR" | sed 's/^/ /' echo echo "NOTE: Full unattended UI payment needs a responsive device/emulator (≥6–8 GiB host RAM recommended)." -echo "This smoke proves: published APK install + deep-link into wallet-core on stack=$STACK." +echo "This smoke proves: APK install + deep-link delivery on stack=$STACK (strict=$SMOKE_STRICT rc=$rc)." +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index a92d739..3bb3e07 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -85,6 +85,7 @@ run_published() { OUT_DIR="$out" \ SERIAL="$SERIAL" \ AUTO_START_EMULATOR=0 \ + SMOKE_STRICT="${SMOKE_STRICT:-1}" \ "$smoke" local rc=$? set -e @@ -110,6 +111,7 @@ run_selfbuild() { ANDROID_SERIAL="$SERIAL" \ AUTO_START_EMULATOR=0 \ PKG=net.taler.wallet.fdroid.debug \ + SMOKE_STRICT="${SMOKE_STRICT:-1}" \ "$ROOT/run-android-build-and-smoke.sh" local rc=$? set -e From 910987731b79dc3761b738cb5d15ec2f55930d48 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:53:53 +0200 Subject: [PATCH 22/38] android-test: stable-self failures are WARN only, not blockers Published F-Droid remains the hard gate (FAIL_FAST). Self-build of the release tag continues the matrix and does not alone fail overall exit. --- .../android-test/GUI-AUTOMATION-NOTES.md | 14 +-- .../taler-monitoring/android-test/README.md | 14 +-- .../run-android-variant-matrix.sh | 107 +++++++++++++----- 3 files changed, 91 insertions(+), 44 deletions(-) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 16023eb..836526d 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -243,14 +243,14 @@ Scripts **today** implement Android only; they should honour: ### Build variants (anti–“weird local build”) -| Variant | Source | Package | Purpose | -|---------|--------|---------|---------| -| **published** | F-Droid APK URL | `net.taler.wallet.fdroid` | What users install | -| **stable-self** | tag `wallet-1.6.1` (override `STABLE_TAG`) rebuilt here | `…fdroid.debug` | Same release tree as stable, **our** Gradle — excludes odd publisher-only or dirty-tree builds | -| **master** | `origin/master` rebuilt here | `…fdroid.debug` | Tip of mainline | -| **fix** (optional) | inference fix branch | `…fdroid.debug` | GOA automation fixes | +| Variant | Source | Severity | Purpose | +|---------|--------|----------|---------| +| **stable-self** | tag `wallet-1.6.1` rebuilt here | **WARN only** | Same release tree, our Gradle — never hard-blocks the matrix | +| **published** | F-Droid APK URL | **BLOCKER** | What users install; FAIL_FAST stops here | +| **master** | `origin/master` | soft fail | Tip of mainline | +| **fix** (optional) | inference fix branch | soft fail | GOA automation fixes | -Default matrix: `published,stable-self,master`. +Default matrix order: `stable-self,published,master`. `./run-android-variant-matrix.sh` — see README. --- diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index fc02faf..3ab8d6f 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -36,18 +36,18 @@ Runs several smokes so a single odd developer build cannot be the only sample: | Variant | What | |---------|------| -| Order | Variant | What | -|------|---------|------| -| **1 (gate)** | **stable-self** | Rebuild tag `wallet-1.6.1` on this host | -| **2 (gate)** | **published** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | -| **3** | **master** | Rebuild `origin/master` (only if gates pass when `FAIL_FAST=1`) | +| Order | Variant | Severity | What | +|------|---------|----------|------| +| **1** | **stable-self** | **WARN only** (not a blocker) | Rebuild tag `wallet-1.6.1` on this host | +| **2** | **published** | **BLOCKER** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | +| **3** | **master** | fail counts / FAIL_FAST | Rebuild `origin/master` | -**Default:** `FAIL_FAST=1` — first non-zero gate stops the matrix (no master if stable fails). +**Default:** `FAIL_FAST=1` only stops on **blocker** failures (`published`). +`stable-self` failure → warning in `SUMMARY.txt`, matrix continues, exit code unaffected by that warn alone. ```bash ./run-android-variant-matrix.sh STACK=goa ./run-android-variant-matrix.sh -# only the hard gates: VARIANTS=stable-self,published FAIL_FAST=1 ./run-android-variant-matrix.sh ``` diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index 3bb3e07..9f80700 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -2,22 +2,23 @@ # Run Android wallet smoke against **several build variants** so a single # "weird" developer tree or a bad local build cannot be the only sample. # -# Default matrix (gate order — fail → hard stop when FAIL_FAST=1): +# Default matrix order: # 1) **stable-self** — rebuild release **tag** (same source as published) +# Failure → **WARN only** (not a hard blocker; continue matrix) # 2) **published** — F-Droid stable APK (what users install) -# 3) **master** — rebuild origin/master (only if 1–2 pass with FAIL_FAST) +# Failure → **BLOCKER** (matrix exit ≠ 0; FAIL_FAST stops here) +# 3) **master** — rebuild origin/master +# Failure → counts toward overall fail; does not skip earlier gates # # Why stable-self in addition to published? -# Rare, but a published APK *or* a one-off developer build can be odd -# (wrong flags, dirty tree, wrong ABI). Rebuilding the release tag on *this* -# machine with the same script path as master isolates "our toolchain" vs -# "F-Droid binary" vs "current master source". +# Isolates "our toolchain rebuild of the release tag" vs "F-Droid binary". +# Rebuild issues (signing flavor, dirty host, low-RAM ANR on debug) must not +# block the published-stable gate — hence warn-only for stable-self. # # Usage: # ./run-android-variant-matrix.sh # STACK=goa ./run-android-variant-matrix.sh # VARIANTS=master,stable-self ./run-android-variant-matrix.sh -# VARIANTS=published,stable-self,master,fix GUI=1 ./run-android-variant-matrix.sh # STABLE_TAG=wallet-1.6.1 MASTER_BRANCH=master ./run-android-variant-matrix.sh # # Env: @@ -27,7 +28,7 @@ # FIX_BRANCH inference fix branch (variant "fix") # VARIANTS comma list: published,stable-self,master,fix # GUI=1 use gui-smoke instead of pay-smoke after each build -# FAIL_FAST=1 stop on first non-zero smoke (default 0 = run all) +# FAIL_FAST=1 stop on first **blocker** failure (default 1) # MATRIX_OUT summary dir (default out-matrix-) # set -euo pipefail @@ -40,7 +41,7 @@ ROOT=$(cd "$(dirname "$0")" && pwd) : "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}" : "${MASTER_BRANCH:=master}" : "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" -# Gate: stable self-build → published F-Droid; then master. FAIL_FAST default on. +# Order: stable-self (warn) → published (blocker) → master. : "${VARIANTS:=stable-self,published,master}" : "${FAIL_FAST:=1}" : "${GUI:=0}" @@ -55,6 +56,7 @@ SUMMARY="$MATRIX_OUT/SUMMARY.txt" echo "=== Android variant matrix ===" | tee -a "$SUMMARY" echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" echo "STABLE_TAG=$STABLE_TAG MASTER_BRANCH=$MASTER_BRANCH" | tee -a "$SUMMARY" +echo "severity: stable-self=WARN published=BLOCKER master=fail-count" | tee -a "$SUMMARY" echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" # Ensure device once (headless AVD by default) @@ -67,13 +69,26 @@ echo "device: $SERIAL" | tee -a "$SUMMARY" IFS=',' read -r -a VLIST <<<"$VARIANTS" declare -a RESULTS=() +declare -a WARNINGS=() overall=0 +# Severity: warn | blocker | soft +# warn — log WARN, never FAIL_FAST, never overall≠0 +# blocker — FAIL_FAST eligible; overall≠0 +# soft — overall≠0 on fail, but FAIL_FAST only if FAIL_FAST=1 and not warn +variant_severity() { + case "$1" in + stable-self|stable|self-stable|release-self) echo warn ;; + published|fdroid|stable-published) echo blocker ;; + *) echo soft ;; + esac +} + run_published() { local out="$MATRIX_OUT/published" mkdir -p "$out" echo "" | tee -a "$SUMMARY" - echo "-------- variant: published (F-Droid stable binary) --------" | tee -a "$SUMMARY" + echo "-------- variant: published (F-Droid stable binary) [BLOCKER] --------" | tee -a "$SUMMARY" local smoke="$ROOT/run-android-pay-smoke.sh" [ "$GUI" = "1" ] && smoke="$ROOT/run-android-gui-smoke.sh" set +e @@ -90,17 +105,23 @@ run_published() { local rc=$? set -e echo "published exit=$rc" | tee -a "$SUMMARY" - echo "published $rc" >>"$MATRIX_OUT/results.tsv" + echo "published $rc blocker" >>"$MATRIX_OUT/results.tsv" return "$rc" } run_selfbuild() { local label="$1" local ref="$2" + local sev="${3:-soft}" local out="$MATRIX_OUT/$label" mkdir -p "$out" echo "" | tee -a "$SUMMARY" - echo "-------- variant: $label (self-build ref=$ref) --------" | tee -a "$SUMMARY" + echo "-------- variant: $label (self-build ref=$ref) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY" + # Warn-only self-build: still collect evidence; SMOKE_STRICT optional (default off for warn) + local strict="${SMOKE_STRICT:-1}" + if [ "$sev" = "warn" ]; then + strict="${SMOKE_STRICT_SELF:-0}" + fi set +e env STACK="$STACK" \ BRANCH="$ref" \ @@ -111,12 +132,12 @@ run_selfbuild() { ANDROID_SERIAL="$SERIAL" \ AUTO_START_EMULATOR=0 \ PKG=net.taler.wallet.fdroid.debug \ - SMOKE_STRICT="${SMOKE_STRICT:-1}" \ + SMOKE_STRICT="$strict" \ "$ROOT/run-android-build-and-smoke.sh" local rc=$? set -e - echo "$label (ref=$ref) exit=$rc" | tee -a "$SUMMARY" - echo "$label $rc $ref" >>"$MATRIX_OUT/results.tsv" + echo "$label (ref=$ref) exit=$rc severity=$sev" | tee -a "$SUMMARY" + echo "$label $rc $ref $sev" >>"$MATRIX_OUT/results.tsv" if [ -f "$out/build-meta.txt" ]; then echo "build-meta:" | tee -a "$SUMMARY" cat "$out/build-meta.txt" | tee -a "$SUMMARY" @@ -128,31 +149,54 @@ for raw in "${VLIST[@]}"; do v=$(echo "$raw" | tr -d '[:space:]') [ -n "$v" ] || continue rc=0 + sev=$(variant_severity "$v") case "$v" in published|fdroid|stable-published) run_published || rc=$? ;; stable-self|stable|self-stable|release-self) - run_selfbuild "stable-self" "$STABLE_TAG" || rc=$? + run_selfbuild "stable-self" "$STABLE_TAG" warn || rc=$? ;; master|main) - run_selfbuild "master" "$MASTER_BRANCH" || rc=$? + run_selfbuild "master" "$MASTER_BRANCH" soft || rc=$? ;; fix|fix-branch|inference) - run_selfbuild "fix" "$FIX_BRANCH" || rc=$? + run_selfbuild "fix" "$FIX_BRANCH" soft || rc=$? ;; *) - # treat as raw git ref / label - run_selfbuild "ref-${v//\//-}" "$v" || rc=$? + run_selfbuild "ref-${v//\//-}" "$v" soft || rc=$? ;; esac - RESULTS+=("$v:$rc") + if [ "$rc" -ne 0 ]; then - overall=1 - if [ "$FAIL_FAST" = "1" ]; then - echo "FAIL_FAST: stopping after $v rc=$rc" | tee -a "$SUMMARY" - break - fi + case "$sev" in + warn) + echo "WARN: variant $v failed rc=$rc — self-build only, NOT a blocker; continuing" | tee -a "$SUMMARY" + WARNINGS+=("$v:$rc") + RESULTS+=("$v:WARN($rc)") + # do not set overall; do not FAIL_FAST + ;; + blocker) + echo "BLOCKER: variant $v failed rc=$rc" | tee -a "$SUMMARY" + RESULTS+=("$v:BLOCKER($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after blocker $v" | tee -a "$SUMMARY" + break + fi + ;; + soft) + echo "FAIL: variant $v rc=$rc (counts toward overall; not stable-self)" | tee -a "$SUMMARY" + RESULTS+=("$v:FAIL($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after $v rc=$rc" | tee -a "$SUMMARY" + break + fi + ;; + esac + else + RESULTS+=("$v:OK") fi done @@ -161,10 +205,13 @@ echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY" for r in "${RESULTS[@]}"; do echo " $r" | tee -a "$SUMMARY" done +if [ "${#WARNINGS[@]}" -gt 0 ]; then + echo "warnings (non-blocking):" | tee -a "$SUMMARY" + for w in "${WARNINGS[@]}"; do + echo " WARN $w" | tee -a "$SUMMARY" + done +fi echo "" echo "Compare OUT_DIR trees under $MATRIX_OUT" -echo "Intent: published vs stable-self should behave similarly;" -echo " master may diverge (new features / fixes)." -echo "A odd stable-self vs published suggests toolchain or unsigned-debug differences;" -echo "a odd master-only failure suggests tip breakage, not F-Droid stable." +echo "stable-self failure = WARN only; published failure = BLOCKER." exit "$overall" From 379baf73e8e90260b66d21c10a6e96223238fe53 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:54:17 +0200 Subject: [PATCH 23/38] android-test: master self-build failures are WARN only too Only published F-Droid remains a matrix blocker; master tip issues warn and continue without failing overall exit alone. --- .../android-test/GUI-AUTOMATION-NOTES.md | 4 ++-- .../taler-monitoring/android-test/README.md | 6 ++--- .../run-android-variant-matrix.sh | 22 ++++++++++--------- 3 files changed, 17 insertions(+), 15 deletions(-) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 836526d..798bb7d 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -245,9 +245,9 @@ Scripts **today** implement Android only; they should honour: | Variant | Source | Severity | Purpose | |---------|--------|----------|---------| -| **stable-self** | tag `wallet-1.6.1` rebuilt here | **WARN only** | Same release tree, our Gradle — never hard-blocks the matrix | +| **stable-self** | tag `wallet-1.6.1` rebuilt here | **WARN only** | Same release tree, our Gradle — never hard-blocks | | **published** | F-Droid APK URL | **BLOCKER** | What users install; FAIL_FAST stops here | -| **master** | `origin/master` | soft fail | Tip of mainline | +| **master** | `origin/master` | **WARN only** | Tip of mainline — never hard-blocks | | **fix** (optional) | inference fix branch | soft fail | GOA automation fixes | Default matrix order: `stable-self,published,master`. diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 3ab8d6f..595a187 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -38,12 +38,12 @@ Runs several smokes so a single odd developer build cannot be the only sample: |---------|------| | Order | Variant | Severity | What | |------|---------|----------|------| -| **1** | **stable-self** | **WARN only** (not a blocker) | Rebuild tag `wallet-1.6.1` on this host | +| **1** | **stable-self** | **WARN only** | Rebuild tag `wallet-1.6.1` on this host | | **2** | **published** | **BLOCKER** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | -| **3** | **master** | fail counts / FAIL_FAST | Rebuild `origin/master` | +| **3** | **master** | **WARN only** | Rebuild `origin/master` | **Default:** `FAIL_FAST=1` only stops on **blocker** failures (`published`). -`stable-self` failure → warning in `SUMMARY.txt`, matrix continues, exit code unaffected by that warn alone. +`stable-self` / `master` failure → warning in `SUMMARY.txt`, matrix continues, exit code unaffected by those warns alone. ```bash ./run-android-variant-matrix.sh diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index 9f80700..baabbf3 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -8,12 +8,12 @@ # 2) **published** — F-Droid stable APK (what users install) # Failure → **BLOCKER** (matrix exit ≠ 0; FAIL_FAST stops here) # 3) **master** — rebuild origin/master -# Failure → counts toward overall fail; does not skip earlier gates +# Failure → **WARN only** (tip breakage / ANR must not block stable gate) # -# Why stable-self in addition to published? -# Isolates "our toolchain rebuild of the release tag" vs "F-Droid binary". -# Rebuild issues (signing flavor, dirty host, low-RAM ANR on debug) must not -# block the published-stable gate — hence warn-only for stable-self. +# Why self-builds in addition to published? +# Isolates "our toolchain rebuild" vs "F-Droid binary". Rebuild issues +# (signing flavor, dirty host, low-RAM ANR on debug) must not block the +# published-stable gate — hence warn-only for stable-self **and** master. # # Usage: # ./run-android-variant-matrix.sh @@ -56,7 +56,7 @@ SUMMARY="$MATRIX_OUT/SUMMARY.txt" echo "=== Android variant matrix ===" | tee -a "$SUMMARY" echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" echo "STABLE_TAG=$STABLE_TAG MASTER_BRANCH=$MASTER_BRANCH" | tee -a "$SUMMARY" -echo "severity: stable-self=WARN published=BLOCKER master=fail-count" | tee -a "$SUMMARY" +echo "severity: stable-self=WARN published=BLOCKER master=WARN" | tee -a "$SUMMARY" echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" # Ensure device once (headless AVD by default) @@ -79,7 +79,9 @@ overall=0 variant_severity() { case "$1" in stable-self|stable|self-stable|release-self) echo warn ;; + master|main) echo warn ;; published|fdroid|stable-published) echo blocker ;; + # optional fix branch etc.: soft fail (counts) unless overridden *) echo soft ;; esac } @@ -158,7 +160,7 @@ for raw in "${VLIST[@]}"; do run_selfbuild "stable-self" "$STABLE_TAG" warn || rc=$? ;; master|main) - run_selfbuild "master" "$MASTER_BRANCH" soft || rc=$? + run_selfbuild "master" "$MASTER_BRANCH" warn || rc=$? ;; fix|fix-branch|inference) run_selfbuild "fix" "$FIX_BRANCH" soft || rc=$? @@ -171,7 +173,7 @@ for raw in "${VLIST[@]}"; do if [ "$rc" -ne 0 ]; then case "$sev" in warn) - echo "WARN: variant $v failed rc=$rc — self-build only, NOT a blocker; continuing" | tee -a "$SUMMARY" + echo "WARN: variant $v failed rc=$rc — self-build, NOT a blocker; continuing" | tee -a "$SUMMARY" WARNINGS+=("$v:$rc") RESULTS+=("$v:WARN($rc)") # do not set overall; do not FAIL_FAST @@ -186,7 +188,7 @@ for raw in "${VLIST[@]}"; do fi ;; soft) - echo "FAIL: variant $v rc=$rc (counts toward overall; not stable-self)" | tee -a "$SUMMARY" + echo "FAIL: variant $v rc=$rc (counts toward overall)" | tee -a "$SUMMARY" RESULTS+=("$v:FAIL($rc)") overall=1 if [ "$FAIL_FAST" = "1" ]; then @@ -213,5 +215,5 @@ if [ "${#WARNINGS[@]}" -gt 0 ]; then fi echo "" echo "Compare OUT_DIR trees under $MATRIX_OUT" -echo "stable-self failure = WARN only; published failure = BLOCKER." +echo "stable-self / master failure = WARN only; published failure = BLOCKER." exit "$overall" From a4f525e40c1f0003654b6e3d51749acec60d13e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:56:01 +0200 Subject: [PATCH 24/38] =?UTF-8?q?android-test:=20smoke=20older=20wallet=20?= =?UTF-8?q?releases;=20blocker=20only=20if=20=E2=89=A414=20days?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expand matrix with previous wallet-* tags (F-Droid APK or self-build). Hard-fail only for released apps younger than BLOCKER_MAX_AGE_DAYS; older releases and master remain warn-only. --- .../android-test/GUI-AUTOMATION-NOTES.md | 13 +- .../taler-monitoring/android-test/README.md | 17 +- .../android-test/lib_release_age.sh | 91 ++++++ .../run-android-variant-matrix.sh | 278 ++++++++++++------ 4 files changed, 294 insertions(+), 105 deletions(-) create mode 100644 scripts/taler-monitoring/android-test/lib_release_age.sh diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 798bb7d..f909ead 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -245,13 +245,14 @@ Scripts **today** implement Android only; they should honour: | Variant | Source | Severity | Purpose | |---------|--------|----------|---------| -| **stable-self** | tag `wallet-1.6.1` rebuilt here | **WARN only** | Same release tree, our Gradle — never hard-blocks | -| **published** | F-Droid APK URL | **BLOCKER** | What users install; FAIL_FAST stops here | -| **master** | `origin/master` | **WARN only** | Tip of mainline — never hard-blocks | -| **fix** (optional) | inference fix branch | soft fail | GOA automation fixes | +| **stable-self** | tag `wallet-1.6.1` rebuilt | WARN | Our rebuild of stable — never hard-blocks | +| **published** | F-Droid current APK | **≤14d → BLOCKER**, else WARN | What users install | +| **older-pub / older-self** | previous `wallet-*` tags | **≤14d → BLOCKER**, else WARN | Regression on recent releases only | +| **master** | `origin/master` | WARN | Tip — never hard-blocks | +| **fix** (optional) | inference branch | soft | GOA automation fixes | -Default matrix order: `stable-self,published,master`. -`./run-android-variant-matrix.sh` — see README. +**Two-week window:** `BLOCKER_MAX_AGE_DAYS=14` (git tag age). Older apps are still tried; only young releases hard-block. +Default matrix: `stable-self,published,older,master`. See README. --- diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 595a187..8e0263f 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -38,17 +38,22 @@ Runs several smokes so a single odd developer build cannot be the only sample: |---------|------| | Order | Variant | Severity | What | |------|---------|----------|------| -| **1** | **stable-self** | **WARN only** | Rebuild tag `wallet-1.6.1` on this host | -| **2** | **published** | **BLOCKER** | F-Droid stable APK (`net.taler.wallet.fdroid` 1.6.1 / 854) | -| **3** | **master** | **WARN only** | Rebuild `origin/master` | +| **1** | **stable-self** | WARN | Rebuild tag `wallet-1.6.1` on this host | +| **2** | **published** | **age rule** | Current F-Droid APK | +| **3** | **older-*** | **age rule** | Previous `wallet-*` tags (APK if on F-Droid, else self-build) | +| **4** | **master** | WARN | Rebuild `origin/master` | -**Default:** `FAIL_FAST=1` only stops on **blocker** failures (`published`). -`stable-self` / `master` failure → warning in `SUMMARY.txt`, matrix continues, exit code unaffected by those warns alone. +**Age rule (released apps):** if git-tag age **≤ 14 days** (`BLOCKER_MAX_AGE_DAYS`) → fail is **BLOCKER** + `FAIL_FAST`. +Older releases are still smoked, but fail only as **WARN**. +Self-build of tip/stable-self is never a hard blocker. ```bash ./run-android-variant-matrix.sh STACK=goa ./run-android-variant-matrix.sh -VARIANTS=stable-self,published FAIL_FAST=1 ./run-android-variant-matrix.sh +# without older history: +INCLUDE_OLDER=0 VARIANTS=stable-self,published,master ./run-android-variant-matrix.sh +# force current published always blocker: +PUBLISHED_FORCE_BLOCKER=1 ./run-android-variant-matrix.sh ``` Artifacts + `SUMMARY.txt` under `out-matrix-/`. diff --git a/scripts/taler-monitoring/android-test/lib_release_age.sh b/scripts/taler-monitoring/android-test/lib_release_age.sh new file mode 100644 index 0000000..52fa2ce --- /dev/null +++ b/scripts/taler-monitoring/android-test/lib_release_age.sh @@ -0,0 +1,91 @@ +# shellcheck shell=bash +# Release age helpers for Android wallet matrix. +# Source after ROOT/lib_android_env.sh (optional). +# +# BLOCKER only when a *released* app is within BLOCKER_MAX_AGE_DAYS (default 14). +# Older releases: try them, but failures are WARN only. + +: "${BLOCKER_MAX_AGE_DAYS:=14}" +: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}" + +# Days since git tag (tagger/committer date). Empty if unknown. +# Usage: release_age_days wallet-1.6.1 → e.g. 28 +release_age_days() { + local tag="$1" + local src="${2:-$TALER_ANDROID_SRC}" + [ -d "$src/.git" ] || return 1 + local epoch + epoch=$(git -C "$src" log -1 --format=%ct "refs/tags/${tag}" 2>/dev/null || true) + [ -n "$epoch" ] || epoch=$(git -C "$src" log -1 --format=%ct "$tag" 2>/dev/null || true) + [ -n "$epoch" ] || return 1 + local now + now=$(date +%s) + echo $(( (now - epoch) / 86400 )) +} + +# severity from age: blocker if age_days <= BLOCKER_MAX_AGE_DAYS, else warn +# Unknown age → warn (never accidental hard fail on mystery builds) +severity_from_age_days() { + local days="${1:-}" + if [ -z "$days" ] || ! [[ "$days" =~ ^[0-9]+$ ]]; then + echo warn + return + fi + if [ "$days" -le "${BLOCKER_MAX_AGE_DAYS}" ]; then + echo blocker + else + echo warn + fi +} + +severity_for_tag() { + local tag="$1" + local days + days=$(release_age_days "$tag" 2>/dev/null || true) + severity_from_age_days "$days" +} + +# List wallet-* release tags newest first (exclude rc/dev). +# Args: max count (default 5) +list_wallet_release_tags() { + local n="${1:-5}" + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + [ -d "$src/.git" ] || return 1 + git -C "$src" fetch --tags origin 2>/dev/null || true + git -C "$src" for-each-ref --sort=-creatordate --format='%(refname:short)' refs/tags \ + | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' \ + | head -n "$n" +} + +# Guess F-Droid APK URL for a wallet version name (e.g. 1.6.1). +# Uses versionCode from tag's build.gradle if possible, else STABLE mapping. +# Prints: versionCode apk_url or returns 1 +fdroid_apk_for_version_name() { + local vn="$1" # 1.6.1 + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + local tag="wallet-${vn}" + local vc="" + if git -C "$src" cat-file -e "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null; then + vc=$(git -C "$src" show "refs/tags/${tag}:wallet/build.gradle" 2>/dev/null \ + | sed -n 's/^[[:space:]]*versionCode[[:space:]]\+\([0-9]\+\).*/\1/p' | head -1) + fi + # ABI x86_64 → +4 in override scheme 10*vc+abi (common F-Droid multi-apk) + # Prefer universal-style code used by F-Droid: for 1.6.1 → 854 = 10*85+4 + local apk_vc="" + if [ -n "$vc" ]; then + apk_vc=$((10 * vc + 4)) + fi + # Known overrides / suggested + case "$vn" in + 1.6.1) apk_vc=854 ;; + esac + [ -n "$apk_vc" ] || return 1 + echo "${apk_vc}|https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk" +} + +# Check HTTP HEAD/GET that APK exists (0 = ok) +fdroid_apk_exists() { + local url="$1" + curl -sfS -m 15 -o /dev/null -r 0-0 "$url" 2>/dev/null \ + || curl -sfS -m 15 -o /dev/null -I "$url" 2>/dev/null +} diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index baabbf3..c658096 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -1,51 +1,49 @@ #!/usr/bin/env bash -# Run Android wallet smoke against **several build variants** so a single -# "weird" developer tree or a bad local build cannot be the only sample. +# Run Android wallet smoke against several build / published variants. # -# Default matrix order: -# 1) **stable-self** — rebuild release **tag** (same source as published) -# Failure → **WARN only** (not a hard blocker; continue matrix) -# 2) **published** — F-Droid stable APK (what users install) -# Failure → **BLOCKER** (matrix exit ≠ 0; FAIL_FAST stops here) -# 3) **master** — rebuild origin/master -# Failure → **WARN only** (tip breakage / ANR must not block stable gate) +# Default order: +# 1) stable-self — rebuild STABLE_TAG (always WARN on fail) +# 2) published — current F-Droid suggested APK +# 3) older-* — previous release tags (APK and/or self-build) if available +# 4) master — tip (always WARN on fail) # -# Why self-builds in addition to published? -# Isolates "our toolchain rebuild" vs "F-Droid binary". Rebuild issues -# (signing flavor, dirty host, low-RAM ANR on debug) must not block the -# published-stable gate — hence warn-only for stable-self **and** master. -# -# Usage: -# ./run-android-variant-matrix.sh -# STACK=goa ./run-android-variant-matrix.sh -# VARIANTS=master,stable-self ./run-android-variant-matrix.sh -# STABLE_TAG=wallet-1.6.1 MASTER_BRANCH=master ./run-android-variant-matrix.sh +# Severity (released apps): +# Age ≤ BLOCKER_MAX_AGE_DAYS (default **14**) → **BLOCKER** on fail (FAIL_FAST) +# Age > 14 days (or unknown) → **WARN** only +# master / pure self-builds without release age → WARN # # Env: -# STABLE_TAG default wallet-1.6.1 (must match published F-Droid release) -# STABLE_APK_URL published APK (same as pay-smoke default) -# MASTER_BRANCH default master -# FIX_BRANCH inference fix branch (variant "fix") -# VARIANTS comma list: published,stable-self,master,fix -# GUI=1 use gui-smoke instead of pay-smoke after each build -# FAIL_FAST=1 stop on first **blocker** failure (default 1) -# MATRIX_OUT summary dir (default out-matrix-) +# STABLE_TAG, STABLE_APK_URL, STABLE_APK_NAME +# MASTER_BRANCH, FIX_BRANCH +# VARIANTS default: stable-self,published,older,master +# INCLUDE_OLDER=1 expand token "older" (default 1) +# OLDER_TAG_COUNT how many recent wallet-* tags to consider (default 4) +# BLOCKER_MAX_AGE_DAYS default 14 +# PUBLISHED_FORCE_BLOCKER=1 always treat current published as blocker (default 0 = age rule) +# FAIL_FAST=1, GUI=0, STACK=… # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) # shellcheck source=lib_android_env.sh . "$ROOT/lib_android_env.sh" +# shellcheck source=lib_release_age.sh +. "$ROOT/lib_release_age.sh" : "${STABLE_TAG:=wallet-1.6.1}" : "${STABLE_APK_URL:=https://f-droid.org/repo/net.taler.wallet.fdroid_854.apk}" : "${STABLE_APK_NAME:=net.taler.wallet.fdroid_854.apk}" : "${MASTER_BRANCH:=master}" : "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" -# Order: stable-self (warn) → published (blocker) → master. -: "${VARIANTS:=stable-self,published,master}" +: "${VARIANTS:=stable-self,published,older,master}" +: "${INCLUDE_OLDER:=1}" +: "${OLDER_TAG_COUNT:=4}" +: "${BLOCKER_MAX_AGE_DAYS:=14}" +: "${PUBLISHED_FORCE_BLOCKER:=0}" : "${FAIL_FAST:=1}" : "${GUI:=0}" : "${STACK:=auto}" +: "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}" +export TALER_ANDROID_SRC BLOCKER_MAX_AGE_DAYS TS=$(date +%Y%m%d-%H%M%S) MATRIX_OUT="${MATRIX_OUT:-$ROOT/out-matrix-$TS}" @@ -55,11 +53,9 @@ SUMMARY="$MATRIX_OUT/SUMMARY.txt" echo "=== Android variant matrix ===" | tee -a "$SUMMARY" echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" -echo "STABLE_TAG=$STABLE_TAG MASTER_BRANCH=$MASTER_BRANCH" | tee -a "$SUMMARY" -echo "severity: stable-self=WARN published=BLOCKER master=WARN" | tee -a "$SUMMARY" +echo "STABLE_TAG=$STABLE_TAG BLOCKER_MAX_AGE_DAYS=$BLOCKER_MAX_AGE_DAYS OLDER_TAG_COUNT=$OLDER_TAG_COUNT" | tee -a "$SUMMARY" echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" -# Ensure device once (headless AVD by default) android_ensure_device || { echo "no adb device — start ./start-android-emulator.sh --wait" >&2 exit 3 @@ -67,63 +63,114 @@ android_ensure_device || { export SERIAL ANDROID_SERIAL echo "device: $SERIAL" | tee -a "$SUMMARY" -IFS=',' read -r -a VLIST <<<"$VARIANTS" +# --- expand VARIANTS (token "older" → concrete older-pub / older-self entries) --- +expand_variants() { + local raw="$1" + local -a out=() + local v tag vn age sev stable_vn + stable_vn="${STABLE_TAG#wallet-}" + + IFS=',' read -r -a toks <<<"$raw" + for v in "${toks[@]}"; do + v=$(echo "$v" | tr -d '[:space:]') + [ -n "$v" ] || continue + if [ "$v" = "older" ] || [ "$v" = "older-releases" ]; then + [ "$INCLUDE_OLDER" = "1" ] || continue + local -a tags=() + mapfile -t tags < <(list_wallet_release_tags "$OLDER_TAG_COUNT" 2>/dev/null || true) + local t + for t in "${tags[@]}"; do + [ -n "$t" ] || continue + # skip current stable tag (already covered by published / stable-self) + [ "$t" = "$STABLE_TAG" ] && continue + vn="${t#wallet-}" + age=$(release_age_days "$t" 2>/dev/null || echo "") + sev=$(severity_from_age_days "$age") + echo " older candidate $t age_days=${age:-?} severity=$sev" | tee -a "$SUMMARY" + # Prefer published APK when downloadable; always also allow self-build token + local meta apk_vc apk_url + meta=$(fdroid_apk_for_version_name "$vn" 2>/dev/null || true) + if [ -n "$meta" ]; then + apk_vc="${meta%%|*}" + apk_url="${meta#*|}" + if fdroid_apk_exists "$apk_url"; then + out+=("older-pub:${vn}:${apk_vc}:${sev}") + else + echo " (no F-Droid APK for $vn at $apk_url — self-build only)" | tee -a "$SUMMARY" + out+=("older-self:${t}:${sev}") + fi + else + out+=("older-self:${t}:${sev}") + fi + done + else + out+=("$v") + fi + done + printf '%s\n' "${out[@]}" +} + +mapfile -t VLIST < <(expand_variants "$VARIANTS") +echo "expanded variants: ${VLIST[*]}" | tee -a "$SUMMARY" + declare -a RESULTS=() declare -a WARNINGS=() overall=0 -# Severity: warn | blocker | soft -# warn — log WARN, never FAIL_FAST, never overall≠0 -# blocker — FAIL_FAST eligible; overall≠0 -# soft — overall≠0 on fail, but FAIL_FAST only if FAIL_FAST=1 and not warn -variant_severity() { - case "$1" in - stable-self|stable|self-stable|release-self) echo warn ;; - master|main) echo warn ;; - published|fdroid|stable-published) echo blocker ;; - # optional fix branch etc.: soft fail (counts) unless overridden - *) echo soft ;; - esac +severity_for_published_current() { + if [ "${PUBLISHED_FORCE_BLOCKER}" = "1" ]; then + echo blocker + return + fi + local days + days=$(release_age_days "$STABLE_TAG" 2>/dev/null || true) + severity_from_age_days "$days" } -run_published() { - local out="$MATRIX_OUT/published" +run_published_apk() { + local label="$1" + local apk_url="$2" + local apk_name="$3" + local sev="$4" + local out="$MATRIX_OUT/$label" mkdir -p "$out" echo "" | tee -a "$SUMMARY" - echo "-------- variant: published (F-Droid stable binary) [BLOCKER] --------" | tee -a "$SUMMARY" + echo "-------- variant: $label (F-Droid APK) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY" + echo " url=$apk_url" | tee -a "$SUMMARY" local smoke="$ROOT/run-android-pay-smoke.sh" [ "$GUI" = "1" ] && smoke="$ROOT/run-android-gui-smoke.sh" + local strict=1 + [ "$sev" = "warn" ] && strict="${SMOKE_STRICT_WARN:-0}" + [ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}" set +e env STACK="$STACK" \ - APK_URL="$STABLE_APK_URL" \ - APK_NAME="$STABLE_APK_NAME" \ + APK_URL="$apk_url" \ + APK_NAME="$apk_name" \ APK_PATH="" \ PKG=net.taler.wallet.fdroid \ OUT_DIR="$out" \ SERIAL="$SERIAL" \ AUTO_START_EMULATOR=0 \ - SMOKE_STRICT="${SMOKE_STRICT:-1}" \ + SMOKE_STRICT="$strict" \ "$smoke" local rc=$? set -e - echo "published exit=$rc" | tee -a "$SUMMARY" - echo "published $rc blocker" >>"$MATRIX_OUT/results.tsv" + echo "$label exit=$rc severity=$sev" | tee -a "$SUMMARY" + echo "$label $rc $sev $apk_url" >>"$MATRIX_OUT/results.tsv" return "$rc" } run_selfbuild() { local label="$1" local ref="$2" - local sev="${3:-soft}" + local sev="${3:-warn}" local out="$MATRIX_OUT/$label" mkdir -p "$out" echo "" | tee -a "$SUMMARY" echo "-------- variant: $label (self-build ref=$ref) [$(echo "$sev" | tr 'a-z' 'A-Z')] --------" | tee -a "$SUMMARY" - # Warn-only self-build: still collect evidence; SMOKE_STRICT optional (default off for warn) local strict="${SMOKE_STRICT:-1}" - if [ "$sev" = "warn" ]; then - strict="${SMOKE_STRICT_SELF:-0}" - fi + [ "$sev" = "warn" ] && strict="${SMOKE_STRICT_SELF:-0}" + [ "$sev" = "blocker" ] && strict="${SMOKE_STRICT:-1}" set +e env STACK="$STACK" \ BRANCH="$ref" \ @@ -135,6 +182,7 @@ run_selfbuild() { AUTO_START_EMULATOR=0 \ PKG=net.taler.wallet.fdroid.debug \ SMOKE_STRICT="$strict" \ + TALER_ANDROID_SRC="$TALER_ANDROID_SRC" \ "$ROOT/run-android-build-and-smoke.sh" local rc=$? set -e @@ -147,63 +195,108 @@ run_selfbuild() { return "$rc" } +record_result() { + local v="$1" rc="$2" sev="$3" + if [ "$rc" -eq 0 ]; then + RESULTS+=("$v:OK($sev)") + return + fi + case "$sev" in + warn) + echo "WARN: $v failed rc=$rc — not a blocker; continuing" | tee -a "$SUMMARY" + WARNINGS+=("$v:$rc") + RESULTS+=("$v:WARN($rc)") + ;; + blocker) + echo "BLOCKER: $v failed rc=$rc (release within ${BLOCKER_MAX_AGE_DAYS}d window)" | tee -a "$SUMMARY" + RESULTS+=("$v:BLOCKER($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after blocker $v" | tee -a "$SUMMARY" + return 2 + fi + ;; + soft) + echo "FAIL: $v rc=$rc" | tee -a "$SUMMARY" + RESULTS+=("$v:FAIL($rc)") + overall=1 + if [ "$FAIL_FAST" = "1" ]; then + echo "FAIL_FAST: stopping after $v" | tee -a "$SUMMARY" + return 2 + fi + ;; + esac + return 0 +} + for raw in "${VLIST[@]}"; do v=$(echo "$raw" | tr -d '[:space:]') [ -n "$v" ] || continue rc=0 - sev=$(variant_severity "$v") + sev=warn + stop=0 + case "$v" in published|fdroid|stable-published) - run_published || rc=$? + sev=$(severity_for_published_current) + age=$(release_age_days "$STABLE_TAG" 2>/dev/null || echo "?") + echo "published current: tag=$STABLE_TAG age_days=$age → severity=$sev" | tee -a "$SUMMARY" + run_published_apk "published" "$STABLE_APK_URL" "$STABLE_APK_NAME" "$sev" || rc=$? ;; stable-self|stable|self-stable|release-self) + # rebuild of stable: always warn (toolchain), age does not make self-build a blocker + sev=warn run_selfbuild "stable-self" "$STABLE_TAG" warn || rc=$? ;; master|main) + sev=warn run_selfbuild "master" "$MASTER_BRANCH" warn || rc=$? ;; fix|fix-branch|inference) + sev=soft run_selfbuild "fix" "$FIX_BRANCH" soft || rc=$? ;; + older-pub:*) + # older-pub:VERSION:VC:SEV + IFS=':' read -r _ vn apk_vc sev <<<"$v" + : "${sev:=warn}" + label="older-pub-${vn}" + url="https://f-droid.org/repo/net.taler.wallet.fdroid_${apk_vc}.apk" + name="net.taler.wallet.fdroid_${apk_vc}.apk" + age=$(release_age_days "wallet-${vn}" 2>/dev/null || echo "?") + echo "older published $vn age_days=$age severity=$sev" | tee -a "$SUMMARY" + run_published_apk "$label" "$url" "$name" "$sev" || rc=$? + v="$label" + ;; + older-self:*) + # older-self:TAG:SEV or older-self:TAG + IFS=':' read -r _ tag sev <<<"$v" + : "${sev:=$(severity_for_tag "$tag")}" + label="older-self-${tag}" + age=$(release_age_days "$tag" 2>/dev/null || echo "?") + echo "older self-build $tag age_days=$age severity=$sev" | tee -a "$SUMMARY" + run_selfbuild "$label" "$tag" "$sev" || rc=$? + v="$label" + ;; *) - run_selfbuild "ref-${v//\//-}" "$v" soft || rc=$? + # raw git ref — warn unless looks like wallet-X.Y and young + if [[ "$v" =~ ^wallet-[0-9] ]]; then + sev=$(severity_for_tag "$v") + run_selfbuild "ref-${v}" "$v" "$sev" || rc=$? + else + sev=warn + run_selfbuild "ref-${v//\//-}" "$v" warn || rc=$? + fi ;; esac - if [ "$rc" -ne 0 ]; then - case "$sev" in - warn) - echo "WARN: variant $v failed rc=$rc — self-build, NOT a blocker; continuing" | tee -a "$SUMMARY" - WARNINGS+=("$v:$rc") - RESULTS+=("$v:WARN($rc)") - # do not set overall; do not FAIL_FAST - ;; - blocker) - echo "BLOCKER: variant $v failed rc=$rc" | tee -a "$SUMMARY" - RESULTS+=("$v:BLOCKER($rc)") - overall=1 - if [ "$FAIL_FAST" = "1" ]; then - echo "FAIL_FAST: stopping after blocker $v" | tee -a "$SUMMARY" - break - fi - ;; - soft) - echo "FAIL: variant $v rc=$rc (counts toward overall)" | tee -a "$SUMMARY" - RESULTS+=("$v:FAIL($rc)") - overall=1 - if [ "$FAIL_FAST" = "1" ]; then - echo "FAIL_FAST: stopping after $v rc=$rc" | tee -a "$SUMMARY" - break - fi - ;; - esac - else - RESULTS+=("$v:OK") - fi + record_result "$v" "$rc" "$sev" || stop=$? + [ "$stop" = "2" ] && break done echo "" | tee -a "$SUMMARY" echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY" +echo "rule: released app age ≤ ${BLOCKER_MAX_AGE_DAYS}d → BLOCKER on fail; older → WARN" | tee -a "$SUMMARY" for r in "${RESULTS[@]}"; do echo " $r" | tee -a "$SUMMARY" done @@ -214,6 +307,5 @@ if [ "${#WARNINGS[@]}" -gt 0 ]; then done fi echo "" -echo "Compare OUT_DIR trees under $MATRIX_OUT" -echo "stable-self / master failure = WARN only; published failure = BLOCKER." +echo "Artifacts: $MATRIX_OUT" exit "$overall" From 984b2d64343aa327e817f989fd16c740602c8db5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:57:01 +0200 Subject: [PATCH 25/38] android-test: sample 3/6/9/12-month wallet tags as WARN only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pick nearest release tags to 90/180/270/365 days for the older matrix slot; failures there never hard-block (only ≤14d releases can). --- .../android-test/GUI-AUTOMATION-NOTES.md | 9 +- .../taler-monitoring/android-test/README.md | 17 ++-- .../android-test/lib_release_age.sh | 50 ++++++++++- .../run-android-variant-matrix.sh | 87 ++++++++++--------- 4 files changed, 107 insertions(+), 56 deletions(-) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index f909ead..0d418be 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -245,14 +245,15 @@ Scripts **today** implement Android only; they should honour: | Variant | Source | Severity | Purpose | |---------|--------|----------|---------| -| **stable-self** | tag `wallet-1.6.1` rebuilt | WARN | Our rebuild of stable — never hard-blocks | +| **stable-self** | stable tag rebuilt | WARN | Our rebuild — never hard-blocks | | **published** | F-Droid current APK | **≤14d → BLOCKER**, else WARN | What users install | -| **older-pub / older-self** | previous `wallet-*` tags | **≤14d → BLOCKER**, else WARN | Regression on recent releases only | +| **older / milestones** | tags ~**3 / 6 / 9 / 12 mo** | **always WARN** | Long-horizon regression, if tags exist | | **master** | `origin/master` | WARN | Tip — never hard-blocks | | **fix** (optional) | inference branch | soft | GOA automation fixes | -**Two-week window:** `BLOCKER_MAX_AGE_DAYS=14` (git tag age). Older apps are still tried; only young releases hard-block. -Default matrix: `stable-self,published,older,master`. See README. +**Blocker window:** only releases ≤ 14 days. +**Milestones:** nearest tags to 90/180/270/365 days (±50d) — warn only. +Default matrix: `stable-self,published,older,master`. --- diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 8e0263f..bb66bce 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -38,22 +38,21 @@ Runs several smokes so a single odd developer build cannot be the only sample: |---------|------| | Order | Variant | Severity | What | |------|---------|----------|------| -| **1** | **stable-self** | WARN | Rebuild tag `wallet-1.6.1` on this host | -| **2** | **published** | **age rule** | Current F-Droid APK | -| **3** | **older-*** | **age rule** | Previous `wallet-*` tags (APK if on F-Droid, else self-build) | +| **1** | **stable-self** | WARN | Rebuild current stable tag | +| **2** | **published** | age ≤14d → **BLOCKER**, else WARN | Current F-Droid APK | +| **3** | **older / milestones** | **always WARN** | Tags nearest **3 / 6 / 9 / 12 months** (if available) | | **4** | **master** | WARN | Rebuild `origin/master` | -**Age rule (released apps):** if git-tag age **≤ 14 days** (`BLOCKER_MAX_AGE_DAYS`) → fail is **BLOCKER** + `FAIL_FAST`. -Older releases are still smoked, but fail only as **WARN**. -Self-build of tip/stable-self is never a hard blocker. +**Milestones:** `MILESTONE_DAYS="90 180 270 365"` (± `MILESTONE_TOLERANCE_DAYS=50`). +Failures there are warnings only — never hard blockers. +**Young releases only** (≤ 14 days): hard blocker on fail. ```bash ./run-android-variant-matrix.sh STACK=goa ./run-android-variant-matrix.sh -# without older history: INCLUDE_OLDER=0 VARIANTS=stable-self,published,master ./run-android-variant-matrix.sh -# force current published always blocker: -PUBLISHED_FORCE_BLOCKER=1 ./run-android-variant-matrix.sh +# custom milestones (days): +MILESTONE_DAYS="90 180 270 365" ./run-android-variant-matrix.sh ``` Artifacts + `SUMMARY.txt` under `out-matrix-/`. diff --git a/scripts/taler-monitoring/android-test/lib_release_age.sh b/scripts/taler-monitoring/android-test/lib_release_age.sh index 52fa2ce..7be935f 100644 --- a/scripts/taler-monitoring/android-test/lib_release_age.sh +++ b/scripts/taler-monitoring/android-test/lib_release_age.sh @@ -46,15 +46,57 @@ severity_for_tag() { } # List wallet-* release tags newest first (exclude rc/dev). -# Args: max count (default 5) +# Args: max count (default 5). Pass 0 or "all" for no limit. list_wallet_release_tags() { local n="${1:-5}" local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" [ -d "$src/.git" ] || return 1 git -C "$src" fetch --tags origin 2>/dev/null || true - git -C "$src" for-each-ref --sort=-creatordate --format='%(refname:short)' refs/tags \ - | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' \ - | head -n "$n" + local cmd=(git -C "$src" for-each-ref --sort=-creatordate --format='%(refname:short)' refs/tags) + if [ "$n" = "all" ] || [ "$n" = "0" ]; then + "${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' + else + "${cmd[@]}" | grep -E '^wallet-[0-9]+\.[0-9]+(\.[0-9]+)?$' | head -n "$n" + fi +} + +# Pick one tag closest to each age milestone (days). Always intended as WARN samples. +# Default milestones: 3,6,9,12 months ≈ 90,180,270,365 days. +# Env: MILESTONE_DAYS="90 180 270 365" MILESTONE_TOLERANCE_DAYS=50 +# Prints lines: TAG AGE_DAYS TARGET_DAYS (only when a tag within tolerance exists) +select_milestone_tags() { + local src="${TALER_ANDROID_SRC:-$HOME/taler/taler-android}" + local milestones="${MILESTONE_DAYS:-90 180 270 365}" + local tol="${MILESTONE_TOLERANCE_DAYS:-50}" + local skip="${1:-}" # optional tag to exclude (e.g. current STABLE_TAG) + local -a tags=() + mapfile -t tags < <(list_wallet_release_tags all 2>/dev/null || true) + [ "${#tags[@]}" -gt 0 ] || return 0 + + local target best_tag best_age best_delta age t delta + local -A used=() + for target in $milestones; do + best_tag="" + best_age="" + best_delta=99999 + for t in "${tags[@]}"; do + [ -n "$t" ] || continue + [ "$t" = "$skip" ] && continue + [ -n "${used[$t]:-}" ] && continue + age=$(release_age_days "$t" "$src" 2>/dev/null || true) + [[ "$age" =~ ^[0-9]+$ ]] || continue + delta=$(( age > target ? age - target : target - age )) + if [ "$delta" -le "$tol" ] && [ "$delta" -lt "$best_delta" ]; then + best_delta=$delta + best_tag=$t + best_age=$age + fi + done + if [ -n "$best_tag" ]; then + used[$best_tag]=1 + echo "$best_tag $best_age $target" + fi + done } # Guess F-Droid APK URL for a wallet version name (e.g. 1.6.1). diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index c658096..976df43 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -4,22 +4,23 @@ # Default order: # 1) stable-self — rebuild STABLE_TAG (always WARN on fail) # 2) published — current F-Droid suggested APK -# 3) older-* — previous release tags (APK and/or self-build) if available -# 4) master — tip (always WARN on fail) +# 3) older / milestones — releases near **3 / 6 / 9 / 12 months** (WARN only) +# 4) master — tip (always WARN on fail) # # Severity (released apps): # Age ≤ BLOCKER_MAX_AGE_DAYS (default **14**) → **BLOCKER** on fail (FAIL_FAST) -# Age > 14 days (or unknown) → **WARN** only -# master / pure self-builds without release age → WARN +# Age > 14 days, incl. 3/6/9/12‑month samples → **WARN** only +# master / stable-self → WARN # # Env: # STABLE_TAG, STABLE_APK_URL, STABLE_APK_NAME # MASTER_BRANCH, FIX_BRANCH # VARIANTS default: stable-self,published,older,master -# INCLUDE_OLDER=1 expand token "older" (default 1) -# OLDER_TAG_COUNT how many recent wallet-* tags to consider (default 4) +# INCLUDE_OLDER=1 expand token "older" / "milestones" (default 1) +# MILESTONE_DAYS default "90 180 270 365" (≈ 3,6,9,12 months) +# MILESTONE_TOLERANCE_DAYS default 50 — max distance to accept a tag # BLOCKER_MAX_AGE_DAYS default 14 -# PUBLISHED_FORCE_BLOCKER=1 always treat current published as blocker (default 0 = age rule) +# PUBLISHED_FORCE_BLOCKER=1 always treat current published as blocker # FAIL_FAST=1, GUI=0, STACK=… # set -euo pipefail @@ -36,14 +37,15 @@ ROOT=$(cd "$(dirname "$0")" && pwd) : "${FIX_BRANCH:=dev/hernani-inference/fix-bank-withdraw-auto-exchange}" : "${VARIANTS:=stable-self,published,older,master}" : "${INCLUDE_OLDER:=1}" -: "${OLDER_TAG_COUNT:=4}" +: "${MILESTONE_DAYS:=90 180 270 365}" +: "${MILESTONE_TOLERANCE_DAYS:=50}" : "${BLOCKER_MAX_AGE_DAYS:=14}" : "${PUBLISHED_FORCE_BLOCKER:=0}" : "${FAIL_FAST:=1}" : "${GUI:=0}" : "${STACK:=auto}" : "${TALER_ANDROID_SRC:=$HOME/taler/taler-android}" -export TALER_ANDROID_SRC BLOCKER_MAX_AGE_DAYS +export TALER_ANDROID_SRC BLOCKER_MAX_AGE_DAYS MILESTONE_DAYS MILESTONE_TOLERANCE_DAYS TS=$(date +%Y%m%d-%H%M%S) MATRIX_OUT="${MATRIX_OUT:-$ROOT/out-matrix-$TS}" @@ -53,7 +55,8 @@ SUMMARY="$MATRIX_OUT/SUMMARY.txt" echo "=== Android variant matrix ===" | tee -a "$SUMMARY" echo "STACK=$STACK GUI=$GUI headless=${EMULATOR_HEADLESS} variants=$VARIANTS" | tee -a "$SUMMARY" -echo "STABLE_TAG=$STABLE_TAG BLOCKER_MAX_AGE_DAYS=$BLOCKER_MAX_AGE_DAYS OLDER_TAG_COUNT=$OLDER_TAG_COUNT" | tee -a "$SUMMARY" +echo "STABLE_TAG=$STABLE_TAG BLOCKER_MAX_AGE_DAYS=$BLOCKER_MAX_AGE_DAYS" | tee -a "$SUMMARY" +echo "milestones(days)=$MILESTONE_DAYS tolerance=$MILESTONE_TOLERANCE_DAYS (always WARN)" | tee -a "$SUMMARY" echo "out: $MATRIX_OUT" | tee -a "$SUMMARY" android_ensure_device || { @@ -64,45 +67,51 @@ export SERIAL ANDROID_SERIAL echo "device: $SERIAL" | tee -a "$SUMMARY" # --- expand VARIANTS (token "older" → concrete older-pub / older-self entries) --- +# Append one older release as WARN-only (pub APK if possible, else self-build). +append_older_warn() { + local t="$1" age="$2" target="${3:-}" + local vn meta apk_vc apk_url + vn="${t#wallet-}" + if [ -n "$target" ]; then + echo " milestone ~${target}d → $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY" + else + echo " older $t (age=${age}d) severity=WARN" | tee -a "$SUMMARY" + fi + meta=$(fdroid_apk_for_version_name "$vn" 2>/dev/null || true) + if [ -n "$meta" ]; then + apk_vc="${meta%%|*}" + apk_url="${meta#*|}" + if fdroid_apk_exists "$apk_url"; then + out+=("older-pub:${vn}:${apk_vc}:warn") + return + fi + echo " (no F-Droid APK for $vn — self-build only)" | tee -a "$SUMMARY" + fi + out+=("older-self:${t}:warn") +} + expand_variants() { local raw="$1" local -a out=() - local v tag vn age sev stable_vn - stable_vn="${STABLE_TAG#wallet-}" + local v tag vn age sev line t target IFS=',' read -r -a toks <<<"$raw" for v in "${toks[@]}"; do v=$(echo "$v" | tr -d '[:space:]') [ -n "$v" ] || continue - if [ "$v" = "older" ] || [ "$v" = "older-releases" ]; then + if [ "$v" = "older" ] || [ "$v" = "older-releases" ] || [ "$v" = "milestones" ]; then [ "$INCLUDE_OLDER" = "1" ] || continue - local -a tags=() - mapfile -t tags < <(list_wallet_release_tags "$OLDER_TAG_COUNT" 2>/dev/null || true) - local t - for t in "${tags[@]}"; do + # Prefer 3/6/9/12-month samples (always WARN). Fall back: nothing if no tags nearby. + local found=0 + while read -r t age target; do [ -n "$t" ] || continue - # skip current stable tag (already covered by published / stable-self) [ "$t" = "$STABLE_TAG" ] && continue - vn="${t#wallet-}" - age=$(release_age_days "$t" 2>/dev/null || echo "") - sev=$(severity_from_age_days "$age") - echo " older candidate $t age_days=${age:-?} severity=$sev" | tee -a "$SUMMARY" - # Prefer published APK when downloadable; always also allow self-build token - local meta apk_vc apk_url - meta=$(fdroid_apk_for_version_name "$vn" 2>/dev/null || true) - if [ -n "$meta" ]; then - apk_vc="${meta%%|*}" - apk_url="${meta#*|}" - if fdroid_apk_exists "$apk_url"; then - out+=("older-pub:${vn}:${apk_vc}:${sev}") - else - echo " (no F-Droid APK for $vn at $apk_url — self-build only)" | tee -a "$SUMMARY" - out+=("older-self:${t}:${sev}") - fi - else - out+=("older-self:${t}:${sev}") - fi - done + append_older_warn "$t" "$age" "$target" + found=1 + done < <(select_milestone_tags "$STABLE_TAG" 2>/dev/null || true) + if [ "$found" = "0" ]; then + echo " (no milestone tags within ±${MILESTONE_TOLERANCE_DAYS}d of $MILESTONE_DAYS)" | tee -a "$SUMMARY" + fi else out+=("$v") fi @@ -296,7 +305,7 @@ done echo "" | tee -a "$SUMMARY" echo "=== matrix done (overall=$overall) ===" | tee -a "$SUMMARY" -echo "rule: released app age ≤ ${BLOCKER_MAX_AGE_DAYS}d → BLOCKER on fail; older → WARN" | tee -a "$SUMMARY" +echo "rule: age ≤ ${BLOCKER_MAX_AGE_DAYS}d → BLOCKER; 3/6/9/12mo milestones + older → WARN only" | tee -a "$SUMMARY" for r in "${RESULTS[@]}"; do echo " $r" | tee -a "$SUMMARY" done From cae26e96b22bcec50084dbf129e45c3110241829 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:57:33 +0200 Subject: [PATCH 26/38] android-test: self-build only current stable, not older releases Milestone/older matrix slots use F-Droid APKs when still hosted; skip if missing. Rebuild remains limited to stable-self (and master/fix). --- .../android-test/GUI-AUTOMATION-NOTES.md | 12 ++--- .../taler-monitoring/android-test/README.md | 11 ++--- .../run-android-variant-matrix.sh | 45 ++++++++++--------- 3 files changed, 36 insertions(+), 32 deletions(-) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 0d418be..8326ac9 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -245,15 +245,15 @@ Scripts **today** implement Android only; they should honour: | Variant | Source | Severity | Purpose | |---------|--------|----------|---------| -| **stable-self** | stable tag rebuilt | WARN | Our rebuild — never hard-blocks | +| **stable-self** | **only** rebuild of current stable tag | WARN | Self-build vs published for **one** release | | **published** | F-Droid current APK | **≤14d → BLOCKER**, else WARN | What users install | -| **older / milestones** | tags ~**3 / 6 / 9 / 12 mo** | **always WARN** | Long-horizon regression, if tags exist | -| **master** | `origin/master` | WARN | Tip — never hard-blocks | +| **older / milestones** | F-Droid APK ~**3 / 6 / 9 / 12 mo** | **always WARN** | No self-build of old tags | +| **master** | `origin/master` rebuild | WARN | Tip — never hard-blocks | | **fix** (optional) | inference branch | soft | GOA automation fixes | -**Blocker window:** only releases ≤ 14 days. -**Milestones:** nearest tags to 90/180/270/365 days (±50d) — warn only. -Default matrix: `stable-self,published,older,master`. +**Self-build:** current stable only (plus master/fix if requested) — not older releases. +**Blocker window:** releases ≤ 14 days. +**Milestones:** nearest tags to 90/180/270/365d with a still-hosted F-Droid APK. --- diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index bb66bce..7b2b0e3 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -38,14 +38,15 @@ Runs several smokes so a single odd developer build cannot be the only sample: |---------|------| | Order | Variant | Severity | What | |------|---------|----------|------| -| **1** | **stable-self** | WARN | Rebuild current stable tag | +| **1** | **stable-self** | WARN | **Only** self-build: current stable tag | | **2** | **published** | age ≤14d → **BLOCKER**, else WARN | Current F-Droid APK | -| **3** | **older / milestones** | **always WARN** | Tags nearest **3 / 6 / 9 / 12 months** (if available) | +| **3** | **older / milestones** | **always WARN** | F-Droid APKs nearest **3 / 6 / 9 / 12 months** (if still hosted) — **no self-build** | | **4** | **master** | WARN | Rebuild `origin/master` | -**Milestones:** `MILESTONE_DAYS="90 180 270 365"` (± `MILESTONE_TOLERANCE_DAYS=50`). -Failures there are warnings only — never hard blockers. -**Young releases only** (≤ 14 days): hard blocker on fail. +**Self-build policy:** only **current published stable** (`stable-self`) + optional **master**. +Historical versions: published F-Droid APK only; skip if APK gone. +**Milestones:** `MILESTONE_DAYS="90 180 270 365"` (±50d), warn only. +**Young releases** (≤ 14 days): hard blocker on fail. ```bash ./run-android-variant-matrix.sh diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index 976df43..3a2fe30 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -67,7 +67,8 @@ export SERIAL ANDROID_SERIAL echo "device: $SERIAL" | tee -a "$SUMMARY" # --- expand VARIANTS (token "older" → concrete older-pub / older-self entries) --- -# Append one older release as WARN-only (pub APK if possible, else self-build). +# Append one older release as WARN-only — **F-Droid APK only** (no self-build). +# Returns 0 if queued, 1 if skipped (no published APK). append_older_warn() { local t="$1" age="$2" target="${3:-}" local vn meta apk_vc apk_url @@ -83,11 +84,13 @@ append_older_warn() { apk_url="${meta#*|}" if fdroid_apk_exists "$apk_url"; then out+=("older-pub:${vn}:${apk_vc}:warn") - return + return 0 fi - echo " (no F-Droid APK for $vn — self-build only)" | tee -a "$SUMMARY" + echo " SKIP $t — no F-Droid APK at ${apk_url:-?} (self-build only for current stable)" | tee -a "$SUMMARY" + else + echo " SKIP $t — cannot map F-Droid versionCode (self-build only for current stable)" | tee -a "$SUMMARY" fi - out+=("older-self:${t}:warn") + return 1 } expand_variants() { @@ -101,16 +104,20 @@ expand_variants() { [ -n "$v" ] || continue if [ "$v" = "older" ] || [ "$v" = "older-releases" ] || [ "$v" = "milestones" ]; then [ "$INCLUDE_OLDER" = "1" ] || continue - # Prefer 3/6/9/12-month samples (always WARN). Fall back: nothing if no tags nearby. - local found=0 + # 3/6/9/12-month samples: published APK only, always WARN. + local found=0 queued=0 while read -r t age target; do [ -n "$t" ] || continue [ "$t" = "$STABLE_TAG" ] && continue - append_older_warn "$t" "$age" "$target" found=1 + if append_older_warn "$t" "$age" "$target"; then + queued=$((queued + 1)) + fi done < <(select_milestone_tags "$STABLE_TAG" 2>/dev/null || true) if [ "$found" = "0" ]; then echo " (no milestone tags within ±${MILESTONE_TOLERANCE_DAYS}d of $MILESTONE_DAYS)" | tee -a "$SUMMARY" + elif [ "$queued" = "0" ]; then + echo " (milestone tags found but none still on F-Droid — no older APK smokes)" | tee -a "$SUMMARY" fi else out+=("$v") @@ -278,24 +285,20 @@ for raw in "${VLIST[@]}"; do v="$label" ;; older-self:*) - # older-self:TAG:SEV or older-self:TAG - IFS=':' read -r _ tag sev <<<"$v" - : "${sev:=$(severity_for_tag "$tag")}" - label="older-self-${tag}" - age=$(release_age_days "$tag" 2>/dev/null || echo "?") - echo "older self-build $tag age_days=$age severity=$sev" | tee -a "$SUMMARY" - run_selfbuild "$label" "$tag" "$sev" || rc=$? - v="$label" + # Explicit older-self is disabled by policy (self-build = current stable only). + echo "SKIP $v — self-build only for current stable ($STABLE_TAG); use older-pub or published" | tee -a "$SUMMARY" + RESULTS+=("$v:SKIP(no-selfbuild-older)") + continue ;; *) - # raw git ref — warn unless looks like wallet-X.Y and young + # raw git ref: only master-like / fix via named tokens; bare wallet-* tags → no self-build if [[ "$v" =~ ^wallet-[0-9] ]]; then - sev=$(severity_for_tag "$v") - run_selfbuild "ref-${v}" "$v" "$sev" || rc=$? - else - sev=warn - run_selfbuild "ref-${v//\//-}" "$v" warn || rc=$? + echo "SKIP self-build $v — only current stable self-build ($STABLE_TAG) is enabled" | tee -a "$SUMMARY" + RESULTS+=("$v:SKIP(no-selfbuild-older)") + continue fi + sev=warn + run_selfbuild "ref-${v//\//-}" "$v" warn || rc=$? ;; esac From a7dd6ee824aac53617b8a1cda0df2c104f5e9564 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:57:42 +0200 Subject: [PATCH 27/38] android-test: document self-build-only-for-current-stable policy in header --- .../android-test/run-android-variant-matrix.sh | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh index 3a2fe30..54fe5dc 100755 --- a/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh +++ b/scripts/taler-monitoring/android-test/run-android-variant-matrix.sh @@ -2,15 +2,14 @@ # Run Android wallet smoke against several build / published variants. # # Default order: -# 1) stable-self — rebuild STABLE_TAG (always WARN on fail) -# 2) published — current F-Droid suggested APK -# 3) older / milestones — releases near **3 / 6 / 9 / 12 months** (WARN only) -# 4) master — tip (always WARN on fail) +# 1) stable-self — **only** self-build of current published stable (WARN) +# 2) published — current F-Droid APK +# 3) older — F-Droid APKs ~3/6/9/12 mo if still hosted (WARN, no self-build) +# 4) master — tip self-build (WARN) # -# Severity (released apps): -# Age ≤ BLOCKER_MAX_AGE_DAYS (default **14**) → **BLOCKER** on fail (FAIL_FAST) -# Age > 14 days, incl. 3/6/9/12‑month samples → **WARN** only -# master / stable-self → WARN +# Self-build policy: current stable + master/fix only — never rebuild old releases. +# +# Severity: age ≤14d → BLOCKER; milestones/older/self-builds → WARN # # Env: # STABLE_TAG, STABLE_APK_URL, STABLE_APK_NAME From 2bc94b471bf3e3265faa97239458be1ecf6e4e55 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 21:59:01 +0200 Subject: [PATCH 28/38] android-test: smoke is withdraw then pay, both first-class Structure Abheben before Bezahlen with multi-round UI; SMOKE_STRICT requires wallet-core signals for both legs when enabled. --- .../android-test/GUI-AUTOMATION-NOTES.md | 4 +- .../taler-monitoring/android-test/README.md | 12 +- .../android-test/run-android-gui-smoke.sh | 130 +++++---- .../android-test/run-android-pay-smoke.sh | 254 ++++++++++++------ 4 files changed, 261 insertions(+), 139 deletions(-) diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 8326ac9..3248573 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -14,7 +14,7 @@ GUI is planned via flags (below), not required to use these scripts today. | F-Droid published APK install | **done** | `net.taler.wallet.fdroid` 1.6.1 / 854 | | Source build + same smoke | **done** | `:wallet:assembleFdroidDebug` → `net.taler.wallet.fdroid.debug` | | Default source branch | **done** | `dev/hernani-inference/fix-bank-withdraw-auto-exchange` (minimal GOA exchange auto-add) | -| Deep-link entry (≡ QR) | **done** | `adb` `VIEW` `taler://withdraw/…` / `taler://pay…` | +| Deep-link entry (≡ QR) | **done** | `adb` `VIEW` **withdraw then pay** (`taler://withdraw/…` → `taler://pay…`) | | Explorer API mint (chain) | **done** | `run-goa-gui-chain.sh` (GOA + stage) | | UI taps via uiautomator | **done** | best-effort Confirm/ToS/Pay; ANR → prefer **Wait** | | Screenshots / logcat artifacts | **done** | under `out*`, `out-gui-chain/` (gitignored) | @@ -38,7 +38,7 @@ Host-CLI-Settlement. |-----------|----------------| | APK installieren | F-Droid `net.taler.wallet.fdroid` **oder** Source `assembleFdroidDebug` (`.debug`) | | GOA-Exchange ohne manuelles Add | Source-Build von `fix-bank-withdraw-auto-exchange` (Default in `run-android-build-and-smoke.sh`) | -| Withdraw/Pay **starten** | `adb` deep-link `taler://withdraw/…` / `taler://pay…` (Ersatz für QR-Scan) | +| **Abheben + Bezahlen** | Beide Beine: erst Withdraw-URI + Taps, dann Pay-URI + Taps (`DO_WITHDRAW`/`DO_PAY`) | | Light UI-Drive | Confirm / Accept / Pay-Taps über Label-Suche (`lib_ui.py`, gui-smoke, chain) | | Multi-Runden-Taps | `run-android-gui-smoke.sh` / `run-goa-gui-chain.sh` (best-effort) | | Explorer-Mint (Kette) | `run-goa-gui-chain.sh` + `EXP_PW_FILE` (GOA + stage) | diff --git a/scripts/taler-monitoring/android-test/README.md b/scripts/taler-monitoring/android-test/README.md index 7b2b0e3..d91d82c 100644 --- a/scripts/taler-monitoring/android-test/README.md +++ b/scripts/taler-monitoring/android-test/README.md @@ -1,12 +1,18 @@ -# Android pay smoke +# Android wallet smoke (Abheben + Bezahlen) -## 1) Published F-Droid wallet +Smokes always cover **withdraw first**, then **pay** (not pay-only). + +| Leg | Default | What | +|-----|---------|------| +| **1 WITHDRAW** | on | `taler://withdraw/…` + Confirm/Abheben taps | +| **2 PAY** | on | `taler://pay…` / pay-template + Pay taps | ```bash # needs adb device/emulator ./run-android-pay-smoke.sh STACK=goa ./run-android-pay-smoke.sh -STACK=stage ./run-android-pay-smoke.sh +DO_PAY=0 ./run-android-pay-smoke.sh # withdraw only +DO_WITHDRAW=0 DO_PAY=1 … # pay only (unusual) ``` APK: F-Droid **1.6.1** (`net.taler.wallet.fdroid`). diff --git a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh index f52c783..a22f1b8 100755 --- a/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-gui-smoke.sh @@ -1,12 +1,9 @@ #!/usr/bin/env bash -# GUI-oriented Android smoke (vanilla level): deep-link entry + UI taps. +# GUI-oriented Android smoke: **withdraw (Abheben)** then **pay**, each with +# deep-link entry + multi-round uiautomator taps. Not pay-only. # -# Entry shortcut (legitimate — same as scanning a QR on the landing): -# adb VIEW taler://withdraw/… and taler://pay… -# Then drives the **graphical UI** via uiautomator dump + taps (Confirm / ToS / Pay). -# -# Systems: STACK=goa | stage | auto -# APK: published F-Droid or APK_PATH=… (from-source) +# 1) taler://withdraw/… + Confirm/Abheben/ToS taps +# 2) taler://pay… + Pay/Confirm taps # # See GUI-AUTOMATION-NOTES.md # @@ -85,20 +82,28 @@ echo "apk: $APK_PATH pkg: $PKG" "${ADB[@]}" shell settings put global transition_animation_scale 0 || true "${ADB[@]}" shell settings put global animator_duration_scale 0 || true -WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') -[ -n "$WURI" ] || { echo "no withdraw uri" >&2; exit 4; } -echo "withdraw: $WURI" -echo "$WURI" >"$OUT_DIR/last-withdraw.uri" +: "${DO_WITHDRAW:=1}" +: "${DO_PAY:=1}" +: "${SMOKE_STRICT:=0}" + +WURI="" +PAY_URI="" +WITHDRAW_OK=0 +PAY_OK=0 "${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true "${ADB[@]}" logcat -c 2>/dev/null || true -# --- Shortcut A: deep-link withdraw (≡ QR scan on landing) --- -"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" | tee "$OUT_DIR/start-withdraw.txt" -sleep 5 - -# --- Graphical: drive UI --- -python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +if [ "$DO_WITHDRAW" = "1" ]; then + echo + echo "======== 1 WITHDRAW (Abheben) ========" + WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') + [ -n "$WURI" ] || { echo "no withdraw uri" >&2; exit 4; } + echo "withdraw: $WURI" + echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" | tee "$OUT_DIR/start-withdraw.txt" + sleep 5 + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' import json, sys, importlib.util from pathlib import Path lib_path, serial, out, rounds, sleep_s = ( @@ -112,26 +117,28 @@ st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, print("gui-withdraw taps:", st.get("taps")) print("gui-withdraw anr:", st.get("anr_dismissals")) PY - -# Pay URI -PAY_URI="" -if [ "$STACK" = "stage" ]; then - if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ - "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then - PAY_URI=$(printf '%s' "$RESP" | python3 -c 'import json,sys;d=json.load(sys.stdin);oid=d.get("order_id")or"";tok=d.get("token")or""; -print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok) if oid and tok else "")') - fi else - PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" + echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========" fi -if [ -n "$PAY_URI" ]; then - echo "pay: $PAY_URI" - echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" - # Shortcut B: deep-link pay (≡ scan pay QR / open paywall link) - "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" | tee "$OUT_DIR/start-pay.txt" - sleep 5 - python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' +if [ "$DO_PAY" = "1" ]; then + echo + echo "======== 2 PAY (Bezahlen) ========" + if [ "$STACK" = "stage" ]; then + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c 'import json,sys;d=json.load(sys.stdin);oid=d.get("order_id")or"";tok=d.get("token")or""; +print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok) if oid and tok else "")') + fi + else + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" + fi + if [ -n "$PAY_URI" ]; then + echo "pay: $PAY_URI" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" | tee "$OUT_DIR/start-pay.txt" + sleep 5 + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$GUI_ROUNDS" "$GUI_SLEEP" <<'PY' import json, sys, importlib.util from pathlib import Path lib_path, serial, out, rounds, sleep_s = ( @@ -145,29 +152,50 @@ st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phas print("gui-pay taps:", st.get("taps")) print("gui-pay anr:", st.get("anr_dismissals")) PY + else + echo "pay: (no URI)" + fi +else + echo "======== 2 PAY skipped (DO_PAY=0) ========" fi -"${ADB[@]}" logcat -d -t 400 >"$OUT_DIR/logcat.txt" || true -grep -iE 'taler-wallet|prepareBank|preparePay|confirm|Error|success' "$OUT_DIR/logcat.txt" \ - | tail -100 >"$OUT_DIR/logcat-taler.txt" || true +"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|Error|success|withdraw|pay' \ + "$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true +if [ "$DO_WITHDRAW" = "1" ] && grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + WITHDRAW_OK=1 +fi +if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ] && \ + grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + PAY_OK=1 +fi + +rc=0 echo -echo "=== GUI smoke summary ($STACK) ===" -echo "package: $PKG" -echo "shortcut withdraw URI: delivered" -echo "GUI withdraw: see $OUT_DIR/gui-withdraw-status.json + screenshots" -[ -n "$PAY_URI" ] && echo "shortcut pay URI: delivered" -[ -n "$PAY_URI" ] && echo "GUI pay: see $OUT_DIR/gui-pay-status.json + screenshots" -if grep -q prepareBankIntegratedWithdrawal "$OUT_DIR/logcat.txt" 2>/dev/null; then - echo "wallet-core: prepareBankIntegratedWithdrawal YES" -else - echo "wallet-core: prepareBankIntegratedWithdrawal no (ANR/slow UI?)" +echo "=== GUI smoke summary ($STACK) — withdraw + pay ===" +echo "package: $PKG DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY STRICT=$SMOKE_STRICT" +if [ "$DO_WITHDRAW" = "1" ]; then + echo "withdraw URI: delivered → $OUT_DIR/gui-withdraw-status.json" + echo "withdraw wallet-core: $([ "$WITHDRAW_OK" = 1 ] && echo YES || echo NO)" + if [ "$SMOKE_STRICT" = "1" ] && [ "$WITHDRAW_OK" != "1" ]; then + echo "FAIL: strict withdraw missing"; rc=11 + fi fi -if grep -qiE 'preparePay|confirmPay|pay-template|preparePurchase' "$OUT_DIR/logcat.txt" 2>/dev/null; then - echo "wallet-core: pay path YES" -else - echo "wallet-core: pay path not confirmed in logcat" +if [ "$DO_PAY" = "1" ]; then + echo "pay URI: ${PAY_URI:-none} → $OUT_DIR/gui-pay-status.json" + echo "pay wallet-core: $([ "$PAY_OK" = 1 ] && echo YES || echo NO)" + if [ "$SMOKE_STRICT" = "1" ] && [ "$PAY_OK" != "1" ]; then + echo "FAIL: strict pay missing"; [ "$rc" -eq 0 ] && rc=13 + fi +fi +if grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \ + || grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" "$OUT_DIR/gui-pay" 2>/dev/null; then + echo "FAIL: ANR"; rc=10 fi echo "artifacts: $OUT_DIR/" ls -la "$OUT_DIR" | sed 's/^/ /' echo "Documented shortcuts → GUI-AUTOMATION-NOTES.md" +exit "$rc" diff --git a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh index a5a85b5..48ec49f 100755 --- a/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh +++ b/scripts/taler-monitoring/android-test/run-android-pay-smoke.sh @@ -1,22 +1,20 @@ #!/usr/bin/env bash -# Android wallet smoke: install APK (published F-Droid **or** from-source), open -# withdraw/pay URIs via adb. +# Android wallet smoke: install APK, then **withdraw (Abheben)** and **pay** via +# deep-link + best-effort UI taps. Both legs are first-class (not pay-only). # -# No root required if: -# - adb is available (package android-tools-adb / platform-tools) -# - a device or emulator is connected (`adb devices` shows "device") -# - optional user-local SDK under $HOME/Android/Sdk (emulator) +# Flow (default both on): +# 1) WITHDRAW taler://withdraw/… from bank demo-withdraw.json + confirm taps +# 2) PAY taler://pay… / pay-template + optional taps +# +# Env: +# DO_WITHDRAW=1 (default) DO_PAY=1 (default) +# SMOKE_STRICT=1 → require wallet-core withdraw (+ pay if DO_PAY) + no ANR +# WITHDRAW_GUI_ROUNDS multi-round Abheben taps (default 6) # # Usage: -# ./android-test/run-android-pay-smoke.sh # published F-Droid APK +# ./android-test/run-android-pay-smoke.sh # STACK=goa ./android-test/run-android-pay-smoke.sh -# STACK=stage ./android-test/run-android-pay-smoke.sh -# # from-source (built first): -# ./android-test/run-android-build-and-smoke.sh -# APK_PATH=/path/to.apk PKG=net.taler.wallet.fdroid.debug ./run-android-pay-smoke.sh -# -# Published APK (default): F-Droid Taler Wallet 1.6.1 (854) -# https://f-droid.org/packages/net.taler.wallet.fdroid/ +# DO_PAY=0 ./android-test/run-android-pay-smoke.sh # withdraw-only # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) @@ -98,32 +96,63 @@ fi echo "apk: $APK_PATH ($(wc -c <"$APK_PATH") bytes)" echo "pkg: $PKG" +: "${DO_WITHDRAW:=1}" +: "${DO_PAY:=1}" +: "${WITHDRAW_GUI_ROUNDS:=6}" +: "${PAY_GUI_ROUNDS:=4}" +: "${SMOKE_STRICT:=0}" + "${ADB[@]}" wait-for-device "${ADB[@]}" install -r "$APK_PATH" "${ADB[@]}" shell settings put global window_animation_scale 0 || true "${ADB[@]}" shell settings put global transition_animation_scale 0 || true "${ADB[@]}" shell settings put global animator_duration_scale 0 || true -# Withdraw URI from live mint -WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') -if [ -z "$WURI" ]; then - echo "FAIL: no taler_withdraw_uri from $WITHDRAW_JSON" >&2 - exit 4 -fi -echo "withdraw: $WURI" -echo "$WURI" >"$OUT_DIR/last-withdraw.uri" - "${ADB[@]}" shell am force-stop "$PKG" 2>/dev/null || true "${ADB[@]}" logcat -c 2>/dev/null || true -"${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" 2>&1 | tee "$OUT_DIR/start-withdraw.txt" -sleep 12 -"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/01-after-withdraw-uri.png" || true -"${ADB[@]}" shell uiautomator dump /sdcard/ui.xml 2>/dev/null || true -"${ADB[@]}" pull /sdcard/ui.xml "$OUT_DIR/01-ui.xml" 2>/dev/null || true +WURI="" +PAY_URI="" +WITHDRAW_OK=0 +PAY_OK=0 -# Best-effort: tap common confirm labels -python3 - "$SERIAL" "$OUT_DIR/01-ui.xml" <<'PY' || true +# ═══════════════════════════════════════════════════════════════ +# 1) WITHDRAW / Abheben (required by default — not pay-only) +# ═══════════════════════════════════════════════════════════════ +if [ "$DO_WITHDRAW" = "1" ]; then + echo + echo "======== 1 WITHDRAW (Abheben) ========" + WURI=$(curl -sfS -m 25 "$WITHDRAW_JSON" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("taler_withdraw_uri") or d.get("qr_payload") or "")') + if [ -z "$WURI" ]; then + echo "FAIL: no taler_withdraw_uri from $WITHDRAW_JSON" >&2 + exit 4 + fi + echo "withdraw URI: $WURI" + echo "$WURI" >"$OUT_DIR/last-withdraw.uri" + + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$WURI" 2>&1 | tee "$OUT_DIR/start-withdraw.txt" + sleep 6 + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/01-after-withdraw-uri.png" || true + "${ADB[@]}" shell uiautomator dump /sdcard/ui.xml 2>/dev/null || true + "${ADB[@]}" pull /sdcard/ui.xml "$OUT_DIR/01-ui.xml" 2>/dev/null || true + + # Multi-round confirm / Abheben / ToS (same idea as GUI smoke) + if [ -f "$ROOT/lib_ui.py" ]; then + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$WITHDRAW_GUI_ROUNDS" 2 <<'PY' || true +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-withdraw", rounds=rounds, sleep_s=sleep_s, phase="wd") +(out / "gui-withdraw-status.json").write_text(json.dumps(st, indent=2)) +print("withdraw GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals")) +PY + else + python3 - "$SERIAL" "$OUT_DIR/01-ui.xml" <<'PY' || true import re, subprocess, sys serial, path = sys.argv[1], sys.argv[2] try: @@ -131,31 +160,46 @@ try: except FileNotFoundError: raise SystemExit(0) labels = [ - "Confirm", "Withdraw", "Accept", "Continue", "OK", - "I accept", "Agree", "Bestätigen", "Abheben", "Akzeptieren", + "Confirm", "Confirm withdrawal", "Withdraw", "Accept", "Continue", "OK", + "I accept", "Agree", "Bestätigen", "Abheben", "Akzeptieren", "Retirer", ] for label in labels: for pat in ( r'text="%s"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"' % re.escape(label), r'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"[^>]*text="%s"' % re.escape(label), ): - for m in re.finditer(pat, xml): + for m in re.finditer(pat, xml, flags=re.I): x1, y1, x2, y2 = map(int, m.groups()) x, y = (x1 + x2) // 2, (y1 + y2) // 2 print(f"tap {label} @ {x},{y}") subprocess.run(["adb", "-s", serial, "shell", "input", "tap", str(x), str(y)], check=False) PY -sleep 8 -"${ADB[@]}" exec-out screencap -p >"$OUT_DIR/02-after-confirm-tap.png" || true + sleep 8 + fi + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/02-after-withdraw-taps.png" || true + # Mid-flow logcat: withdraw must register before pay + "${ADB[@]}" logcat -d -t 400 >"$OUT_DIR/logcat-after-withdraw.txt" || true + if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat-after-withdraw.txt" 2>/dev/null; then + WITHDRAW_OK=1 + echo "withdraw: wallet-core activity YES" + else + echo "withdraw: wallet-core activity NOT seen yet (onboarding/ANR/slow?)" + fi +else + echo "======== 1 WITHDRAW skipped (DO_WITHDRAW=0) ========" +fi -# Optional public pay template (stage fixed product / GOA shop if amount known) -# Stage: open monnaies shop is browser-side; deep link pay after template POST if available. -PAY_URI="" -if [ "$STACK" = "stage" ]; then - # fixed template panier-legumes → taler://pay/… - if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ - "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then - PAY_URI=$(printf '%s' "$RESP" | python3 -c ' +# ═══════════════════════════════════════════════════════════════ +# 2) PAY (after withdraw; still first-class, but second leg) +# ═══════════════════════════════════════════════════════════════ +if [ "$DO_PAY" = "1" ]; then + echo + echo "======== 2 PAY (Bezahlen) ========" + if [ "$STACK" = "stage" ]; then + if RESP=$(curl -sfS -m 20 -X POST -H 'Content-Type: application/json' -d '{}' \ + "$MERCHANT/instances/fermes-des-collines/templates/panier-legumes" 2>/dev/null); then + PAY_URI=$(printf '%s' "$RESP" | python3 -c ' import json,sys d=json.load(sys.stdin) oid=d.get("order_id") or "" @@ -163,58 +207,102 @@ tok=d.get("token") or "" if oid and tok: print("taler://pay/stage.monnaie.lefrancpaysan.ch/instances/fermes-des-collines/%s/?c=%s"%(oid,tok)) ' 2>/dev/null || true) + fi + elif [ "$STACK" = "goa" ]; then + PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" fi -elif [ "$STACK" = "goa" ]; then - # Paivana pay-template deep link (wallet prepares order) - PAY_URI="taler://pay-template/taler.hacktivism.ch/instances/goa-shop/paivana" -fi -if [ -n "$PAY_URI" ]; then - echo "pay: $PAY_URI ($PAY_HINT)" - echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" - "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" 2>&1 | tee "$OUT_DIR/start-pay.txt" - sleep 12 - "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/03-after-pay-uri.png" || true + if [ -n "$PAY_URI" ]; then + echo "pay URI: $PAY_URI ($PAY_HINT)" + echo "$PAY_URI" >"$OUT_DIR/last-pay.uri" + "${ADB[@]}" shell am start -a android.intent.action.VIEW -d "$PAY_URI" 2>&1 | tee "$OUT_DIR/start-pay.txt" + sleep 6 + if [ -f "$ROOT/lib_ui.py" ]; then + python3 - "$ROOT/lib_ui.py" "$SERIAL" "$OUT_DIR" "$PAY_GUI_ROUNDS" 2 <<'PY' || true +import json, sys, importlib.util +from pathlib import Path +lib_path, serial, out, rounds, sleep_s = ( + Path(sys.argv[1]), sys.argv[2], Path(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]) +) +spec = importlib.util.spec_from_file_location("lib_ui", lib_path) +lib = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lib) +st = lib.gui_drive(serial, out / "gui-pay", rounds=rounds, sleep_s=sleep_s, phase="pay") +(out / "gui-pay-status.json").write_text(json.dumps(st, indent=2)) +print("pay GUI taps:", st.get("taps"), "anr:", st.get("anr_dismissals")) +PY + fi + "${ADB[@]}" exec-out screencap -p >"$OUT_DIR/03-after-pay-uri.png" || true + else + echo "pay: (no public template URI for stack=$STACK)" + fi else - echo "pay: (skipped — no public template order)" + echo "======== 2 PAY skipped (DO_PAY=0) ========" fi -# Evidence from logcat -"${ADB[@]}" logcat -d -t 300 >"$OUT_DIR/logcat.txt" || true -grep -iE 'taler-wallet|prepareBank|preparePay|confirm|error|Error|success' "$OUT_DIR/logcat.txt" \ - | tail -80 >"$OUT_DIR/logcat-taler.txt" || true +# Final evidence +"${ADB[@]}" logcat -d -t 500 >"$OUT_DIR/logcat.txt" || true +grep -iE 'taler-wallet|prepareBank|acceptWithdrawal|confirmWithdrawal|preparePay|confirmPay|error|Error|success|withdraw|pay' \ + "$OUT_DIR/logcat.txt" | tail -100 >"$OUT_DIR/logcat-taler.txt" || true + +# Re-evaluate withdraw/pay from full logcat +if [ "$DO_WITHDRAW" = "1" ]; then + if grep -qE 'prepareBankIntegratedWithdrawal|acceptWithdrawal|confirmWithdrawal' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + WITHDRAW_OK=1 + fi +fi +if [ "$DO_PAY" = "1" ] && [ -n "$PAY_URI" ]; then + if grep -qiE 'preparePay|confirmPay|preparePurchase|checkPay|pay-template|PayForTemplate' \ + "$OUT_DIR/logcat.txt" 2>/dev/null; then + PAY_OK=1 + fi +fi -# SMOKE_STRICT=1 (matrix gate default): ANR or missing wallet-core withdraw = hard fail -: "${SMOKE_STRICT:=0}" rc=0 echo -echo "=== summary ===" -echo "package installed: $PKG" -echo "withdraw intent delivered: $WURI" -if grep -qiE "isn't responding|isn.t responding|System UI isn" "$OUT_DIR/01-ui.xml" 2>/dev/null \ - || grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null; then - echo "FAIL: System UI / app ANR (host too small or emulator hung)" - rc=10 -fi -if grep -q 'prepareBankIntegratedWithdrawal' "$OUT_DIR/logcat.txt" 2>/dev/null; then - echo "wallet-core: prepareBankIntegratedWithdrawal seen (wallet accepted withdraw URI)" -else - echo "wallet-core: prepareBankIntegratedWithdrawal NOT seen (UI may still show onboarding/ANR on small RAM)" - if [ "$SMOKE_STRICT" = "1" ]; then - echo "FAIL: SMOKE_STRICT=1 requires prepareBankIntegratedWithdrawal in logcat" - rc=11 - fi -fi -if [ -n "$PAY_URI" ]; then - if grep -qiE 'preparePay|confirmPay|pay-template|preparePurchase' "$OUT_DIR/logcat.txt" 2>/dev/null; then - echo "wallet-core: pay path activity seen" +echo "=== summary (withdraw + pay) ===" +echo "package: $PKG stack: $STACK" +echo "DO_WITHDRAW=$DO_WITHDRAW DO_PAY=$DO_PAY SMOKE_STRICT=$SMOKE_STRICT" +if [ "$DO_WITHDRAW" = "1" ]; then + echo "withdraw URI: ${WURI:-(none)}" + if [ "$WITHDRAW_OK" = "1" ]; then + echo "withdraw wallet-core: YES" else - echo "wallet-core: pay path not confirmed in logcat (check $OUT_DIR/03-after-pay-uri.png)" + echo "withdraw wallet-core: NO" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires withdraw (prepareBank/accept/confirm) in logcat" + rc=11 + fi fi fi +if [ "$DO_PAY" = "1" ]; then + echo "pay URI: ${PAY_URI:-(none)}" + if [ -z "$PAY_URI" ]; then + echo "pay: no URI (template/API)" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires a pay URI when DO_PAY=1" + rc=12 + fi + elif [ "$PAY_OK" = "1" ]; then + echo "pay wallet-core: YES" + else + echo "pay wallet-core: NO (intent may still have been delivered)" + if [ "$SMOKE_STRICT" = "1" ]; then + echo "FAIL: SMOKE_STRICT requires pay path activity in logcat" + [ "$rc" -eq 0 ] && rc=13 + fi + fi +fi +if grep -qiE "isn't responding|isn.t responding|System UI isn" "$OUT_DIR/01-ui.xml" 2>/dev/null \ + || grep -qiE "isn't responding|not responding" "$OUT_DIR/logcat.txt" 2>/dev/null \ + || grep -rqiE "isn't responding" "$OUT_DIR/gui-withdraw" 2>/dev/null; then + echo "FAIL: System UI / app ANR" + rc=10 +fi echo "artifacts: $OUT_DIR/" ls -la "$OUT_DIR" | sed 's/^/ /' echo -echo "NOTE: Full unattended UI payment needs a responsive device/emulator (≥6–8 GiB host RAM recommended)." -echo "This smoke proves: APK install + deep-link delivery on stack=$STACK (strict=$SMOKE_STRICT rc=$rc)." +echo "NOTE: Full unattended Abheben+Bezahlen needs responsive device (≥6–8 GiB host RAM)." +echo "Smoke legs: withdraw=$( [ "$DO_WITHDRAW" = 1 ] && echo on || echo off) pay=$( [ "$DO_PAY" = 1 ] && echo on || echo off) rc=$rc" exit "$rc" From b9c6fb81a6c8e06723473bc915eec6ac2cb59391 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 22:04:39 +0200 Subject: [PATCH 29/38] docs(android-test): dual-stack GOA/stage run results (ANR fail) Record time-bounded F-Droid smoke on both stacks: landings/URIs OK, unattended withdraw+pay blocked by System UI ANR on low-RAM emulator. --- .../taler-monitoring/android-test/.gitignore | 1 + .../android-test/GUI-AUTOMATION-NOTES.md | 46 +++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index 6d4e62e..fc1fb44 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -7,3 +7,4 @@ out-gui/ out-source-gui/ out-gui-chain/ out-matrix-*/ +out-stack-run-*/ diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index 3248573..b30472b 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -264,6 +264,51 @@ Scripts **today** implement Android only; they should honour: | **GOA** | explorer mint / `demo-withdraw.json` @ bank.hacktivism.ch | goa-shop templates / Paivana pay-template | | **stage** | explorer / demo-withdraw @ stage.bank… | fermes / jardin public templates | +### Dual-stack run result (2026-07-17 · headless AVD) + +**What ran:** published F-Droid wallet **1.6.1** (`net.taler.wallet.fdroid`), +`run-android-pay-smoke.sh` with **Abheben + Bezahlen**, `SMOKE_STRICT=1`, +per-stack wall **480 s** (no full matrix / no Gradle — time-bounded). +Device: `emulator-5554` (TalerWallet34, headless, SwiftShader). +Host RAM ~**3.7 GiB** (~150 MiB free during run) — under recommended ≥6–8 GiB. + +| Stack | Bank / pay entry | Exit | Elapsed | Withdraw URI | Pay URI | wallet-core withdraw | wallet-core pay | ANR | Overall | +|-------|------------------|------|---------|--------------|---------|----------------------|-----------------|-----|---------| +| **goa** | bank.hacktivism.ch · paivana pay-template | **10** | ~106 s | delivered | delivered | **NO** | **NO** | **YES** (System UI) | **FAIL** | +| **stage** (*lefrancpaysan*) | stage.bank… · fermes `panier-legumes` order | **10** | ~176 s | delivered | delivered | **NO** | **NO** | **YES** (Android System) | **FAIL** | + +Neither stack hit the 480 s timeout; both finished under 3 min with hard fail. + +**What worked (infra / landings):** + +- adb install F-Droid APK +- GOA + stage **demo-withdraw.json** returned usable `taler://withdraw/…` +- GOA **pay-template** paivana URI built +- stage merchant **template POST** → `taler://pay/…/fermes-des-collines/…` +- Intent delivery (`am start VIEW`) for withdraw and pay +- Artifacts under `out-stack-run-20260717-215934/{goa,stage}/` (gitignored) + +**What failed (app/GUI on this host):** + +- **System UI / Android System ANR** → only Close/Wait; no Confirm/Abheben/Pay taps +- **wallet-core** never showed `prepareBankIntegratedWithdrawal` / pay prepare in logcat +- Chooser noise: **Taler Wallet** + **Taler Wallet Debug** both installed (“Open with”) +- Unattended **Abheben + Bezahlen** not green on this emulator/RAM + +**Interpretation:** + +| Layer | GOA | stage | +|-------|-----|-------| +| HTTP / bank / merchant reachability for smoke inputs | OK | OK | +| Deep-link plumbing | OK | OK | +| Unattended GUI Confirm → settled | **blocked (ANR)** | **blocked (ANR)** | +| Strict smoke (`SMOKE_STRICT=1`) | fail rc=10 | fail rc=10 | + +**Not a stack-specific regression** between GOA and stage in this run — same failure mode (host/emulator). +Settlement proof remains **host CLI e2e/ladder**, not this Android GUI path, until a device with enough RAM is used. + +**Retry when:** physical phone or host with more free RAM; ideally uninstall one of fdroid / fdroid.debug to avoid “Open with”; optional `WINDOWED=0` stays default (headless). + --- ## Upstream branches (`taler-android` `dev/hernani-inference/*`) @@ -282,6 +327,7 @@ Scripts **today** implement Android only; they should honour: - [x] Minimal variant documented - [x] Minimalvariante: was geht / was nicht (Beschränkungen) - [x] Two stacks (goa / stage) +- [x] Dual-stack run documented (2026-07-17: both FAIL ANR, landings OK) - [x] Flags design (`AUTO_ANDROID` / `AUTO_GUI` / platform defaults) - [x] **Headless emulator default** (`EMULATOR_HEADLESS=1`, SwiftShader) - [x] Linux default = Android suite only From a2b9908c79e0e94bcf7ba754715157190885481b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 22:18:19 +0200 Subject: [PATCH 30/38] docs(android-test): as-is survey GOA+stage stable/master/inference Record dual-stack overview: landings OK, all app variants ANR-blocked on low-RAM emulator; older F-Droid APKs missing; local master+inference branch built but not greener (not pushed). --- .../taler-monitoring/android-test/.gitignore | 1 + .../android-test/GUI-AUTOMATION-NOTES.md | 92 ++++++++++++------- 2 files changed, 60 insertions(+), 33 deletions(-) diff --git a/scripts/taler-monitoring/android-test/.gitignore b/scripts/taler-monitoring/android-test/.gitignore index fc1fb44..49b08e5 100644 --- a/scripts/taler-monitoring/android-test/.gitignore +++ b/scripts/taler-monitoring/android-test/.gitignore @@ -8,3 +8,4 @@ out-source-gui/ out-gui-chain/ out-matrix-*/ out-stack-run-*/ +out-survey-*/ diff --git a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md index b30472b..18fe398 100644 --- a/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md +++ b/scripts/taler-monitoring/android-test/GUI-AUTOMATION-NOTES.md @@ -264,50 +264,76 @@ Scripts **today** implement Android only; they should honour: | **GOA** | explorer mint / `demo-withdraw.json` @ bank.hacktivism.ch | goa-shop templates / Paivana pay-template | | **stage** | explorer / demo-withdraw @ stage.bank… | fermes / jardin public templates | -### Dual-stack run result (2026-07-17 · headless AVD) +### Lageübersicht GOA + stage (as-is survey · 2026-07-17 ~22:05–22:18) -**What ran:** published F-Droid wallet **1.6.1** (`net.taler.wallet.fdroid`), -`run-android-pay-smoke.sh` with **Abheben + Bezahlen**, `SMOKE_STRICT=1`, -per-stack wall **480 s** (no full matrix / no Gradle — time-bounded). -Device: `emulator-5554` (TalerWallet34, headless, SwiftShader). -Host RAM ~**3.7 GiB** (~150 MiB free during run) — under recommended ≥6–8 GiB. +**Device:** headless `emulator-5554` · host ~3.7 GiB RAM (often <300 MiB free) · timeout per leg ≤420 s. +**Legs:** Abheben + Bezahlen (`DO_WITHDRAW=1 DO_PAY=1`). Artefakte: `out-survey-20260717-220527/` (gitignored). -| Stack | Bank / pay entry | Exit | Elapsed | Withdraw URI | Pay URI | wallet-core withdraw | wallet-core pay | ANR | Overall | -|-------|------------------|------|---------|--------------|---------|----------------------|-----------------|-----|---------| -| **goa** | bank.hacktivism.ch · paivana pay-template | **10** | ~106 s | delivered | delivered | **NO** | **NO** | **YES** (System UI) | **FAIL** | -| **stage** (*lefrancpaysan*) | stage.bank… · fermes `panier-legumes` order | **10** | ~176 s | delivered | delivered | **NO** | **NO** | **YES** (Android System) | **FAIL** | +#### Varianten -Neither stack hit the 480 s timeout; both finished under 3 min with hard fail. +| Variante | Quelle | Bemerkung | +|----------|--------|-----------| +| **stable (published)** | F-Droid `net.taler.wallet.fdroid` **1.6.1 / 854** | as-is | +| **master as-is** | rebuild `origin/master` @ `f65a976c` → `.fdroid.debug` | as-is, no patches | +| **ältere Releases** (3/6/9/12 mon) | F-Droid APKs für `1.5.0`/`1.3.0`/`1.1.0`/`1.0.8` | **nicht auffindbar** (HTTP miss) → **SKIP** (kein Selbstbau älterer Tags) | +| **master + inference** (extra) | local branch `local/master-with-inference` @ `3ebe8b09` | merges `fix-bank-withdraw-auto-exchange` + `fix-withdraw-spinner-fallback` · **not pushed** | -**What worked (infra / landings):** +#### Ergebnis-Matrix -- adb install F-Droid APK -- GOA + stage **demo-withdraw.json** returned usable `taler://withdraw/…` -- GOA **pay-template** paivana URI built -- stage merchant **template POST** → `taler://pay/…/fermes-des-collines/…` -- Intent delivery (`am start VIEW`) for withdraw and pay -- Artifacts under `out-stack-run-20260717-215934/{goa,stage}/` (gitignored) +| Variant | Stack | Install / deep-link | Withdraw URI | Pay URI | wallet-core Abheben | wallet-core Pay | ANR | Exit | +|---------|-------|---------------------|--------------|---------|---------------------|-----------------|-----|------| +| stable published | **goa** (hacktivism) | OK | OK bank.hacktivism.ch | OK paivana template | **NO** | soft/YES once* | **YES** | **10** | +| stable published | **stage** (*.lefrancpaysan) | OK | OK stage.bank… | OK fermes order | **NO** | **NO** | **YES** | **10** | +| master as-is | **goa** | OK | OK | OK paivana | **NO** | **NO** | **YES** | **10** | +| master as-is | **stage** | OK | OK | OK fermes | **NO** | **NO** | **YES** | **10** | +| master+inference | **goa** | OK | OK | OK | **NO** | **NO** | **YES** | **10** | +| master+inference | **stage** | OK | OK | OK | **NO** | **NO** | **YES** | **10** | -**What failed (app/GUI on this host):** +\*stable-goa console once reported pay wallet-core YES mid-run; final score still ANR-dominated / not reliable settlement. -- **System UI / Android System ANR** → only Close/Wait; no Confirm/Abheben/Pay taps -- **wallet-core** never showed `prepareBankIntegratedWithdrawal` / pay prepare in logcat -- Chooser noise: **Taler Wallet** + **Taler Wallet Debug** both installed (“Open with”) -- Unattended **Abheben + Bezahlen** not green on this emulator/RAM +#### Was pro Stack **steht** (Infra) -**Interpretation:** +| Check | **goa / hacktivism** | **stage / \*.lefrancpaysan** | +|-------|----------------------|------------------------------| +| Bank `…/intro/demo-withdraw.json` → `taler://withdraw/…` | **OK** | **OK** | +| Pay entry | paivana `pay-template` **OK** | fermes `panier-legumes` POST → `taler://pay/…` **OK** | +| adb `VIEW` Intent delivery | **OK** | **OK** | +| Unattended Confirm/Abheben/Pay GUI | **FAIL (ANR)** | **FAIL (ANR)** | +| wallet-core prepare/accept withdraw in logcat | **not seen** | **not seen** | -| Layer | GOA | stage | -|-------|-----|-------| -| HTTP / bank / merchant reachability for smoke inputs | OK | OK | -| Deep-link plumbing | OK | OK | -| Unattended GUI Confirm → settled | **blocked (ANR)** | **blocked (ANR)** | -| Strict smoke (`SMOKE_STRICT=1`) | fail rc=10 | fail rc=10 | +**Fazit as-is:** Beide Stacks liefern die **Eingänge** (Withdraw- + Pay-URIs). Die **GUI/wallet-core-Strecke** ist auf diesem Emulator für **alle** getesteten App-Varianten (stable, master, inference) **gleich tot** — System UI / Android System ANR, keine brauchbaren Taps. Das ist **kein Stack-Unterschied** GOA vs stage und **kein** belastbarer App-Versions-Vergleich. -**Not a stack-specific regression** between GOA and stage in this run — same failure mode (host/emulator). -Settlement proof remains **host CLI e2e/ladder**, not this Android GUI path, until a device with enough RAM is used. +#### Inference-Patch-Versuch (lokal, **kein Push**) -**Retry when:** physical phone or host with more free RAM; ideally uninstall one of fdroid / fdroid.debug to avoid “Open with”; optional `WINDOWED=0` stays default (headless). +Weil **nichts** der as-is-Varianten wallet-core-grün war, wurde zusätzlich gebaut: + +```text +taler-android branch: local/master-with-inference # NOT pushed +base: origin/master +merged: origin/dev/hernani-inference/fix-bank-withdraw-auto-exchange + origin/dev/hernani-inference/fix-withdraw-spinner-fallback +commit: 3ebe8b091379 +APK: apks/wallet-master-inference-3ebe8b091379.apk (local) +``` + +| Stack | Inference smoke | Besser als master/stable? | +|-------|-----------------|---------------------------| +| goa | exit 10, ANR, no withdraw core | **Nein** | +| stage | exit 10, ANR, no withdraw core | **Nein** | + +**Kein grünes Inference-Signal** auf diesem Host → wir können **nicht** belegen, dass die Patches Withdraw/Pay „retten“. +Ebenso können wir **nicht** belegen, dass master ohne Patches „schuld“ ist: die Umgebung erreicht wallet-core gar nicht. + +> **Wenn später auf Gerät/mehr RAM die Inference-APK plötzlich grün und master/stable rot ist:** +> das ist der **klare Fix-Hinweis** — Patches aus `dev/hernani-inference/*` upstreamen / in master holen. +> **Heute:** noch **nicht** der Fall; primärer Blocker = **Emulator/RAM/ANR**. + +#### Nächste sinnvolle Schritte (später) + +1. Physisches Gerät oder Host ≥6–8 GiB free RAM +2. Nur **eine** Wallet-Package-ID installiert (kein Open-with) +3. Survey wiederholen: stable · master · `local/master-with-inference` auf **goa + stage** +4. Ältere F-Droid-APKs nur wenn wieder gehostet; kein Selbstbau alter Tags --- From 6fb8dc6d43005ee9b6d125adb47494b691e5f516 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 22:36:54 +0200 Subject: [PATCH 31/38] docs/ops: merchant portal instance ids must be lowercase (avoid 401) Document that mixed-case instance ids yield odd HTTP 401 in webui/API. Lowercase MERCHANT_INSTANCE / shop paths defensively in monitoring and landings. --- 2026/2026-07-17.md | 2 ++ configs/bank-landing/shop-pay.js | 2 +- configs/merchant-landing/shop-pay.js | 5 +++-- configs/taler-hacktivism/README.md | 17 +++++++++++++++++ .../demo-instance-goa-demo-cp4zqk.md | 4 ++++ scripts/taler-merchant/README.md | 7 +++++++ scripts/taler-merchant/setup_credit_facade.sh | 3 ++- .../android-test/run-goa-gui-chain.sh | 8 ++++---- scripts/taler-monitoring/lib.sh | 4 ++++ 9 files changed, 44 insertions(+), 8 deletions(-) diff --git a/2026/2026-07-17.md b/2026/2026-07-17.md index 33ee314..4423c96 100644 --- a/2026/2026-07-17.md +++ b/2026/2026-07-17.md @@ -8,3 +8,5 @@ 2. Start vor Merchant-Apps → 502 3. Pasta AAAA/timeout → pin `172.17.0.1` Live-Stack war gesund (302/402); Ursache/Fix nicht wiederholen. +- **Merchant portal:** instance ids **lowercase only** — capitals → odd **401**; + documented in taler-hacktivism + stagepaysan config READMEs; monitoring lowercases `MERCHANT_INSTANCE`. diff --git a/configs/bank-landing/shop-pay.js b/configs/bank-landing/shop-pay.js index 2c0573b..5ebc439 100644 --- a/configs/bank-landing/shop-pay.js +++ b/configs/bank-landing/shop-pay.js @@ -10,7 +10,7 @@ (function () { var MERCHANT_HOST = "taler.hacktivism.ch"; /* goa-shop: dedicated instance with fixed-order product templates */ - var INSTANCE = "goa-shop"; + var INSTANCE = String("goa-shop").toLowerCase(); var SHOP_PAYTO = "payto://x-taler-bank/bank.hacktivism.ch/goa-shop?receiver-name=GOA%20Shop"; var SHOP_PAYTO_HTTPS = "https://bank.hacktivism.ch/webui/"; diff --git a/configs/merchant-landing/shop-pay.js b/configs/merchant-landing/shop-pay.js index 2c0573b..af62a30 100644 --- a/configs/merchant-landing/shop-pay.js +++ b/configs/merchant-landing/shop-pay.js @@ -9,8 +9,9 @@ */ (function () { var MERCHANT_HOST = "taler.hacktivism.ch"; - /* goa-shop: dedicated instance with fixed-order product templates */ - var INSTANCE = "goa-shop"; + /* goa-shop: dedicated instance with fixed-order product templates. + * Instance id must stay lowercase (portal/API: capitals → odd 401s). */ + var INSTANCE = String("goa-shop").toLowerCase(); var SHOP_PAYTO = "payto://x-taler-bank/bank.hacktivism.ch/goa-shop?receiver-name=GOA%20Shop"; var SHOP_PAYTO_HTTPS = "https://bank.hacktivism.ch/webui/"; diff --git a/configs/taler-hacktivism/README.md b/configs/taler-hacktivism/README.md index 127cd32..06a110e 100644 --- a/configs/taler-hacktivism/README.md +++ b/configs/taler-hacktivism/README.md @@ -56,6 +56,23 @@ After conf/hosts changes: `taler-merchant-exchangekeyupdate -t` and `taler-merch | 9010 | nginx TLS in container (podman) | | unix sock | `taler-merchant-httpd` | +## Merchant portal (SPA `/webui/`) — instance IDs **lowercase only** + +The merchant **webui** (and private API paths under `/instances/{id}/…`) must use +**only lowercase** instance ids (`a-z`, digits, hyphens). + +| Do | Do not | +|----|--------| +| `goa-shop`, `goa-demo-cp4zqk`, `fermes-des-collines` | `Goa-Shop`, `GOA-DEMO`, mixed case | + +**Why:** uppercase (or mixed-case) ids produce **odd HTTP 401** failures on +login / private API even when the password/token is correct. Auth looks like +“wrong secret” but the real issue is path / instance string case. + +Create instances, login fields, `Authorization` targets, and `taler://pay…` +paths with the **exact lowercase** id. Display names may use capitals; +**ids must not**. + ## Demo instance (2026-07-09) Self-provisioned shop **`goa-demo-cp4zqk`**, bank account same username, payto linked, seed **GOA:5000**. diff --git a/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md b/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md index 95aa009..109c0bb 100644 --- a/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md +++ b/configs/taler-hacktivism/demo-instance-goa-demo-cp4zqk.md @@ -15,6 +15,10 @@ Created **2026-07-09** via self-provisioning (`POST /instances`, no SMS TAN). Login: SPA → instance id + password (or bearer `secret-token:$PASSWORD`). +**Instance id must be all-lowercase** (`goa-demo-cp4zqk`). Typing capitals in +the portal login field (or calling `/instances/Goa-Demo-…/private/…`) yields +**strange 401s** even with the right password — use exact lowercase only. + ## Bank account (linked) | Field | Value | diff --git a/scripts/taler-merchant/README.md b/scripts/taler-merchant/README.md index 339d693..4ddea4a 100644 --- a/scripts/taler-merchant/README.md +++ b/scripts/taler-merchant/README.md @@ -30,6 +30,13 @@ Automatic import needs: SMS backends are symlinks into `/var/taler-src/...` (not copied). +## Portal / instance IDs: **lowercase only** + +Merchant SPA (`/webui/`) and `/instances/{id}/…` require **lowercase** instance +ids. Uppercase or mixed case → **weird HTTP 401** (looks like bad password). +Display names may be mixed case; **ids must be lowercase** (`goa-shop`, not +`Goa-Shop`). See `configs/taler-hacktivism/README.md`. + ## Usage ```bash diff --git a/scripts/taler-merchant/setup_credit_facade.sh b/scripts/taler-merchant/setup_credit_facade.sh index 3ab0403..bf5ffb4 100755 --- a/scripts/taler-merchant/setup_credit_facade.sh +++ b/scripts/taler-merchant/setup_credit_facade.sh @@ -16,7 +16,8 @@ # - restarts taler-merchant-wirewatch (once, after facade is set) set -euo pipefail -INST="${MERCHANT_INSTANCE:-goa-demo-cp4zqk}" +# Instance id must be lowercase (portal/API: capitals → odd 401s). +INST=$(printf '%s' "${MERCHANT_INSTANCE:-goa-demo-cp4zqk}" | tr '[:upper:]' '[:lower:]') BANK_USER="${BANK_USER:-$INST}" MERCHANT_PW_FILE="${MERCHANT_PW_FILE:-/root/merchant-${INST}-password.txt}" BANK_PW_FILE="${BANK_PW_FILE:-/root/bank-${BANK_USER}-password.txt}" diff --git a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh index 82c002d..1c20a63 100755 --- a/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh +++ b/scripts/taler-monitoring/android-test/run-goa-gui-chain.sh @@ -52,8 +52,8 @@ case "$STACK" in PKG="${PKG:-net.taler.wallet.fdroid.debug}" BANK="${BANK:-https://bank.hacktivism.ch}" MERCHANT="${MERCHANT:-https://taler.hacktivism.ch}" - INSTANCE="${INSTANCE:-goa-shop}" - DONATE_INSTANCE="${DONATE_INSTANCE:-goa-demo-cp4zqk}" + INSTANCE=$(printf '%s' "${INSTANCE:-goa-shop}" | tr '[:upper:]' '[:lower:]') + DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-goa-demo-cp4zqk}" | tr '[:upper:]' '[:lower:]') DONATE_TEMPLATE="${DONATE_TEMPLATE:-goa-free}" DEFAULT_AMOUNTS=("GOA:10" "GOA:20") DEFAULT_PAYS=("product:orbit-sticker" "product:nebula-coffee" "product:voidwave-playlist" "donate:GOA:12") @@ -63,8 +63,8 @@ case "$STACK" in PKG="${PKG:-net.taler.wallet.fdroid.debug}" BANK="${BANK:-https://stage.bank.lefrancpaysan.ch}" MERCHANT="${MERCHANT:-https://stage.monnaie.lefrancpaysan.ch}" - INSTANCE="${INSTANCE:-fermes-des-collines}" - DONATE_INSTANCE="${DONATE_INSTANCE:-fermes-des-collines}" + INSTANCE=$(printf '%s' "${INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]') + DONATE_INSTANCE=$(printf '%s' "${DONATE_INSTANCE:-fermes-des-collines}" | tr '[:upper:]' '[:lower:]') DONATE_TEMPLATE="${DONATE_TEMPLATE:-don-panier-libre}" DEFAULT_AMOUNTS=("TESTPAYSAN:10" "TESTPAYSAN:20") DEFAULT_PAYS=("product:panier-legumes" "product:fromage-chevre" "product:oeufs-6") diff --git a/scripts/taler-monitoring/lib.sh b/scripts/taler-monitoring/lib.sh index a113212..841d6e8 100755 --- a/scripts/taler-monitoring/lib.sh +++ b/scripts/taler-monitoring/lib.sh @@ -14,6 +14,9 @@ # When LAN Host "koopa" is unreachable, try WAN DNAT (see ~/.ssh/config Host koopa-external). : "${KOOPA_SSH_FALLBACKS:=koopa-external}" : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}" +# Merchant portal /private API: instance ids must be lowercase or you get odd 401s. +# (Display names may use capitals; path segment /instances/{id}/ must not.) +MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower:]') : "${WITHDRAW_AMT:=GOA:20}" # single-shot fallback; e2e ladder uses ATM notes : "${PAY_AMT:=GOA:0.01}" : "${CREDIT_AMT:=GOA:4700}" # covers ATM ladder 20+50+100+200+4200 (paivana) @@ -201,6 +204,7 @@ apply_taler_domain() { MERCHANT_INSTANCE=fermes-des-collines ;; esac + MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower:]') WITHDRAW_AMT="${WITHDRAW_AMT:-TESTPAYSAN:50}" PAY_AMT="${PAY_AMT:-TESTPAYSAN:5}" # Cover oeufs(5)+fromage(8.5)+jus(6)+shop pick(~5–12) + fees From 7549eea81d4ef3d4c9b4075bf1f170fb4e9e6aed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 22:58:04 +0200 Subject: [PATCH 32/38] monitoring: check_auth401 WebUI signup/login/pwchange/case matrix --- scripts/taler-monitoring/check_auth401.sh | 704 ++++++++++++++++++++++ 1 file changed, 704 insertions(+) create mode 100755 scripts/taler-monitoring/check_auth401.sh diff --git a/scripts/taler-monitoring/check_auth401.sh b/scripts/taler-monitoring/check_auth401.sh new file mode 100755 index 0000000..dfca606 --- /dev/null +++ b/scripts/taler-monitoring/check_auth401.sh @@ -0,0 +1,704 @@ +#!/usr/bin/env bash +# auth401 — merchant HTTP 401 / case-sensitivity matrix (WebUI-shaped flows) +# +# Exercises the same APIs the merchant SPA uses for: +# • self-provision signup (POST /instances) — mixed-case id +# • login / createAccessToken (POST …/private/token) — Basic user case +# • password change (POST …/private/auth) +# • admin-create (POST /management/instances) when admin bearer available +# • access-token + private/ with Bearer +# +# Backend fact (#11340): Basic-auth *username* is case-sensitive; URL path is +# usually not. SPA must always lowercase the username (createAccessToken, signup +# id, admin-create, pw change re-login). These tests assert the *raw API* +# behaviour so a missing SPA lowercasing shows up as 401 where 200 is wanted +# only after lowercasing. +# +# ./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401 +# ./taler-monitoring.sh -d hacktivism.ch auth401 +# +# Continue through all groups / collect every ERROR (optional): +# AUTH401_CONTINUE=1 ./taler-monitoring.sh -d stage… auth401 +# CONTINUE_ON_ERROR=1 ./taler-monitoring.sh -d stage… auth401 # same +# +# Env: see secrets.env.example (AUTH401_*). +# +# NOTE: we intentionally do NOT use `set -e`. A single failed curl/grep under +# set -e would abort with no ERROR summary — the opposite of what monitoring +# needs. Optional CONTINUE_ON_ERROR still controls hard prereq early-exit. +set -uo pipefail +ROOT=$(cd "$(dirname "$0")" && pwd) +# shellcheck source=lib.sh +source "$ROOT/lib.sh" +load_monitoring_secrets_env 2>/dev/null || true + +tmp=$(mktemp -d) +AUTH401_FINISHED=0 +_auth401_finish() { + [ "${AUTH401_FINISHED}" = "1" ] && return 0 + AUTH401_FINISHED=1 + # summary() returns non-zero when FAIL_N>0 — never let that skip the print + summary || true + rm -rf "${tmp:-}" + if [ "${FAIL_N:-0}" -gt 0 ]; then + exit 1 + fi + exit 0 +} +trap '_auth401_finish' EXIT + +# --------------------------------------------------------------------------- +# Parameters +# --------------------------------------------------------------------------- +: "${AUTH401_TIMEOUT:=${TIMEOUT:-20}}" +: "${AUTH401_CREATE:=1}" +: "${AUTH401_INSTANCE:=}" +: "${AUTH401_PASSWORD:=${MERCHANT_INSTANCE_PASSWORD:-${AUTH401_PASS:-}}}" +: "${AUTH401_ID_PREFIX:=mon401}" +: "${AUTH401_SKIP_IF_MFA:=1}" +: "${AUTH401_BEARER:=${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}}" +: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}" +: "${AUTH401_DURABLE_ID:=mon401}" +# Prefer durable alone (skip throwaway) — default 0 so full matrix always runs +: "${AUTH401_PREFER_DURABLE:=0}" +# Run expanded groups (signup/login/pwchange/admin/webui). Default 1. +: "${AUTH401_FULL:=1}" +# Admin bearer for POST /management/instances (optional) +: "${AUTH401_ADMIN_TOKEN:=${MERCHANT_ADMIN_TOKEN:-}}" +: "${AUTH401_ADMIN_PASSWORD:=}" +: "${AUTH401_ADMIN_USER:=admin}" +# 1 = do not abort on hard prereq fail; run remaining groups (default 0) +# Alias: CONTINUE_ON_ERROR=1 +: "${AUTH401_CONTINUE:=${CONTINUE_ON_ERROR:-0}}" +# Never rotate password on durable account (default 1 — keep mon401.password valid) +: "${AUTH401_PWCHANGE_THROWAY_ONLY:=1}" + +TIMEOUT="$AUTH401_TIMEOUT" +export TIMEOUT + +_auth401_abort_or_continue() { + local why="${1:-prereq failed}" + if [ "${AUTH401_CONTINUE}" = "1" ]; then + warn "continue" "${why} (AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1)" + return 0 + fi + # hard stop — EXIT trap prints summary + all ERRORS collected so far + exit 1 +} + +BASE="${MERCHANT_PUBLIC%/}" +if [ -z "$BASE" ]; then + echo "MERCHANT_PUBLIC empty — set -d domain or --merchant" >&2 + exit 2 +fi + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- +expect_http() { + local label="$1" want="$2" got="$3" detail="${4:-}" + case ",$want," in + *",$got,"*) ok "$label" "HTTP $got${detail:+ · $detail}" ;; + *) fail "$label" "HTTP $got (want $want)${detail:+ · $detail}" ;; + esac +} + +to_lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; } +to_upper() { printf '%s' "$1" | tr '[:lower:]' '[:upper:]'; } + +# First char upper, rest lower → synthetic MIX for Basic user (always ≠ low if len>0) +synth_mix() { + python3 -c "s='''$1'''; print((s[:1].upper()+s[1:].lower()) if s else s)" +} + +# First 3 upper (partial mix) +synth_partmix() { + python3 -c "s='''$1'''; print(s[:3].upper()+s[3:] if len(s)>=3 else s.upper())" +} + +# Normalize curl status: "200", "000", never "200000" / "000000" +_http_norm() { + local raw="$1" dig + dig=$(printf '%s' "$raw" | tr -cd '0-9') + if [ -z "$dig" ]; then + printf '000' + else + # first 3 digits = real code (curl 200 then || echo 000 → 200000) + printf '%s' "$dig" | head -c 3 + fi +} + +token_http() { + local path_id="$1" user="$2" password="$3" + local b64 raw + b64=$(printf '%s' "${user}:${password}" | base64 -w0 2>/dev/null \ + || printf '%s' "${user}:${password}" | base64) + raw=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/last_token.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${path_id}/private/token" \ + -H "Authorization: Basic ${b64}" \ + -H 'Content-Type: application/json' \ + -d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \ + 2>/dev/null || true) + _http_norm "${raw:-}" +} + +json_code_hint() { + python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true +import json, sys +try: + d = json.load(open(sys.argv[1])) +except Exception: + print("") + raise SystemExit +c = d.get("code", "") +h = (d.get("hint") or d.get("detail") or "")[:70] +print(f"code={c} {h}".strip()) +PY +} + +extract_token() { + python3 - "$tmp/last_token.json" <<'PY' 2>/dev/null || true +import json, sys +try: + d = json.load(open(sys.argv[1])) +except Exception: + print("") + raise SystemExit(0) +print(d.get("access_token") or d.get("token") or "") +PY +} + +instance_body() { + local id="$1" pass="$2" name="${3:-auth401 probe}" + python3 - "$id" "$pass" "$name" <<'PY' +import json, sys +print(json.dumps({ + "address": {}, + "auth": {"method": "token", "password": sys.argv[2]}, + "default_pay_delay": {"d_us": 60000000}, + "default_wire_transfer_delay": {"d_us": 30000000}, + "default_refund_delay": {"d_us": 30000000}, + "id": sys.argv[1], + "jurisdiction": {}, + "name": sys.argv[3], + "use_stefan": True, +})) +PY +} + +auth_change_http() { + local path_id="$1" bearer="$2" new_pass="$3" + local tok="$bearer" body + case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac + body=$(NEW_PASS="$new_pass" python3 -c 'import json,os; print(json.dumps({"method":"token","password":os.environ["NEW_PASS"]}))') + curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/auth_change.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${path_id}/private/auth" \ + -H "Authorization: Bearer ${tok}" \ + -H 'Content-Type: application/json' \ + -d "$body" \ + 2>/dev/null || echo 000 +} + +# --------------------------------------------------------------------------- +# Stage secrets (quiet) +# --------------------------------------------------------------------------- +_auth401_stage_secret_roots() { + printf '%s\n' \ + "${FRANCPAYSAN_SECRETS:-}" \ + "${HOME}/francpaysan-secrets" \ + "${HOME}/src/francpaysan-secrets" \ + "${HOME}/taler/src/francpaysan-secrets" +} + +_auth401_resolve_secrets_quiet() { + local root inst_id pass + for root in $(_auth401_stage_secret_roots); do + [ -n "$root" ] && [ -d "$root" ] || continue + # durable ids: prefer mon401fix when mon401 was rotated by old tests + if [ -z "${DURABLE_PASS:-}" ]; then + for inst_id in "${AUTH401_DURABLE_ID}" mon401fix mon401 "${MERCHANT_INSTANCE:-}"; do + [ -n "$inst_id" ] || continue + pass="" + if [ -f "$root/stage/instances/${inst_id}.password" ]; then + pass=$(tr -d '\n\r' <"$root/stage/instances/${inst_id}.password") + fi + # skip empty password files + if [ -n "$pass" ]; then + # verify file is for this id (mon401.password may be stale copy of mon401fix) + DURABLE_PASS="$pass" + DURABLE_ID=$(to_lower "$inst_id") + AUTH401_SECRET_NOTE="durable ${inst_id} from ${root}/stage/instances/" + # only set AUTH401_PASSWORD if caller did not pin an instance + if [ -z "${AUTH401_PASSWORD}" ] && [ -z "${AUTH401_INSTANCE}" ]; then + AUTH401_PASSWORD="$pass" + fi + break + fi + done + fi + if [ -z "${AUTH401_BEARER}" ] && [ -f "$root/stage/default-instance-token.txt" ]; then + AUTH401_BEARER=$(tr -d '\n\r' <"$root/stage/default-instance-token.txt") + AUTH401_BEARER_NOTE="default-instance-token.txt" + fi + if [ -z "${AUTH401_ADMIN_TOKEN}" ] && [ -f "$root/stage/admin-instance-token.txt" ]; then + AUTH401_ADMIN_TOKEN=$(tr -d '\n\r' <"$root/stage/admin-instance-token.txt") + AUTH401_ADMIN_NOTE="admin-instance-token.txt" + fi + [ -n "${DURABLE_PASS:-}" ] && break + done + # also keep AUTH401_INSTANCE for durable-only mode + if [ "${AUTH401_PREFER_DURABLE}" = "1" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + AUTH401_INSTANCE="${AUTH401_INSTANCE:-$DURABLE_ID}" + AUTH401_PASSWORD="${AUTH401_PASSWORD:-$DURABLE_PASS}" + AUTH401_CREATE=0 + AUTH401_MODE_NOTE="PREFER_DURABLE=1 → only durable ${AUTH401_INSTANCE}" + fi +} + +_auth401_resolve_secrets_quiet +: "${AUTH401_CREATE_PASSWORD:=${AUTH401_PASSWORD:-Mon401-Test-Pass!}}" +export AUTH401_INSTANCE AUTH401_PASSWORD AUTH401_BEARER AUTH401_CREATE AUTH401_ADMIN_TOKEN + +# --------------------------------------------------------------------------- +set_area auth401 +section "auth401 · merchant 401 / case matrix (WebUI APIs) · ${BASE}" + +if [ -n "${AUTH401_SECRET_NOTE:-}" ]; then info "stage secret" "$AUTH401_SECRET_NOTE"; fi +if [ -n "${AUTH401_BEARER_NOTE:-}" ]; then info "stage secret" "AUTH401_BEARER from ${AUTH401_BEARER_NOTE}"; fi +if [ -n "${AUTH401_ADMIN_NOTE:-}" ]; then info "stage secret" "AUTH401_ADMIN_TOKEN from ${AUTH401_ADMIN_NOTE}"; fi +if [ -n "${AUTH401_MODE_NOTE:-}" ]; then info "auth401 mode" "$AUTH401_MODE_NOTE"; fi +info "auth401 scope" "FULL=${AUTH401_FULL} CREATE=${AUTH401_CREATE} PREFER_DURABLE=${AUTH401_PREFER_DURABLE} CONTINUE=${AUTH401_CONTINUE}" +if [ "${AUTH401_CONTINUE}" = "1" ]; then + info "continue mode" "CONTINUE_ON_ERROR/AUTH401_CONTINUE=1 — hard prereqs soft; always full ERROR list via EXIT trap" +fi + +# =========================================================================== +set_group config +# =========================================================================== +cfg_code=$(http_body "$BASE/config" "$tmp/mer-config.json") +expect_http "merchant /config" 200 "$cfg_code" "$BASE/config" + +HAVE_SP=0 +MANDATORY_TAN=0 +if [ "$cfg_code" = "200" ]; then + eval "$(python3 - "$tmp/mer-config.json" <<'PY' +import json, sys +d = json.load(open(sys.argv[1])) +sp = 1 if d.get("have_self_provisioning") else 0 +tans = d.get("mandatory_tan_channels") or [] +print(f"HAVE_SP={sp}") +print(f"MANDATORY_TAN={1 if tans else 0}") +print(f"TAN_CH={','.join(tans) if tans else ''}") +print(f"API_VER={d.get('version','')}") +PY +)" + info "self-provision" "have_self_provisioning=${HAVE_SP} mandatory_tan=${MANDATORY_TAN}${TAN_CH:+ ($TAN_CH)} api=${API_VER:-?}" +else + fail "merchant config required for auth401" "HTTP $cfg_code" + _auth401_abort_or_continue "no /config" +fi + +# =========================================================================== +set_group webui +# SPA surface: /webui/ loads + version stamp; optional toLowerCase in bundle +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + wcode=$(http_code "$BASE/webui/") + case "$wcode" in + 200|301|302) ok "webui /" "HTTP $wcode" ;; + *) warn "webui /" "HTTP $wcode" ;; + esac + vcode=$(http_body "$BASE/webui/version.txt" "$tmp/spa-ver.txt" 2>/dev/null || echo 000) + if [ "$vcode" = "200" ] && [ -s "$tmp/spa-ver.txt" ]; then + ok "webui version.txt" "$(tr -d '\n\r' <"$tmp/spa-ver.txt")" + else + warn "webui version.txt" "HTTP ${vcode:-000}" + fi + # Download a slice of index.js — look for lowercasing mitigation (#11340) + jcode=$(curl -skS -m 60 -o "$tmp/spa-index.js" -w '%{http_code}' "$BASE/webui/index.js" 2>/dev/null || echo 000) + jcode=$(printf '%s' "$jcode" | tr -cd '0-9' | head -c 3) + if [ "$jcode" = "200" ] && [ -s "$tmp/spa-index.js" ]; then + ok "webui index.js" "HTTP 200 · $(wc -c <"$tmp/spa-index.js" | tr -d ' ') bytes" + if grep -q 'toLowerCase' "$tmp/spa-index.js" 2>/dev/null; then + ok "webui bundle toLowerCase" "present (SPA may mitigate #11340)" + else + warn "webui bundle toLowerCase" "not found in index.js — login MIX may 401 without lowercasing" + fi + else + warn "webui index.js" "HTTP ${jcode:-000}" + fi +fi + +# =========================================================================== +set_group signup +# WebUI self-provision: POST /instances with mixed-case id (as typed) +# =========================================================================== +CREATED=0 +USE_ID="" +USE_PASS="" +ID_MIX="" +ID_LOW="" +CREATE_PASS="$AUTH401_CREATE_PASSWORD" +stamp=$(date +%H%M%S) +# Mixed-case id like the SPA would send before lowercasing (or if it forgot) +ID_MIX="$(printf '%s' "${AUTH401_ID_PREFIX}" | awk '{print toupper(substr($0,1,1)) substr($0,2)}')X${stamp}" +ID_LOW=$(to_lower "$ID_MIX") + +if [ "${AUTH401_FULL}" = "1" ] && [ "$AUTH401_CREATE" = "1" ] && [ "$HAVE_SP" = "1" ]; then + if [ "$MANDATORY_TAN" = "1" ] && [ "$AUTH401_SKIP_IF_MFA" = "1" ]; then + warn "signup skipped" "mandatory_tan_channels set — set AUTH401_INSTANCE+PASSWORD" + else + body=$(instance_body "$ID_MIX" "$CREATE_PASS" "auth401 signup MIX") + ccode=$(_http_norm "$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/create.json" -w '%{http_code}' \ + -X POST "${BASE}/instances" \ + -H 'Content-Type: application/json' \ + -d "$body" 2>/dev/null || true)") + case "$ccode" in + 200|204) + ok "signup POST /instances id=MIX" "HTTP $ccode id=$ID_MIX (SPA self-provision)" + CREATED=1 + USE_ID="$ID_MIX" + USE_PASS="$CREATE_PASS" + # Immediate login as SPA after signup: must use *lowercase* Basic user + code=$(token_http "$ID_LOW" "$ID_LOW" "$CREATE_PASS") + expect_http "signup→login SPA-sim path=low user=low → 200" 200 "$code" "$(json_code_hint)" + # Bug path: SPA forgot toLowerCase on login after signup with MIX id typed + code=$(token_http "$ID_LOW" "$ID_MIX" "$CREATE_PASS") + expect_http "signup→login RAW user=MIX (no lower) → 401" 401 "$code" "$(json_code_hint)" + ;; + 202) + warn "signup needs MFA" "HTTP 202 — use AUTH401_INSTANCE" + ;; + *) + fail "signup POST /instances" "HTTP $ccode $(head -c 120 "$tmp/create.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi +elif [ -n "${AUTH401_INSTANCE:-}" ] && [ -n "${AUTH401_PASSWORD:-}" ]; then + USE_ID=$(to_lower "$AUTH401_INSTANCE") + USE_PASS="$AUTH401_PASSWORD" + ID_LOW=$(to_lower "$USE_ID") + ID_MIX=$(synth_mix "$ID_LOW") + info "signup" "using existing AUTH401_INSTANCE=$USE_ID (no POST /instances)" + ok "using existing instance" "$USE_ID" +else + warn "signup skipped" "CREATE=${AUTH401_CREATE} HAVE_SP=${HAVE_SP} (need instance for matrix)" +fi + +# Fallback durable +if [ -z "$USE_ID" ] && [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + USE_ID="$DURABLE_ID" + USE_PASS="$DURABLE_PASS" + ID_LOW=$(to_lower "$USE_ID") + ID_MIX=$(synth_mix "$ID_LOW") + info "fallback durable" "$USE_ID" + ok "using durable instance" "$USE_ID" +fi + +if [ -z "$USE_ID" ]; then + fail "no instance under test" "need signup OK or AUTH401_INSTANCE+PASSWORD or mon401.password" + _auth401_abort_or_continue "no instance under test" + # CONTINUE: synthesize dummy ids so later groups still run (expect many 404/401) + ID_LOW_USE="auth401-missing-instance" + ID_USER_MIX=$(synth_mix "$ID_LOW_USE") + ID_PATH_MIX="$ID_USER_MIX" + ID_UPPER=$(to_upper "$ID_LOW_USE") + ID_PARTMIX=$(synth_partmix "$ID_LOW_USE") + PASS_OK="invalid" + PASS_BAD='auth401-wrong-password-NOT-VALID' + USE_ID="$ID_LOW_USE" + USE_PASS="$PASS_OK" +fi + +ID_LOW_USE=$(to_lower "$USE_ID") +# Always synthetic case variants (even for mon401) so MIX ≠ low +ID_USER_MIX=$(synth_mix "$ID_LOW_USE") +ID_PATH_MIX="$ID_USER_MIX" +ID_UPPER=$(to_upper "$ID_LOW_USE") +ID_PARTMIX=$(synth_partmix "$ID_LOW_USE") +PASS_OK="$USE_PASS" +PASS_BAD='auth401-wrong-password-NOT-VALID' + +info "probe ids" "low=${ID_LOW_USE} MIX_user=${ID_USER_MIX} UPPER=${ID_UPPER} created=${CREATED} pass_len=${#PASS_OK}" + +# =========================================================================== +set_group login +# WebUI login = POST …/private/token with Basic (username, password) +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + expect_http "login SPA-sim user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK") + expect_http "login RAW user=MIX pwd=OK → 401 (#11340)" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK") + expect_http "login RAW user=UPPER pwd=OK → 401" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK") + expect_http "login path=MIX user=low pwd=OK → 200 (path tolerant)" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK") + expect_http "login path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" +fi + +# =========================================================================== +set_group case +# Full Basic case matrix (always hard expects — synthetic MIX) +# =========================================================================== +code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_OK") +expect_http "path=MIX user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_PATH_MIX" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=MIX user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$PASS_OK") +expect_http "path=low user=MIX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=low user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_UPPER" "$ID_LOW_USE" "$PASS_OK") +expect_http "path=UPPER user=low pwd=OK → 200" 200 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_UPPER" "$PASS_OK") +expect_http "path=low user=UPPER pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_PARTMIX" "$PASS_OK") +expect_http "path=low user=MiX pwd=OK → 401 (case)" 401 "$code" "$(json_code_hint)" + +# =========================================================================== +set_group password +# =========================================================================== +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_BAD") +expect_http "pwd=BAD → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "") +expect_http "pwd=EMPTY → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_PATH_MIX" "$ID_USER_MIX" "$PASS_BAD") +expect_http "path=MIX user=MIX pwd=BAD → 401" 401 "$code" "$(json_code_hint)" + +# =========================================================================== +set_group pwchange +# WebUI password change: POST private/auth then re-login +# Only on throwaway signup instances — never rotate durable mon401.password +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + if [ "${AUTH401_PWCHANGE_THROWAY_ONLY}" = "1" ] && [ "${CREATED:-0}" != "1" ]; then + info "pwchange skipped" "not a throwaway instance (CREATED=0) — refuse to rotate durable password" + else + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + if [ "$code" != "200" ]; then + fail "pwchange: login for token" "HTTP $code — skip pwchange" + else + TOK=$(extract_token) + if [ -z "$TOK" ]; then + fail "pwchange: parse token" "empty access_token" + else + ok "pwchange: got bearer" "len=${#TOK}" + NEW_PASS="Auth401-New-${stamp}!" + ac=$(_http_norm "$(auth_change_http "$ID_LOW_USE" "$TOK" "$NEW_PASS")") + case "$ac" in + 200|204) + ok "pwchange POST private/auth" "HTTP $ac (SPA password form)" + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") + expect_http "pwchange: old pwd → 401" 401 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$NEW_PASS") + expect_http "pwchange: new pwd user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$ID_LOW_USE" "$ID_USER_MIX" "$NEW_PASS") + expect_http "pwchange: new pwd user=MIX → 401" 401 "$code" "$(json_code_hint)" + PASS_OK="$NEW_PASS" + USE_PASS="$NEW_PASS" + ;; + 202) + warn "pwchange needs MFA" "HTTP 202" + ;; + *) + fail "pwchange POST private/auth" "HTTP $ac $(head -c 100 "$tmp/auth_change.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi + fi + fi +fi + +# =========================================================================== +set_group admin +# Admin UI: POST /management/instances (needs admin bearer) +# =========================================================================== +if [ "${AUTH401_FULL}" = "1" ]; then + ADMIN_TOK="${AUTH401_ADMIN_TOKEN:-}" + # Optional: login as admin user with password + if [ -z "$ADMIN_TOK" ] && [ -n "${AUTH401_ADMIN_PASSWORD}" ]; then + code=$(token_http "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_USER" "$AUTH401_ADMIN_PASSWORD") + if [ "$code" = "200" ]; then + ADMIN_TOK=$(extract_token) + ok "admin login user=${AUTH401_ADMIN_USER}" "got token" + else + # try lower only + au=$(to_lower "$AUTH401_ADMIN_USER") + code=$(token_http "$au" "$au" "$AUTH401_ADMIN_PASSWORD") + if [ "$code" = "200" ]; then + ADMIN_TOK=$(extract_token) + ok "admin login user=${au}" "got token" + else + warn "admin login" "HTTP $code — set AUTH401_ADMIN_TOKEN" + fi + fi + fi + + if [ -z "$ADMIN_TOK" ]; then + info "admin-create skipped" "no AUTH401_ADMIN_TOKEN / AUTH401_ADMIN_PASSWORD (stage default is not admin)" + else + case "$ADMIN_TOK" in secret-token:*) ;; *) ADMIN_TOK="secret-token:${ADMIN_TOK}" ;; esac + AID_MIX="Adm401X${stamp}" + AID_LOW=$(to_lower "$AID_MIX") + APASS="Adm401-Pass-${stamp}!" + body=$(instance_body "$AID_MIX" "$APASS" "auth401 admin-create MIX") + acode=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/admin_create.json" -w '%{http_code}' \ + -X POST "${BASE}/management/instances" \ + -H "Authorization: Bearer ${ADMIN_TOK}" \ + -H 'Content-Type: application/json' \ + -d "$body" 2>/dev/null || echo 000) + case "$acode" in + 200|204) + ok "admin-create POST /management/instances id=MIX" "HTTP $acode id=$AID_MIX" + code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS") + expect_http "admin-create→login user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$AID_LOW" "$AID_MIX" "$APASS") + expect_http "admin-create→login user=MIX → 401" 401 "$code" "$(json_code_hint)" + # admin-set auth on new instance + code=$(token_http "$AID_LOW" "$AID_LOW" "$APASS") + if [ "$code" = "200" ]; then + AT=$(extract_token) + NEW_A="Adm401-New-${stamp}!" + ch=$(auth_change_http "$AID_LOW" "$AT" "$NEW_A") + case "$ch" in + 200|204) + ok "admin-created instance pwchange" "HTTP $ch" + code=$(token_http "$AID_LOW" "$AID_LOW" "$NEW_A") + expect_http "admin-created re-login new pwd → 200" 200 "$code" "$(json_code_hint)" + ;; + *) warn "admin-created pwchange" "HTTP $ch" ;; + esac + fi + ;; + 401|403) + warn "admin-create unauthorized" "HTTP $acode — token not admin (expected on stage default)" + ;; + 202) + warn "admin-create MFA" "HTTP 202" + ;; + *) + fail "admin-create" "HTTP $acode $(head -c 120 "$tmp/admin_create.json" 2>/dev/null | tr '\n' ' ')" + ;; + esac + fi +fi + +# =========================================================================== +set_group missing +# =========================================================================== +code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/last_token.json" -w '%{http_code}' \ + -X POST "${BASE}/instances/${ID_LOW_USE}/private/token" \ + -H 'Content-Type: application/json' \ + -d '{"scope":"write","duration":{"d_us":3600000000},"refreshable":true}' \ + 2>/dev/null || echo 000) +expect_http "private/token NO_AUTH → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "$ID_LOW_USE" "auth401-not-a-real-user" "$PASS_OK") +expect_http "user=OTHER pwd=OK → 401" 401 "$code" "$(json_code_hint)" + +code=$(token_http "auth401-does-not-exist-${stamp}" "$ID_LOW_USE" "$PASS_OK") +# Must not look like auth failure (401). Prefer 404; 502 from proxy is soft. +case "$code" in + 404) ok "path=missing → 404 (not 401)" "HTTP 404 · $(json_code_hint)" ;; + 401) fail "path=missing must not be 401" "HTTP 401 · $(json_code_hint)" ;; + 502|503) warn "path=missing" "HTTP $code (want 404; proxy flake, not auth)" ;; + *) fail "path=missing → 404 (not 401)" "HTTP $code (want 404) · $(json_code_hint)" ;; +esac + +# =========================================================================== +set_group bearer +# =========================================================================== +code=$(token_http "$ID_LOW_USE" "$ID_LOW_USE" "$PASS_OK") +if [ "$code" = "200" ]; then + TOKEN=$(extract_token) + if [ -n "$TOKEN" ]; then + ok "got access_token" "len=${#TOKEN}" + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" \ + -H "Authorization: Bearer ${TOKEN}" 2>/dev/null || echo 000) + expect_http "GET private/ Bearer OK → 200" 200 "$code" + + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" \ + -H "Authorization: Bearer secret-token:AUTH401INVALID000" 2>/dev/null || echo 000) + expect_http "GET private/ Bearer BAD → 401" 401 "$code" + + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv.json" -w '%{http_code}' \ + "${BASE}/instances/${ID_LOW_USE}/private/" 2>/dev/null || echo 000) + expect_http "GET private/ NO_AUTH → 401" 401 "$code" + else + fail "parse access_token" "HTTP $code but no token field" + fi +else + fail "token for bearer tests" "HTTP $code" +fi + +code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/mgmt.json" -w '%{http_code}' \ + "${BASE}/management/instances" 2>/dev/null || echo 000) +expect_http "GET /management/instances NO_AUTH → 401" 401 "$code" + +if [ -n "${AUTH401_BEARER:-}" ]; then + if [ -z "${AUTH401_BEARER_INSTANCE:-}" ]; then + case "${AUTH401_BEARER_NOTE:-}" in + *default-instance-token*) AUTH401_BEARER_INSTANCE=default ;; + *) AUTH401_BEARER_INSTANCE="${DURABLE_ID:-default}" ;; + esac + fi + inst=$(to_lower "$AUTH401_BEARER_INSTANCE") + tok="$AUTH401_BEARER" + case "$tok" in secret-token:*) ;; *) tok="secret-token:${tok}" ;; esac + code=$(curl -skS --max-redirs 0 -m "${TIMEOUT}" \ + -o "$tmp/priv2.json" -w '%{http_code}' \ + "${BASE}/instances/${inst}/private/" \ + -H "Authorization: Bearer ${tok}" 2>/dev/null || echo 000) + case "$code" in + 200) ok "GET private/ secrets Bearer" "HTTP 200 instance=$inst" ;; + 401) warn "secrets Bearer rejected" "HTTP 401 on $inst" ;; + *) warn "secrets Bearer" "HTTP $code on $inst" ;; + esac +fi + +# =========================================================================== +set_group durable +# Extra: durable mon401 still works after throwaway tests (if present) +# =========================================================================== +if [ -n "${DURABLE_ID:-}" ] && [ -n "${DURABLE_PASS:-}" ]; then + dlow=$(to_lower "$DURABLE_ID") + dmix=$(synth_mix "$dlow") + code=$(token_http "$dlow" "$dlow" "$DURABLE_PASS") + expect_http "durable ${dlow} login user=low → 200" 200 "$code" "$(json_code_hint)" + code=$(token_http "$dlow" "$dmix" "$DURABLE_PASS") + expect_http "durable ${dlow} login user=MIX → 401" 401 "$code" "$(json_code_hint)" +else + info "durable" "no mon401.password — skipped" +fi + +# =========================================================================== +set_group report +# =========================================================================== +info "summary matrix" "401: Basic user case≠low | bad/empty pwd | no auth | bad bearer | wrong user · 200: user=low + path any case · 404: missing instance · SPA must toLowerCase username on login/signup/admin/pw-relogin" +if [ "$CREATED" = "1" ]; then + info "throwaway left" "id=${ID_MIX:-?} low=${ID_LOW_USE} (password rotated if pwchange ran)" +fi + +# fall through → EXIT trap runs _auth401_finish (summary + full ERROR list) +: From 7d24efcb64951138c3f25ebb663ea33461b01b01 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 23:22:18 +0200 Subject: [PATCH 33/38] monitoring: wire auth401 + full phases; CONTINUE_ON_ERROR export --- scripts/taler-monitoring/taler-monitoring.sh | 39 +++++++++++++++++++- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/scripts/taler-monitoring/taler-monitoring.sh b/scripts/taler-monitoring/taler-monitoring.sh index e63c922..8b20e72 100755 --- a/scripts/taler-monitoring/taler-monitoring.sh +++ b/scripts/taler-monitoring/taler-monitoring.sh @@ -21,13 +21,16 @@ Usage: Phases: urls public HTTPS (no SSH) ← default with --domain + (includes merchant /webui/ SPA fingerprints: version.txt, overlay, assets) inside container status via SSH (local stack only) versions taler packages vs deb.taler.net trixie + repo availability sanity public + optional server server server-side only (SSH) e2e withdraw + pay (small amounts; remote aborts on login/KYC) ladder withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire) + auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance) all urls + inside + versions + sanity + e2e (SSH phases only on koopa) + full all + server + ladder + auth401 (maximum; long-running, needs secrets) Options: -d, --domain DOMAIN load profile from domains.conf (bank/exchange/merchant) @@ -63,6 +66,23 @@ Env (same meaning): QR_ECC=M qrencode ECC level (default M) KOOPA_SSH KOOPA_SSH_FALLBACKS (default koopa → koopa-external) METRICS_LOAD=0 skip host/container RAM/CPU probes (e2e/ladder/inside) + EXPECT_WEBUI_VERSION=1.6.11 pin merchant /webui/version.txt (urls) + EXPECT_WEBUI_OVERLAY=selfbuild require substr in version-overlay.txt + WEBUI_OVERLAY_DENY='master-11340' fail if overlay matches regex + CHECK_WEBUI_SPA=0 skip SPA fingerprint block in urls + AUTH401_* see check_auth401.sh / secrets.env.example + AUTH401_CONTINUE=1 / CONTINUE_ON_ERROR=1 + auth401: run all groups, collect every ERROR (no mid-run halt) + +Full load (GOA / hacktivism): + ./taler-monitoring.sh -d hacktivism.ch full + # or explicit: + ./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401 + +SPA pin after selfbuild: + EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \\ + WEBUI_OVERLAY_DENY='c778af04|master-11340' \\ + ./taler-monitoring.sh -d hacktivism.ch urls EOF } @@ -110,7 +130,7 @@ while [ $# -gt 0 ]; do CURRENCY_OVERRIDE="$2"; shift 2 ;; --no-probe) NO_PROBE=1; shift ;; - urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|all) PHASES+=("$1"); shift ;; + urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|auth401|all|full) PHASES+=("$1"); shift ;; *) # bare domain shorthand: ./taler-monitoring.sh taler.net if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then @@ -183,6 +203,9 @@ export LADDER_MAX_AMOUNT LADDER_INCLUDE_ZERO LADDER_INCLUDE_MAX export LADDER_HIGH_FROM LADDER_HIGH_RUNGS LADDER_LOAD export LADDER_PAY LADDER_WITHDRAW_SCALE LADDER_PAY_SETTLE_ROUNDS export TALER_DOMAIN_APPLIED=1 +# Optional: auth401 and other phases may honor CONTINUE_ON_ERROR +export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-}" +export AUTH401_CONTINUE="${AUTH401_CONTINUE:-${CONTINUE_ON_ERROR:-}}" # Default phases # - local GOA (koopa): urls + inside + versions + e2e @@ -212,6 +235,15 @@ for p in "${PHASES[@]}"; do # remote without inside SSH: public + optional e2e OUT_PHASES+=(urls versions e2e) fi + elif [ "$p" = "full" ]; then + # Maximum coverage. Long. Needs secrets for e2e/ladder/auth401. + if [ "${LOCAL_STACK}" = "1" ]; then + OUT_PHASES+=(urls inside versions sanity server e2e ladder auth401) + elif [ -n "${INSIDE_SSH:-}" ] || [ "${INSIDE_PROFILE:-}" = "stage-lfp" ]; then + OUT_PHASES+=(urls inside versions e2e ladder auth401) + else + OUT_PHASES+=(urls versions e2e auth401) + fi else OUT_PHASES+=("$p") fi @@ -247,7 +279,7 @@ if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then _pt=0 for p in "${PHASES[@]}"; do case "$p" in - urls) _pt=$((_pt + 85)) ;; # +qr group + urls) _pt=$((_pt + 95)) ;; # +qr + webui SPA fingerprints inside) _pt=$((_pt + 30)) ;; versions) _pt=$((_pt + 25)) ;; sanity) _pt=$((_pt + 35)) ;; @@ -255,6 +287,7 @@ if [ "${PROGRESS_TOTAL:-0}" = "0" ] || [ -z "${PROGRESS_TOTAL:-}" ]; then # e2e: ATM ladder emits many INFO lines (coins before/after each note) e2e) _pt=$((_pt + 240)) ;; ladder|goa-ladder) _pt=$((_pt + 120)) ;; + auth401) _pt=$((_pt + 70)) ;; esac done set_progress_total "$_pt" @@ -266,6 +299,7 @@ printf ' bank %s\n' "$BANK_PUBLIC" printf ' exchange %s\n' "$EXCHANGE_PUBLIC" printf ' merchant %s\n' "$MERCHANT_PUBLIC" printf ' currency expect=%s skip_ssh=%s\n' "${EXPECT_CURRENCY:-any}" "$SKIP_SSH" +printf ' phases %s\n' "${PHASES[*]}" printf ' progress total≈%s (set PROGRESS_TOTAL= to override; PROGRESS_OFF=1 to hide)\n' "${PROGRESS_TOTAL:-0}" chmod +x "$ROOT"/check_*.sh 2>/dev/null || true @@ -280,6 +314,7 @@ for p in "${PHASES[@]}"; do server) "$ROOT/check_server.sh" || ec=1 ;; e2e) "$ROOT/check_e2e.sh" || ec=1 ;; ladder|goa-ladder) "$ROOT/check_goa_ladder.sh" || ec=1 ;; + auth401) "$ROOT/check_auth401.sh" || ec=1 ;; esac done exit "$ec" From 52b1dc0f38f226309707811ae44710f98ca1f55c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 17 Jul 2026 23:41:31 +0200 Subject: [PATCH 34/38] monitoring: merchant SPA version.txt/overlay fingerprints in urls --- scripts/taler-monitoring/check_urls.sh | 91 ++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/scripts/taler-monitoring/check_urls.sh b/scripts/taler-monitoring/check_urls.sh index 7191609..36520d4 100755 --- a/scripts/taler-monitoring/check_urls.sh +++ b/scripts/taler-monitoring/check_urls.sh @@ -841,6 +841,97 @@ PY fi check_url_soft "merchant /webui/" 200 "$MERCHANT_PUBLIC/webui/" check_url_soft "merchant /" 302,301,200 "$MERCHANT_PUBLIC/" + + # --- merchant SPA fingerprints (selfbuild / package overlay) --- + # version.txt = SPA package version (e.g. 1.6.11) + # version-overlay.txt = optional deploy stamp (selfbuild-v1.6.11-0e4807845) + # Env: + # EXPECT_WEBUI_VERSION=1.6.11 → fail if version.txt ≠ + # EXPECT_WEBUI_OVERLAY=substr → fail if overlay missing that substr + # WEBUI_OVERLAY_DENY=regex → fail if overlay matches (e.g. master-11340) + # CHECK_WEBUI_SPA=0 → skip this block + if [ "${CHECK_WEBUI_SPA:-1}" = "1" ]; then + set_group webui + section "www · merchant SPA /webui/ fingerprints" + vcode=$(http_body "$MERCHANT_PUBLIC/webui/version.txt" "$tmp/webui-version.txt") + if [ "$vcode" = "200" ] && [ -s "$tmp/webui-version.txt" ]; then + got_ver=$(tr -d ' \n\r\t' <"$tmp/webui-version.txt") + ok "merchant /webui/version.txt" "HTTP 200 · ${got_ver}" + if [ -n "${EXPECT_WEBUI_VERSION:-}" ]; then + if [ "$got_ver" = "${EXPECT_WEBUI_VERSION}" ]; then + ok "merchant webui version pin" "EXPECT_WEBUI_VERSION=${EXPECT_WEBUI_VERSION}" + else + fail "merchant webui version pin" "got ${got_ver} want ${EXPECT_WEBUI_VERSION}" + fi + else + info "merchant webui version" "${got_ver} (set EXPECT_WEBUI_VERSION= to pin)" + fi + else + if [ "${LOCAL_STACK:-1}" = "1" ]; then + fail "merchant /webui/version.txt" "HTTP ${vcode:-000} (SPA missing version stamp)" + else + warn "merchant /webui/version.txt" "HTTP ${vcode:-000}" + fi + fi + + ocode=$(http_body "$MERCHANT_PUBLIC/webui/version-overlay.txt" "$tmp/webui-overlay.txt") + if [ "$ocode" = "200" ] && [ -s "$tmp/webui-overlay.txt" ]; then + got_ov=$(tr -d '\n\r' <"$tmp/webui-overlay.txt" | head -c 200) + ok "merchant /webui/version-overlay.txt" "HTTP 200 · ${got_ov}" + if [ -n "${EXPECT_WEBUI_OVERLAY:-}" ]; then + case "$got_ov" in + *"${EXPECT_WEBUI_OVERLAY}"*) + ok "merchant webui overlay pin" "contains ${EXPECT_WEBUI_OVERLAY}" + ;; + *) + fail "merchant webui overlay pin" "got «${got_ov}» want substr ${EXPECT_WEBUI_OVERLAY}" + ;; + esac + fi + if [ -n "${WEBUI_OVERLAY_DENY:-}" ]; then + if printf '%s' "$got_ov" | grep -qE -- "${WEBUI_OVERLAY_DENY}"; then + fail "merchant webui overlay deny" "«${got_ov}» matches WEBUI_OVERLAY_DENY=${WEBUI_OVERLAY_DENY}" + else + ok "merchant webui overlay deny" "no match for ${WEBUI_OVERLAY_DENY}" + fi + fi + else + # Overlay is optional for pure package installs; warn on local selfbuild stacks + if [ "${LOCAL_STACK:-1}" = "1" ]; then + warn "merchant /webui/version-overlay.txt" "HTTP ${ocode:-000} (optional; selfbuild writes it)" + else + info "merchant /webui/version-overlay.txt" "HTTP ${ocode:-000} (optional)" + fi + fi + + # Core SPA assets (paths used by selfbuild dist/prod). + # index.js is large — longer timeout; normalize curl "200000" (200 + ||000). + _webui_asset_code() { + local url="$1" tmo="${2:-${TIMEOUT}}" + local raw + raw=$(curl -skS --max-redirs 0 -m "$tmo" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || true) + # take first 3 digits only (avoid 200000 glue when curl exits non-zero after writing code) + printf '%s' "$raw" | tr -cd '0-9' | head -c 3 + [ -n "$(printf '%s' "$raw" | tr -cd '0-9')" ] || printf '000' + } + for asset in index.html index.js; do + tmo="${TIMEOUT}" + [ "$asset" = "index.js" ] && tmo="${WEBUI_INDEX_JS_TIMEOUT:-60}" + acode=$(_webui_asset_code "$MERCHANT_PUBLIC/webui/${asset}" "$tmo") + case "$acode" in + 200) ok "merchant /webui/${asset}" "HTTP 200" ;; + *) + if [ "${LOCAL_STACK:-1}" = "1" ]; then + fail "merchant /webui/${asset}" "HTTP ${acode:-000}" + else + warn "merchant /webui/${asset}" "HTTP ${acode:-000}" + fi + ;; + esac + done + # return group for any later merchant checks in this block + set_group merchant + fi fi # Merchant legal docs From 4f7d8164f8455e8fa53692aa7f1bcd2d02aa6a3a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Sat, 18 Jul 2026 00:03:29 +0200 Subject: [PATCH 35/38] monitoring: docs + secrets.env for auth401 and webui pins --- scripts/taler-monitoring/README.md | 20 +++++++++++++ scripts/taler-monitoring/TESTS.md | 31 +++++++++++++++++++- scripts/taler-monitoring/secrets.env.example | 26 ++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) diff --git a/scripts/taler-monitoring/README.md b/scripts/taler-monitoring/README.md index eea5e12..578b683 100644 --- a/scripts/taler-monitoring/README.md +++ b/scripts/taler-monitoring/README.md @@ -109,6 +109,26 @@ Each check prints **global** and **grouped** ids: # Landing stats.json freshness (outside-in; age via generated_at_unix) STATS_STALE_SECS=900 STATS_FAIL_SECS=3600 ./taler-monitoring.sh -d stage.lefrancpaysan.ch urls + +# --- Full load (GOA / hacktivism) --- +# all = urls + inside + versions + sanity + e2e +# full = all + server + ladder + auth401 (long; needs secrets.env) +./taler-monitoring.sh -d hacktivism.ch full +./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401 + +# SPA pin after selfbuild-webui.sh (container): +EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \ + WEBUI_OVERLAY_DENY='c778af04|master-11340' \ + ./taler-monitoring.sh -d hacktivism.ch urls + +# Only 401 matrix (may create throwaway instance): +./taler-monitoring.sh -d hacktivism.ch auth401 + +# Stage FrancPaysan — durable mon401 account (password in francpaysan-secrets): +./taler-monitoring.sh -d stage.monnaie.lefrancpaysan.ch auth401 +# secrets: ~/francpaysan-secrets/stage/instances/mon401.password +# ~/francpaysan-secrets/stage/default-instance-token.txt +# force throwaway create instead: AUTH401_PREFER_DURABLE=0 AUTH401_CREATE=1 … ``` ### Landing `stats.json` (www.stats) diff --git a/scripts/taler-monitoring/TESTS.md b/scripts/taler-monitoring/TESTS.md index 4ad45e9..fb829d0 100644 --- a/scripts/taler-monitoring/TESTS.md +++ b/scripts/taler-monitoring/TESTS.md @@ -52,6 +52,7 @@ Numbering follows **executed** checks (early skip may shift later NN inside the | **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three | | **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` | | **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** | +| **www.webui-** | SPA fingerprints: **`/webui/version.txt`**, **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` | | **www.paivana-** | local GOA paywall front (redirect to template) | | **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local | | **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` | @@ -106,7 +107,7 @@ When filing an issue, quote the full id + label, e.g. --- -## versions / sanity / ladder / server +## versions / sanity / ladder / server / auth401 | Area.group | Meaning | |------------|---------| @@ -116,6 +117,34 @@ When filing an issue, quote the full id + label, e.g. | **sanity.bank-** / **.exchange-** / **.merchant-** | public + optional server-side per component | | **ladder.plan-** / **.load-** / **.withdraw-** / **.pay-** / **.report-** | amount ladder (GOA ceiling or stage TESTPAYSAN `max_wire`) | | **server-** | SSH host ports / processes (flat unless grouped later) | +| **auth401.*** | merchant 401 / case matrix (WebUI APIs): **signup** `POST /instances` MIX id, **login** Basic user low vs MIX/UPPER, full **case** matrix, **password** bad/empty, **pwchange** `POST private/auth` + re-login, **admin-create** `POST /management/instances` (if admin token), **bearer**, **durable mon401**, **webui** SPA `toLowerCase` probe (`check_auth401.sh`) | + +### SPA pin (after selfbuild-webui) + +```bash +EXPECT_WEBUI_VERSION=1.6.11 \ +EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \ +WEBUI_OVERLAY_DENY='c778af04|master-11340' \ + ./taler-monitoring.sh -d hacktivism.ch urls +``` + +Skip SPA block: `CHECK_WEBUI_SPA=0`. + +### Full load (maximum phases) + +| Alias | Expands to (local GOA) | +|-------|-------------------------| +| **`all`** | `urls inside versions sanity e2e` | +| **`full`** | `urls inside versions sanity server e2e ladder auth401` | + +```bash +cd ~/src/koopa/koopa-admin-log/scripts/taler-monitoring +# secrets.env with bank admin + merchant tokens (see secrets.env.example) +./taler-monitoring.sh -d hacktivism.ch full +# same, explicit: +./taler-monitoring.sh -d hacktivism.ch \ + urls inside versions sanity server e2e ladder auth401 +``` --- diff --git a/scripts/taler-monitoring/secrets.env.example b/scripts/taler-monitoring/secrets.env.example index 212a7f2..87ec88c 100644 --- a/scripts/taler-monitoring/secrets.env.example +++ b/scripts/taler-monitoring/secrets.env.example @@ -43,6 +43,32 @@ # PROGRESS_SHOW_EVERY=8 # PROGRESS_OFF=1 +# --- Merchant SPA /webui/ fingerprints (urls phase, group www.webui) --- +# EXPECT_WEBUI_VERSION=1.6.11 +# EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 +# WEBUI_OVERLAY_DENY=c778af04|master-11340 +# CHECK_WEBUI_SPA=0 + +# --- auth401 phase (WebUI signup/login/pwchange/admin + case matrix) --- +# Default FULL=1 creates throwaway MIX instance + runs all groups. +# Stage also loads durable mon401 for extra durable.* checks: +# AUTH401_FULL=1 +# AUTH401_CONTINUE=1 # optional: hard prereqs soft; full ERROR list always via EXIT trap +# CONTINUE_ON_ERROR=1 # alias for AUTH401_CONTINUE +# (auth401 no longer uses set -e — silent mid-run abort fixed) +# AUTH401_CREATE=1 +# AUTH401_PREFER_DURABLE=0 # 1 = only mon401, skip throwaway signup +# AUTH401_INSTANCE=mon401 +# AUTH401_PASSWORD= +# AUTH401_DURABLE_ID=mon401 +# AUTH401_CREATE_PASSWORD=Mon401-Test-Pass! +# AUTH401_SKIP_IF_MFA=1 +# AUTH401_BEARER= # default-instance-token (not admin) +# AUTH401_ADMIN_TOKEN= # required for admin-create group +# AUTH401_ADMIN_PASSWORD= # or login as AUTH401_ADMIN_USER=admin +# AUTH401_ADMIN_USER=admin +# FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets + # --- Ladder (GOA withdraw ladder · explorer pool) --- # Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt). # Overrides only if needed: From fb504edc1a5c8fb87f01487280709b33a8ce7d6f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Sat, 18 Jul 2026 14:19:30 +0200 Subject: [PATCH 36/38] merchant: in-container selfbuild-webui by tag/commit --- scripts/taler-merchant/selfbuild-webui.sh | 197 ++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100755 scripts/taler-merchant/selfbuild-webui.sh diff --git a/scripts/taler-merchant/selfbuild-webui.sh b/scripts/taler-merchant/selfbuild-webui.sh new file mode 100755 index 0000000..bd1cfda --- /dev/null +++ b/scripts/taler-merchant/selfbuild-webui.sh @@ -0,0 +1,197 @@ +#!/usr/bin/env bash +# Build taler-merchant-webui SPA *inside* this container and install it. +# +# Usage (as root in taler-hacktivism): +# selfbuild-webui.sh [REF] +# selfbuild-webui.sh v1.6.11 +# selfbuild-webui.sh 0e4807845fc06f7c705711974650d67ef30e7919 +# WEBUI_REF=master selfbuild-webui.sh +# +# Env: +# WEBUI_REF tag / branch / commit (default: first arg or v1.6.11) +# TTC_GIT default https://git.taler.net/taler-typescript-core.git +# TTC_DIR default /var/taler-src/taler-typescript-core +# SPA_DEST default /usr/share/taler-merchant-webui +# PNPM_VERSION default 10.33.4 +# SKIP_NODE_INSTALL 1 = do not apt/npm install node/pnpm +# SKIP_DEPLOY 1 = build only, leave dist/prod in tree +# EXPECT_VERSION if set, die unless dist version.txt matches +# +set -euo pipefail + +export DEBIAN_FRONTEND=noninteractive +export PATH="/usr/local/bin:/usr/bin:/bin:${HOME}/.local/bin:${PATH:-}" + +WEBUI_REF="${WEBUI_REF:-${1:-v1.6.11}}" +TTC_GIT="${TTC_GIT:-https://git.taler.net/taler-typescript-core.git}" +TTC_DIR="${TTC_DIR:-/var/taler-src/taler-typescript-core}" +SPA_DEST="${SPA_DEST:-/usr/share/taler-merchant-webui}" +PNPM_VERSION="${PNPM_VERSION:-10.33.4}" +SKIP_NODE_INSTALL="${SKIP_NODE_INSTALL:-0}" +SKIP_DEPLOY="${SKIP_DEPLOY:-0}" +EXPECT_VERSION="${EXPECT_VERSION:-}" + +log() { printf '+ %s\n' "$*"; } +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } + +[ "$(id -u)" -eq 0 ] || die "run as root in the merchant container" + +# --------------------------------------------------------------------------- +# Toolchain: node + pnpm +# --------------------------------------------------------------------------- +ensure_node() { + if command -v node >/dev/null 2>&1 && command -v npm >/dev/null 2>&1; then + log "node $(node -v) npm $(npm -v)" + return 0 + fi + [ "${SKIP_NODE_INSTALL}" = "1" ] && die "node/npm missing and SKIP_NODE_INSTALL=1" + + log "install nodejs + npm via apt" + apt-get update -qq + apt-get install -y --no-install-recommends nodejs npm ca-certificates git \ + python3 curl ca-certificates 2>&1 | tail -20 + command -v node >/dev/null 2>&1 || die "node still missing after apt" + log "node $(node -v) npm $(npm -v)" +} + +ensure_pnpm() { + if command -v pnpm >/dev/null 2>&1; then + log "pnpm $(pnpm -v)" + return 0 + fi + [ "${SKIP_NODE_INSTALL}" = "1" ] && die "pnpm missing and SKIP_NODE_INSTALL=1" + + log "install pnpm@${PNPM_VERSION} globally" + npm install -g "pnpm@${PNPM_VERSION}" 2>&1 | tail -15 + # npm global bin may be /usr/local/bin or under /usr/lib + hash -r 2>/dev/null || true + export PATH="$(npm prefix -g)/bin:${PATH}" + command -v pnpm >/dev/null 2>&1 || die "pnpm still missing after npm install -g" + log "pnpm $(pnpm -v)" +} + +# --------------------------------------------------------------------------- +# Source tree at REF +# --------------------------------------------------------------------------- +checkout_ttc() { + mkdir -p /var/taler-src + if [ -d "${TTC_DIR}/.git" ]; then + log "update existing ${TTC_DIR}" + cd "${TTC_DIR}" + git remote set-url origin "${TTC_GIT}" 2>/dev/null \ + || git remote add origin "${TTC_GIT}" || true + git fetch --tags --force origin + else + log "clone ${TTC_GIT} → ${TTC_DIR}" + rm -rf "${TTC_DIR}" + git clone "${TTC_GIT}" "${TTC_DIR}" + cd "${TTC_DIR}" + fi + + log "checkout ${WEBUI_REF}" + # Accept tag, branch, or raw commit + if git rev-parse --verify --quiet "refs/tags/${WEBUI_REF}" >/dev/null; then + git checkout -f "refs/tags/${WEBUI_REF}" + elif git rev-parse --verify --quiet "origin/${WEBUI_REF}" >/dev/null; then + git checkout -f -B "${WEBUI_REF}" "origin/${WEBUI_REF}" + elif git rev-parse --verify --quiet "${WEBUI_REF}" >/dev/null; then + git checkout -f "${WEBUI_REF}" + else + # maybe short hash only available after fetch + git fetch origin "${WEBUI_REF}" 2>/dev/null || true + git checkout -f "${WEBUI_REF}" || die "cannot checkout WEBUI_REF=${WEBUI_REF}" + fi + + # assets submodule (needed by build.mjs) + if [ -f .gitmodules ] && grep -q 'taler-assets' .gitmodules 2>/dev/null; then + log "submodule contrib/taler-assets" + git submodule update --init contrib/taler-assets \ + || git submodule update --init --recursive contrib/taler-assets \ + || true + fi + if [ ! -f contrib/taler-assets/svg/logo/qr-logo.svg ]; then + # fallback: shallow clone assets next to known relative url + log "assets missing — try direct clone" + rm -rf contrib/taler-assets + mkdir -p contrib + git clone --depth 1 https://git.taler.net/taler-assets.git contrib/taler-assets \ + || die "taler-assets incomplete" + fi + test -f contrib/taler-assets/svg/logo/qr-logo.svg || die "taler-assets still incomplete" + + HEAD_SHORT=$(git rev-parse --short HEAD) + HEAD_FULL=$(git rev-parse HEAD) + PKG_VER=$(python3 -c "import json; print(json.load(open('packages/taler-merchant-webui/package.json'))['version'])") + log "HEAD=${HEAD_SHORT} (${HEAD_FULL})" + log "package.json version=${PKG_VER}" +} + +# --------------------------------------------------------------------------- +# Build + install +# --------------------------------------------------------------------------- +build_spa() { + cd "${TTC_DIR}" + log "pnpm install + compile @gnu-taler/taler-merchant-webui" + pnpm install --frozen-lockfile --filter @gnu-taler/taler-merchant-webui... + pnpm run --filter @gnu-taler/taler-merchant-webui... compile + + PROD="${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + test -f "${PROD}/index.js" || die "dist/prod/index.js missing" + test -f "${PROD}/version.txt" || die "dist/prod/version.txt missing" + GOT=$(tr -d ' \n\r' <"${PROD}/version.txt") + log "built version.txt=${GOT}" + if [ -n "${EXPECT_VERSION}" ] && [ "${GOT}" != "${EXPECT_VERSION}" ]; then + die "expected version ${EXPECT_VERSION}, got ${GOT}" + fi + # sensible default when building known tag + case "${WEBUI_REF}" in + v1.6.11|1.6.11) + [ "${GOT}" = "1.6.11" ] || die "expected 1.6.11 for ref ${WEBUI_REF}, got ${GOT}" + ;; + esac + BUILT_VER="${GOT}" +} + +deploy_spa() { + cd "${TTC_DIR}" + PROD="${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + HEAD_SHORT=$(git rev-parse --short HEAD) + ts=$(date +%Y%m%d-%H%M%S) + + if [ -d "${SPA_DEST}" ] && [ "$(ls -A "${SPA_DEST}" 2>/dev/null || true)" ]; then + bak="${SPA_DEST}.bak-selfbuild-${ts}" + log "backup ${SPA_DEST} → ${bak}" + cp -a "${SPA_DEST}" "${bak}" + fi + mkdir -p "${SPA_DEST}" + find "${SPA_DEST}" -mindepth 1 -maxdepth 1 ! -name 'bak-*' -exec rm -rf {} + + + log "install SPA → ${SPA_DEST}" + cp -a "${PROD}/." "${SPA_DEST}/" + printf 'selfbuild-%s-%s\n' "${WEBUI_REF}" "${HEAD_SHORT}" >"${SPA_DEST}/version-overlay.txt" + chown -R root:root "${SPA_DEST}" + + echo "=== deployed ===" + cat "${SPA_DEST}/version.txt" + echo + cat "${SPA_DEST}/version-overlay.txt" + echo + wc -c "${SPA_DEST}/index.js" +} + +# --------------------------------------------------------------------------- +main() { + log "selfbuild-webui REF=${WEBUI_REF}" + ensure_node + ensure_pnpm + checkout_ttc + build_spa + if [ "${SKIP_DEPLOY}" = "1" ]; then + log "SKIP_DEPLOY=1 — left at ${TTC_DIR}/packages/taler-merchant-webui/dist/prod" + else + deploy_spa + fi + log "DONE webui ${WEBUI_REF} → version ${BUILT_VER:-?} @ ${SPA_DEST}" +} + +main "$@" From 3daf69bd2f0ab27fb7c7d76f5019d2a838173aa6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Sat, 18 Jul 2026 14:30:56 +0200 Subject: [PATCH 37/38] merchant: host build-deploy webui tag + merchant master --- ...ild-deploy-webui-1.6.11-merchant-master.sh | 217 ++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100755 scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh diff --git a/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh b/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh new file mode 100755 index 0000000..bfd0b05 --- /dev/null +++ b/scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh @@ -0,0 +1,217 @@ +#!/usr/bin/env bash +# Build taler-merchant-webui from tag v1.6.11 + taler-merchant from master, +# deploy into podman container taler-hacktivism on koopa, restart services. +# +# Run from clementine (or any host with: git, node/pnpm, ssh koopa, podman on koopa): +# ./scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +# +# Env: +# KOOPA_SSH=koopa +# MERCHANT_CTR=taler-hacktivism +# TTC_GIT=git@git.taler.net:taler-typescript-core.git +# MERCHANT_GIT=https://git.taler.net/merchant.git # or git@… +# WEBUI_TAG=v1.6.11 +# MERCHANT_REF=master +# SKIP_WEBUI=0 SKIP_MERCHANT=0 SKIP_RESTART=0 +# PNPM=$HOME/.local/bin/pnpm +# WORK_ROOT=/tmp/taler-selfbuild-$$ +# +set -euo pipefail + +KOOPA_SSH="${KOOPA_SSH:-koopa}" +MERCHANT_CTR="${MERCHANT_CTR:-taler-hacktivism}" +TTC_GIT="${TTC_GIT:-git@git.taler.net:taler-typescript-core.git}" +MERCHANT_GIT="${MERCHANT_GIT:-https://git.taler.net/merchant.git}" +WEBUI_TAG="${WEBUI_TAG:-v1.6.11}" +MERCHANT_REF="${MERCHANT_REF:-master}" +SKIP_WEBUI="${SKIP_WEBUI:-0}" +SKIP_MERCHANT="${SKIP_MERCHANT:-0}" +SKIP_RESTART="${SKIP_RESTART:-0}" +PNPM="${PNPM:-$HOME/.local/bin/pnpm}" +WORK_ROOT="${WORK_ROOT:-/tmp/taler-selfbuild-$$}" +SPA_DIR="${WORK_ROOT}/spa-prod" +TTC_DIR="${WORK_ROOT}/taler-typescript-core" +export PATH="${HOME}/.local/bin:${PATH}" + +log() { printf '+ %s\n' "$*"; } +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } + +need() { command -v "$1" >/dev/null 2>&1 || die "missing command: $1"; } + +ssh_koopa() { + ssh -o BatchMode=yes -o ConnectTimeout=15 "${KOOPA_SSH}" "$@" +} + +ctr_root() { + ssh_koopa podman exec -u root "${MERCHANT_CTR}" bash -c "$*" +} + +log "work root: ${WORK_ROOT}" +mkdir -p "${WORK_ROOT}" +need git +need ssh +need tar +need curl + +# --------------------------------------------------------------------------- +# 1) WebUI from tag +# --------------------------------------------------------------------------- +if [ "${SKIP_WEBUI}" != "1" ]; then + need node + if [ ! -x "${PNPM}" ]; then + log "installing pnpm to ~/.local …" + npm install -g pnpm@10.33.4 --prefix "${HOME}/.local" + fi + need pnpm || [ -x "${PNPM}" ] + PNPM_BIN=$(command -v pnpm || echo "${PNPM}") + + log "clone ${TTC_GIT} @ ${WEBUI_TAG} (full clone, not worktree — build.mjs needs real .git)" + rm -rf "${TTC_DIR}" + git clone --branch "${WEBUI_TAG}" --depth 1 "${TTC_GIT}" "${TTC_DIR}" + cd "${TTC_DIR}" + git submodule update --init contrib/taler-assets + test -f contrib/taler-assets/svg/logo/qr-logo.svg || die "taler-assets submodule incomplete" + + ver=$(python3 -c "import json; print(json.load(open('packages/taler-merchant-webui/package.json'))['version'])") + log "package.json version=${ver} HEAD=$(git rev-parse --short HEAD)" + + log "pnpm install + compile merchant-webui …" + "${PNPM_BIN}" install --frozen-lockfile --filter @gnu-taler/taler-merchant-webui... + "${PNPM_BIN}" run --filter @gnu-taler/taler-merchant-webui... compile + + test -f packages/taler-merchant-webui/dist/prod/index.js || die "dist/prod/index.js missing" + test -f packages/taler-merchant-webui/dist/prod/version.txt || die "version.txt missing" + got=$(tr -d '\n' < packages/taler-merchant-webui/dist/prod/version.txt) + log "built SPA version.txt=${got}" + # expect 1.6.11 for default tag + case "${WEBUI_TAG}" in + v1.6.11|1.6.11) + [ "${got}" = "1.6.11" ] || die "expected version 1.6.11, got ${got}" + ;; + esac + + rm -rf "${SPA_DIR}" + mkdir -p "${SPA_DIR}" + cp -a packages/taler-merchant-webui/dist/prod/. "${SPA_DIR}/" + printf 'selfbuild-%s-%s\n' "${WEBUI_TAG}" "$(git rev-parse --short HEAD)" >"${SPA_DIR}/version-overlay.txt" + + log "deploy SPA into ${MERCHANT_CTR}:/usr/share/taler-merchant-webui" + ts=$(date +%Y%m%d-%H%M%S) + ctr_root " + set -e + if [ -d /usr/share/taler-merchant-webui ]; then + cp -a /usr/share/taler-merchant-webui /usr/share/taler-merchant-webui.bak-selfbuild-${ts} + fi + mkdir -p /usr/share/taler-merchant-webui + # clear previous SPA files (keep directory) + find /usr/share/taler-merchant-webui -mindepth 1 -maxdepth 1 ! -name 'bak-*' -exec rm -rf {} + + " + tar -C "${SPA_DIR}" -czf - . | ssh_koopa "podman exec -i -u root ${MERCHANT_CTR} bash -c ' + set -e + cd /usr/share/taler-merchant-webui + tar -xzf - + chown -R root:root /usr/share/taler-merchant-webui + echo deployed: + cat version.txt + cat version-overlay.txt + wc -c index.js + '" + log "webui deploy done" +else + log "SKIP_WEBUI=1" +fi + +# --------------------------------------------------------------------------- +# 2) Merchant backend from master (build inside container) +# --------------------------------------------------------------------------- +if [ "${SKIP_MERCHANT}" != "1" ]; then + log "build taler-merchant ${MERCHANT_REF} inside ${MERCHANT_CTR}" + # Install build deps if missing; update /var/taler-src/merchant; bootstrap; make install + ssh_koopa "podman exec -u root ${MERCHANT_CTR} bash -s" <&1 | tail -30 || true + +# exchange headers sometimes via package names that differ +apt-get install -y --no-install-recommends libtalerutil-dev 2>/dev/null || true + +echo '=== source tree ===' +mkdir -p /var/taler-src +if [ -d "\$SRC/.git" ]; then + cd "\$SRC" + git remote set-url origin "\$GIT_URL" 2>/dev/null || git remote add origin "\$GIT_URL" || true + git fetch --tags origin + git checkout "\$REF" + git pull --ff-only origin "\$REF" || git reset --hard "origin/\$REF" +else + rm -rf "\$SRC" + git clone "\$GIT_URL" "\$SRC" + cd "\$SRC" + git checkout "\$REF" +fi +# submodules if any +git submodule update --init --recursive 2>/dev/null || true +echo "merchant HEAD=\$(git rev-parse --short HEAD) \$(git log -1 --oneline)" + +echo '=== bootstrap / configure / make ===' +if [ ! -x configure ]; then + ./bootstrap +fi +# Prefer /usr prefix to replace package binaries +./configure --prefix=/usr --disable-doc 2>&1 | tail -40 +make -j"\$(nproc)" 2>&1 | tail -50 +make install 2>&1 | tail -30 + +echo '=== installed httpd ===' +ls -la /usr/bin/taler-merchant-httpd +taler-merchant-httpd --help 2>&1 | head -5 || true +# record build id +git rev-parse HEAD > /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD 2>/dev/null \\ + || mkdir -p /usr/share/taler-merchant && git rev-parse HEAD > /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD +echo "SELFBUILD_MERCHANT_HEAD=\$(cat /usr/share/taler-merchant/SELFBUILD_MERCHANT_HEAD)" +REMOTE + log "merchant build/install done" +else + log "SKIP_MERCHANT=1" +fi + +# --------------------------------------------------------------------------- +# 3) Restart + health +# --------------------------------------------------------------------------- +if [ "${SKIP_RESTART}" != "1" ]; then + log "restart merchant services" + ssh_koopa "podman exec -u root ${MERCHANT_CTR} bash -c ' + set -e + if id taler-merchant-httpd >/dev/null 2>&1; then + runuser -u taler-merchant-httpd -- taler-merchant-dbinit 2>&1 | tail -15 || true + fi + runuser -u taler-merchant-httpd -- /usr/local/bin/start_merchant.sh --restart + '" + log "smoke" + ssh_koopa 'curl -skS -m 8 -o /dev/null -w "merchant_local %{http_code}\n" https://127.0.0.1:9010/config || true' + curl -skS -m 12 -o /dev/null -w "merchant_public %{http_code}\n" https://taler.hacktivism.ch/config || true + curl -skS -m 12 -o /dev/null -w "webui %{http_code}\n" https://taler.hacktivism.ch/webui/ || true + pubver=$(curl -skS -m 12 https://taler.hacktivism.ch/webui/version.txt 2>/dev/null || true) + pubov=$(curl -skS -m 12 https://taler.hacktivism.ch/webui/version-overlay.txt 2>/dev/null || true) + log "public version.txt=${pubver}" + log "public version-overlay.txt=${pubov}" +else + log "SKIP_RESTART=1" +fi + +log "DONE" +log " webui: tag ${WEBUI_TAG} → container /usr/share/taler-merchant-webui" +log " merchant: ${MERCHANT_REF} → /usr/bin/taler-merchant-*" +log " work left at ${WORK_ROOT} (delete when satisfied)" From 6b1e8f8662bf5e8dae115c878ce08f364692bc01 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Sat, 18 Jul 2026 14:32:29 +0200 Subject: [PATCH 38/38] merchant: document selfbuild webui/merchant deploy --- scripts/taler-merchant/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/scripts/taler-merchant/README.md b/scripts/taler-merchant/README.md index 4ddea4a..2173a28 100644 --- a/scripts/taler-merchant/README.md +++ b/scripts/taler-merchant/README.md @@ -30,6 +30,27 @@ Automatic import needs: SMS backends are symlinks into `/var/taler-src/...` (not copied). +## Self-build deploy (webui tag + merchant master) + +From a laptop with `ssh koopa` + `pnpm`/`node` (e.g. clementine): + +```bash +cd ~/src/koopa/koopa-admin-log +chmod +x scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +./scripts/taler-merchant/build-deploy-webui-1.6.11-merchant-master.sh +``` + +What it does: + +1. **Clone** `taler-typescript-core` **tag `v1.6.11`**, build SPA, install into + `taler-hacktivism:/usr/share/taler-merchant-webui` (backup `…bak-selfbuild-*`). +2. **Inside the container:** update `/var/taler-src/merchant` to **master**, + `./bootstrap && ./configure --prefix=/usr && make && make install`. +3. **`start_merchant.sh --restart`** + smoke `/config` + `/webui/`. + +Env knobs: `SKIP_WEBUI=1`, `SKIP_MERCHANT=1`, `SKIP_RESTART=1`, +`WEBUI_TAG=…`, `MERCHANT_REF=master`, `KOOPA_SSH=koopa`. + ## Portal / instance IDs: **lowercase only** Merchant SPA (`/webui/`) and `/instances/{id}/…` require **lowercase** instance