monitoring: resolve ladder explorer secret without laptop-hardcoded path

Ladder used /Users/newkamek/… for bank-explorer password and wallet-cli. Use SECRETS_ROOT/read_secret, EXP_PW/EXP_PW_FILE overrides, and find_wallet_cli. Document Android automation prereqs; stage shop assets also require qrcode.min.js.
2026-07-17 19:58:10 +02:00 · 2026-07-17 19:58:10 +02:00 · 8c38fbaf59
commit 8c38fbaf59
parent 08c55e71a7
7 changed files with 174 additions and 14 deletions
--- a/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md
+++ b/scripts/taler-monitoring/GIT-AUTOMATION-NOTES.md
@ -0,0 +1,87 @@
+# Git / automation notes (taler-monitoring + mobile)
+
+Notes for **automated checks** around the Taler stacks (GOA, stage
+TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet
+UI automation is separate tooling** and is **not** installed on the
+default ops laptop yet.
+
+## What runs without Android
+
+| Phase | Needs |
+|-------|--------|
+| `urls` / `sanity` | public HTTPS only |
+| `inside` / `versions` | SSH (`koopa` or `francpaysan-stage-user`) |
+| `e2e` / `ladder` | `taler-wallet-cli` + bank/merchant secrets |
+
+```bash
+./taler-monitoring.sh -d stage.lefrancpaysan.ch urls
+./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e   # after secrets.env
+```
+
+Mobile app checks (manual withdraw/pay, then automation) are **out of
+band** until the software below is present.
+
+---
+
+## Android automation software — **required** (not installed yet)
+
+To automate **GNU Taler Wallet** on Android (stable + master builds;
+manual then auto withdraw; fixed + variable pay) against stage
+TESTPAYSAN, the following must be available on the runner host.
+
+### Status on this host (2026-07-17)
+
+| Component | Status |
+|-----------|--------|
+| Java (OpenJDK 21) | present (`/usr/bin/java`) |
+| `aapt` / `aapt2` | present |
+| **`adb`** (platform-tools) | **missing** (package was removed: `android-tools-adb` rc) |
+| **Android SDK** (`ANDROID_HOME` / `ANDROID_SDK_ROOT`) | **missing** / empty |
+| **Emulator + AVD** | **missing** |
+| Appium / Maestro / `uiautomator2` | **missing** |
+| scrcpy | missing (optional) |
+
+### Must install before mobile automation
+
+1. **Android platform-tools** — at least `adb`  
+   - Debian: `android-tools-adb` (and preferably full SDK platform-tools)
+2. **Android SDK command-line tools** — `sdkmanager`, platforms, build-tools  
+   - set `ANDROID_HOME` (or `ANDROID_SDK_ROOT`) permanently
+3. **System image + AVD** — for emulator runs (x86_64 or host-matching ABI)  
+   - `emulator`, `avdmanager`, KVM/acceleration if available
+4. **UI automation stack** (pick one, prefer the leaner path for wallet flows):  
+   - **Maestro**, or  
+   - **Appium** + **UiAutomator2** (+ Node/Python bindings as needed)
+5. **Optional:** `scrcpy` for live view during manual debugging
+
+### Explicitly out of scope until freigabe
+
+- Installing production wallet APKs and driving real stage withdraw/pay  
+  without a human go-ahead  
+- Committing secrets, device serials, or APK binaries into git
+
+### Operator still needs to supply
+
+- Stable / master **APK** (or store/build URLs) for the wallet  
+- USB debugging consent on a physical device, **or** agreement to use AVD only  
+- Which flows are in-scope first (withdraw communal QR → shop pay, …)
+
+### Suggested freigabe prompt (for an agent)
+
+See session notes: full copy-paste block under “Android-Automatisierung
+vollständig einrichten” — install tooling, verify `adb devices` + emulator,
+document paths; **no** real wallet payments until approved.
+
+---
+
+## Related docs
+
+| Doc | Role |
+|-----|------|
+| [README.md](./README.md) | how to run `taler-monitoring.sh` |
+| [TESTS.md](./TESTS.md) | grouped check IDs |
+| [secrets.env.example](./secrets.env.example) | e2e secrets layout |
+| `francpaysan-admin-log/TODO.md` | stage wallet mobile test backlog |
+
+When Android tooling is installed, append a short “installed on …”
+section here (date, `adb version`, AVD name, `ANDROID_HOME`).
--- a/scripts/taler-monitoring/README.md
+++ b/scripts/taler-monitoring/README.md
@ -13,7 +13,8 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs**

 IDs reset per **group** inside an area — so “too many www tests” become
 `www.exchange-*`, `www.bank-*`, `www.landing-*`, etc.  
-Catalog: **[TESTS.md](./TESTS.md)**.
+Catalog: **[TESTS.md](./TESTS.md)**.  
+Android UI automation prerequisites: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)** (tools **not** installed by default).

 ```text
 ┌ OK    ┐ www.exchange-01  exchange /config · HTTP 200
--- a/scripts/taler-monitoring/TESTS.md
+++ b/scripts/taler-monitoring/TESTS.md
@ -53,7 +53,7 @@ Numbering follows **executed** checks (early skip may shift later NN inside the
 | **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` |
 | **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** |
 | **www.paivana-** | local GOA paywall front (redirect to template) |
-| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; cross-links local |
+| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local |
 | **www.qr-** | QR payloads: harvest `taler://` / `payto://` / app `data-qr-url` from landings + mint JSON; **form** check; **qrencode → zbarimg** exact roundtrip; optional static QR images. Needs `qrencode` + `zbar-tools`. Skip: `QR_CHECK=0` |

 **QR rule:** `taler://withdraw/HOST/taler-integration/UUID` (no default `:443`/`:80`); `taler://pay/` / `pay-template/`; `payto://…` shape OK; decoded PNG must equal payload. Form errors on withdraw/pay/payto are **ERROR** on local/GOA.
--- a/scripts/taler-monitoring/check_goa_ladder.sh
+++ b/scripts/taler-monitoring/check_goa_ladder.sh
@ -21,13 +21,17 @@
 #   LADDER_TIMEOUT_S         default 3600
 #   LADDER_LOAD=0            skip host load snapshots
 #   LADDER_SETTLE_ROUNDS / LADDER_SETTLE_SLEEP — balance poll only (no shepherd)
-#   EXP_PW_FILE, LADDER_REPORT_DIR, MERCHANT_INSTANCE, …
+#   EXP_PW / EXP_PW_FILE / EXP_USER — explorer pool (default: secrets tree)
+#   CLI_JS / WALLET_CLI — wallet-cli .mjs (default: find_wallet_cli)
+#   LADDER_REPORT_DIR, MERCHANT_INSTANCE, …
 #
 # Phase:  ./taler-monitoring.sh ladder
 set -euo pipefail
 ROOT=$(cd "$(dirname "$0")" && pwd)
 # shellcheck source=lib.sh
 source "$ROOT/lib.sh"
+# When invoked standalone (not via taler-monitoring.sh), still load secrets.env
+load_monitoring_secrets_env 2>/dev/null || true

 # Area ladder.* — GOA withdraw/pay ladder
 # Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report
@ -44,8 +48,10 @@ elapsed_ms() {
 : "${LADDER_SETTLE_ROUNDS:=18}"
 : "${LADDER_SETTLE_SLEEP:=2}"
 : "${EXP_USER:=explorer}"
-: "${EXP_PW_FILE:=/Users/newkamek/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt}"
-: "${CLI_JS:=/Users/newkamek/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs}"
+# EXP_PW_FILE / EXP_PW optional overrides; otherwise read_secret + SECRETS_ROOT
+: "${EXP_PW_FILE:=}"
+: "${EXP_PW:=}"
+: "${CLI_JS:=}"
 : "${LADDER_MAX_AMOUNT:=4503599627370496}"
 : "${LADDER_STEPS:=23}"
 : "${LADDER_LOAD:=1}"
@ -58,6 +64,15 @@ elapsed_ms() {
 : "${LADDER_PAY_SETTLE_ROUNDS:=6}"
 : "${MERCHANT_INSTANCE:=goa-demo-cp4zqk}"

+# Resolve wallet-cli .mjs (no hardcoded laptop path)
+if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then
+  CLI_JS=$(find_wallet_cli 2>/dev/null || true)
+fi
+if [ -z "${CLI_JS}" ] || [ ! -f "${CLI_JS}" ]; then
+  # allow PATH wrapper as last resort (wcli falls back to taler-wallet-cli)
+  CLI_JS=""
+fi
+
 CUR="${EXPECT_CURRENCY:-GOA}"
 BANK="${BANK_PUBLIC%/}"
 EX="${EXCHANGE_PUBLIC%/}/"
@ -82,13 +97,42 @@ ladder_over() {
 }

 wcli() {
-  if [ -f "$CLI_JS" ]; then
+  if [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then
    node "$CLI_JS" --wallet-db="$WDB" --no-throttle "$@"
  else
    taler-wallet-cli --wallet-db="$WDB" --no-throttle "$@"
  fi
 }

+# Explorer pool password: env → file override → secrets tree → SSH koopa
+resolve_explorer_pw() {
+  local pw="" f=""
+  if [ -n "${EXP_PW:-}" ]; then
+    printf '%s' "$EXP_PW"
+    return 0
+  fi
+  if [ -n "${EXP_PW_FILE:-}" ] && [ -f "$EXP_PW_FILE" ]; then
+    tr -d '\n\r' <"$EXP_PW_FILE"
+    return 0
+  fi
+  # Prefer shared read_secret (SECRETS_ROOT / ~/.config / koopa SSH)
+  if pw=$(read_secret "taler-bank/bank-explorer-password.txt" 2>/dev/null) && [ -n "$pw" ]; then
+    printf '%s' "$pw"
+    return 0
+  fi
+  # Extra local candidates (sibling of admin-log, taler-landing config)
+  for f in \
+    "${SECRETS_ROOT:+${SECRETS_ROOT}/taler-bank/bank-explorer-password.txt}" \
+    "${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt" \
+    "${HOME}/.config/taler-landing/bank-explorer-password.txt"
+  do
+    [ -n "$f" ] && [ -f "$f" ] || continue
+    tr -d '\n\r' <"$f"
+    return 0
+  done
+  return 1
+}
+
 wallet_avail() {
  wcli balance 2>/dev/null | python3 -c '
 import json,sys
@ -270,11 +314,18 @@ metrics_report_load "$LOAD_BEFORE" "ladder-start" || true
 # Fresh wallet per rung — baseline empty (or last-rung DB if re-used later)
 metrics_report_coins "ladder-start" || true

-if [ ! -f "$EXP_PW_FILE" ]; then
-  err bank "explorer password missing" "$EXP_PW_FILE"
+if ! EXP_PW=$(resolve_explorer_pw); then
+  err bank "explorer password missing" \
+    "set EXP_PW / EXP_PW_FILE or SECRETS_ROOT=…/koopa/host-root (taler-bank/bank-explorer-password.txt)${SECRETS_ROOT:+ · SECRETS_ROOT=${SECRETS_ROOT}}"
+  secrets_hint 2>/dev/null || true
  exit 1
 fi
-EXP_PW=$(tr -d '\n' <"$EXP_PW_FILE")
+info "explorer secret" "${SECRETS_ROOT:+SECRETS_ROOT=${SECRETS_ROOT}}${SECRETS_ROOT:-via EXP_PW/EXP_PW_FILE/SSH}"
+if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then
+  info "wallet-cli" "$CLI_JS"
+else
+  info "wallet-cli" "PATH taler-wallet-cli ($(command -v taler-wallet-cli 2>/dev/null || echo missing))"
+fi

 # --- auto-account ---
 t0=$(now_ms)
--- a/scripts/taler-monitoring/check_urls.sh
+++ b/scripts/taler-monitoring/check_urls.sh
@ -1236,7 +1236,8 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
  fi
 fi

-# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js + shops.css
+# Merchant shop assets — GOA: shop-pay.*; stage TESTPAYSAN: /assets/shop-ui.js +
+# shops.css + QR encoder (shop-ui modal needs /intro/qrcode.min.js or /qrcode.min.js).
 _ma=0
 _ma_need=2
 _ma_label="shop-pay.js + .css"
@ -1244,9 +1245,16 @@ if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
  _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.js" && _ma=$((_ma + 1))
  _landing_probe "$MERCHANT_PUBLIC/intro/shop-pay.css" && _ma=$((_ma + 1))
 else
-  _ma_label="shop-ui.js + shops.css"
+  # stage farmer shops: UI + stylesheet + qrcode (pay modal QR_Taler)
+  _ma_need=3
+  _ma_label="shop-ui.js + shops.css + qrcode.min.js"
  _landing_probe "$MERCHANT_PUBLIC/assets/shop-ui.js" && _ma=$((_ma + 1))
  _landing_probe "$MERCHANT_PUBLIC/assets/shops.css" && _ma=$((_ma + 1))
+  # Prefer /intro/ (always on landings tree); root alias is Caddy-only convenience
+  if _landing_probe "$MERCHANT_PUBLIC/intro/qrcode.min.js" \
+    || _landing_probe "$MERCHANT_PUBLIC/qrcode.min.js"; then
+    _ma=$((_ma + 1))
+  fi
 fi
 if [ "$_ma" -eq "$_ma_need" ]; then
  ok "landing merchant shop assets" "${_ma_label}"
@ -1254,9 +1262,11 @@ else
  if [ "${EXPECT_CURRENCY:-}" = "GOA" ] || [ "${LOCAL_STACK:-0}" = "1" ]; then
    warn "landing merchant shop assets" "${_ma}/${_ma_need} present (soft) · want ${_ma_label}"
  else
-    # stage: soft INFO if missing; WARN only if zero
-    if [ "$_ma" -eq 0 ]; then
-      warn "landing merchant shop assets" "0/${_ma_need} · want ${_ma_label}"
+    # stage: hard-ish — missing qrcode broke pay QR («bibliothèque manquante»)
+    if [ "$_ma" -lt 2 ]; then
+      fail "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}"
+    elif [ "$_ma" -lt "$_ma_need" ]; then
+      warn "landing merchant shop assets" "${_ma}/${_ma_need} · want ${_ma_label}"
    else
      info "landing merchant shop assets" "${_ma}/${_ma_need} · ${_ma_label}"
    fi
--- a/scripts/taler-monitoring/lib.sh
+++ b/scripts/taler-monitoring/lib.sh
@ -1115,8 +1115,11 @@ find_wallet_cli() {
  local c cand
  for c in \
    "${HOME}/.local/bin/taler-wallet-cli.mjs" \
+    "${HOME}/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
+    "${HOME}/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
    "${HOME}/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs" \
    /home/hernani/taler/opt/taler-wallet-cli/usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
+    /home/hernani/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \
    /Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs \
    /usr/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs \
    /usr/share/taler-wallet-cli/bin/taler-wallet-cli.mjs
--- a/scripts/taler-monitoring/secrets.env.example
+++ b/scripts/taler-monitoring/secrets.env.example
@ -43,6 +43,14 @@
 # PROGRESS_SHOW_EVERY=8
 # PROGRESS_OFF=1

+# --- Ladder (GOA withdraw ladder · explorer pool) ---
+# Prefer SECRETS_ROOT / KOOPA_ADMIN_SECRETS (bank-explorer-password.txt).
+# Overrides only if needed:
+# EXP_USER=explorer
+# EXP_PW=
+# EXP_PW_FILE=$HOME/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-explorer-password.txt
+# CLI_JS=$HOME/taler/src/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs
+
 # Values that typically appear on a full stack (for orientation only):
 #   bank-admin-password.txt
 #   bank-explorer-password.txt