diff --git a/.gitignore b/.gitignore index eb91f32..7a7b906 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ secrets.env +out/ __pycache__/ *.pyc android-test/apks/ diff --git a/CLI-AUTOMATION-NOTES.md b/CLI-AUTOMATION-NOTES.md index 5529c87..5a542b1 100644 --- a/CLI-AUTOMATION-NOTES.md +++ b/CLI-AUTOMATION-NOTES.md @@ -228,6 +228,26 @@ Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro | **Workaround** | Create CHF:0 `stickers` template on stage.my (merchant token). Until then phase ERROR is correct. | | **Wanted** | Same public template as prod on stage. | +### 9c. public GET paid = `StatusPaidResponse` (no `paid` / `order_status`) + +| | | +|--|--| +| **Seen** | TESTPAYSAN token-pay 2026-08-30/31; e2e WARN not settled; body `{refunded:false, refund_taken:…}` | +| **Area** | **ops** | +| **Problem** | Suite required `paid` / `order_status` on **public** GET. Spec does not (`get-orders-ORDER_ID.rst` `StatusPaidResponse`: refund_* only). `order_status:"paid"` is **private** (`CheckPaymentPaidResponse`). | +| **Workaround** | `merchant_order_json_is_paid`: refunded/refund_taken, no `taler_pay_uri`. Do not trust `handle-uri` exit 0. | +| **Wanted** | wallet-cli non-zero on Payment expired/failed. | + +### 9d. Merchant `/keys` parse fail → 2010 then recover from DB + +| | | +|--|--| +| **Seen** | 2026-08-30 TESTPAYSAN `taler-merchant-httpd` after pasta hairpin + public `EXCHANGE_BASE_URL` | +| **Area** | **ops** / **core** | +| **Problem** | First fetch of `https://stage.exchange.lefrancpaysan.ch/keys` from inside the Quadlet failed (`version:0` / json_helper) → merchant **2010**. After keys-change notification, `/keys` loaded from DB; `GET /exchanges` then **http 200 ec 0** for the public URL. Leftover **list** rows still show demo.taler.net, taler-ops.ch, `http://127.0.0.1:9031/` (package kudos/tops + old DB). Config has **one** `[merchant-exchange-testpaysan]` HTTPS URL; kudos/chf `DISABLED=YES` in **overrides**, never `/usr/share`. | +| **Workaround** | Pasta `/etc/hosts` → `169.254.1.2` for `stage.{exchange,bank,monnaie}.lefrancpaysan.ch`. Persist `/data/etc/hosts-public-https`. Restart **httpd only**, not Quadlet. | +| **Wanted** | Merchant drop stale exchange list rows when `DISABLED=YES`; parse `/keys` error that names URL + HTTP status. | + --- ## 10. Amount / currency parsing edge cases @@ -285,11 +305,23 @@ Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro | **Seen** | e2e pay settle loops; ladder pay settle rounds | | **Area** | **cli** / **core** | | **Problem** | After `handle-uri` pay, success is “order paid” / balance drop / merchant order status — not one CLI wait. | -| **Workaround** | Short poll loops; never run-until-done. | +| **Workaround** | Short poll loops; never run-until-done. TESTPAYSAN token-pay: private merchant status, see §9c. | | **Wanted** | Same as §1–2: bounded wait on transaction id / order id. | --- +## 14b. Own sqlite wallet (Android / iOS export) + +| | | +|--|--| +| **Seen** | 2026-08-30 magikoopa; real CHF / TESTPAYSAN pay from a phone SQL dump | +| **Area** | **cli** / **ops** | +| **Problem** | Suite e2e/ladder/zero-pay used a fresh temp sqlite, so a funded export could not pay. | +| **Workaround** | `./taler-monitoring.sh --wallet-db PATH` (same flag as **taler-wallet-cli** / **taler-harness**). Native mobile sqlite: `--wallet-migrate` → `TALER_WALLET_MIGRATE_NATIVE_DB=1`. Not `*.json` (§15). zero-pay: `PARALLEL=1`. | +| **Wanted** | Keep harness flag names when adding wallet CLI to the suite. | + +--- + ## 15. `--wallet-db=*.json` is rejected (“memory backend not supported”) | | | diff --git a/README.md b/README.md index d472e79..2fb5ae1 100644 --- a/README.md +++ b/README.md @@ -143,6 +143,7 @@ Each check prints **global** and **grouped** ids: # Admin credit for e2e: E2E_BANK_ADMIN_PASS=… or stage host secrets # (bank-admin-password.txt under /mnt/data/stagepaysan/bank/secrets/) # Override: INSIDE_SSH=… INSIDE_PROFILE=stage-lfp +# Live stack names + mini ladder: README_testpaysan.md (do not duplicate here). # Stage TOPS (default) · public-template CHF:0 — phase zero-pay # (same HTTP path as e2e_one_pay_public_template; no merchant token) @@ -155,6 +156,7 @@ COUNT=100 PARALLEL=4 ./pay_talersticker_zero.sh # 4 wallets, 4 pays per COUNT=100 PARALLEL=10 GAP_S=0.5 ./pay_talersticker_zero.sh COUNT=100 PARALLEL=all ./pay_talersticker_zero.sh # one block, COUNT wallets # OUTDIR=/tmp/sticker-zero COUNT=3 ./pay_talersticker_zero.sh # smoke +# Prod vs stage.my, CHF:0, real-CHF mini: README_mytops.md (do not duplicate here). # Other domains — public HTTPS only (pass e2e/inside explicitly) ./taler-monitoring.sh -d taler.net diff --git a/README_mytops.md b/README_mytops.md new file mode 100644 index 0000000..c577b1c --- /dev/null +++ b/README_mytops.md @@ -0,0 +1,23 @@ +# mytops / TOPS (CHF) — deltas + +Generic CLI: [README.md](./README.md). **cwd = this tree.** Instance/template only via env (`INST` `TEMPLATE` `MERCHANT`) — not baked as suite defaults here. + +| `-d` | Money | +|------|--------| +| `stage.my.taler-ops.ch` / `stage.taler-ops.ch` | stage CHF | +| `my.taler-ops.ch` | **prod, real CHF** — on purpose only | + +```bash +./taler-monitoring.sh -d my.taler-ops.ch urls +./taler-monitoring.sh -d stage.my.taler-ops.ch urls +# CHF:0 / mini ladder: set INST TEMPLATE MERCHANT; LADDER_STEPS=5; cap LADDER_MAX_AMOUNT +# own sqlite: --wallet-db PATH [--wallet-migrate] (same flag as taler-wallet-cli / harness) +# Prod CHF:0 stickers instance is the one that exists on my.taler-ops.ch +# stage.my: that template is missing (HTTP 2018). Wrapper default COUNT=100 + stage.my → expect fail until template exists. +``` + +| | | +|--|--| +| last100tx | `paid=true` without deposit often omitted; amount-sort hides CHF:0 | +| Betel | SSH `taler-betel`, stage only | +| Lifeline | no login without Freigabe | diff --git a/README_testpaysan.md b/README_testpaysan.md new file mode 100644 index 0000000..764cff1 --- /dev/null +++ b/README_testpaysan.md @@ -0,0 +1,25 @@ +# TESTPAYSAN (FP stage) — deltas + +Generic CLI: [README.md](./README.md). **cwd = this tree.** Instance/template via env, not in this file. + +| | | +|--|--| +| `-d` | `stage.lefrancpaysan.ch` (same as `stage.monnaie…`) | +| Currency | TESTPAYSAN | +| Public | `https://stage.{bank,exchange,monnaie}.lefrancpaysan.ch` | +| Live UNIX | **`testpaysan`** · Quadlet **`stage-testpaysan`** · ports **9030–9032** · data **`/mnt/data/testpaysan`** | +| SSH (this laptop) | Host **`francpaysan`**, then `sudo -n -u testpaysan` | +| Stale | `stagepaysan` / `stage-lfp-*` / `francpaysan-stage-user` + `id_ed` — harness `-d` still prints those INSIDE_* flags; public URLs are correct | +| HALT | no Quadlet recreate of `stage-testpaysan`; no prod `*.lefrancpaysan.ch` without `stage.` | +| Config | persist under `/data/etc/…-overrides.conf`, never `/usr/share` | +| Pasta | guest `/etc/hosts` → `169.254.1.2` for those FQDNs | + +```bash +./taler-monitoring.sh -d stage.lefrancpaysan.ch urls +E2E_WALLET_SERVE=1 E2E_TIMEOUT=480 ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e +# mini ladder: LADDER_STEPS=5 (not 3). Pay instance = env LADDER_PAY_TEMPLATE_INSTANCE +COUNT=1 ./taler-monitoring.sh -d stage.lefrancpaysan.ch zero-pay # needs a funded wallet +# own sqlite (Android/iOS export): --wallet-db PATH [--wallet-migrate] +``` + +**Smoke 2026-08-30:** `urls` HTTPS OK; **3 ERROR** `stats.json` stuck **2026-08-29 13:36 CEST**. `zero-pay` COUNT=1 **FAIL** insufficient balance without prior `e2e` withdraw. diff --git a/VERSION b/VERSION index 53cc1a6..dd43a14 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.24.0 +1.26.1 diff --git a/VERSIONS.md b/VERSIONS.md index 0bfac91..cb1174e 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -17,6 +17,8 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre | Tag | Date (UTC) | Notes | |-----|------------|--------| +| **v1.26.1** | 2026-08-31 | **Bugfix:** public order settle accepts `StatusPaidResponse` (`refunded`/`refund_taken`); `paid`/`order_status` not on public GET. | +| **v1.26.0** | 2026-08-30 | **Feature:** `--wallet-db PATH` / `--wallet-migrate` (taler-wallet-cli / taler-harness names) for e2e, ladder, zero-pay. TESTPAYSAN: one Quadlet `stage-testpaysan`; e2e token-pay via env `E2E_TOKEN_*` / `--choice-index`. Stack READMEs `README_testpaysan.md` / `README_mytops.md`. | | **v1.24.0** | 2026-08-30 | **Feature / zero-pay:** public-template pays as phase `zero-pay` (`check_zero_pay.sh`). Default merchant **stage.my.taler-ops.ch** (CHF:0 stickers). Wrapper `pay_talersticker_zero.sh` COUNT=100. TESTPAYSAN: empty POST `{}` + auto-confirm of bank wires (host `auto-confirm-withdrawals`). Not in default 4h PHASES (use COUNT=1). | | **v1.23.9** | 2026-08-30 | **Bugfix / versions:** `INSIDE_PODMAN=1` on stagepaysan still used GOA container names → empty `dpkg-query` → **versions.host-04** ERROR on `/monitoring_err`. Stage-lfp names apply in host-podman too; `run-fp-stage-monitoring.sh` exports `INSIDE_*_CTR`. | | **v1.23.8** | 2026-08-29 | **Bugfix / ladder:** harness-style wallet daemon — `ladder/lib_wallet_serve.sh` (`advanced serve` + `--wallet-connection`, default `LADDER_WALLET_SERVE=1`). Mint sends `exchange_url`; settle/pay-wait skip one-shot `run-pending` when serve is up. Aligns mon ladder with taler-harness WalletService + e2e serve path (no run-until-done). CLI-AUTOMATION-NOTES §1a. | diff --git a/check_amount_ladder.sh b/check_amount_ladder.sh index c91528b..36c0972 100755 --- a/check_amount_ladder.sh +++ b/check_amount_ladder.sh @@ -210,7 +210,7 @@ export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_HIGH_RUNGS LADDER export LADDER_FREE_TEMPLATE LADDER_FREE_TEMPLATE_INSTANCE LADDER_USE_FREE_TEMPLATE LADDER_PAY_WAIT_AVAILABLE_S SCRATCH=$(mktemp -d) -WDB="$SCRATCH/wallet.sqlite3" +WDB=$(taler_mon_resolve_wallet_db "$SCRATCH/wallet.sqlite3") || exit 2 REPORT_DIR="${LADDER_REPORT_DIR:-$SCRATCH}" mkdir -p "$REPORT_DIR" TSV="$REPORT_DIR/ladder-results.tsv" diff --git a/check_e2e.sh b/check_e2e.sh index 9e20c62..7596691 100755 --- a/check_e2e.sh +++ b/check_e2e.sh @@ -8,8 +8,8 @@ source "$ROOT/lib.sh" # shellcheck source=metrics.sh source "$ROOT/metrics.sh" -: "${E2E_TIMEOUT:=180}" # Bank withdraw ladder + wirewatch lag needs room (was 55 — too tight) -: "${E2E_PAY_SECS:=22}" # hard reserve for handle-uri + settle (do not starve pay) +: "${E2E_TIMEOUT:=360}" # Bank withdraw + refresh/pay + shop sample + token-pay +: "${E2E_PAY_SECS:=60}" # handle-uri + coin refresh; macOS perl SIGALRM kills at this cap : "${E2E_SETTLE_ROUNDS:=18}" : "${E2E_SETTLE_SLEEP:=3}" E2E_START=$(date +%s) @@ -307,7 +307,7 @@ PAY_AMT=$(printf '%s' "$PAY_LIST" | awk '{print $1}') if [ "$E2E_VARIABLE" = "1" ]; then n_w=$(printf '%s' "$WITHDRAW_LIST" | wc -w | tr -d ' ') n_p=$(printf '%s' "$PAY_LIST" | wc -w | tr -d ' ') - need=$(( 50 + n_w * 40 + n_p * 20 + 60 )) # + room for paivana pay + need=$(( 80 + n_w * 50 + n_p * 50 + 180 )) # shop sample + token-pay + refresh room if [ "${E2E_TIMEOUT}" -lt "$need" ]; then E2E_TIMEOUT=$need fi @@ -330,8 +330,9 @@ fi BANK_HOST=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$BANK_PUBLIC"'").hostname or "bank")' 2>/dev/null || echo bank) SCRATCH=$(mktemp -d) -WDB="$SCRATCH/wallet.sqlite3" -WSOCK="$SCRATCH/wallet.sock" +WDB=$(taler_mon_resolve_wallet_db "$SCRATCH/wallet.sqlite3") || exit 2 +# macOS AF_UNIX path cap ~104 bytes. Do not use $TMPDIR (/var/folders/...). +WSOCK="/tmp/e2e-wcli-$$.sock" WSERVE_PID="" # Client/server wallet (advanced serve): long-lived shepherd without run-until-done. # Default on — wallet-cli only completes withdraw/pay while a serve process is up. @@ -598,20 +599,26 @@ start_wallet_serve() { [ -n "${WALLET_CLI:-}" ] && [ -f "${WALLET_CLI}" ] || return 1 stop_wallet_serve rm -f "$WSOCK" + # helper shebang is /usr/bin/env python3 — macOS /usr/bin/python3 is 3.9 (FATAL). + export PATH="/opt/homebrew/bin:/usr/local/bin:${HOME}/.local/bin:${PATH}" + export PYTHON="${PYTHON:-$(command -v python3)}" # serve holds the DB open; clients use --wallet-connection only "$NODE_BIN" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults \ advanced serve --unix-path "$WSOCK" \ >"$SCRATCH/wallet-serve.log" 2>&1 & WSERVE_PID=$! local i - for i in $(seq 1 40); do - if [ -S "$WSOCK" ] && kill -0 "$WSERVE_PID" 2>/dev/null; then + for i in $(seq 1 80); do + if kill -0 "$WSERVE_PID" 2>/dev/null && { [ -S "$WSOCK" ] || grep -q "serving at" "$SCRATCH/wallet-serve.log" 2>/dev/null; }; then ok "wallet serve" "pid=$WSERVE_PID sock=$WSOCK (no run-until-done)" return 0 fi + if ! kill -0 "$WSERVE_PID" 2>/dev/null; then + break + fi sleep 0.15 done - warn "wallet serve" "socket not ready — falling back to one-shot CLI (coins may stay pending)" + warn "wallet serve" "socket not ready — $(head -c 180 "$SCRATCH/wallet-serve.log" 2>/dev/null | tr '\n' ' ')" stop_wallet_serve return 1 } @@ -658,20 +665,10 @@ wcli() { fi } -# Pay must not be starved: use reserved seconds even if global budget is tight +# Pay must not be starved: always keep the reserved handle-uri cap +# (leftover e2e budget used to shrink this to 12–14s → SIGALRM mid-pay). wcli_pay() { local cap=$E2E_PAY_SECS - local left - left=$(e2e_left) - left=$(printf '%s' "$left" | awk '/^[0-9]+$/{print; exit} {for(i=1;i<=NF;i++) if($i+0==$i){print $i; exit}}') - if [ -n "$left" ] && [ "$left" -gt "$cap" ] 2>/dev/null; then - cap=$E2E_PAY_SECS - elif [ -n "$left" ] && [ "$left" -ge 8 ] 2>/dev/null; then - cap=$left - else - # still try once with floor 12s so Alarm clock is not the blocker - cap=12 - fi info "pay wallet cap" "${cap}s" if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then _wcli_exec "$cap" --wallet-connection="$WSOCK" --no-throttle "$@" @@ -1203,16 +1200,7 @@ except Exception: print("") fi if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$tag.out" 2>/dev/null \ || grep -qiE 'done|paid|success|Payment' "$SCRATCH/pay-$tag.out" 2>/dev/null \ - || python3 -c 'import json,sys -for p in sys.argv[1:]: - try: - d=json.load(open(p)) - if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid": - sys.exit(0) - except Exception: - pass -sys.exit(1) -' "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null; then + || merchant_order_json_is_paid "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null; then ok "payment settled $PAY_AMT ($PAY_SUM · order $OID)" metrics_report_coins "after-pay-${tag}" || true metrics_record_flow spent "$PAY_AMT" || true @@ -1343,16 +1331,7 @@ except Exception: print("") "${MERCHANT_PUBLIC}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true fi curl -skS -m 8 -o "$SCRATCH/ord-pub-$tag.json" "$statusUrl" 2>/dev/null || true - if python3 -c 'import json,sys -for p in sys.argv[1:]: - try: - d=json.load(open(p)) - if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid": - sys.exit(0) - except Exception: - pass -sys.exit(1) -' "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null \ + if merchant_order_json_is_paid "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null \ || grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$tag.out" 2>/dev/null \ || grep -qiE 'done|paid|success|Payment' "$SCRATCH/pay-$tag.out" 2>/dev/null; then ok "shop $pname" "payment settled ($pamt · order $OID)" @@ -1717,12 +1696,120 @@ else info "paivana summary" "${PAIVANA_REPORT:-?} · instance ${E2E_PAIVANA_INSTANCE}" fi +# --------------------------------------------------------------------------- +# Loyalty-token pays (generic). TESTPAYSAN default = stage token-order API. +# Override: E2E_TOKEN_ORDER_URL E2E_TOKEN_PRODUCT E2E_TOKEN_PAID_N E2E_TOKEN_FREE_N +# --------------------------------------------------------------------------- +if [ "${CUR:-}" = "TESTPAYSAN" ] && [ -z "${E2E_TOKEN_ENABLE:-}" ]; then + E2E_TOKEN_ENABLE=1 +fi +: "${E2E_TOKEN_ORDER_URL:=https://stage.monnaie.lefrancpaysan.ch/shops/api/kiosque-order}" +: "${E2E_TOKEN_PRODUCT:=prod_banana}" +: "${E2E_TOKEN_PAID_N:=2}" +: "${E2E_TOKEN_FREE_N:=1}" +TOKEN_OK=0 +TOKEN_OK_N=0 +TOKEN_FAIL_N=0 +TOKEN_REPORT="" + +e2e_one_token_order() { + local product="$1" choice="$2" tag="$3" + local body create_f payuri oid otok inst + body=$(python3 -c 'import json,sys; print(json.dumps({"product_id":sys.argv[1]}))' "$product") + create_f="$SCRATCH/token-$tag.json" + curl -skS -m 20 -o "$create_f" -X POST \ + -H 'Content-Type: application/json' -d "$body" \ + "$E2E_TOKEN_ORDER_URL" 2>"$SCRATCH/token-$tag.err" || true + oid=$(python3 -c 'import json,sys +try: + d=json.load(open(sys.argv[1])); print(d.get("order_id") or "") +except Exception: print("")' "$create_f" 2>/dev/null || true) + otok=$(python3 -c 'import json,sys +try: + d=json.load(open(sys.argv[1])); print(d.get("token") or "") +except Exception: print("")' "$create_f" 2>/dev/null || true) + payuri=$(python3 -c 'import json,sys +try: + d=json.load(open(sys.argv[1])); print((d.get("taler_pay_uri") or "").replace(":443","")) +except Exception: print("")' "$create_f" 2>/dev/null || true) + if [ -z "$oid" ] || [ -z "$payuri" ]; then + warn "token-pay $tag" "order POST failed — $(head -c 120 "$create_f" 2>/dev/null | tr '\n' ' ')" + return 1 + fi + info "token-pay $tag" "order $oid choice=$choice product=$product" + if ! wcli_pay handle-uri --yes --non-interactive --choice-index="$choice" "$payuri" \ + >"$SCRATCH/token-pay-$tag.out" 2>&1; then + warn "token-pay $tag" "handle-uri failed — $(tail -c 140 "$SCRATCH/token-pay-$tag.out" | tr '\n' ' ')" + return 1 + fi + # wallet-cli can print Payment expired/failed and still exit 0 + if grep -qiE 'Payment expired|Payment failed|Payment aborted|insufficient balance' \ + "$SCRATCH/token-pay-$tag.out" 2>/dev/null; then + warn "token-pay $tag" "handle-uri not paid — $(tail -c 140 "$SCRATCH/token-pay-$tag.out" | tr '\n' ' ')" + return 1 + fi + inst="${E2E_TOKEN_INSTANCE:-kiosque-banane}" + local r + for r in 1 2 3 4 5 6 8 10 12 15 18 20; do + if [ -n "$otok" ]; then + curl -skS -m 8 -o "$SCRATCH/token-st-$tag.json" \ + "${MERCHANT_PUBLIC}/instances/${inst}/orders/$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$oid")?token=$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$otok")" \ + 2>/dev/null || true + if merchant_order_json_is_paid "$SCRATCH/token-st-$tag.json" 2>/dev/null; then + ok "token-pay $tag" "paid choice=$choice oid=$oid" + return 0 + fi + fi + sleep 2 + done + warn "token-pay $tag" "not settled for order $oid — $(head -c 80 "$SCRATCH/token-st-$tag.json" 2>/dev/null | tr '\n' ' ')" + return 1 +} + +if [ "${E2E_TOKEN_ENABLE}" = "1" ]; then + set_group token + section "e2e · token pays (public order API)" + info "token-pay" "POST $E2E_TOKEN_ORDER_URL product=$E2E_TOKEN_PRODUCT paid_n=$E2E_TOKEN_PAID_N free_n=$E2E_TOKEN_FREE_N" + _ti=1 + while [ "$_ti" -le "$E2E_TOKEN_PAID_N" ]; do + if e2e_one_token_order "$E2E_TOKEN_PRODUCT" 0 "paid${_ti}"; then + TOKEN_OK_N=$((TOKEN_OK_N + 1)) + TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }paid${_ti}=OK" + else + TOKEN_FAIL_N=$((TOKEN_FAIL_N + 1)) + TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }paid${_ti}=FAIL" + fi + _ti=$((_ti + 1)) + done + _ti=1 + while [ "$_ti" -le "$E2E_TOKEN_FREE_N" ]; do + if e2e_one_token_order "$E2E_TOKEN_PRODUCT" 1 "free${_ti}"; then + TOKEN_OK_N=$((TOKEN_OK_N + 1)) + TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }free${_ti}=OK" + else + TOKEN_FAIL_N=$((TOKEN_FAIL_N + 1)) + TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }free${_ti}=FAIL" + fi + _ti=$((_ti + 1)) + done + unset _ti + if [ "$TOKEN_FAIL_N" = "0" ] && [ "$TOKEN_OK_N" -gt 0 ]; then + TOKEN_OK=1 + PAY_OK=1 + PAY_OK_N=$((PAY_OK_N + TOKEN_OK_N)) + ok "token-pay summary" "$TOKEN_REPORT" + else + warn "token-pay summary" "$TOKEN_REPORT" + fi +fi + set_group report section "e2e · report" info "user" "$USER" info "Bank withdraws" "$WITHDRAW_REPORT" info "payments" "$PAY_REPORT" info "shop" "${SHOP_REPORT:-(n/a)}" +info "token-pay" "${TOKEN_REPORT:-(n/a)}" info "paivana" "${PAIVANA_REPORT:-(n/a)}" info "final balance" "$(fmt_bal "$SCRATCH/bal-live.out" 2>/dev/null || echo "(n/a)")" info "scratch" "$SCRATCH" diff --git a/check_inside.sh b/check_inside.sh index 38f145f..c1ba27a 100755 --- a/check_inside.sh +++ b/check_inside.sh @@ -58,9 +58,9 @@ info "flags" "INSIDE_ACCESS=${INSIDE_ACCESS} INSIDE_PODMAN=${INSIDE_PODMAN:-0} I # --------------------------------------------------------------------------- if [ "$PROFILE" = "stage-lfp" ]; then SSH_HOST="${INSIDE_SSH:-}" - BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}" - EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}" - MER_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}" + BANK_CTR="${INSIDE_BANK_CTR:-stage-testpaysan}" + EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" + MER_CTR="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" BANK_PORT="${INSIDE_BANK_PORT:-9032}" EX_PORT="${INSIDE_EXCHANGE_PORT:-9031}" MER_PORT="${INSIDE_MERCHANT_PORT:-9030}" @@ -101,9 +101,9 @@ hasp() { podman exec "$1" pgrep -f "$2" >/dev/null 2>&1; } BANK="${BANK_CTR}" EX="${EX_CTR}" MER="${MER_CTR}" -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$BANK" || BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*bank|lfp-bank' | head -1) -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$EX" || EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*exchange|lfp-exchange' | head -1) -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$MER" || MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*merchant|lfp-merchant' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$BANK" || BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*bank|lfp-bank' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$EX" || EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*exchange|lfp-exchange' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$MER" || MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*merchant|lfp-merchant' | head -1) check_dns() { local comp="$1" ctr="$2" host="$3" diff --git a/check_versions.sh b/check_versions.sh index 24c20be..8fbc2e6 100755 --- a/check_versions.sh +++ b/check_versions.sh @@ -187,9 +187,9 @@ fi # GOA names → empty dpkg list → versions.host-04 ERROR (public /monitoring_err). if [ "${INSIDE_PROFILE:-}" = "stage-lfp" ] \ || [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ]; then - _VERS_BANK="${INSIDE_BANK_CTR:-stage-lfp-bank}" - _VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}" - _VERS_MER="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}" + _VERS_BANK="${INSIDE_BANK_CTR:-stage-testpaysan}" + _VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" + _VERS_MER="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" fi if [ "$_VERS_LOCAL" = "1" ]; then diff --git a/check_zero_pay.sh b/check_zero_pay.sh index 2c99066..fb53894 100755 --- a/check_zero_pay.sh +++ b/check_zero_pay.sh @@ -73,7 +73,16 @@ fi now_ms() { python3 -c 'import time; print(int(time.time()*1000))'; } -slot_db() { echo "$OUTDIR/wallet-$1.sqlite3"; } +if [ -n "${WALLET_DB:-}" ] && [ "$PARALLEL" -gt 1 ]; then + die "--wallet-db / WALLET_DB needs PARALLEL=1" +fi +slot_db() { + if [ -n "${WALLET_DB:-}" ] && [ "$1" = "0" ]; then + taler_mon_resolve_wallet_db "$OUTDIR/wallet-0.sqlite3" || exit 2 + else + echo "$OUTDIR/wallet-$1.sqlite3" + fi +} slot_sock() { echo "$OUTDIR/wallet-$1.sock"; } slot_log() { echo "$OUTDIR/wallet-serve-$1.log"; } @@ -214,17 +223,7 @@ order_public() { } order_is_paid() { - python3 -c 'import json,sys -try: - d=json.load(open(sys.argv[1])) -except Exception: - raise SystemExit(1) -if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid": - raise SystemExit(0) -if "refunded" in d or "refund_amount" in d: - raise SystemExit(0) -raise SystemExit(1) -' "$1" + merchant_order_json_is_paid "$1" } wallet_handle_done() { diff --git a/host-agent/run-fp-stage-monitoring.sh b/host-agent/run-fp-stage-monitoring.sh index 8968af7..58b4006 100755 --- a/host-agent/run-fp-stage-monitoring.sh +++ b/host-agent/run-fp-stage-monitoring.sh @@ -14,9 +14,10 @@ export TALER_DOMAIN="${TALER_DOMAIN:-stage.lefrancpaysan.ch}" export INSIDE_PODMAN="${INSIDE_PODMAN:-1}" export INSIDE_MODE="${INSIDE_MODE:-local-podman}" export INSIDE_PROFILE="${INSIDE_PROFILE:-stage-lfp}" -export INSIDE_BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}" -export INSIDE_EXCHANGE_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}" -export INSIDE_MERCHANT_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}" +# One Quadlet stack on testpaysan (not three stage-lfp-* names). +export INSIDE_BANK_CTR="${INSIDE_BANK_CTR:-stage-testpaysan}" +export INSIDE_EXCHANGE_CTR="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" +export INSIDE_MERCHANT_CTR="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" export LOCAL_STACK="${LOCAL_STACK:-0}" export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}" # monpages obligatory (ERROR): only FP stage mon hosts below — never GOA inventory diff --git a/lib.sh b/lib.sh index 0513d82..eab70c7 100755 --- a/lib.sh +++ b/lib.sh @@ -4,6 +4,46 @@ # Machine paths/SSH hosts: only from env (~/.config/taler-monitoring/env or # TALER_MONITORING_ENV=…), installed via taler-monitoring-env — never hardcode. +# Wallet sqlite path (same flag as taler-wallet-cli / taler-harness: --wallet-db). +# Android/iOS SQL export or any existing sqlite. Not *.json (memory backend). +taler_mon_resolve_wallet_db() { + local fallback="${1:-}" + local p="${WALLET_DB:-}" + if [ -z "$p" ]; then + printf '%s\n' "$fallback" + return 0 + fi + case "$p" in + *.json|*.JSON) + echo "ERROR: --wallet-db must be sqlite (.sqlite3 / .db), not .json" >&2 + return 2 + ;; + esac + printf '%s\n' "$p" +} + +# Merchant order JSON is paid? Public GET: StatusPaidResponse (refunded / refund_taken). +# Spec does not put paid / order_status there (those are private CheckPaymentPaidResponse). +merchant_order_json_is_paid() { + python3 -c ' +import json, sys +for p in sys.argv[1:]: + try: + d = json.load(open(p)) + except Exception: + continue + if not isinstance(d, dict): + continue + if d.get("paid") is True or str(d.get("order_status") or "").lower() == "paid": + raise SystemExit(0) + if d.get("taler_pay_uri"): + continue + if "refunded" in d or "refund_amount" in d or "refund_taken" in d: + raise SystemExit(0) +raise SystemExit(1) +' "$@" +} + # Load host env once (before any := defaults) if [ "${_TALER_MON_ENV_LOADED:-0}" != "1" ]; then _mon_cfg="${TALER_MONITORING_ENV:-${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env}" @@ -66,8 +106,8 @@ MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower : "${PAY_AMT:=GOA:0.01}" : "${CREDIT_AMT:=GOA:4700}" # covers bank withdraw ladder 20+50+100+200+4200 (paivana) : "${TIMEOUT:=12}" -: "${E2E_TIMEOUT:=55}" # whole e2e budget; skip rest when exceeded (e2e raises as needed) -: "${E2E_PAY_SECS:=22}" # dedicated seconds for pay handle-uri (avoid Alarm clock) +: "${E2E_TIMEOUT:=360}" # whole e2e budget; skip rest when exceeded (e2e raises as needed) +: "${E2E_PAY_SECS:=60}" # dedicated seconds for pay handle-uri (avoid SIGALRM) # Whole-run wall clock for taler-monitoring.sh (seconds). 0 = unlimited. # Host-agent default phases (urls/inside/versions) should finish under this. : "${RUN_TIMEOUT:=600}" diff --git a/pay_talersticker_zero.sh b/pay_talersticker_zero.sh index ae231de..9040e7e 100755 --- a/pay_talersticker_zero.sh +++ b/pay_talersticker_zero.sh @@ -8,4 +8,4 @@ set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) export COUNT="${COUNT:-100}" export TALER_DOMAIN="${TALER_DOMAIN:-stage.taler-ops.ch}" -exec "$ROOT/taler-monitoring.sh" -d "$TALER_DOMAIN" zero-pay +exec "$ROOT/taler-monitoring.sh" -d "$TALER_DOMAIN" "$@" zero-pay diff --git a/taler-monitoring.sh b/taler-monitoring.sh index 58a4929..90af7ac 100755 --- a/taler-monitoring.sh +++ b/taler-monitoring.sh @@ -125,6 +125,10 @@ Options: --exchange URL exchange base — overrides profile --merchant URL merchant-backend base — overrides profile --currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only + --wallet-db PATH sqlite wallet (taler-wallet-cli / taler-harness same flag). + Android/iOS SQL export or a test DB. Not *.json. + Env WALLET_DB=PATH. e2e / ladder / zero-pay (PARALLEL=1). + --wallet-migrate TALER_WALLET_MIGRATE_NATIVE_DB=1 (native mobile sqlite) --no-probe do not probe alternate hosts (unknown domains only) --lang en|fr|de UI/console language (sticky bar + badges; de available, not a profile default) overrides auto (FrancPaysan→fr, else en) @@ -268,6 +272,16 @@ while [ $# -gt 0 ]; do [ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; } CURRENCY_OVERRIDE="$2"; shift 2 ;; + --wallet-db) + [ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; } + WALLET_DB="$2" + export WALLET_DB + shift 2 + ;; + --wallet-migrate) + export TALER_WALLET_MIGRATE_NATIVE_DB=1 + shift + ;; --no-probe) NO_PROBE=1; shift ;; urls|inside|versions|sanity|server|e2e|zero-pay|zeropay|sticker-pay|template-zero|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;; *)