diff --git a/.gitignore b/.gitignore index 7a7b906..eb91f32 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,4 @@ secrets.env -out/ __pycache__/ *.pyc android-test/apks/ diff --git a/CLI-AUTOMATION-NOTES.md b/CLI-AUTOMATION-NOTES.md index 5a542b1..22f75e5 100644 --- a/CLI-AUTOMATION-NOTES.md +++ b/CLI-AUTOMATION-NOTES.md @@ -35,18 +35,6 @@ Legend: --- - -## 1a. Harness-style wallet daemon (mon ladder + e2e) - -| | | -|--|--| -| **Seen** | Ladder `OK_BANK` with `avail=0` while mon e2e / harness get full coins; force-select 400 without `selected_exchange` | -| **Area** | **cli** / **core** / **ops** | -| **Problem** | Harness uses a long-lived **WalletService** (unix socket client). One-shot `wallet-cli` starts and stops the shepherd each call — after bank `transfer_done`, coins often stay `pendingIncoming` until something keeps the wallet awake. `run-until-done` is banned (hangs). | -| **Workaround** | Mon: `advanced serve --unix-path $WSOCK` then `--wallet-connection=$WSOCK` (e2e default; ladder `LADDER_WALLET_SERVE=1` via `ladder/lib_wallet_serve.sh`). Force-select body must be `{"reserve_pub":"…","selected_exchange":""}`. Mint may include `"exchange_url":"$EX"`. Poll balance + bank status; never `run-until-done`. | -| **Wanted** | Documented wallet-cli “serve for automation” profile matching harness WalletClient. | - - ## 2. Withdraw completion is not observable from CLI alone | | | @@ -208,46 +196,6 @@ Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro | **Workaround** | Monitoring builds URI externally then `handle-uri --yes`. | | **Wanted** | Documented recipe only, or `wallet-cli testing pay-template --base=… --instance=… --id=…` for demos. | -### 9a. TESTPAYSAN public template POST `{}` → HTTP 451 (demo exchange) - -| | | -|--|--| -| **Seen** | 2026-08-30 magikoopa; `/monitoring_err` e2e.pay; `zero-pay` | -| **Area** | **ops** | -| **Problem** | `POST …/instances/jardin-du-creux/templates/oeufs-6` with `{}` → **451** code **2513** hard legal limits. `exchange_rejections` include **https://exchange.demo.taler.net/** currency mismatch. Merchant still trusts demo KUDOS, not `stage.exchange.lefrancpaysan.ch`. Sending `amount`/`summary` → **409** 2860/2861 (template already has contract). | -| **Workaround** | Empty POST is correct (same as shop-ui). Fix merchant: enable `[merchant-exchange-testpaysan]`, disable demo exchanges. Bank auto-confirm is a different hop (withdraw). | -| **Wanted** | Stage merchant only TESTPAYSAN exchange. | - -### 9b. stage.my has no `stickers` template - -| | | -|--|--| -| **Seen** | 2026-08-30 `zero-pay` `-d stage.taler-ops.ch` instance `admin` | -| **Area** | **ops** | -| **Problem** | Public POST `/templates/stickers` → **2018** template not found. Production `my.taler-ops.ch` instance `talersticker` works. Stage instance `tops` exists without that template. | -| **Workaround** | Create CHF:0 `stickers` template on stage.my (merchant token). Until then phase ERROR is correct. | -| **Wanted** | Same public template as prod on stage. | - -### 9c. public GET paid = `StatusPaidResponse` (no `paid` / `order_status`) - -| | | -|--|--| -| **Seen** | TESTPAYSAN token-pay 2026-08-30/31; e2e WARN not settled; body `{refunded:false, refund_taken:…}` | -| **Area** | **ops** | -| **Problem** | Suite required `paid` / `order_status` on **public** GET. Spec does not (`get-orders-ORDER_ID.rst` `StatusPaidResponse`: refund_* only). `order_status:"paid"` is **private** (`CheckPaymentPaidResponse`). | -| **Workaround** | `merchant_order_json_is_paid`: refunded/refund_taken, no `taler_pay_uri`. Do not trust `handle-uri` exit 0. | -| **Wanted** | wallet-cli non-zero on Payment expired/failed. | - -### 9d. Merchant `/keys` parse fail → 2010 then recover from DB - -| | | -|--|--| -| **Seen** | 2026-08-30 TESTPAYSAN `taler-merchant-httpd` after pasta hairpin + public `EXCHANGE_BASE_URL` | -| **Area** | **ops** / **core** | -| **Problem** | First fetch of `https://stage.exchange.lefrancpaysan.ch/keys` from inside the Quadlet failed (`version:0` / json_helper) → merchant **2010**. After keys-change notification, `/keys` loaded from DB; `GET /exchanges` then **http 200 ec 0** for the public URL. Leftover **list** rows still show demo.taler.net, taler-ops.ch, `http://127.0.0.1:9031/` (package kudos/tops + old DB). Config has **one** `[merchant-exchange-testpaysan]` HTTPS URL; kudos/chf `DISABLED=YES` in **overrides**, never `/usr/share`. | -| **Workaround** | Pasta `/etc/hosts` → `169.254.1.2` for `stage.{exchange,bank,monnaie}.lefrancpaysan.ch`. Persist `/data/etc/hosts-public-https`. Restart **httpd only**, not Quadlet. | -| **Wanted** | Merchant drop stale exchange list rows when `DISABLED=YES`; parse `/keys` error that names URL + HTTP status. | - --- ## 10. Amount / currency parsing edge cases @@ -305,23 +253,11 @@ Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro | **Seen** | e2e pay settle loops; ladder pay settle rounds | | **Area** | **cli** / **core** | | **Problem** | After `handle-uri` pay, success is “order paid” / balance drop / merchant order status — not one CLI wait. | -| **Workaround** | Short poll loops; never run-until-done. TESTPAYSAN token-pay: private merchant status, see §9c. | +| **Workaround** | Short poll loops; never run-until-done. | | **Wanted** | Same as §1–2: bounded wait on transaction id / order id. | --- -## 14b. Own sqlite wallet (Android / iOS export) - -| | | -|--|--| -| **Seen** | 2026-08-30 magikoopa; real CHF / TESTPAYSAN pay from a phone SQL dump | -| **Area** | **cli** / **ops** | -| **Problem** | Suite e2e/ladder/zero-pay used a fresh temp sqlite, so a funded export could not pay. | -| **Workaround** | `./taler-monitoring.sh --wallet-db PATH` (same flag as **taler-wallet-cli** / **taler-harness**). Native mobile sqlite: `--wallet-migrate` → `TALER_WALLET_MIGRATE_NATIVE_DB=1`. Not `*.json` (§15). zero-pay: `PARALLEL=1`. | -| **Wanted** | Keep harness flag names when adding wallet CLI to the suite. | - ---- - ## 15. `--wallet-db=*.json` is rejected (“memory backend not supported”) | | | diff --git a/README.md b/README.md index 2fb5ae1..2c08f4e 100644 --- a/README.md +++ b/README.md @@ -143,20 +143,6 @@ Each check prints **global** and **grouped** ids: # Admin credit for e2e: E2E_BANK_ADMIN_PASS=… or stage host secrets # (bank-admin-password.txt under /mnt/data/stagepaysan/bank/secrets/) # Override: INSIDE_SSH=… INSIDE_PROFILE=stage-lfp -# Live stack names + mini ladder: README_testpaysan.md (do not duplicate here). - -# Stage TOPS (default) · public-template CHF:0 — phase zero-pay -# (same HTTP path as e2e_one_pay_public_template; no merchant token) -./taler-monitoring.sh -d stage.taler-ops.ch zero-pay -COUNT=100 ./pay_talersticker_zero.sh -# TESTPAYSAN shop (empty POST {}; bank auto-confirm on stagepaysan) -./taler-monitoring.sh -d stage.lefrancpaysan.ch zero-pay -COUNT=100 GAP_S=0.5 ./pay_talersticker_zero.sh # 0.5s idle between pays/blocks -COUNT=100 PARALLEL=4 ./pay_talersticker_zero.sh # 4 wallets, 4 pays per block -COUNT=100 PARALLEL=10 GAP_S=0.5 ./pay_talersticker_zero.sh -COUNT=100 PARALLEL=all ./pay_talersticker_zero.sh # one block, COUNT wallets -# OUTDIR=/tmp/sticker-zero COUNT=3 ./pay_talersticker_zero.sh # smoke -# Prod vs stage.my, CHF:0, real-CHF mini: README_mytops.md (do not duplicate here). # Other domains — public HTTPS only (pass e2e/inside explicitly) ./taler-monitoring.sh -d taler.net diff --git a/README_mytops.md b/README_mytops.md deleted file mode 100644 index c577b1c..0000000 --- a/README_mytops.md +++ /dev/null @@ -1,23 +0,0 @@ -# mytops / TOPS (CHF) — deltas - -Generic CLI: [README.md](./README.md). **cwd = this tree.** Instance/template only via env (`INST` `TEMPLATE` `MERCHANT`) — not baked as suite defaults here. - -| `-d` | Money | -|------|--------| -| `stage.my.taler-ops.ch` / `stage.taler-ops.ch` | stage CHF | -| `my.taler-ops.ch` | **prod, real CHF** — on purpose only | - -```bash -./taler-monitoring.sh -d my.taler-ops.ch urls -./taler-monitoring.sh -d stage.my.taler-ops.ch urls -# CHF:0 / mini ladder: set INST TEMPLATE MERCHANT; LADDER_STEPS=5; cap LADDER_MAX_AMOUNT -# own sqlite: --wallet-db PATH [--wallet-migrate] (same flag as taler-wallet-cli / harness) -# Prod CHF:0 stickers instance is the one that exists on my.taler-ops.ch -# stage.my: that template is missing (HTTP 2018). Wrapper default COUNT=100 + stage.my → expect fail until template exists. -``` - -| | | -|--|--| -| last100tx | `paid=true` without deposit often omitted; amount-sort hides CHF:0 | -| Betel | SSH `taler-betel`, stage only | -| Lifeline | no login without Freigabe | diff --git a/README_testpaysan.md b/README_testpaysan.md deleted file mode 100644 index 764cff1..0000000 --- a/README_testpaysan.md +++ /dev/null @@ -1,25 +0,0 @@ -# TESTPAYSAN (FP stage) — deltas - -Generic CLI: [README.md](./README.md). **cwd = this tree.** Instance/template via env, not in this file. - -| | | -|--|--| -| `-d` | `stage.lefrancpaysan.ch` (same as `stage.monnaie…`) | -| Currency | TESTPAYSAN | -| Public | `https://stage.{bank,exchange,monnaie}.lefrancpaysan.ch` | -| Live UNIX | **`testpaysan`** · Quadlet **`stage-testpaysan`** · ports **9030–9032** · data **`/mnt/data/testpaysan`** | -| SSH (this laptop) | Host **`francpaysan`**, then `sudo -n -u testpaysan` | -| Stale | `stagepaysan` / `stage-lfp-*` / `francpaysan-stage-user` + `id_ed` — harness `-d` still prints those INSIDE_* flags; public URLs are correct | -| HALT | no Quadlet recreate of `stage-testpaysan`; no prod `*.lefrancpaysan.ch` without `stage.` | -| Config | persist under `/data/etc/…-overrides.conf`, never `/usr/share` | -| Pasta | guest `/etc/hosts` → `169.254.1.2` for those FQDNs | - -```bash -./taler-monitoring.sh -d stage.lefrancpaysan.ch urls -E2E_WALLET_SERVE=1 E2E_TIMEOUT=480 ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e -# mini ladder: LADDER_STEPS=5 (not 3). Pay instance = env LADDER_PAY_TEMPLATE_INSTANCE -COUNT=1 ./taler-monitoring.sh -d stage.lefrancpaysan.ch zero-pay # needs a funded wallet -# own sqlite (Android/iOS export): --wallet-db PATH [--wallet-migrate] -``` - -**Smoke 2026-08-30:** `urls` HTTPS OK; **3 ERROR** `stats.json` stuck **2026-08-29 13:36 CEST**. `zero-pay` COUNT=1 **FAIL** insufficient balance without prior `e2e` withdraw. diff --git a/TESTS.md b/TESTS.md index f7b199e..7364968 100644 --- a/TESTS.md +++ b/TESTS.md @@ -25,7 +25,7 @@ Every check line has a **global** run number and a **grouped** id: | **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` | | **server** | `check_server.sh` | (flat `server-NN` or host groups) | | **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `bankwd` `settle` `pay` `shop` `paivana` `dig` `report` | -| **ladder** | `check_amount_ladder.sh` + `ladder/lib_pay.sh` + `ladder/lib_wallet_serve.sh` | `plan` `load` `withdraw` `pay` `report` — classic / max-search; harness-style `advanced serve` (`LADDER_WALLET_SERVE=1`); phases `ladder` · `max-ladder` · `goa-ladder` (alias) | +| **ladder** | `check_amount_ladder.sh` + `ladder/lib_pay.sh` | `plan` `load` `withdraw` `pay` `report` — classic / max-search; phases `ladder` · `max-ladder` · `goa-ladder` (alias) | **Why groups:** flat `www-001`…`www-080` was hard to talk about. `www.bank-04` / `e2e.bankwd-02` pin the failure to a logical block. @@ -121,8 +121,7 @@ When filing an issue, quote the full id + label, e.g. | Area.group | Meaning | |------------|---------| | **versions.outside-** | deb.taler.net suite index | -| **versions.host-** | host-podman: packages via local `podman exec` (`INSIDE_PODMAN=1`). Stage-lfp uses `stage-lfp-*` names even in this mode (not GOA `taler-hacktivism*`) | -| **zero-pay.** | public-template pays (`check_zero_pay.sh`); COUNT=1 in phase | +| **versions.host-** | host-podman: packages via local `podman exec` (`INSIDE_PODMAN=1`) | | **versions.ssh-** | ssh: packages via `KOOPA_SSH` / `INSIDE_SSH` from laptop/remote | | **versions.compare-** | installed vs suite | | **aptdeploy.trixie-fresh-** | fresh rebuild container `…-apt-src-trixie` | diff --git a/VERSION b/VERSION index dd43a14..ca8ec41 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.26.1 +1.23.5 diff --git a/VERSIONS.md b/VERSIONS.md index cb1174e..1506e64 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -17,13 +17,6 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre | Tag | Date (UTC) | Notes | |-----|------------|--------| -| **v1.26.1** | 2026-08-31 | **Bugfix:** public order settle accepts `StatusPaidResponse` (`refunded`/`refund_taken`); `paid`/`order_status` not on public GET. | -| **v1.26.0** | 2026-08-30 | **Feature:** `--wallet-db PATH` / `--wallet-migrate` (taler-wallet-cli / taler-harness names) for e2e, ladder, zero-pay. TESTPAYSAN: one Quadlet `stage-testpaysan`; e2e token-pay via env `E2E_TOKEN_*` / `--choice-index`. Stack READMEs `README_testpaysan.md` / `README_mytops.md`. | -| **v1.24.0** | 2026-08-30 | **Feature / zero-pay:** public-template pays as phase `zero-pay` (`check_zero_pay.sh`). Default merchant **stage.my.taler-ops.ch** (CHF:0 stickers). Wrapper `pay_talersticker_zero.sh` COUNT=100. TESTPAYSAN: empty POST `{}` + auto-confirm of bank wires (host `auto-confirm-withdrawals`). Not in default 4h PHASES (use COUNT=1). | -| **v1.23.9** | 2026-08-30 | **Bugfix / versions:** `INSIDE_PODMAN=1` on stagepaysan still used GOA container names → empty `dpkg-query` → **versions.host-04** ERROR on `/monitoring_err`. Stage-lfp names apply in host-podman too; `run-fp-stage-monitoring.sh` exports `INSIDE_*_CTR`. | -| **v1.23.8** | 2026-08-29 | **Bugfix / ladder:** harness-style wallet daemon — `ladder/lib_wallet_serve.sh` (`advanced serve` + `--wallet-connection`, default `LADDER_WALLET_SERVE=1`). Mint sends `exchange_url`; settle/pay-wait skip one-shot `run-pending` when serve is up. Aligns mon ladder with taler-harness WalletService + e2e serve path (no run-until-done). CLI-AUTOMATION-NOTES §1a. | -| **v1.23.7** | 2026-07-22 | **FP stage host-agent:** ship `taler-monitoring-fp-stage.{service,timer}`; `install-host-agent.sh --fp-stage` (stagepaysan) enables FP mon only, disables GOA timers on that host, enforces suite tip knobs. Same sticky/publish pipeline as GOA. | -| **v1.23.6** | 2026-07-22 | **Strict suite tip for mon pages:** every host-agent run must `fetch`+`reset` to `origin/main` and **refuse** if HEAD ≠ remote tip (`SUITE_REQUIRE_REMOTE_TIP=1`). Overlay is refreshed **from** the tip after update (never re-applied over tip by default — was publishing stale site-gen). `git clean` after reset. Mon HTML still embeds that COMMIT in sticky/meta. | | **v1.23.5** | 2026-07-22 | **Bugfix / host-agent:** multi-agent shared `/var/www/monitoring-sites` — stop full-tree `rsync --delete` (wiped sibling pages e.g. `/monitoring-max-ladder/` after main/surface/aptdeploy runs). Publish **only this job’s** `HTML_OK_DIR`/`HTML_ERR_DIR` per host; `apply-monitoring-live.sh` merges without tree-delete. | | **v1.23.4** | 2026-07-19 | **Bugfix / mon:** shared **normalize_taler_uri** (strip taler:// host :443/:80); ladder/e2e/pay use it. **urls** scans bank+exchange+merchant intros + demo-withdraw/auto-account/stats for default ports (**ERROR** on GOA). Unit test tests/test_normalize_taler_uri.sh. | | **v1.23.3** | 2026-07-19 | **Bugfix:** payments stale because max-ladder never got coins — root cause on koopa was **taler-wallet-cli missing** (all mint-OK → SKIP_ACCEPT). Hard-fail ladder prereq if CLI/helper missing; settle keeps polling after transfer_done + short run-pending; pay wait default 180s; max-ladder wrapper sets PATH/WALLET_CLI. | diff --git a/check_amount_ladder.sh b/check_amount_ladder.sh index 36c0972..2f5da98 100755 --- a/check_amount_ladder.sh +++ b/check_amount_ladder.sh @@ -10,8 +10,8 @@ # Flow (bank landings): # 1) GET /intro/auto-account.json → personal *account-* (balance 0) # 2) Mint pool withdrawals as explorer + confirm when selected -# 3) wallet-cli accept-uri via harness-style advanced serve (no run-until-done) -# 4) bank confirm when selected; settle = balance + transfer_done (serve keeps shepherd) +# 3) wallet-cli accept-uri only (no run-until-done) +# 4) bank confirm when selected; settle = balance + transfer_done # 5) pay via free-amount template (GOA: goa-free) or private orders # # Modes (LADDER_MODE): @@ -89,9 +89,6 @@ elapsed_ms() { : "${LADDER_MAX_SEED:=}" # After withdraw, wait for spendable available>0 before pay (secs total; shows countdown) : "${LADDER_PAY_WAIT_AVAILABLE_S:=180}" -# Harness-style long-lived wallet daemon (advanced serve). 1=on (default). -# Without serve, coins often stay pendingIncoming after bank transfer_done. -: "${LADDER_WALLET_SERVE:=1}" # Historic libeufin-ish absolute ceiling used as default LADDER_MAX_AMOUNT on GOA LADDER_ABS_CEILING="4503599627370496" @@ -210,7 +207,7 @@ export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_HIGH_RUNGS LADDER export LADDER_FREE_TEMPLATE LADDER_FREE_TEMPLATE_INSTANCE LADDER_USE_FREE_TEMPLATE LADDER_PAY_WAIT_AVAILABLE_S SCRATCH=$(mktemp -d) -WDB=$(taler_mon_resolve_wallet_db "$SCRATCH/wallet.sqlite3") || exit 2 +WDB="$SCRATCH/wallet.sqlite3" REPORT_DIR="${LADDER_REPORT_DIR:-$SCRATCH}" mkdir -p "$REPORT_DIR" TSV="$REPORT_DIR/ladder-results.tsv" @@ -228,18 +225,13 @@ ladder_over() { } wcli() { - # Prefer harness-style serve socket (long-lived shepherd); else one-shot --wallet-db. - # PATH: taler-helper-sqlite3 needs Python ≥3.11 (Homebrew/local on macOS). + # taler-helper-sqlite3 is Python ≥3.11; prefer Homebrew/local python on macOS + # so Apple /usr/bin/python3 3.9 does not FATAL the sqlite backend (no reservePub). local _path="${PATH:-}" case ":${_path}:" in *:/opt/homebrew/bin:*) ;; - *) _path="/opt/homebrew/bin:/usr/local/bin:${HOME}/.local/bin:${_path}" ;; + *) _path="/opt/homebrew/bin:/usr/local/bin:${_path}" ;; esac - PATH="$_path" - if command -v wallet_serve_wcli >/dev/null 2>&1; then - wallet_serve_wcli "$@" - return $? - fi if [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then PATH="$_path" node "$CLI_JS" --wallet-db="$WDB" --no-throttle "$@" else @@ -489,8 +481,6 @@ source "$ROOT/metrics.sh" # Payment module (classic + max-search payable) # shellcheck source=ladder/lib_pay.sh source "$ROOT/ladder/lib_pay.sh" -# shellcheck source=ladder/lib_wallet_serve.sh -source "$ROOT/ladder/lib_wallet_serve.sh" METRICS_DIR="$REPORT_DIR" ALT_UNITS_FILE="${REPORT_DIR}/alt_unit_names.json" export METRICS_DIR CUR WDB CLI_JS ALT_UNITS_FILE @@ -588,31 +578,18 @@ wallet_prepare() { local label="${1:-wallet}" WDB="$SCRATCH/wallet-${label}.sqlite3" export WDB - # Start harness-style daemon once per DB (shepherd stays up for wirewatch settle). - if [ ! -f "$WDB" ] || [ -z "${WSERVE_PID:-}" ] || ! kill -0 "${WSERVE_PID:-0}" 2>/dev/null; then - wallet_serve_install_trap - if wallet_serve_start; then - if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then - ok "wallet serve" "pid=$WSERVE_PID sock=$WSOCK (harness-style; no run-until-done)" - fi - else - if [ "${LADDER_WALLET_SERVE:-1}" = "1" ]; then - warn wallet "wallet serve unavailable" "problem: advanced serve did not come up — falling back to one-shot CLI (coins may stay pendingIncoming). See $SCRATCH/wallet-serve.log" - fi - fi + if [ ! -f "$WDB" ]; then + wcli exchanges add "$EX" >"$SCRATCH/ex-add-$label.out" 2>&1 || true + wcli exchanges update "$EX" >"$SCRATCH/ex-upd-$label.out" 2>&1 || true + wcli exchanges accept-tos "$EX" >"$SCRATCH/ex-tos-$label.out" 2>&1 || true fi - # ToS / exchange — via serve socket when up - wcli exchanges add "$EX" >"$SCRATCH/ex-add-$label.out" 2>&1 || true - wcli exchanges update "$EX" >"$SCRATCH/ex-upd-$label.out" 2>&1 || true - wcli exchanges accept-tos "$EX" >"$SCRATCH/ex-tos-$label.out" 2>&1 || true } t0=$(now_ms) rm -f "$SCRATCH/wallet-main.sqlite3" -wallet_serve_stop 2>/dev/null || true wallet_prepare "main" ms_tos=$(elapsed_ms "$t0") -ok "wallet exchange + ToS (${ms_tos}ms) — cumulative DB for withdraw+pay (serve=${LADDER_WALLET_SERVE:-1})" +ok "wallet exchange + ToS (${ms_tos}ms) — cumulative DB for withdraw+pay (no run-until-done)" : "${LADDER_MAX_RUNGS:=99}" if [ "${LADDER_MODE}" = "max" ]; then @@ -687,11 +664,10 @@ ladder_withdraw_rung() { # mint from explorer pool t0=$(now_ms) - # e2e/harness: pass exchange_url so bank binds the op to our exchange code=$(curl -sS -m 30 -o "$SCRATCH/wd-$tag.json" -w '%{http_code}' \ -H "Authorization: Bearer ${TOK}" \ -H 'Content-Type: application/json' \ - -d "{\"amount\":\"${AMT}\",\"exchange_url\":\"${EX}\"}" \ + -d "{\"amount\":\"${AMT}\"}" \ "${BANK}/accounts/${EXP_USER}/withdrawals") ms_mint=$(elapsed_ms "$t0") WID=$(python3 -c 'import json;d=json.load(open("'"$SCRATCH"'/wd-'"$tag"'.json"));print(d.get("withdrawal_id") or "")' 2>/dev/null || true) @@ -984,14 +960,11 @@ sys.exit(0 if Decimal(sys.argv[1]) > Decimal(sys.argv[2]) else 1) | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("transfer_done"), d.get("status"))' 2>/dev/null || echo "?") if echo "$xfer" | grep -qi True; then bank_done=1 - # With harness-style serve, shepherd is already running — poll balance only. - # One-shot CLI: bounded run-pending (never run-until-done). + # nudge wallet to pick up wire (bounded; not run-until-done) wcli transactions >"$SCRATCH/tx-settle-$tag.out" 2>&1 || true - if [ -z "${WSERVE_PID:-}" ] || [ ! -S "${WSOCK:-}" ]; then - if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then - _to=$(command -v gtimeout 2>/dev/null || command -v timeout) - "$_to" 8 wcli advanced run-pending >"$SCRATCH/run-pending-settle.out" 2>&1 || true - fi + if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then + _to=$(command -v gtimeout 2>/dev/null || command -v timeout) + "$_to" 8 wcli advanced run-pending >"$SCRATCH/run-pending-settle.out" 2>&1 || true fi # after bank done, still prefer coins; allow remaining rounds for available note="bank transfer_done waiting coins avail=${after} $xfer" diff --git a/check_e2e.sh b/check_e2e.sh index 7596691..9e20c62 100755 --- a/check_e2e.sh +++ b/check_e2e.sh @@ -8,8 +8,8 @@ source "$ROOT/lib.sh" # shellcheck source=metrics.sh source "$ROOT/metrics.sh" -: "${E2E_TIMEOUT:=360}" # Bank withdraw + refresh/pay + shop sample + token-pay -: "${E2E_PAY_SECS:=60}" # handle-uri + coin refresh; macOS perl SIGALRM kills at this cap +: "${E2E_TIMEOUT:=180}" # Bank withdraw ladder + wirewatch lag needs room (was 55 — too tight) +: "${E2E_PAY_SECS:=22}" # hard reserve for handle-uri + settle (do not starve pay) : "${E2E_SETTLE_ROUNDS:=18}" : "${E2E_SETTLE_SLEEP:=3}" E2E_START=$(date +%s) @@ -307,7 +307,7 @@ PAY_AMT=$(printf '%s' "$PAY_LIST" | awk '{print $1}') if [ "$E2E_VARIABLE" = "1" ]; then n_w=$(printf '%s' "$WITHDRAW_LIST" | wc -w | tr -d ' ') n_p=$(printf '%s' "$PAY_LIST" | wc -w | tr -d ' ') - need=$(( 80 + n_w * 50 + n_p * 50 + 180 )) # shop sample + token-pay + refresh room + need=$(( 50 + n_w * 40 + n_p * 20 + 60 )) # + room for paivana pay if [ "${E2E_TIMEOUT}" -lt "$need" ]; then E2E_TIMEOUT=$need fi @@ -330,9 +330,8 @@ fi BANK_HOST=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$BANK_PUBLIC"'").hostname or "bank")' 2>/dev/null || echo bank) SCRATCH=$(mktemp -d) -WDB=$(taler_mon_resolve_wallet_db "$SCRATCH/wallet.sqlite3") || exit 2 -# macOS AF_UNIX path cap ~104 bytes. Do not use $TMPDIR (/var/folders/...). -WSOCK="/tmp/e2e-wcli-$$.sock" +WDB="$SCRATCH/wallet.sqlite3" +WSOCK="$SCRATCH/wallet.sock" WSERVE_PID="" # Client/server wallet (advanced serve): long-lived shepherd without run-until-done. # Default on — wallet-cli only completes withdraw/pay while a serve process is up. @@ -599,26 +598,20 @@ start_wallet_serve() { [ -n "${WALLET_CLI:-}" ] && [ -f "${WALLET_CLI}" ] || return 1 stop_wallet_serve rm -f "$WSOCK" - # helper shebang is /usr/bin/env python3 — macOS /usr/bin/python3 is 3.9 (FATAL). - export PATH="/opt/homebrew/bin:/usr/local/bin:${HOME}/.local/bin:${PATH}" - export PYTHON="${PYTHON:-$(command -v python3)}" # serve holds the DB open; clients use --wallet-connection only "$NODE_BIN" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults \ advanced serve --unix-path "$WSOCK" \ >"$SCRATCH/wallet-serve.log" 2>&1 & WSERVE_PID=$! local i - for i in $(seq 1 80); do - if kill -0 "$WSERVE_PID" 2>/dev/null && { [ -S "$WSOCK" ] || grep -q "serving at" "$SCRATCH/wallet-serve.log" 2>/dev/null; }; then + for i in $(seq 1 40); do + if [ -S "$WSOCK" ] && kill -0 "$WSERVE_PID" 2>/dev/null; then ok "wallet serve" "pid=$WSERVE_PID sock=$WSOCK (no run-until-done)" return 0 fi - if ! kill -0 "$WSERVE_PID" 2>/dev/null; then - break - fi sleep 0.15 done - warn "wallet serve" "socket not ready — $(head -c 180 "$SCRATCH/wallet-serve.log" 2>/dev/null | tr '\n' ' ')" + warn "wallet serve" "socket not ready — falling back to one-shot CLI (coins may stay pending)" stop_wallet_serve return 1 } @@ -665,10 +658,20 @@ wcli() { fi } -# Pay must not be starved: always keep the reserved handle-uri cap -# (leftover e2e budget used to shrink this to 12–14s → SIGALRM mid-pay). +# Pay must not be starved: use reserved seconds even if global budget is tight wcli_pay() { local cap=$E2E_PAY_SECS + local left + left=$(e2e_left) + left=$(printf '%s' "$left" | awk '/^[0-9]+$/{print; exit} {for(i=1;i<=NF;i++) if($i+0==$i){print $i; exit}}') + if [ -n "$left" ] && [ "$left" -gt "$cap" ] 2>/dev/null; then + cap=$E2E_PAY_SECS + elif [ -n "$left" ] && [ "$left" -ge 8 ] 2>/dev/null; then + cap=$left + else + # still try once with floor 12s so Alarm clock is not the blocker + cap=12 + fi info "pay wallet cap" "${cap}s" if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then _wcli_exec "$cap" --wallet-connection="$WSOCK" --no-throttle "$@" @@ -1200,7 +1203,16 @@ except Exception: print("") fi if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$tag.out" 2>/dev/null \ || grep -qiE 'done|paid|success|Payment' "$SCRATCH/pay-$tag.out" 2>/dev/null \ - || merchant_order_json_is_paid "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null; then + || python3 -c 'import json,sys +for p in sys.argv[1:]: + try: + d=json.load(open(p)) + if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid": + sys.exit(0) + except Exception: + pass +sys.exit(1) +' "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null; then ok "payment settled $PAY_AMT ($PAY_SUM · order $OID)" metrics_report_coins "after-pay-${tag}" || true metrics_record_flow spent "$PAY_AMT" || true @@ -1331,7 +1343,16 @@ except Exception: print("") "${MERCHANT_PUBLIC}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true fi curl -skS -m 8 -o "$SCRATCH/ord-pub-$tag.json" "$statusUrl" 2>/dev/null || true - if merchant_order_json_is_paid "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null \ + if python3 -c 'import json,sys +for p in sys.argv[1:]: + try: + d=json.load(open(p)) + if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid": + sys.exit(0) + except Exception: + pass +sys.exit(1) +' "$SCRATCH/ord-paid-$tag.json" "$SCRATCH/ord-pub-$tag.json" 2>/dev/null \ || grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$tag.out" 2>/dev/null \ || grep -qiE 'done|paid|success|Payment' "$SCRATCH/pay-$tag.out" 2>/dev/null; then ok "shop $pname" "payment settled ($pamt · order $OID)" @@ -1696,120 +1717,12 @@ else info "paivana summary" "${PAIVANA_REPORT:-?} · instance ${E2E_PAIVANA_INSTANCE}" fi -# --------------------------------------------------------------------------- -# Loyalty-token pays (generic). TESTPAYSAN default = stage token-order API. -# Override: E2E_TOKEN_ORDER_URL E2E_TOKEN_PRODUCT E2E_TOKEN_PAID_N E2E_TOKEN_FREE_N -# --------------------------------------------------------------------------- -if [ "${CUR:-}" = "TESTPAYSAN" ] && [ -z "${E2E_TOKEN_ENABLE:-}" ]; then - E2E_TOKEN_ENABLE=1 -fi -: "${E2E_TOKEN_ORDER_URL:=https://stage.monnaie.lefrancpaysan.ch/shops/api/kiosque-order}" -: "${E2E_TOKEN_PRODUCT:=prod_banana}" -: "${E2E_TOKEN_PAID_N:=2}" -: "${E2E_TOKEN_FREE_N:=1}" -TOKEN_OK=0 -TOKEN_OK_N=0 -TOKEN_FAIL_N=0 -TOKEN_REPORT="" - -e2e_one_token_order() { - local product="$1" choice="$2" tag="$3" - local body create_f payuri oid otok inst - body=$(python3 -c 'import json,sys; print(json.dumps({"product_id":sys.argv[1]}))' "$product") - create_f="$SCRATCH/token-$tag.json" - curl -skS -m 20 -o "$create_f" -X POST \ - -H 'Content-Type: application/json' -d "$body" \ - "$E2E_TOKEN_ORDER_URL" 2>"$SCRATCH/token-$tag.err" || true - oid=$(python3 -c 'import json,sys -try: - d=json.load(open(sys.argv[1])); print(d.get("order_id") or "") -except Exception: print("")' "$create_f" 2>/dev/null || true) - otok=$(python3 -c 'import json,sys -try: - d=json.load(open(sys.argv[1])); print(d.get("token") or "") -except Exception: print("")' "$create_f" 2>/dev/null || true) - payuri=$(python3 -c 'import json,sys -try: - d=json.load(open(sys.argv[1])); print((d.get("taler_pay_uri") or "").replace(":443","")) -except Exception: print("")' "$create_f" 2>/dev/null || true) - if [ -z "$oid" ] || [ -z "$payuri" ]; then - warn "token-pay $tag" "order POST failed — $(head -c 120 "$create_f" 2>/dev/null | tr '\n' ' ')" - return 1 - fi - info "token-pay $tag" "order $oid choice=$choice product=$product" - if ! wcli_pay handle-uri --yes --non-interactive --choice-index="$choice" "$payuri" \ - >"$SCRATCH/token-pay-$tag.out" 2>&1; then - warn "token-pay $tag" "handle-uri failed — $(tail -c 140 "$SCRATCH/token-pay-$tag.out" | tr '\n' ' ')" - return 1 - fi - # wallet-cli can print Payment expired/failed and still exit 0 - if grep -qiE 'Payment expired|Payment failed|Payment aborted|insufficient balance' \ - "$SCRATCH/token-pay-$tag.out" 2>/dev/null; then - warn "token-pay $tag" "handle-uri not paid — $(tail -c 140 "$SCRATCH/token-pay-$tag.out" | tr '\n' ' ')" - return 1 - fi - inst="${E2E_TOKEN_INSTANCE:-kiosque-banane}" - local r - for r in 1 2 3 4 5 6 8 10 12 15 18 20; do - if [ -n "$otok" ]; then - curl -skS -m 8 -o "$SCRATCH/token-st-$tag.json" \ - "${MERCHANT_PUBLIC}/instances/${inst}/orders/$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$oid")?token=$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$otok")" \ - 2>/dev/null || true - if merchant_order_json_is_paid "$SCRATCH/token-st-$tag.json" 2>/dev/null; then - ok "token-pay $tag" "paid choice=$choice oid=$oid" - return 0 - fi - fi - sleep 2 - done - warn "token-pay $tag" "not settled for order $oid — $(head -c 80 "$SCRATCH/token-st-$tag.json" 2>/dev/null | tr '\n' ' ')" - return 1 -} - -if [ "${E2E_TOKEN_ENABLE}" = "1" ]; then - set_group token - section "e2e · token pays (public order API)" - info "token-pay" "POST $E2E_TOKEN_ORDER_URL product=$E2E_TOKEN_PRODUCT paid_n=$E2E_TOKEN_PAID_N free_n=$E2E_TOKEN_FREE_N" - _ti=1 - while [ "$_ti" -le "$E2E_TOKEN_PAID_N" ]; do - if e2e_one_token_order "$E2E_TOKEN_PRODUCT" 0 "paid${_ti}"; then - TOKEN_OK_N=$((TOKEN_OK_N + 1)) - TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }paid${_ti}=OK" - else - TOKEN_FAIL_N=$((TOKEN_FAIL_N + 1)) - TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }paid${_ti}=FAIL" - fi - _ti=$((_ti + 1)) - done - _ti=1 - while [ "$_ti" -le "$E2E_TOKEN_FREE_N" ]; do - if e2e_one_token_order "$E2E_TOKEN_PRODUCT" 1 "free${_ti}"; then - TOKEN_OK_N=$((TOKEN_OK_N + 1)) - TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }free${_ti}=OK" - else - TOKEN_FAIL_N=$((TOKEN_FAIL_N + 1)) - TOKEN_REPORT="${TOKEN_REPORT:+$TOKEN_REPORT }free${_ti}=FAIL" - fi - _ti=$((_ti + 1)) - done - unset _ti - if [ "$TOKEN_FAIL_N" = "0" ] && [ "$TOKEN_OK_N" -gt 0 ]; then - TOKEN_OK=1 - PAY_OK=1 - PAY_OK_N=$((PAY_OK_N + TOKEN_OK_N)) - ok "token-pay summary" "$TOKEN_REPORT" - else - warn "token-pay summary" "$TOKEN_REPORT" - fi -fi - set_group report section "e2e · report" info "user" "$USER" info "Bank withdraws" "$WITHDRAW_REPORT" info "payments" "$PAY_REPORT" info "shop" "${SHOP_REPORT:-(n/a)}" -info "token-pay" "${TOKEN_REPORT:-(n/a)}" info "paivana" "${PAIVANA_REPORT:-(n/a)}" info "final balance" "$(fmt_bal "$SCRATCH/bal-live.out" 2>/dev/null || echo "(n/a)")" info "scratch" "$SCRATCH" diff --git a/check_inside.sh b/check_inside.sh index c1ba27a..38f145f 100755 --- a/check_inside.sh +++ b/check_inside.sh @@ -58,9 +58,9 @@ info "flags" "INSIDE_ACCESS=${INSIDE_ACCESS} INSIDE_PODMAN=${INSIDE_PODMAN:-0} I # --------------------------------------------------------------------------- if [ "$PROFILE" = "stage-lfp" ]; then SSH_HOST="${INSIDE_SSH:-}" - BANK_CTR="${INSIDE_BANK_CTR:-stage-testpaysan}" - EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" - MER_CTR="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" + BANK_CTR="${INSIDE_BANK_CTR:-stage-lfp-bank}" + EX_CTR="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}" + MER_CTR="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}" BANK_PORT="${INSIDE_BANK_PORT:-9032}" EX_PORT="${INSIDE_EXCHANGE_PORT:-9031}" MER_PORT="${INSIDE_MERCHANT_PORT:-9030}" @@ -101,9 +101,9 @@ hasp() { podman exec "$1" pgrep -f "$2" >/dev/null 2>&1; } BANK="${BANK_CTR}" EX="${EX_CTR}" MER="${MER_CTR}" -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$BANK" || BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*bank|lfp-bank' | head -1) -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$EX" || EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*exchange|lfp-exchange' | head -1) -podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$MER" || MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage-testpaysan|stage.*merchant|lfp-merchant' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$BANK" || BANK=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*bank|lfp-bank' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$EX" || EX=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*exchange|lfp-exchange' | head -1) +podman ps --format '{{.Names}}' 2>/dev/null | grep -qx "$MER" || MER=$(podman ps --format '{{.Names}}' 2>/dev/null | grep -iE 'stage.*merchant|lfp-merchant' | head -1) check_dns() { local comp="$1" ctr="$2" host="$3" diff --git a/check_versions.sh b/check_versions.sh index 8fbc2e6..3abc9ae 100755 --- a/check_versions.sh +++ b/check_versions.sh @@ -179,18 +179,13 @@ elif [ "${INSIDE_PROFILE:-}" = "stage-lfp" ] \ || [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ] \ || [ -n "${INSIDE_SSH:-}" ]; then _VERS_SSH_HOST="${INSIDE_SSH:-}" + _VERS_BANK="${INSIDE_BANK_CTR:-stage-lfp-bank}" + _VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-lfp-exchange-ansible}" + _VERS_MER="${INSIDE_MERCHANT_CTR:-stage-lfp-merchant}" elif [ "${LOCAL_STACK:-0}" = "1" ] && [ "${SKIP_SSH:-0}" != "1" ]; then # laptop → koopa via SSH (not host-podman) _VERS_SSH_HOST="${KOOPA_SSH}" fi -# Stage names independent of access mode. host-podman on stagepaysan used to keep -# GOA names → empty dpkg list → versions.host-04 ERROR (public /monitoring_err). -if [ "${INSIDE_PROFILE:-}" = "stage-lfp" ] \ - || [ "${EXPECT_CURRENCY:-}" = "TESTPAYSAN" ]; then - _VERS_BANK="${INSIDE_BANK_CTR:-stage-testpaysan}" - _VERS_EX="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" - _VERS_MER="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" -fi if [ "$_VERS_LOCAL" = "1" ]; then VERS_ACCESS=host-podman diff --git a/check_zero_pay.sh b/check_zero_pay.sh deleted file mode 100755 index fb53894..0000000 --- a/check_zero_pay.sh +++ /dev/null @@ -1,471 +0,0 @@ -#!/usr/bin/env bash -# Phase zero-pay: public-template pays (CHF:0 stickers / TESTPAYSAN shop). -# Same path as e2e_one_pay_public_template. Never run-until-done. -# Wrapper: pay_talersticker_zero.sh (COUNT default 100). -# -# Env: COUNT (phase default 1) GAP_S PARALLEL MERCHANT INST TEMPLATE AMOUNT -# EXCHANGE ZERO_PAY_POST=amount|empty HANDLE_SECS SETTLE_TRIES SETTLE_SLEEP -# -set -euo pipefail - -ROOT=$(cd "$(dirname "$0")" && pwd) -# shellcheck source=lib.sh -source "$ROOT/lib.sh" -# shellcheck source=metrics.sh -source "$ROOT/metrics.sh" -set_area zero-pay - -# Homebrew python3 (>=3.11) must beat /usr/bin/python3 3.9 — taler-helper-sqlite3. -export PATH="/opt/homebrew/bin:${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin:${PATH:-}" - -die() { echo "ERROR: $*" >&2; fail "zero-pay" "$*" 2>/dev/null || true; exit 1; } - -COUNT="${COUNT:-${ZERO_PAY_COUNT:-1}}" -GAP_S="${GAP_S:-0}" -PARALLEL="${PARALLEL:-1}" -MERCHANT="${MERCHANT:-${MERCHANT_PUBLIC:-https://stage.my.taler-ops.ch}}" -MERCHANT="${MERCHANT%/}" -CUR="${EXPECT_CURRENCY:-${CUR:-}}" -if [ "$CUR" = "TESTPAYSAN" ]; then - INST="${INST:-${ZERO_PAY_INST:-jardin-du-creux}}" - TEMPLATE="${TEMPLATE:-${ZERO_PAY_TEMPLATE:-oeufs-6}}" - AMOUNT="${AMOUNT:-${ZERO_PAY_AMOUNT:-TESTPAYSAN:5}}" - EXCHANGE="${EXCHANGE:-${EXCHANGE_PUBLIC:-https://stage.exchange.lefrancpaysan.ch/}}" - ZERO_PAY_POST="${ZERO_PAY_POST:-empty}" -else - INST="${INST:-${ZERO_PAY_INST:-admin}}" - TEMPLATE="${TEMPLATE:-${ZERO_PAY_TEMPLATE:-stickers}}" - AMOUNT="${AMOUNT:-${ZERO_PAY_AMOUNT:-CHF:0}}" - EXCHANGE="${EXCHANGE:-${EXCHANGE_PUBLIC:-https://exchange.stage.taler-ops.ch/}}" - ZERO_PAY_POST="${ZERO_PAY_POST:-amount}" -fi -EXCHANGE="${EXCHANGE%/}/" -HANDLE_SECS="${HANDLE_SECS:-25}" -SETTLE_TRIES="${SETTLE_TRIES:-40}" -SETTLE_SLEEP="${SETTLE_SLEEP:-0.15}" -case "$GAP_S" in - ''|*[!0-9.]*|*.*.*) die "GAP_S must be a non-negative number (got $GAP_S)" ;; -esac -case "$COUNT" in - ''|*[!0-9]*) die "COUNT must be a positive integer (got $COUNT)" ;; -esac -[ "$COUNT" -ge 1 ] || die "COUNT must be >= 1" -case "$PARALLEL" in - all|ALL) PARALLEL="$COUNT" ;; - ''|*[!0-9]*) die "PARALLEL must be a positive integer or all (got $PARALLEL)" ;; -esac -[ "$PARALLEL" -ge 1 ] || die "PARALLEL must be >= 1" -if [ "$PARALLEL" -gt "$COUNT" ]; then - PARALLEL="$COUNT" -fi - -OUTDIR="${OUTDIR:-/tmp/talersticker-zero-$(date +%Y%m%d-%H%M%S)}" -mkdir -p "$OUTDIR" -TSV="$OUTDIR/results.tsv" -LOG="$OUTDIR/run.log" -SCRATCH="$OUTDIR/scratch" -mkdir -p "$SCRATCH" -SERVE_PIDFILE="$OUTDIR/serve.pids" - -if [ -z "${TALER_MON_STATE:-}" ]; then - exec > >(tee -a "$LOG") 2>&1 -fi - -now_ms() { python3 -c 'import time; print(int(time.time()*1000))'; } - -if [ -n "${WALLET_DB:-}" ] && [ "$PARALLEL" -gt 1 ]; then - die "--wallet-db / WALLET_DB needs PARALLEL=1" -fi -slot_db() { - if [ -n "${WALLET_DB:-}" ] && [ "$1" = "0" ]; then - taler_mon_resolve_wallet_db "$OUTDIR/wallet-0.sqlite3" || exit 2 - else - echo "$OUTDIR/wallet-$1.sqlite3" - fi -} -slot_sock() { echo "$OUTDIR/wallet-$1.sock"; } -slot_log() { echo "$OUTDIR/wallet-serve-$1.log"; } - -stop_wallet_serve() { - if [ -f "$SERVE_PIDFILE" ]; then - while read -r pid; do - [ -n "$pid" ] || continue - kill "$pid" 2>/dev/null || true - wait "$pid" 2>/dev/null || true - done <"$SERVE_PIDFILE" - rm -f "$SERVE_PIDFILE" - fi - rm -f "$OUTDIR"/wallet-*.sock 2>/dev/null || true -} -trap 'stop_wallet_serve' EXIT - -_wcli_exec() { - local cap="$1" - shift - local -a cmd - cmd=( "$NODE_BIN" "$WALLET_CLI" "$@" ) - if command -v timeout >/dev/null 2>&1; then - timeout -k 2 "$cap" "${cmd[@]}" - elif command -v gtimeout >/dev/null 2>&1; then - gtimeout --kill-after=2 "$cap" "${cmd[@]}" - else - perl -e 'alarm shift; exec @ARGV' "$cap" "${cmd[@]}" - fi -} - -wcli_slot() { - local slot="$1" - local cap="$2" - shift 2 - local sock db - sock=$(slot_sock "$slot") - db=$(slot_db "$slot") - if [ -S "$sock" ]; then - _wcli_exec "$cap" --wallet-connection="$sock" --no-throttle "$@" - else - _wcli_exec "$cap" --wallet-db="$db" --no-throttle --skip-defaults "$@" - fi -} - -start_wallet_slot() { - local slot="$1" - local sock db log - sock=$(slot_sock "$slot") - db=$(slot_db "$slot") - log=$(slot_log "$slot") - rm -f "$sock" - "$NODE_BIN" "$WALLET_CLI" --wallet-db="$db" --no-throttle --skip-defaults \ - advanced serve --unix-path "$sock" \ - >"$log" 2>&1 & - local pid=$! - echo "$pid" >>"$SERVE_PIDFILE" - local i - for i in $(seq 1 50); do - if [ -S "$sock" ] && kill -0 "$pid" 2>/dev/null; then - info "wallet serve slot=$slot pid=$pid" - return 0 - fi - sleep 0.1 - done - echo "WARN: wallet serve slot=$slot socket not ready" - return 1 -} - -find_wallet_cli() { - if [ -n "${WALLET_CLI:-}" ] && [ -f "$WALLET_CLI" ]; then - printf '%s\n' "$WALLET_CLI" - return 0 - fi - local c - for c in \ - "${HOME}/src/llm-qwythos/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ - "${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ - "${HOME}/.local/bin/taler-wallet-cli.mjs" \ - "${HOME}/src/llm-qwythos/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli-bundled.cjs" - do - [ -f "$c" ] && { printf '%s\n' "$c"; return 0; } - done - return 1 -} - -NODE_BIN=$(command -v node) || die "node not found" -python3 - <<'PY' || die "python3 >= 3.11 required (taler-helper-sqlite3)" -import sys -raise SystemExit(0 if sys.version_info >= (3, 11) else 1) -PY -command -v taler-helper-sqlite3 >/dev/null || die "taler-helper-sqlite3 not on PATH" -WALLET_CLI=$(find_wallet_cli) || die "taler-wallet-cli .mjs not found (set WALLET_CLI=)" - -json_get() { - python3 -c 'import json,sys -p,k=sys.argv[1],sys.argv[2] -try: - d=json.load(open(p)) - v=d - for part in k.split("."): - if isinstance(v, dict): - v=v.get(part) - else: - v=None - break - print("" if v is None else v) -except Exception: - print("") -' "$@" -} - -create_template_order() { - local summary="$1" out="$2" - local body url - if [ "${ZERO_PAY_POST}" = "empty" ]; then - body='{}' - else - body=$(python3 -c 'import json,sys; print(json.dumps({"amount":sys.argv[1],"summary":sys.argv[2]}))' "$AMOUNT" "$summary") - fi - url="${MERCHANT}/instances/${INST}/templates/${TEMPLATE}" - if [ "$INST" = "admin" ] || [ "$INST" = "default" ] || [ -z "$INST" ]; then - url="${MERCHANT}/templates/${TEMPLATE}" - fi - curl -sS -m 20 -X POST -H "Content-Type: application/json" -d "$body" "$url" >"$out" -} - -order_public() { - local oid="$1" tok="$2" out="$3" - local qoid qtok base - qoid=$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$oid") - qtok=$(python3 -c 'import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=""))' "$tok") - if [ "$INST" = "admin" ] || [ "$INST" = "default" ] || [ -z "$INST" ]; then - base="${MERCHANT}/orders/${qoid}?token=${qtok}" - else - base="${MERCHANT}/instances/${INST}/orders/${qoid}?token=${qtok}" - fi - curl -sS -m 12 "$base" >"$out" -} - -order_is_paid() { - merchant_order_json_is_paid "$1" -} - -wallet_handle_done() { - grep -q "Finished with status .done." "$1" 2>/dev/null -} - -write_row() { - # n oid create handle settle total status payuri - printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$@" -} - -# One payment. Args: n slot. Always writes scratch/row.N.tsv and scratch/msg.N.txt. -do_one_pay() { - local n="$1" slot="$2" - local tag - tag=$(printf '%03d' "$n") - local nonce summary create_f status_f pay_f row_f msg_f - nonce="${n}-$(date +%s)-$RANDOM" - summary="taler-monitoring zero-pay #${n} ${nonce}" - create_f="$SCRATCH/ord-$tag.json" - status_f="$SCRATCH/st-$tag.json" - pay_f="$SCRATCH/pay-$tag.out" - row_f="$SCRATCH/row.$n.tsv" - msg_f="$SCRATCH/msg.$n.txt" - - local t0 t1 t_create t_h0 t_h1 t_s0 t_s1 - local create_ms handle_ms settle_ms total_ms - local oid tok payuri hint hc status - local settled merchant_paid i fail_tail - - t0=$(now_ms) - if ! create_template_order "$summary" "$create_f"; then - t1=$(now_ms) - create_ms=$((t1-t0)) - write_row "$n" "" "$create_ms" "" "" "$create_ms" "CREATE_HTTP_FAIL" "" >"$row_f" - echo "[$tag] CREATE_HTTP_FAIL ${create_ms}ms slot=$slot" >"$msg_f" - return 0 - fi - oid=$(json_get "$create_f" order_id) - tok=$(json_get "$create_f" token) - t_create=$(now_ms) - create_ms=$((t_create-t0)) - if [ -z "$oid" ] || [ -z "$tok" ]; then - hint=$(json_get "$create_f" hint) - write_row "$n" "" "$create_ms" "" "" "$create_ms" "CREATE_FAIL" "" >"$row_f" - echo "[$tag] CREATE_FAIL ${create_ms}ms hint=${hint} slot=$slot" >"$msg_f" - return 0 - fi - - order_public "$oid" "$tok" "$status_f" || true - payuri=$(json_get "$status_f" taler_pay_uri) - if [ -z "$payuri" ]; then - local mh - mh=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$MERCHANT"'").hostname or "")') - payuri="taler://pay/${mh}/instances/${INST}/${oid}/?c=${tok}" - fi - payuri=$(printf '%s' "$payuri" | sed 's/:443\//\//g; s/:443?/?/g') - - t_h0=$(now_ms) - set +e - wcli_slot "$slot" "$HANDLE_SECS" handle-uri --yes --non-interactive "$payuri" >"$pay_f" 2>&1 - hc=$? - set -e - t_h1=$(now_ms) - handle_ms=$((t_h1-t_h0)) - if grep -q "Payment failed" "$pay_f" 2>/dev/null; then - hc=1 - fi - - settled=0 - merchant_paid=0 - t_s0=$(now_ms) - i=1 - while [ "$i" -le "$SETTLE_TRIES" ]; do - order_public "$oid" "$tok" "$status_f" || true - if order_is_paid "$status_f"; then - merchant_paid=1 - settled=1 - break - fi - if wallet_handle_done "$pay_f"; then - settled=1 - fi - if [ "$settled" = "1" ] && [ "$i" -ge 3 ]; then - break - fi - sleep "$SETTLE_SLEEP" - i=$((i+1)) - done - t_s1=$(now_ms) - settle_ms=$((t_s1-t_s0)) - total_ms=$((t_s1-t0)) - - if [ "$merchant_paid" = "1" ] || [ "$settled" = "1" ]; then - status=PAID - elif [ "$hc" = "0" ]; then - status=HANDLE_OK_UNSETTLED - else - status=HANDLE_FAIL - fi - write_row "$n" "$oid" "$create_ms" "$handle_ms" "$settle_ms" "$total_ms" "$status" "$payuri" >"$row_f" - echo "[$tag] $status oid=$oid create=${create_ms}ms handle=${handle_ms}ms settle=${settle_ms}ms total=${total_ms}ms slot=$slot" >"$msg_f" - if [ "$status" != "PAID" ]; then - fail_tail=$(tail -c 180 "$pay_f" | tr '\n' ' ') - echo " handle-exit=${hc} tail=${fail_tail}" >>"$msg_f" - fi -} - -prep_slot_exchange() { - local slot="$1" - wcli_slot "$slot" 40 exchanges add "$EXCHANGE" >"$OUTDIR/exchanges-add-$slot.out" 2>&1 || true - wcli_slot "$slot" 40 exchanges tos "$EXCHANGE" >"$OUTDIR/exchanges-tos-$slot.out" 2>&1 || true - wcli_slot "$slot" 40 exchanges accept-tos "$EXCHANGE" >"$OUTDIR/exchanges-accept-tos-$slot.out" 2>&1 || true -} - -section "zero-pay · ${AMOUNT} x ${COUNT} · ${INST}/${TEMPLATE}" -info "merchant $MERCHANT" -info "instance $INST" -info "template $TEMPLATE amount=$AMOUNT" -info "wallet $WALLET_CLI" -info "python $(command -v python3) $(python3 -c 'import sys; print(".".join(map(str,sys.version_info[:3])))')" -info "helper $(command -v taler-helper-sqlite3)" -info "outdir $OUTDIR" -info "gap ${GAP_S}s between blocks (not in total_ms)" -info "parallel $PARALLEL per block (own wallet serve per slot)" -if [ "$PARALLEL" -gt 20 ]; then - echo "WARN: PARALLEL=$PARALLEL starts that many wallet-cli node processes" -fi - -wcli_slot 0 25 version >"$OUTDIR/wallet-version.txt" 2>&1 || true -info "wallet-cli $(head -1 "$OUTDIR/wallet-version.txt" 2>/dev/null || echo '?')" - -: >"$SERVE_PIDFILE" -slot=0 -while [ "$slot" -lt "$PARALLEL" ]; do - start_wallet_slot "$slot" || true - prep_slot_exchange "$slot" - slot=$((slot+1)) -done - -wcli_slot 0 20 balance >"$OUTDIR/balance-before.txt" 2>&1 || true -info "balance before: $(tr '\n' ' ' < "$OUTDIR/balance-before.txt" | tail -c 200)" - -printf 'n\torder_id\tcreate_ms\thandle_ms\tsettle_ms\ttotal_ms\tstatus\tpay_uri\n' >"$TSV" - -ok_n=0 -fail_n=0 -RUN_START_MS=$(now_ms) - -n=1 -while [ "$n" -le "$COUNT" ]; do - block_end=$((n + PARALLEL - 1)) - if [ "$block_end" -gt "$COUNT" ]; then - block_end="$COUNT" - fi - slot=0 - k="$n" - pay_pids="" - while [ "$k" -le "$block_end" ]; do - do_one_pay "$k" "$slot" & - pay_pids="$pay_pids $!" - slot=$((slot+1)) - k=$((k+1)) - done - set +e - for pid in $pay_pids; do - wait "$pid" - done - set -e - k="$n" - while [ "$k" -le "$block_end" ]; do - if [ -f "$SCRATCH/msg.$k.txt" ]; then - cat "$SCRATCH/msg.$k.txt" - else - echo "[$(printf '%03d' "$k")] MISSING (worker died)" - fi - if [ -f "$SCRATCH/row.$k.tsv" ]; then - cat "$SCRATCH/row.$k.tsv" >>"$TSV" - else - write_row "$k" "" "" "" "" "" "WORKER_DIED" "" >>"$TSV" - fi - k=$((k+1)) - done - n=$((block_end + 1)) - if [ "$n" -le "$COUNT" ]; then - case "$GAP_S" in - 0|0.0|0.00) ;; - *) sleep "$GAP_S" ;; - esac - fi -done - -RUN_END_MS=$(now_ms) -wall_ms=$((RUN_END_MS-RUN_START_MS)) - -ok_n=$(awk -F'\t' 'NR>1 && $7=="PAID" {c++} END {print c+0}' "$TSV") -fail_n=$((COUNT - ok_n)) - -section "zero-pay · summary" -info "ok=$ok_n fail=$fail_n count=$COUNT parallel=$PARALLEL gap_s=$GAP_S wall_ms=$wall_ms out=$TSV" -python3 - "$TSV" "$ok_n" "$fail_n" "$COUNT" "$wall_ms" "$OUTDIR/summary.txt" "$GAP_S" "$PARALLEL" <<'PY' -import sys -tsv, ok, fail, count, wall, outp, gap_s, parallel = sys.argv[1:] -ok, fail, count, wall = int(ok), int(fail), int(count), int(wall) -rows=[] -with open(tsv) as f: - next(f) - for line in f: - p=line.rstrip('\n').split('\t') - if len(p)<7: continue - rows.append(p) -paid=[r for r in rows if r[6]=='PAID'] -def nums(idx, rs): - xs=[int(r[idx]) for r in rs if r[idx].isdigit()] - return xs -def fmt(xs): - if not xs: return "n/a" - xs=sorted(xs) - med=xs[len(xs)//2] - return "n=%d min=%d med=%d max=%d mean=%.1f" % (len(xs), xs[0], med, xs[-1], sum(xs)/len(xs)) -lines=[] -lines.append("ok=%d fail=%d count=%d parallel=%s wall_ms=%d wall_s=%.1f gap_s=%s" % ( - ok, fail, count, parallel, wall, wall/1000.0, gap_s)) -lines.append("create_ms " + fmt(nums(2, paid))) -lines.append("handle_ms " + fmt(nums(3, paid))) -lines.append("settle_ms " + fmt(nums(4, paid))) -lines.append("total_ms " + fmt(nums(5, paid))) -if paid: - lines.append("first_paid_oid=%s last_paid_oid=%s" % (paid[0][1], paid[-1][1])) -text="\n".join(lines)+"\n" -open(outp,"w").write(text) -print(text) -PY - -wcli_slot 0 20 balance >"$OUTDIR/balance-after.txt" 2>&1 || true -info "balance after: $(tr '\n' ' ' < "$OUTDIR/balance-after.txt" | tail -c 200)" - -if [ "$ok_n" -lt "$COUNT" ]; then - fail "zero-pay" "ok=$ok_n fail=$fail_n count=$COUNT out=$TSV" - summary - exit 1 -fi -ok "zero-pay" "ok=$ok_n count=$COUNT wall_ms=$wall_ms out=$TSV" -summary -exit 0 diff --git a/host-agent/README.md b/host-agent/README.md index bb438d6..dde3b1c 100644 --- a/host-agent/README.md +++ b/host-agent/README.md @@ -4,9 +4,8 @@ | Behaviour | Default | |-----------|---------| -| Suite refresh | **`update-suite.sh` every run** → `git fetch --tags` + hard reset to `origin/main` + clean; **HEAD must equal remote tip** (`SUITE_REQUIRE_REMOTE_TIP=1`) before mon HTML is generated | +| Suite refresh | **`update-suite.sh` every run** → `git fetch --tags` + hard reset to `origin/main` | | Update strict | `SUITE_UPDATE_STRICT=1` (default): **abort** if fetch/reset fails — no silent old tree | -| Overlay | After tip reset, suite-overlay is **copied from** remote tip (not applied over it). Emergency only: `SUITE_ALLOW_STALE_OVERLAY=1` | | Wall clock | `RUN_TIMEOUT=600` | | Log | line-buffered `run-*.log` (continuous write) | | HTML | always (first run / empty tree too) + commit URL; **bootstrap install stubs are monpages ERROR** | @@ -20,7 +19,7 @@ Thin wrappers only set stack defaults: | Wrapper | Stack | Typical user | |---------|-------|----------------| | `run-hacktivism-monitoring.sh` | GOA `hacktivism.ch` | `hernani` @ koopa | -| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` — install: `install-host-agent.sh --fp-stage` (`taler-monitoring-fp-stage.timer`, 4h) | +| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` | | `run-host-report.sh` + `taler-monitoring-mytops-stage.{service,timer}` | mytops **stage / betel** only | `taler-monitoring` @ betel (4h) | | `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh $DEPLOY_SSH** | | `run-aptdeploy-monitoring.sh` | apt-src tests → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h) | diff --git a/host-agent/env/stagepaysan.env.example b/host-agent/env/stagepaysan.env.example index a2d01d7..abfcdc3 100644 --- a/host-agent/env/stagepaysan.env.example +++ b/host-agent/env/stagepaysan.env.example @@ -25,11 +25,7 @@ SUITE_DIR=$HOME/src/taler-monitoring SUITE_GIT_URL=https://git.hacktivism.ch/hernani/taler-monitoring.git SUITE_GIT_REF=main SUITE_UPDATE_MODE=reset -SUITE_UPDATE_STRICT=1 -# Mon HTML sticky must pin remote tip (same as GOA host-agent) -SUITE_REQUIRE_REMOTE_TIP=1 SOURCE_REPO_WEB=https://git.hacktivism.ch/hernani/taler-monitoring -SOURCE_REPO_GIT=https://git.hacktivism.ch/hernani/taler-monitoring.git FRANCPAYSAN_SECRETS=$HOME/francpaysan-secrets WALLET_CLI=$HOME/.local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs diff --git a/host-agent/install-host-agent.sh b/host-agent/install-host-agent.sh index a5d6e65..118ef7f 100755 --- a/host-agent/install-host-agent.sh +++ b/host-agent/install-host-agent.sh @@ -1,118 +1,20 @@ #!/usr/bin/env bash -# install-host-agent.sh — install user systemd mon units +# install-host-agent.sh — install hernani user systemd units on koopa # -# # GOA / koopa as hernani (default): +# # on koopa as hernani: # ~/src/taler-monitoring/host-agent/install-host-agent.sh -# # FrancPaysan stage as stagepaysan: -# ~/src/taler-monitoring/host-agent/install-host-agent.sh --fp-stage -# # from laptop: +# # or from laptop: # ./install-host-agent.sh --remote $DEPLOY_SSH -# ./install-host-agent.sh --remote $DEPLOY_SSH --fp-stage # # Suite tree is ONLY ~/src/taler-monitoring (standalone repo). No koopa-admin-log. -# Both stacks share run-host-report.sh → update-suite tip enforcement + per-dir publish. # set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) -REMOTE="" -STACK="${HOST_AGENT_STACK:-goa}" -while [ $# -gt 0 ]; do - case "$1" in - --remote) - REMOTE="${2:-${DEPLOY_SSH:-}}" - shift 2 - ;; - --fp-stage|--fp|--stagepaysan) - STACK=fp-stage - shift - ;; - --goa|goa) - STACK=goa - shift - ;; - *) - # bare host = remote (legacy) - if [ -z "$REMOTE" ] && [ "$1" != "--remote" ]; then - REMOTE="$1" - fi - shift - ;; - esac -done +REMOTE="${1:-}" +[ "${1:-}" = "--remote" ] && REMOTE="${2:-${DEPLOY_SSH:-}}" SUITE_URL="${SUITE_GIT_URL:-https://git.hacktivism.ch/hernani/taler-monitoring.git}" -install_fp_stage() { - local mon home_unit - mon=$(cd "$ROOT/.." && pwd) - if [ -d "$HOME/src/taler-monitoring/.git" ] || [ -d "$HOME/src/taler-monitoring" ]; then - mon="$HOME/src/taler-monitoring" - fi - home_unit="$HOME/.config/systemd/user" - mkdir -p "$home_unit" \ - "$HOME/.local/state/taler-monitoring-stage-lfp" \ - "$HOME/monitoring-sites-staging" \ - "$HOME/.config/taler-monitoring" \ - "$HOME/src" - - if [ ! -f "$HOME/.config/taler-monitoring/env" ] && [ -f "$ROOT/env/stagepaysan.env.example" ]; then - cp "$ROOT/env/stagepaysan.env.example" "$HOME/.config/taler-monitoring/env" - echo "created ~/.config/taler-monitoring/env from stagepaysan.env.example" - fi - # Ensure tip-enforcement knobs present (idempotent append if missing) - if [ -f "$HOME/.config/taler-monitoring/env" ]; then - grep -q '^SUITE_UPDATE_MODE=' "$HOME/.config/taler-monitoring/env" 2>/dev/null \ - || echo 'SUITE_UPDATE_MODE=reset' >>"$HOME/.config/taler-monitoring/env" - grep -q '^SUITE_UPDATE_STRICT=' "$HOME/.config/taler-monitoring/env" 2>/dev/null \ - || echo 'SUITE_UPDATE_STRICT=1' >>"$HOME/.config/taler-monitoring/env" - grep -q '^SUITE_REQUIRE_REMOTE_TIP=' "$HOME/.config/taler-monitoring/env" 2>/dev/null \ - || echo 'SUITE_REQUIRE_REMOTE_TIP=1' >>"$HOME/.config/taler-monitoring/env" - fi - - if [ ! -d "$HOME/src/taler-monitoring/.git" ]; then - echo "clone $SUITE_URL → $HOME/src/taler-monitoring" - git clone "$SUITE_URL" "$HOME/src/taler-monitoring" \ - || echo "WARN: clone failed — using tree at $mon if present" - fi - mon="$HOME/src/taler-monitoring" - ln -sfn "$mon" "$HOME/taler-monitoring" 2>/dev/null || true - ( - cd "$mon" - git fetch --tags --force --prune origin 2>/dev/null || true - git checkout -B main origin/main 2>/dev/null || git checkout main 2>/dev/null || true - git reset --hard origin/main 2>/dev/null || true - ) - chmod +x "$mon"/taler-monitoring.sh "$mon"/check_*.sh 2>/dev/null || true - chmod +x "$mon/host-agent"/*.sh 2>/dev/null || true - chmod +x "$mon/site-gen"/*.sh "$mon/site-gen/console_to_html.py" 2>/dev/null || true - - install -m 644 "$ROOT/taler-monitoring-fp-stage.service" "$home_unit/" - install -m 644 "$ROOT/taler-monitoring-fp-stage.timer" "$home_unit/" - chmod +x "$ROOT/run-fp-stage-monitoring.sh" 2>/dev/null || true - - # FP host must not run GOA timers (would write GOA mon dirs / wrong inventory) - for u in taler-monitoring-hacktivism taler-monitoring-surface \ - taler-monitoring-aptdeploy taler-monitoring-max-ladder \ - taler-monitoring-mail taler-monitoring-mattermost; do - systemctl --user disable --now "${u}.timer" 2>/dev/null || true - systemctl --user disable --now "${u}.path" 2>/dev/null || true - systemctl --user stop "${u}.service" 2>/dev/null || true - rm -f "$home_unit/${u}.service" "$home_unit/${u}.timer" "$home_unit/${u}.path" 2>/dev/null || true - done - - systemctl --user daemon-reload - systemctl --user enable --now taler-monitoring-fp-stage.timer - systemctl --user start --no-block taler-monitoring-fp-stage.service || true - - echo "--- FP stage status ---" - systemctl --user status taler-monitoring-fp-stage.timer --no-pager -l | head -12 || true - systemctl --user list-timers --all | grep taler-monitoring || true - echo "OK host-agent FP stage:" - echo " · fp-stage 4h → /monitoring on stage.{bank,exchange,monnaie}.lefrancpaysan.ch" - echo " · suite tip: $mon (SUITE_REQUIRE_REMOTE_TIP=1 via run-host-report)" - echo "manual: systemctl --user start taler-monitoring-fp-stage.service" -} - install_local() { local mon home_unit mon=$(cd "$ROOT/.." && pwd) @@ -213,7 +115,7 @@ install_local() { } if [ -n "$REMOTE" ] && [ "$REMOTE" != "--remote" ]; then - echo "install on $REMOTE (stack=$STACK) …" + echo "install on $REMOTE …" # ROOT = …/taler-monitoring/host-agent → suite root is parent rsync -az --delete \ --exclude secrets.env \ @@ -221,17 +123,8 @@ if [ -n "$REMOTE" ] && [ "$REMOTE" != "--remote" ]; then --exclude '.git' \ "$ROOT/../" \ "${REMOTE}:src/taler-monitoring/" - if [ "$STACK" = "fp-stage" ]; then - ssh -o BatchMode=yes "$REMOTE" \ - 'bash $HOME/src/taler-monitoring/host-agent/install-host-agent.sh --fp-stage' - else - ssh -o BatchMode=yes "$REMOTE" \ - 'bash $HOME/src/taler-monitoring/host-agent/install-host-agent.sh' - fi + ssh -o BatchMode=yes "$REMOTE" \ + 'bash $HOME/src/taler-monitoring/host-agent/install-host-agent.sh' else - if [ "$STACK" = "fp-stage" ]; then - install_fp_stage - else - install_local - fi + install_local fi diff --git a/host-agent/run-fp-stage-monitoring.sh b/host-agent/run-fp-stage-monitoring.sh index 58b4006..e7f95e8 100755 --- a/host-agent/run-fp-stage-monitoring.sh +++ b/host-agent/run-fp-stage-monitoring.sh @@ -14,10 +14,6 @@ export TALER_DOMAIN="${TALER_DOMAIN:-stage.lefrancpaysan.ch}" export INSIDE_PODMAN="${INSIDE_PODMAN:-1}" export INSIDE_MODE="${INSIDE_MODE:-local-podman}" export INSIDE_PROFILE="${INSIDE_PROFILE:-stage-lfp}" -# One Quadlet stack on testpaysan (not three stage-lfp-* names). -export INSIDE_BANK_CTR="${INSIDE_BANK_CTR:-stage-testpaysan}" -export INSIDE_EXCHANGE_CTR="${INSIDE_EXCHANGE_CTR:-stage-testpaysan}" -export INSIDE_MERCHANT_CTR="${INSIDE_MERCHANT_CTR:-stage-testpaysan}" export LOCAL_STACK="${LOCAL_STACK:-0}" export CONTINUE_ON_ERROR="${CONTINUE_ON_ERROR:-1}" # monpages obligatory (ERROR): only FP stage mon hosts below — never GOA inventory diff --git a/host-agent/run-host-report.sh b/host-agent/run-host-report.sh index 88bee07..13df1ca 100755 --- a/host-agent/run-host-report.sh +++ b/host-agent/run-host-report.sh @@ -152,38 +152,25 @@ UPD_LOG="$LOG_DIR/suite-update-${STAMP}.log" : >"$LOG" # --- suite auto-upgrade BEFORE the HTML log (git noise must not pollute mon pages) --- -# Always download newest taler-monitoring from Forgejo first. Mon HTML landing pages -# embed COMMIT/VERSION from this tree — never generate pages from a lagging clone. OVERLAY_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/suite-overlay" - -# Prefer update-suite from the live suite tree if present (may itself be slightly -# old until fetch); fall back to agent-bundled copy. -_upd_src="$AGENT_DIR/update-suite.sh" -if [ -f "${SUITE_DIR:-$HOME/src/taler-monitoring}/host-agent/update-suite.sh" ]; then - _upd_src="${SUITE_DIR:-$HOME/src/taler-monitoring}/host-agent/update-suite.sh" +if [ -f "$AGENT_MON/taler-monitoring.sh" ]; then + mkdir -p "$OVERLAY_DIR" + rsync -a \ + --exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \ + --exclude 'android-test/out*' --exclude '__pycache__' \ + "$AGENT_MON/" "$OVERLAY_DIR/" 2>/dev/null || true fi + # shellcheck source=update-suite.sh -if ! source "$_upd_src" >"$UPD_LOG" 2>&1; then +if ! source "$AGENT_DIR/update-suite.sh" >"$UPD_LOG" 2>&1; then echo "ERROR: suite auto-update failed — refusing to run with stale tree" >&2 echo " detail: $UPD_LOG" >&2 tail -n 30 "$UPD_LOG" >&2 || true echo " fix git/network or set SUITE_UPDATE_STRICT=0 only as emergency" >&2 exit 1 fi -unset _upd_src if [ -d "${SUITE_DIR:-}/host-agent" ]; then AGENT_DIR="${SUITE_DIR}/host-agent" - AGENT_MON="${SUITE_DIR}" -fi -# After tip reset: refresh suite-overlay FROM remote tip (never the reverse by default). -# Old behaviour rsynced local agent → overlay *before* fetch, then could re-apply a -# stale overlay over a fresh Forgejo tree and publish outdated mon-page generators. -if [ -f "${SUITE_DIR}/taler-monitoring.sh" ]; then - mkdir -p "$OVERLAY_DIR" - rsync -a --delete \ - --exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \ - --exclude 'android-test/out*' --exclude '__pycache__' \ - "${SUITE_DIR}/" "$OVERLAY_DIR/" 2>/dev/null || true fi # Now open the public HTML log (clean agent header + suite summary only) @@ -228,11 +215,9 @@ if [ -f "$OVERLAY_DIR/host-agent/run-host-report.sh" ] \ && grep -q '_wrap_keys\|_WRAP_SET_' "$OVERLAY_DIR/host-agent/run-host-report.sh" 2>/dev/null; then _need_overlay=1 fi -# Default: never re-apply overlay over a just-fetched remote tip (v1.23.6+). -# Emergency only: SUITE_ALLOW_STALE_OVERLAY=1 restores old "overlay wins" path. -if [ "$_need_overlay" = "1" ] && [ "${SUITE_ALLOW_STALE_OVERLAY:-0}" = "1" ] \ - && [ -f "$OVERLAY_DIR/taler-monitoring.sh" ]; then - echo "WARN: SUITE_ALLOW_STALE_OVERLAY=1 — re-applying suite-overlay over ${COMMIT_SHORT:-?}" >&2 +if [ "$_need_overlay" = "1" ] && [ -f "$OVERLAY_DIR/taler-monitoring.sh" ]; then + echo "WARN: Forgejo @ ${COMMIT_SHORT:-?} lagging local suite-overlay (phases=[$PHASES])" >&2 + echo " re-applying suite-overlay → $SUITE_MON" >&2 mkdir -p "$SUITE_MON" rsync -a \ --exclude secrets.env --exclude 'android-test/artifacts' --exclude 'android-test/apks' \ @@ -240,12 +225,11 @@ if [ "$_need_overlay" = "1" ] && [ "${SUITE_ALLOW_STALE_OVERLAY:-0}" = "1" ] \ "$OVERLAY_DIR/" "$SUITE_MON/" ROOT="$SUITE_MON" elif [ "$_need_overlay" = "1" ]; then - echo "WARN: suite @ ${COMMIT_SHORT:-?} may lack phase files for [$PHASES] — using remote tip anyway (push missing features; set SUITE_ALLOW_STALE_OVERLAY=1 only as emergency)" >&2 - ROOT="$SUITE_MON" - echo "using suite mon @ ${COMMIT_SHORT:-?} · $ROOT" + echo "WARN: Forgejo missing features and no suite-overlay — using agent mon $AGENT_MON" >&2 + ROOT="$AGENT_MON" else echo "using suite mon @ ${COMMIT_SHORT:-?} · $ROOT" - # Never rsync suite-overlay over a fresh Forgejo tree — that reintroduced + # Do NOT rsync suite-overlay over a fresh Forgejo tree — that reintroduced # stale site-gen/host-agent (bootstrap sticky, old monpages) on stage/betel. fi unset _need_overlay diff --git a/host-agent/taler-monitoring-fp-stage.service b/host-agent/taler-monitoring-fp-stage.service deleted file mode 100644 index bf38942..0000000 --- a/host-agent/taler-monitoring-fp-stage.service +++ /dev/null @@ -1,24 +0,0 @@ -[Unit] -Description=taler-monitoring FrancPaysan stage (urls/inside/versions/monpages/e2e) -Documentation=file:%h/src/taler-monitoring/host-agent/README.md -After=network-online.target -Wants=network-online.target - -[Service] -Type=oneshot -# Quote PHASES (spaces). Do not override PATH here — EnvironmentFile PATH= can break bash. -Environment="PHASES=urls inside versions monpages e2e" -Environment=RUN_TIMEOUT=1800 -Environment=E2E_TIMEOUT=900 -Environment=E2E_USE_TEMPLATES=1 -Environment=FRANCPAYSAN_SECRETS=%h/francpaysan-secrets -Environment=WALLET_CLI=%h/.local/lib/taler-wallet-cli/node_modules/taler-wallet-cli/bin/taler-wallet-cli.mjs -# Shared pipeline: fetch/reset suite tip (SUITE_REQUIRE_REMOTE_TIP) then mon HTML publish -EnvironmentFile=-%h/.config/taler-monitoring/env -WorkingDirectory=%h/src/taler-monitoring -ExecStart=/bin/bash %h/src/taler-monitoring/host-agent/run-fp-stage-monitoring.sh -Nice=10 -TimeoutStartSec=40min - -[Install] -WantedBy=default.target diff --git a/host-agent/taler-monitoring-fp-stage.timer b/host-agent/taler-monitoring-fp-stage.timer deleted file mode 100644 index e30d7c7..0000000 --- a/host-agent/taler-monitoring-fp-stage.timer +++ /dev/null @@ -1,11 +0,0 @@ -[Unit] -Description=taler-monitoring FP stage every 4h -Documentation=file:%h/src/taler-monitoring/host-agent/README.md - -[Timer] -OnBootSec=10min -OnUnitActiveSec=4h -Persistent=true - -[Install] -WantedBy=timers.target diff --git a/host-agent/update-suite.sh b/host-agent/update-suite.sh index 4f0d39b..9c5f3ea 100755 --- a/host-agent/update-suite.sh +++ b/host-agent/update-suite.sh @@ -76,9 +76,6 @@ case "$SUITE_UPDATE_MODE" in && ! git reset --hard FETCH_HEAD; then _upd_die "git reset --hard origin/$SUITE_GIT_REF failed" fi - # Drop untracked junk that could shadow suite files (never secrets). - git clean -fd -e secrets.env -e 'android-test/artifacts' -e 'android-test/apks' \ - -e 'android-test/out*' 2>/dev/null || true ;; pull) if ! git pull --ff-only origin "$SUITE_GIT_REF" 2>&1; then @@ -101,23 +98,6 @@ SOURCE_REPO_WEB="${SOURCE_REPO_WEB:-https://git.hacktivism.ch/hernani/taler-moni COMMIT_URL="${SOURCE_REPO_WEB}/src/commit/${COMMIT}" export COMMIT COMMIT_SHORT COMMIT_URL SOURCE_REPO_WEB -# Strict tip check (default): mon pages must be generated from remote tip, not a lagging clone. -# SUITE_REQUIRE_REMOTE_TIP=0 only for emergency offline/dev. -: "${SUITE_REQUIRE_REMOTE_TIP:=1}" -if [ "${SUITE_UPDATE_MODE}" != "keep" ] && [ "${SUITE_UPDATE_MODE}" != "skip" ] \ - && [ "${SUITE_UPDATE_MODE}" != "off" ] && [ "${SUITE_UPDATE_MODE}" != "none" ] \ - && [ "${SUITE_REQUIRE_REMOTE_TIP}" = "1" ]; then - _remote=$(git rev-parse "origin/${SUITE_GIT_REF}" 2>/dev/null || git rev-parse FETCH_HEAD 2>/dev/null || true) - if [ -z "$_remote" ]; then - _upd_die "cannot resolve origin/${SUITE_GIT_REF} after fetch — refuse mon-page generate on unknown tip" - elif [ "$COMMIT" != "$_remote" ]; then - _upd_die "suite HEAD ${COMMIT_SHORT} != origin/${SUITE_GIT_REF} $(git rev-parse --short=12 "$_remote" 2>/dev/null) — refuse stale mon pages" - else - echo " remote tip OK: HEAD == origin/${SUITE_GIT_REF} (${COMMIT_SHORT})" - fi - unset _remote -fi - # Release tag for sticky bar (exact tag if HEAD is tagged, else nearest) SUITE_VERSION="" if command -v git >/dev/null 2>&1; then diff --git a/ladder/README.md b/ladder/README.md index 171454f..3c6e826 100644 --- a/ladder/README.md +++ b/ladder/README.md @@ -30,16 +30,3 @@ There is **no** `check_goa_ladder.sh` — removed; use the phases above. | `LADDER_FREE_TEMPLATE` | `goa-free` | template id (hacktivism) | | `LADDER_FREE_TEMPLATE_INSTANCE` | merchant instance | e.g. `goa-demo-cp4zqk` | | `LADDER_PAY_WAIT_AVAILABLE_S` | `90` | after withdraw, wait for spendable **available**>0; logs **withdraw board in start order** (OK/OK_BANK) with live `pendingIncoming`→`done` + timer (same wallet DB) | - - -## Wallet daemon (harness-style) - -taler-harness keeps a **WalletService** up for the whole withdraw/confirm/wirewatch path. -Mon ladder defaults to the same idea: - -| Env | Default | Meaning | -|-----|---------|---------| -| `LADDER_WALLET_SERVE` | `1` | `advanced serve` + `--wallet-connection` (no `run-until-done`) | -| `LADDER_WALLET_SERVE=0` | — | one-shot CLI only (coins may stay `pendingIncoming` / `OK_BANK`) | - -Implementation: `ladder/lib_wallet_serve.sh` (shared with the e2e pattern). diff --git a/ladder/lib_pay.sh b/ladder/lib_pay.sh index c21d4d1..01ba819 100644 --- a/ladder/lib_pay.sh +++ b/ladder/lib_pay.sh @@ -371,14 +371,12 @@ ladder_wait_available() { info pay "timer 0s left — stop waiting (available still ${CUR}:0)" break fi - # serve: shepherd already running (harness-style). one-shot: bounded run-pending. + # nudge wallet (same DB); do not run-until-done (can hang) wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true - if [ -z "${WSERVE_PID:-}" ] || [ ! -S "${WSOCK:-}" ]; then - if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then - local _to - _to=$(command -v gtimeout 2>/dev/null || command -v timeout) - "$_to" 8 wcli advanced run-pending >"$SCRATCH/run-pending.out" 2>&1 || true - fi + if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then + local _to + _to=$(command -v gtimeout 2>/dev/null || command -v timeout) + "$_to" 8 wcli advanced run-pending >"$SCRATCH/run-pending.out" 2>&1 || true fi # progress board every ~5s (tick every 2s → every 3rd) if [ $((tick % 3)) -eq 0 ]; then diff --git a/ladder/lib_wallet_serve.sh b/ladder/lib_wallet_serve.sh deleted file mode 100644 index e38bfaa..0000000 --- a/ladder/lib_wallet_serve.sh +++ /dev/null @@ -1,156 +0,0 @@ -# ladder/lib_wallet_serve.sh — long-lived wallet daemon (harness-style) -# -# taler-harness uses WalletService + WalletClient over a unix socket so the -# wallet shepherd stays up while bank confirm + wirewatch complete. Mon e2e -# already mirrors that with `advanced serve`. Ladder historically used one-shot -# CLI only → coins often stayed pendingIncoming (OK_BANK) until pay-wait. -# -# Source after lib.sh. Callers set: SCRATCH, WDB, CLI_JS or WALLET_CLI, NODE_BIN. -# Optional: LADDER_WALLET_SERVE=1 (default), E2E_WALLET_SERVE (e2e alias). -# -# Public: -# wallet_serve_resolve_bins — NODE_BIN + CLI path -# wallet_serve_start — advanced serve --unix-path -# wallet_serve_stop — kill daemon -# wallet_serve_wcli args… — client via socket if up, else --wallet-db -# wallet_serve_install_trap — EXIT cleanup (idempotent) - -: "${LADDER_WALLET_SERVE:=1}" -# e2e reuses same default name -: "${E2E_WALLET_SERVE:=${LADDER_WALLET_SERVE}}" - -WSOCK="${WSOCK:-}" -WSERVE_PID="${WSERVE_PID:-}" -_WALLET_SERVE_TRAP_INSTALLED="${_WALLET_SERVE_TRAP_INSTALLED:-0}" - -wallet_serve_resolve_bins() { - # Prefer explicit WALLET_CLI / CLI_JS (.mjs); else find_wallet_cli from lib.sh - if [ -z "${CLI_JS:-}" ] || [ ! -f "${CLI_JS:-}" ]; then - if [ -n "${WALLET_CLI:-}" ] && [ -f "${WALLET_CLI}" ]; then - CLI_JS="$WALLET_CLI" - elif command -v find_wallet_cli >/dev/null 2>&1; then - CLI_JS=$(find_wallet_cli 2>/dev/null || true) - fi - fi - if [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then - WALLET_CLI="$CLI_JS" - fi - if [ -z "${NODE_BIN:-}" ] || [ ! -x "${NODE_BIN:-}" ]; then - NODE_BIN=$(command -v node 2>/dev/null || true) - fi - # PATH for taler-helper-sqlite3 (harness/e2e preflight) - local _path="${PATH:-}" - case ":${_path}:" in - *:/opt/homebrew/bin:*) ;; - *) _path="/opt/homebrew/bin:/usr/local/bin:${HOME}/.local/bin:${_path}" ;; - esac - PATH="$_path" - export PATH NODE_BIN CLI_JS WALLET_CLI -} - -wallet_serve_stop() { - if [ -n "${WSERVE_PID:-}" ] && kill -0 "$WSERVE_PID" 2>/dev/null; then - kill "$WSERVE_PID" 2>/dev/null || true - wait "$WSERVE_PID" 2>/dev/null || true - fi - WSERVE_PID="" - if [ -n "${WSOCK:-}" ]; then - rm -f "$WSOCK" 2>/dev/null || true - fi -} - -wallet_serve_start() { - # Returns 0 if serve is up (or intentionally disabled); 1 if wanted but failed. - wallet_serve_resolve_bins - local want="${LADDER_WALLET_SERVE:-1}" - # e2e callers may only set E2E_WALLET_SERVE - if [ "${E2E_WALLET_SERVE:-}" = "0" ]; then - want=0 - fi - if [ "$want" != "1" ]; then - return 0 - fi - if [ -z "${NODE_BIN:-}" ] || [ ! -x "${NODE_BIN}" ]; then - return 1 - fi - if [ -z "${WALLET_CLI:-}" ] || [ ! -f "${WALLET_CLI}" ]; then - return 1 - fi - if [ -z "${WDB:-}" ]; then - return 1 - fi - if [ -z "${SCRATCH:-}" ]; then - SCRATCH=$(mktemp -d) - fi - WSOCK="${WSOCK:-$SCRATCH/wallet.sock}" - wallet_serve_stop - rm -f "$WSOCK" - # harness: daemon owns DB; clients use --wallet-connection only (no concurrent --wallet-db) - "$NODE_BIN" "$WALLET_CLI" --wallet-db="$WDB" --no-throttle --skip-defaults \ - advanced serve --unix-path "$WSOCK" \ - >"${SCRATCH}/wallet-serve.log" 2>&1 & - WSERVE_PID=$! - local i - for i in $(seq 1 50); do - if [ -S "$WSOCK" ] && kill -0 "$WSERVE_PID" 2>/dev/null; then - return 0 - fi - # died early - if ! kill -0 "$WSERVE_PID" 2>/dev/null; then - break - fi - sleep 0.1 - done - wallet_serve_stop - return 1 -} - -wallet_serve_install_trap() { - [ "${_WALLET_SERVE_TRAP_INSTALLED}" = "1" ] && return 0 - _WALLET_SERVE_TRAP_INSTALLED=1 - # Append to existing EXIT trap if any — stop serve first - # shellcheck disable=SC2064 - trap 'wallet_serve_stop' EXIT -} - -# Run wallet-cli: prefer live serve socket (harness WalletClient path). -# Optional first arg: max seconds (integer) → wrapped with timeout when available. -wallet_serve_wcli() { - wallet_serve_resolve_bins - local maxc="" - if [[ "${1:-}" =~ ^[0-9]+$ ]]; then - maxc=$1 - shift - fi - local -a cmd - if [ -n "${NODE_BIN:-}" ] && [ -x "${NODE_BIN}" ] && [ -n "${WALLET_CLI:-}" ] && [ -f "${WALLET_CLI}" ]; then - if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ] && kill -0 "$WSERVE_PID" 2>/dev/null; then - cmd=( "$NODE_BIN" "$WALLET_CLI" --wallet-connection="$WSOCK" --no-throttle "$@" ) - else - cmd=( "$NODE_BIN" "$WALLET_CLI" --wallet-db="${WDB:?}" --no-throttle --skip-defaults "$@" ) - fi - elif [ -n "${CLI_JS:-}" ] && [ -f "$CLI_JS" ]; then - if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then - cmd=( node "$CLI_JS" --wallet-connection="$WSOCK" --no-throttle "$@" ) - else - cmd=( node "$CLI_JS" --wallet-db="${WDB:?}" --no-throttle "$@" ) - fi - else - if [ -n "${WSERVE_PID:-}" ] && [ -S "${WSOCK:-}" ]; then - cmd=( taler-wallet-cli --wallet-connection="$WSOCK" --no-throttle "$@" ) - else - cmd=( taler-wallet-cli --wallet-db="${WDB:?}" --no-throttle "$@" ) - fi - fi - if [ -n "$maxc" ]; then - if command -v timeout >/dev/null 2>&1; then - timeout -k 2 "$maxc" "${cmd[@]}" - elif command -v gtimeout >/dev/null 2>&1; then - gtimeout --kill-after=2 "$maxc" "${cmd[@]}" - else - "${cmd[@]}" - fi - else - "${cmd[@]}" - fi -} diff --git a/lib.sh b/lib.sh index eab70c7..9187afd 100755 --- a/lib.sh +++ b/lib.sh @@ -4,46 +4,6 @@ # Machine paths/SSH hosts: only from env (~/.config/taler-monitoring/env or # TALER_MONITORING_ENV=…), installed via taler-monitoring-env — never hardcode. -# Wallet sqlite path (same flag as taler-wallet-cli / taler-harness: --wallet-db). -# Android/iOS SQL export or any existing sqlite. Not *.json (memory backend). -taler_mon_resolve_wallet_db() { - local fallback="${1:-}" - local p="${WALLET_DB:-}" - if [ -z "$p" ]; then - printf '%s\n' "$fallback" - return 0 - fi - case "$p" in - *.json|*.JSON) - echo "ERROR: --wallet-db must be sqlite (.sqlite3 / .db), not .json" >&2 - return 2 - ;; - esac - printf '%s\n' "$p" -} - -# Merchant order JSON is paid? Public GET: StatusPaidResponse (refunded / refund_taken). -# Spec does not put paid / order_status there (those are private CheckPaymentPaidResponse). -merchant_order_json_is_paid() { - python3 -c ' -import json, sys -for p in sys.argv[1:]: - try: - d = json.load(open(p)) - except Exception: - continue - if not isinstance(d, dict): - continue - if d.get("paid") is True or str(d.get("order_status") or "").lower() == "paid": - raise SystemExit(0) - if d.get("taler_pay_uri"): - continue - if "refunded" in d or "refund_amount" in d or "refund_taken" in d: - raise SystemExit(0) -raise SystemExit(1) -' "$@" -} - # Load host env once (before any := defaults) if [ "${_TALER_MON_ENV_LOADED:-0}" != "1" ]; then _mon_cfg="${TALER_MONITORING_ENV:-${XDG_CONFIG_HOME:-$HOME/.config}/taler-monitoring/env}" @@ -106,8 +66,8 @@ MERCHANT_INSTANCE=$(printf '%s' "${MERCHANT_INSTANCE}" | tr '[:upper:]' '[:lower : "${PAY_AMT:=GOA:0.01}" : "${CREDIT_AMT:=GOA:4700}" # covers bank withdraw ladder 20+50+100+200+4200 (paivana) : "${TIMEOUT:=12}" -: "${E2E_TIMEOUT:=360}" # whole e2e budget; skip rest when exceeded (e2e raises as needed) -: "${E2E_PAY_SECS:=60}" # dedicated seconds for pay handle-uri (avoid SIGALRM) +: "${E2E_TIMEOUT:=55}" # whole e2e budget; skip rest when exceeded (e2e raises as needed) +: "${E2E_PAY_SECS:=22}" # dedicated seconds for pay handle-uri (avoid Alarm clock) # Whole-run wall clock for taler-monitoring.sh (seconds). 0 = unlimited. # Host-agent default phases (urls/inside/versions) should finish under this. : "${RUN_TIMEOUT:=600}" @@ -1605,7 +1565,6 @@ find_wallet_cli() { # shellcheck disable=SC2086 for c in ${WALLET_CLI_CANDIDATES:-} \ "${HOME}/.local/bin/taler-wallet-cli.mjs" \ - "${HOME}/src/llm-qwythos/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ "${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs" \ "${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli.mjs" \ "${HOME}/src/taler/taler-typescript-core/packages/taler-wallet-cli/dist/taler-wallet-cli-local.mjs" \ diff --git a/pay_talersticker_zero.sh b/pay_talersticker_zero.sh deleted file mode 100755 index 9040e7e..0000000 --- a/pay_talersticker_zero.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/usr/bin/env bash -# Thin wrapper → phase zero-pay (COUNT default 100). -# Default merchant: https://stage.my.taler-ops.ch (not production). -# COUNT=3 ./pay_talersticker_zero.sh -# ./pay_talersticker_zero.sh # 100 pays, stage.my stickers CHF:0 -# ./taler-monitoring.sh -d stage.lefrancpaysan.ch zero-pay -set -euo pipefail -ROOT=$(cd "$(dirname "$0")" && pwd) -export COUNT="${COUNT:-100}" -export TALER_DOMAIN="${TALER_DOMAIN:-stage.taler-ops.ch}" -exec "$ROOT/taler-monitoring.sh" -d "$TALER_DOMAIN" "$@" zero-pay diff --git a/taler-monitoring.sh b/taler-monitoring.sh index 90af7ac..5379f3f 100755 --- a/taler-monitoring.sh +++ b/taler-monitoring.sh @@ -93,8 +93,6 @@ Phases: sanity public + optional server server server-side only (SSH) e2e withdraw + pay (small amounts; remote aborts on login/KYC) - zero-pay public-template pays (stage.my CHF:0 stickers; TESTPAYSAN shop). COUNT=1 in phase; wrapper COUNT=100 - sticker-pay alias of zero-pay ladder amount ladder classic (default 23-rung withdraw + pay; any currency) max-ladder max-search: highest withdrawable + highest payable (≤ available) goa-ladder alias of ladder (legacy phase name; same check_amount_ladder.sh) @@ -125,10 +123,6 @@ Options: --exchange URL exchange base — overrides profile --merchant URL merchant-backend base — overrides profile --currency CODE expected currency (GOA, KUDOS, CHF, …); empty = report only - --wallet-db PATH sqlite wallet (taler-wallet-cli / taler-harness same flag). - Android/iOS SQL export or a test DB. Not *.json. - Env WALLET_DB=PATH. e2e / ladder / zero-pay (PARALLEL=1). - --wallet-migrate TALER_WALLET_MIGRATE_NATIVE_DB=1 (native mobile sqlite) --no-probe do not probe alternate hosts (unknown domains only) --lang en|fr|de UI/console language (sticky bar + badges; de available, not a profile default) overrides auto (FrancPaysan→fr, else en) @@ -272,18 +266,8 @@ while [ $# -gt 0 ]; do [ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; } CURRENCY_OVERRIDE="$2"; shift 2 ;; - --wallet-db) - [ $# -ge 2 ] || { echo "missing arg for $1" >&2; exit 2; } - WALLET_DB="$2" - export WALLET_DB - shift 2 - ;; - --wallet-migrate) - export TALER_WALLET_MIGRATE_NATIVE_DB=1 - shift - ;; --no-probe) NO_PROBE=1; shift ;; - urls|inside|versions|sanity|server|e2e|zero-pay|zeropay|sticker-pay|template-zero|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;; + urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;; *) # bare domain shorthand: ./taler-monitoring.sh taler.net if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then @@ -461,7 +445,6 @@ _progress_estimate_phase() { server) n=20 ;; # e2e: bank withdraw ladder emits many INFO lines (coins before/after each note) e2e) n=240 ;; - zero-pay|zeropay|sticker-pay|template-zero) n=15 ;; ladder|goa-ladder|max-ladder|maxladder) n=120 ;; auth401) n=70 ;; aptdeploy) n=40 ;; @@ -638,9 +621,6 @@ while [ "$_phase_idx" -lt "$_n_phases" ]; do sanity) run_phase sanity "$ROOT/check_sanity.sh" || ec=1 ;; server) run_phase server "$ROOT/check_server.sh" || ec=1 ;; e2e) run_phase e2e "$ROOT/check_e2e.sh" || ec=1 ;; - zero-pay|zeropay|sticker-pay|template-zero) - run_phase zero-pay "$ROOT/check_zero_pay.sh" || ec=1 - ;; ladder|goa-ladder) run_phase ladder "$ROOT/check_amount_ladder.sh" || ec=1 ;;