From 76a253613a91b637fedbc3948ef6d0550960776c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Sun, 19 Jul 2026 14:25:24 +0200 Subject: [PATCH] release 1.22.0: modular ladder withdraw + pay max-search Split pay into ladder/lib_pay.sh; force-select extract in ladder/extract_rpubs.py. check_goa_ladder.sh is only a compat shim. max-ladder now hunts highest withdrawable and highest payable (never above wallet available), using public free-amount template goa-free on hacktivism. Classic remains default 23-rung wd+pay. Pay phase always runs when LADDER_PAY=1 (visible SKIP_BALANCE when no spendable coins). Report includes best_pay / alt names. --- README.md | 6 +- VERSION | 2 +- VERSIONS.md | 3 +- check_amount_ladder.sh | 320 +++------------- check_goa_ladder.sh | 4 +- ladder/README.md | 25 ++ .../extract_rpubs.py | 0 ladder/lib_pay.sh | 361 ++++++++++++++++++ taler-monitoring.sh | 2 +- 9 files changed, 449 insertions(+), 274 deletions(-) create mode 100644 ladder/README.md rename ladder_extract_rpubs.py => ladder/extract_rpubs.py (100%) create mode 100644 ladder/lib_pay.sh diff --git a/README.md b/README.md index d8cf093..5af4d7f 100644 --- a/README.md +++ b/README.md @@ -277,7 +277,7 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC. # Amount ladder (any currency): classic 0 → … → max, then optional pay ./taler-monitoring.sh ladder # classic (current domain) ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima -# Max-search: start high, narrow toward highest withdrawable +# Max-search: highest withdrawable + highest payable (≤ available; GOA free template goa-free) ./taler-monitoring.sh max-ladder # Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01). # Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=). @@ -396,10 +396,12 @@ LADDER_LOAD=0 ./taler-monitoring.sh ladder | `METRICS_LOAD` | `1` | `0` = skip all host/container load probes | | `LADDER_LOAD` | `1` | `0` = skip load in ladder only | | `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws | -| `LADDER_MODE` | `classic` | default **classic** 23-rung; `max` = hunt highest withdrawable (`max-ladder`) | +| `LADDER_MODE` | `classic` | default **classic** 23-rung wd+pay; `max` = max-search wd+pay (`max-ladder`) | | `LADDER_MAX_PROBES` | `32` | max-search probe budget; `0` = until converge / timeout | | `LADDER_HIGH_RUNGS` | `6` | max-search: first N probes random-high, then bisect | | `LADDER_MAX_TOL_REL` | `0.001` | max-search relative lo/hi gap to stop | +| `LADDER_FREE_TEMPLATE` | `goa-free` | public free-amount pay template (hacktivism) | +| `LADDER_USE_FREE_TEMPLATE` | `1` | create pays via template POST `{"amount":…}` | | `LADDER_WITHDRAW_SCALE` | `1.5` | classic: withdraw mids / pay mids ratio (funding headroom) | | `LADDER_STEPS` | `23` | classic: rungs for withdraw and pay (0 + mids + max−1 + max) | | `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python | diff --git a/VERSION b/VERSION index 3500250..57807d6 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.21.0 +1.22.0 diff --git a/VERSIONS.md b/VERSIONS.md index f8d32cd..2f327d9 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -17,7 +17,8 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre | Tag | Date (UTC) | Notes | |-----|------------|--------| -| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external `ladder_extract_rpubs.py` (no stdin SyntaxError); force-select multi-rpub + raw_decode; 409 tries next rpub; soft `CEILING_REJECT` on mid/high mint 5110/P0001; report python never aborts ladder; helper PATH python≥3.11; wallet-db not `*.json`. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. | +| **v1.22.0** | 2026-07-19 | **Feature:** modular amount ladder — `ladder/lib_pay.sh` (pay) + withdraw in `check_amount_ladder.sh`; `ladder/extract_rpubs.py`. **max-ladder** hunts highest **withdrawable** and highest **payable** (≤ wallet available). GOA pays use public free-amount template **`goa-free`** (`LADDER_FREE_TEMPLATE`). Classic still default 23-rung wd+pay. `check_goa_ladder.sh` remains a thin compat shim only. | +| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external extract (no stdin SyntaxError); force-select multi-rpub; soft `CEILING_REJECT` on mint 5110/P0001; crash-proof report. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. | | **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. *(Superseded notes folded into **v1.21.0** feature release.)* | | **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. | | **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. | diff --git a/check_amount_ladder.sh b/check_amount_ladder.sh index 9c7e2c1..a4ad861 100755 --- a/check_amount_ladder.sh +++ b/check_amount_ladder.sh @@ -11,24 +11,26 @@ # 4) bank confirm when selected; settle = balance + transfer_done # # Modes (LADDER_MODE): -# classic — random strictly increasing rungs + pins 0, max-1, max (default) -# max — free max-search: start randomly high, narrow toward highest -# amount that mint+accept+confirm (+ bank transfer_done) succeeds +# classic — random strictly increasing rungs + pins 0, max-1, max (default 23) +# max — max-search: highest withdrawable AND highest payable +# (pay never exceeds wallet available; GOA uses public free-amount +# template goa-free by default) # -# Phase A: withdraw. Phase B: pay (classic only by default; off in max mode). +# Phase A: withdraw. Phase B: pay (classic plan list, or max-search pay). # # Stacks auto-tune via bank /config max_wire + exchange min denom when possible. # # Usage: # ./taler-monitoring.sh ladder -# ./taler-monitoring.sh max-ladder # LADDER_MODE=max +# ./taler-monitoring.sh max-ladder # LADDER_MODE=max (wd + pay hunt) # ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder -# LADDER_MODE=max LADDER_MAX_PROBES=16 ./taler-monitoring.sh ladder +# LADDER_MODE=max LADDER_MAX_PROBES=32 ./taler-monitoring.sh ladder # LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # # Env: LADDER_MODE, LADDER_STEPS, LADDER_MAX_AMOUNT, LADDER_MIN_AMOUNT, # LADDER_MAX_PROBES, LADDER_MAX_TOL_REL, LADDER_HIGH_RUNGS, LADDER_STACK_AUTO, -# LADDER_PAY, EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, … +# LADDER_PAY, LADDER_FREE_TEMPLATE, LADDER_FREE_TEMPLATE_INSTANCE, +# EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, … set -euo pipefail ROOT=$(cd "$(dirname "$0")" && pwd) # shellcheck source=lib.sh @@ -71,7 +73,12 @@ elapsed_ms() { # Public variable template for stage pays (optional; fixed templates used when amount maps) : "${LADDER_PAY_TEMPLATE:=}" : "${LADDER_PAY_TEMPLATE_INSTANCE:=fermes-des-collines}" -# classic | max (max = find highest withdrawable; phase alias max-ladder) +# Free-amount public template (hacktivism: goa-free on goa-demo-cp4zqk — any GOA amount) +: "${LADDER_FREE_TEMPLATE:=goa-free}" +: "${LADDER_FREE_TEMPLATE_INSTANCE:=}" +# 1 = create pays via public free template POST (amount in body); 0 = private orders only +: "${LADDER_USE_FREE_TEMPLATE:=1}" +# classic | max (max = highest withdrawable + highest payable; phase alias max-ladder) : "${LADDER_MODE:=classic}" # max-search: probe budget (0 = run until convergence / LADDER_TIMEOUT_S only) : "${LADDER_MAX_PROBES:=32}" @@ -80,6 +87,8 @@ elapsed_ms() { : "${LADDER_HIGH_RUNGS:=6}" # optional seed for reproducible max-search (empty = time-based) : "${LADDER_MAX_SEED:=}" +# After withdraw, wait for spendable available>0 before pay (secs total) +: "${LADDER_PAY_WAIT_AVAILABLE_S:=60}" # Historic libeufin-ish absolute ceiling used as default LADDER_MAX_AMOUNT on GOA LADDER_ABS_CEILING="4503599627370496" @@ -180,13 +189,10 @@ case "${LADDER_MODE}" in LADDER_MODE=classic ;; esac -# max mode: pay ladder off unless caller explicitly set LADDER_PAY=1 *and* LADDER_PAY_WITH_MAX=1 -if [ "$LADDER_MODE" = "max" ]; then - if [ "${LADDER_PAY_WITH_MAX:-0}" != "1" ]; then - LADDER_PAY=0 - fi -fi +# max mode: withdraw + pay hunts (LADDER_PAY=0 still disables pay entirely) +: "${LADDER_FREE_TEMPLATE_INSTANCE:=${MERCHANT_INSTANCE}}" export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_HIGH_RUNGS LADDER_MAX_SEED LADDER_PAY +export LADDER_FREE_TEMPLATE LADDER_FREE_TEMPLATE_INSTANCE LADDER_USE_FREE_TEMPLATE LADDER_PAY_WAIT_AVAILABLE_S SCRATCH=$(mktemp -d) WDB="$SCRATCH/wallet.sqlite3" @@ -435,6 +441,9 @@ PY # shellcheck source=metrics.sh source "$ROOT/metrics.sh" +# Payment module (classic + max-search payable) +# shellcheck source=ladder/lib_pay.sh +source "$ROOT/ladder/lib_pay.sh" METRICS_DIR="$REPORT_DIR" ALT_UNITS_FILE="${REPORT_DIR}/alt_unit_names.json" export METRICS_DIR CUR WDB CLI_JS ALT_UNITS_FILE @@ -540,8 +549,9 @@ ok "wallet exchange + ToS (${ms_tos}ms) — cumulative DB for withdraw+pay (no r : "${LADDER_MAX_RUNGS:=99}" if [ "${LADDER_MODE}" = "max" ]; then - info "ladder mode" "max-search (find highest withdrawable ${CUR})" + info "ladder mode" "max-search (highest withdrawable + highest payable ${CUR})" info "max-search params" "probes≤${LADDER_MAX_PROBES} high_rand=${LADDER_HIGH_RUNGS} tol_rel=${LADDER_MAX_TOL_REL} ceiling=${CUR}:${LADDER_MAX_AMOUNT}" + info "max-search pay" "LADDER_PAY=${LADDER_PAY} free_template=${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE:-$INST}" LADDER_LIST="" PAY_LIST="" LADDER_N=0 @@ -757,8 +767,8 @@ ladder_withdraw_rung() { # and regex on *both* accept and transactions output (reservePub lives in tx, not accept). extract_rpubs_for_wid() { # External module — never feed extract via bash heredoc (refactor tools - # previously turned Python continue into return → SyntaxError line 91). - local _ext="$ROOT/ladder_extract_rpubs.py" + # lives in ladder/extract_rpubs.py). + local _ext="$ROOT/ladder/extract_rpubs.py" if [ ! -f "$_ext" ]; then warn bank "force-select extract missing" "problem: $_ext not found" return 0 @@ -1122,9 +1132,9 @@ print(min(a,h)) "problem: no OK/OK_BANK within ${rung} probes; hi=$(format_amount_alt "${CUR}:${MAX_HI}")" fi { - echo "# max-search mode" - echo "best=${MAX_BEST_AMT:-}" - echo "best_alt=$(format_amount_alt "${MAX_BEST_AMT:-${CUR}:0}")" + echo "# max-search mode (withdraw)" + echo "best_wd=${MAX_BEST_AMT:-}" + echo "best_wd_alt=$(format_amount_alt "${MAX_BEST_AMT:-${CUR}:0}")" echo "lo=${MAX_LO}" echo "lo_alt=$(format_amount_alt "${CUR}:${MAX_LO}")" echo "hi=${MAX_HI}" @@ -1133,6 +1143,7 @@ print(min(a,h)) } >"$SCRATCH/ladder-plan.txt" printf '%s\n' "${MAX_BEST_AMT:-${CUR}:0}" >"$SCRATCH/ladder-wd-plan.txt" : >"$SCRATCH/ladder-pay-plan.txt" + # pay phase uses max-search (not classic PAY_LIST) PAY_LIST="" else section "ladder · phase A · withdraw (${LADDER_N} rungs)" @@ -1163,255 +1174,15 @@ fi # --------------------------------------------------------------------------- -# Phase B — payment ladder (same shape: 0 → low → … → max-1 → max) +# Phase B — payment (ladder/lib_pay.sh) +# classic: planned pay list · max: highest payable ≤ available (goa-free) # --------------------------------------------------------------------------- PAY_OK_N=0 PAY_FAIL_N=0 -if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; then - set_group pay - section "ladder · phase B · pay" - metrics_report_coins "before-pay-ladder" || true - # Merchant secret (same as e2e) — stage often has only public shop templates - MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}" - if [ -z "$MPW" ]; then - MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true) - fi - if [ -z "$MPW" ]; then - MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true) - fi - if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then - if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then - MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt") - fi - if [ -z "$MPW" ]; then - _remote_mer="${FRANCPAYSAN_REMOTE_MERCHANT_TOKEN:-}" - [ -z "$_remote_mer" ] && [ -n "${FRANCPAYSAN_REMOTE_SECRETS_ROOT:-}" ] && \ - _remote_mer="${FRANCPAYSAN_REMOTE_SECRETS_ROOT}/merchant/secrets/default-instance-token.txt" - for host in ${INSIDE_SSH:+"$INSIDE_SSH"} ${FRANCPAYSAN_SSH:+"$FRANCPAYSAN_SSH"}; do - [ -n "$host" ] && [ -n "$_remote_mer" ] || continue - MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \ - "tr -d '\\n\\r' <${_remote_mer} 2>/dev/null || true" \ - 2>/dev/null || true) - [ -n "$MPW" ] && break - done - unset _remote_mer - fi - fi - if [ -z "$MPW" ]; then - if [ "${CUR}" = "TESTPAYSAN" ]; then - # Stage: withdraw ladder is the main probe; pays need instance token or - # public templates for *exact* face values (not continuous ladder amounts). - info pay "no merchant token — skip pay ladder on TESTPAYSAN (withdraw maxima still covered)" - info pay "hint: set E2E_MERCHANT_TOKEN or use ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e for shop templates" - else - warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)" - fi - else - ok "merchant token" "instance ${INST}" - case "$MPW" in - secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;; - *) AUTH="Authorization: Bearer secret-token:${MPW}" ;; - esac - wallet_prepare "main" - # shellcheck disable=SC2086 - set -- $PAY_LIST - PAY_N=$# - _bal0=$(wallet_avail) - info "pay rungs" "$PAY_N · $*" - info "pay budget" "wallet available ${CUR}:${_bal0} · $(format_amount_alt "${CUR}:${_bal0}") — never order more than this" - prung=0 - for PAMT in "$@"; do - prung=$((prung + 1)) - if ladder_over; then - warn pay "time budget exhausted" "after ${PAY_OK_N} ok pays" - break - fi - section "ladder · pay rung $prung $PAMT · $(format_amount_alt "$PAMT")" - ptag=$(printf '%s' "$PAMT" | tr '.:' '__') - if [ "$prung" -eq 1 ]; then - prange="pin:0" - elif [ "$prung" -eq "$PAY_N" ]; then - prange="pin:max" - elif [ "$prung" -eq $((PAY_N - 1)) ] && [ "$PAY_N" -ge 3 ]; then - prange="pin:max-1" - else - prange="random" - fi - PNUM=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$PAMT") - IS_PZERO=0 - python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])==0 else 1)' "$PNUM" 2>/dev/null && IS_PZERO=1 - IS_PMAX=0 - [ "$PNUM" = "${LADDER_MAX_AMOUNT}" ] && IS_PMAX=1 - t_pay=$(now_ms) - ms_order=0 ms_handle=0 ms_psettle=0 - pnote="" - pstatus="FAIL" - OID="-" - - metrics_report_coins "before-pay-${ptag}" || true - bal=$(wallet_avail) - - if [ "$IS_PZERO" = "1" ]; then - pstatus="ZERO_SKIP" - pnote="zero pay probe skipped" - warn pay "pay $PAMT skipped" "problem: zero amount order not useful. Ladder continues." - ms_total=$(elapsed_ms "$t_pay") - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - PAY_OK_N=$((PAY_OK_N + 1)) - continue - fi - - # Never spend more than wallet available (spendable coins only — not pendingIncoming). - # Soft-skip any over-budget rung; do not hard-fail the ladder for insufficient funds. - if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>=Decimal(sys.argv[2]) else 1)' "$bal" "$PNUM" 2>/dev/null; then - pnote="skip: would spend more than available avail=${CUR}:${bal} need=${PAMT}" - ms_total=$(elapsed_ms "$t_pay") - pstatus="SKIP_BALANCE" - warn pay "pay $PAMT skipped (balance)" \ - "problem: pay amount exceeds wallet available — never overspend. $(format_amount_alt "${CUR}:${bal}") free vs $(format_amount_alt "$PAMT"). detail: $pnote" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t0\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV" - continue - fi - - t0=$(now_ms) - SUM_JSON=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "ladder pay ${PAMT}" 2>/dev/null || echo '"ladder pay"') - curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \ - -H "$AUTH" -H 'Content-Type: application/json' \ - -d "{\"order\":{\"summary\":${SUM_JSON},\"amount\":\"${PAMT}\",\"fulfillment_message\":\"ok\"},\"create_token\":true}" \ - "${MER}/instances/${INST}/private/orders" 2>"$SCRATCH/ord-$ptag.err" || true - ms_order=$(elapsed_ms "$t0") - OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() -try: print(json.loads(t).get("order_id") or "") -except Exception: - m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") -' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) - OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() -try: print(json.loads(t).get("token") or "") -except Exception: - m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") -' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) - if [ -z "$OID" ]; then - pnote="order create failed $(head -c 80 "$SCRATCH/ord-$ptag.json" 2>/dev/null | tr '\n\"' ' ')" - ms_total=$(elapsed_ms "$t_pay") - if [ "$IS_PMAX" = "1" ]; then - pstatus="CEILING_REJECT" - warn pay "pay $PAMT rejected (ceiling)" "$pnote" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV" - continue - fi - err pay "order $PAMT" "$pnote" - pstatus="FAIL_ORDER" - PAY_FAIL_N=$((PAY_FAIL_N + 1)) - FAIL_N_L=$((FAIL_N_L + 1)) - STOP_REASON="$pnote" - STOP_AMOUNT="$PAMT" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV" - break - fi - ok "order $OID ($PAMT) ${ms_order}ms" - - curl -skS -m 12 -o "$SCRATCH/ord-det-$ptag.json" -H "$AUTH" \ - "${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true - PAYURI=$(python3 -c 'import json,sys -try: print(json.load(open(sys.argv[1])).get("taler_pay_uri") or "") -except Exception: print("") -' "$SCRATCH/ord-det-$ptag.json" 2>/dev/null || true) - if [ -z "$PAYURI" ] && [ -n "$OTOK" ]; then - MH=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$MER"'").hostname or "taler.hacktivism.ch")' 2>/dev/null || echo "taler.hacktivism.ch") - PAYURI="taler://pay/${MH}/instances/${INST}/${OID}/?c=${OTOK}" - fi - PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g') - if [ -z "$PAYURI" ]; then - pnote="no pay URI for $OID" - ms_total=$(elapsed_ms "$t_pay") - err pay "uri $PAMT" "$pnote" - pstatus="FAIL_URI" - PAY_FAIL_N=$((PAY_FAIL_N + 1)) - FAIL_N_L=$((FAIL_N_L + 1)) - STOP_REASON="$pnote" - STOP_AMOUNT="$PAMT" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - break - fi - - t0=$(now_ms) - if ! wcli handle-uri --yes "$PAYURI" >"$SCRATCH/pay-$ptag.out" 2>&1; then - ms_handle=$(elapsed_ms "$t0") - pnote="handle-uri failed $(tail -c 100 "$SCRATCH/pay-$ptag.out" | tr '\n\"' ' ')" - ms_total=$(elapsed_ms "$t_pay") - if [ "$IS_PMAX" = "1" ]; then - pstatus="CEILING_REJECT" - warn pay "pay $PAMT handle failed (ceiling)" "$pnote" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - continue - fi - err pay "handle $PAMT" "$pnote" - pstatus="FAIL_HANDLE" - PAY_FAIL_N=$((PAY_FAIL_N + 1)) - FAIL_N_L=$((FAIL_N_L + 1)) - STOP_REASON="$pnote" - STOP_AMOUNT="$PAMT" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - break - fi - ms_handle=$(elapsed_ms "$t0") - ok "handle-uri $PAMT ${ms_handle}ms" - - # Short settle polls: merchant order + wallet tx only — never run-until-done - t0=$(now_ms) - settled=0 - r=0 - while [ "$r" -lt "${LADDER_PAY_SETTLE_ROUNDS}" ]; do - r=$((r + 1)) - wcli transactions >"$SCRATCH/tx-$ptag.out" 2>&1 || true - curl -skS -m 8 -o "$SCRATCH/ord-paid-$ptag.json" -H "$AUTH" \ - "${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true - if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$ptag.out" 2>/dev/null \ - || python3 -c 'import json,sys -d=json.load(open(sys.argv[1])) -sys.exit(0 if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid" else 1) -' "$SCRATCH/ord-paid-$ptag.json" 2>/dev/null; then - settled=1 - break - fi - sleep 0.5 - done - ms_psettle=$(elapsed_ms "$t0") - ms_total=$(elapsed_ms "$t_pay") - after=$(wallet_avail) - if [ "$settled" = "1" ]; then - pstatus="OK" - pnote="avail=${CUR}:${after}" - ok "pay settled $PAMT → bal ${CUR}:${after} (total ${ms_total}ms)" - PAY_OK_N=$((PAY_OK_N + 1)) - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - metrics_report_coins "after-pay-${ptag}" || true - metrics_record_flow spent "$PAMT" || true - else - pnote="not settled order=$OID avail=${CUR}:${after}" - if [ "$IS_PMAX" = "1" ]; then - pstatus="CEILING_REJECT" - warn pay "pay $PAMT not settled (ceiling)" "$pnote" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - continue - fi - err pay "settle $PAMT" "$pnote" - pstatus="FAIL_SETTLE" - PAY_FAIL_N=$((PAY_FAIL_N + 1)) - FAIL_N_L=$((FAIL_N_L + 1)) - STOP_REASON="$pnote" - STOP_AMOUNT="$PAMT" - echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV" - metrics_report_coins "after-pay-fail-${ptag}" || true - break - fi - done - metrics_report_coins "after-pay-ladder" || true - info "pay summary" "ok=$PAY_OK_N fail=$PAY_FAIL_N tsv=$PAY_TSV" - fi -elif [ "${LADDER_PAY}" = "1" ] && [ "$FAIL_N_L" -gt 0 ]; then - warn pay "skipped pay ladder" "withdraw phase already failed" -fi +MAX_PAY_BEST_AMT="" +MAX_PAY_LO="0" +MAX_PAY_HI="0" +ladder_run_pay_phase ms_phase=$(python3 -c 'import sys,time; print(int((time.time()-float(sys.argv[1]))*1000))' "$SECTION_T0") @@ -1433,6 +1204,10 @@ export LADDER_REPORT_STOP_AMOUNT="${STOP_AMOUNT:-}" export LADDER_REPORT_STOP_REASON="${STOP_REASON:-}" export LADDER_REPORT_MODE="${LADDER_MODE:-classic}" export LADDER_REPORT_MAX_BEST="${MAX_BEST_AMT:-}" +export LADDER_REPORT_MAX_PAY_BEST="${MAX_PAY_BEST_AMT:-}" +export LADDER_REPORT_MAX_PAY_LO="${MAX_PAY_LO:-}" +export LADDER_REPORT_MAX_PAY_HI="${MAX_PAY_HI:-}" +export LADDER_REPORT_MAX_PAY_BEST_ALT="$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}" 2>/dev/null || true)" export LADDER_REPORT_MAX_LO="${MAX_LO:-}" export LADDER_REPORT_MAX_HI="${MAX_HI:-}" # Human alt_unit_names (e.g. 8.15 Tera-GOA (GOA:…)) for report + final lines @@ -1457,6 +1232,10 @@ try: max_best_alt = os.environ.get("LADDER_REPORT_MAX_BEST_ALT") or None max_lo_alt = os.environ.get("LADDER_REPORT_MAX_LO_ALT") or None max_hi_alt = os.environ.get("LADDER_REPORT_MAX_HI_ALT") or None + max_pay_best = os.environ.get("LADDER_REPORT_MAX_PAY_BEST") or None + max_pay_lo = os.environ.get("LADDER_REPORT_MAX_PAY_LO") or None + max_pay_hi = os.environ.get("LADDER_REPORT_MAX_PAY_HI") or None + max_pay_best_alt = os.environ.get("LADDER_REPORT_MAX_PAY_BEST_ALT") or None alt_path = os.environ.get("LADDER_REPORT_ALT_UNITS") or "" alt = {} if alt_path: @@ -1578,6 +1357,7 @@ try: _best = as_amt(max_best) if max_best else None _lo = as_amt(max_lo) if max_lo is not None and str(max_lo) != "" else None _hi = as_amt(max_hi) if max_hi is not None and str(max_hi) != "" else None + _pb = as_amt(max_pay_best) if max_pay_best else None _max_block = { "best": _best, "best_alt": max_best_alt or (format_alt(_best) if _best else None), @@ -1585,6 +1365,10 @@ try: "lo_alt": max_lo_alt or (format_alt(_lo) if _lo else None), "hi": max_hi, "hi_alt": max_hi_alt or (format_alt(_hi) if _hi else None), + "best_pay": _pb, + "best_pay_alt": max_pay_best_alt or (format_alt(_pb) if _pb else None), + "pay_lo": max_pay_lo, + "pay_hi": max_pay_hi, } report = { @@ -1662,6 +1446,8 @@ try: print(" lo_alt=%s" % la) print(" hi=%s" % (max_hi if max_hi is not None else "-")) print(" hi_alt=%s" % ha) + print(" best_pay=%s" % (max_pay_best or "-")) + print(" best_pay_alt=%s" % (max_pay_best_alt or (format_alt(as_amt(max_pay_best) or "") if max_pay_best else "-"))) except Exception as e: print(" max-search print error: %s" % (e,)) if stop_amt: @@ -1722,7 +1508,7 @@ if [ "$OK_N" -eq 0 ]; then exit 1 fi if [ "${LADDER_MODE}" = "max" ] && [ -n "${MAX_BEST_AMT:-}" ]; then - ok "ladder finished mode=max best=$(format_amount_alt "$MAX_BEST_AMT") · lo=$(format_amount_alt "${CUR}:${MAX_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_HI}") · withdraw_ok=$OK_N phase=${ms_phase}ms" + ok "ladder finished mode=max wd_best=$(format_amount_alt "$MAX_BEST_AMT") · pay_best=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}") · withdraw_ok=$OK_N pay_ok=$PAY_OK_N phase=${ms_phase}ms" else ok "ladder finished mode=${LADDER_MODE} withdraw_ok=$OK_N pay_ok=$PAY_OK_N phase=${ms_phase}ms" fi diff --git a/check_goa_ladder.sh b/check_goa_ladder.sh index c32f686..1640dc5 100755 --- a/check_goa_ladder.sh +++ b/check_goa_ladder.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Compat shim — use check_amount_ladder.sh (generic currency amount ladder). -# Kept so older host-agent / docs invoking check_goa_ladder.sh still work. +# Compat shim only — prefer: ./taler-monitoring.sh ladder | max-ladder +# Real implementation: check_amount_ladder.sh (+ ladder/ modules). ROOT=$(cd "$(dirname "$0")" && pwd) exec bash "$ROOT/check_amount_ladder.sh" "$@" diff --git a/ladder/README.md b/ladder/README.md new file mode 100644 index 0000000..0ef8e26 --- /dev/null +++ b/ladder/README.md @@ -0,0 +1,25 @@ +# Amount ladder modules + +Glued by `../check_amount_ladder.sh` (phase entry: `ladder` / `max-ladder`). + +| Path | Role | +|------|------| +| `extract_rpubs.py` | Force-select: scrape `reservePub` from wallet dumps (no bash heredoc) | +| `lib_pay.sh` | **Payment** ladder: free-amount template orders, classic pay list, max-search highest **payable** ≤ available | +| *(withdraw stays in `check_amount_ladder.sh`)* | **Withdraw** ladder: classic 23-rung plan + max-search highest **withdrawable** | + +`check_goa_ladder.sh` at repo root is only a **compat shim** (`exec` → `check_amount_ladder.sh`). Not required for new scripts; phase alias `goa-ladder` already uses the amount ladder. + +## Modes + +- **classic** (default): withdraw plan 0→mids→max-1→max, then matching pay plan (never overspend). +- **max** (`max-ladder`): adaptive max withdraw, then adaptive max pay via public free template (hacktivism: `goa-free`). + +## Env (pay) + +| Variable | Default | Meaning | +|----------|---------|---------| +| `LADDER_PAY` | `1` | `0` = skip all pays | +| `LADDER_USE_FREE_TEMPLATE` | `1` | public POST template with `{"amount":…}` | +| `LADDER_FREE_TEMPLATE` | `goa-free` | template id (hacktivism) | +| `LADDER_FREE_TEMPLATE_INSTANCE` | merchant instance | e.g. `goa-demo-cp4zqk` | diff --git a/ladder_extract_rpubs.py b/ladder/extract_rpubs.py similarity index 100% rename from ladder_extract_rpubs.py rename to ladder/extract_rpubs.py diff --git a/ladder/lib_pay.sh b/ladder/lib_pay.sh new file mode 100644 index 0000000..9e8e157 --- /dev/null +++ b/ladder/lib_pay.sh @@ -0,0 +1,361 @@ +# ladder/lib_pay.sh — payment side of amount ladder (sourced by check_amount_ladder.sh) +# Expects: lib.sh + metrics.sh already sourced; CUR BANK MER INST EX SCRATCH WDB +# wcli wallet_avail format_amount_alt ladder_over now_ms elapsed_ms +# LADDER_* env, AUTH optional, PAY_TSV PAY_OK_N PAY_FAIL_N +# +# Public free-amount template (hacktivism): goa-free @ goa-demo-cp4zqk +# Never spends more than wallet available. + +: "${LADDER_FREE_TEMPLATE:=goa-free}" +: "${LADDER_FREE_TEMPLATE_INSTANCE:=}" +: "${LADDER_USE_FREE_TEMPLATE:=1}" +: "${LADDER_PAY_WAIT_AVAILABLE_S:=60}" +: "${LADDER_PAY_SETTLE_ROUNDS:=6}" + +MAX_PAY_BEST_AMT="${MAX_PAY_BEST_AMT:-}" +MAX_PAY_LO="${MAX_PAY_LO:-0}" +MAX_PAY_HI="${MAX_PAY_HI:-0}" + +ladder_pay_resolve_merchant() { + # Sets AUTH (may be empty if free template only) + MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}" + if [ -z "$MPW" ]; then + MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true) + fi + if [ -z "$MPW" ]; then + MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true) + fi + if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then + if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then + MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt") + fi + fi + AUTH="" + if [ -n "$MPW" ]; then + case "$MPW" in + secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;; + *) AUTH="Authorization: Bearer secret-token:${MPW}" ;; + esac + fi + : "${LADDER_FREE_TEMPLATE_INSTANCE:=${INST}}" +} + +# Create order for PAMT → OID OTOK PAYURI. Prefers public free template. +ladder_create_order() { + local pamt="$1" ptag="$2" + local inst mh body tpl_https sum_json + OID=""; OTOK=""; PAYURI="" + inst="${LADDER_FREE_TEMPLATE_INSTANCE:-$INST}" + mh=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"${MER}"'").hostname or "taler.hacktivism.ch")' 2>/dev/null || echo "taler.hacktivism.ch") + + if [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; then + tpl_https="${MER}/instances/${inst}/templates/${LADDER_FREE_TEMPLATE}" + body=$(python3 -c 'import json,sys; print(json.dumps({"amount":sys.argv[1]}))' "$pamt" 2>/dev/null || printf '{"amount":"%s"}' "$pamt") + curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \ + -H 'Content-Type: application/json' \ + -d "$body" \ + "$tpl_https" 2>"$SCRATCH/ord-$ptag.err" || true + OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() +try: print(json.loads(t).get("order_id") or "") +except Exception: + m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") +' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) + OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() +try: print(json.loads(t).get("token") or "") +except Exception: + m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") +' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) + if [ -n "$OID" ] && [ -n "$OTOK" ]; then + PAYURI="taler://pay/${mh}/instances/${inst}/${OID}/?c=${OTOK}" + PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g') + return 0 + fi + fi + + [ -n "${AUTH:-}" ] || return 1 + sum_json=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "ladder pay ${pamt}" 2>/dev/null || echo '"ladder pay"') + curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \ + -H "$AUTH" -H 'Content-Type: application/json' \ + -d "{\"order\":{\"summary\":${sum_json},\"amount\":\"${pamt}\",\"fulfillment_message\":\"ok\"},\"create_token\":true}" \ + "${MER}/instances/${INST}/private/orders" 2>"$SCRATCH/ord-$ptag.err" || true + OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() +try: print(json.loads(t).get("order_id") or "") +except Exception: + m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") +' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) + OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read() +try: print(json.loads(t).get("token") or "") +except Exception: + m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "") +' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true) + [ -n "$OID" ] || return 1 + curl -skS -m 12 -o "$SCRATCH/ord-det-$ptag.json" -H "$AUTH" \ + "${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true + PAYURI=$(python3 -c 'import json,sys +try: print(json.load(open(sys.argv[1])).get("taler_pay_uri") or "") +except Exception: print("") +' "$SCRATCH/ord-det-$ptag.json" 2>/dev/null || true) + if [ -z "$PAYURI" ] && [ -n "$OTOK" ]; then + PAYURI="taler://pay/${mh}/instances/${INST}/${OID}/?c=${OTOK}" + fi + PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g') + [ -n "$PAYURI" ] +} + +# One pay attempt; never overspends. Always soft (no hard ladder stop). +ladder_pay_attempt() { + local pamt="$1" prange="$2" prung="$3" + local ptag pnum bal t_pay t0 ms_order ms_handle ms_psettle after settled r pstatus pnote + ptag=$(printf '%s' "$pamt" | tr '.:' '__') + pnum=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$pamt") + t_pay=$(now_ms) + ms_order=0; ms_handle=0; ms_psettle=0 + pnote=""; pstatus="FAIL"; OID="-" + + bal=$(wallet_avail) + if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[1])>=Decimal(sys.argv[2]) else 1)' "$bal" "$pnum" 2>/dev/null; then + pstatus="SKIP_BALANCE" + pnote="skip: would spend more than available avail=${CUR}:${bal} need=${pamt}" + warn pay "pay $pamt skipped (balance)" \ + "problem: never overspend. $(format_amount_alt "${CUR}:${bal}") free vs $(format_amount_alt "$pamt")" + echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t0\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV" + return 0 + fi + + t0=$(now_ms) + if ! ladder_create_order "$pamt" "$ptag"; then + ms_order=$(elapsed_ms "$t0") + pstatus="FAIL_ORDER" + pnote="order create failed $(head -c 100 "$SCRATCH/ord-$ptag.json" 2>/dev/null | tr '\n\"' ' ')" + warn pay "order $pamt" "$pnote" + echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV" + return 0 + fi + ms_order=$(elapsed_ms "$t0") + ok "order $OID ($pamt) ${ms_order}ms · $(format_amount_alt "$pamt")" + + t0=$(now_ms) + if ! wcli handle-uri --yes "$PAYURI" >"$SCRATCH/pay-$ptag.out" 2>&1; then + ms_handle=$(elapsed_ms "$t0") + pstatus="FAIL_HANDLE" + pnote="handle-uri failed $(tail -c 100 "$SCRATCH/pay-$ptag.out" | tr '\n\"' ' ')" + warn pay "handle $pamt" "$pnote" + echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV" + return 0 + fi + ms_handle=$(elapsed_ms "$t0") + ok "handle-uri $pamt ${ms_handle}ms" + + t0=$(now_ms) + settled=0 + r=0 + while [ "$r" -lt "${LADDER_PAY_SETTLE_ROUNDS}" ]; do + r=$((r + 1)) + wcli transactions >"$SCRATCH/tx-pay-$ptag.out" 2>&1 || true + if [ -n "${AUTH:-}" ]; then + curl -skS -m 8 -o "$SCRATCH/ord-paid-$ptag.json" -H "$AUTH" \ + "${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true + fi + if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-pay-$ptag.out" 2>/dev/null \ + || python3 -c 'import json,sys +try: + d=json.load(open(sys.argv[1])) + sys.exit(0 if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid" else 1) +except Exception: + sys.exit(1) +' "$SCRATCH/ord-paid-$ptag.json" 2>/dev/null; then + settled=1 + break + fi + sleep 0.5 + done + ms_psettle=$(elapsed_ms "$t0") + after=$(wallet_avail) + if [ "$settled" = "1" ]; then + pstatus="OK" + pnote="avail=${CUR}:${after}" + ok "pay settled $pamt · $(format_amount_alt "$pamt") → bal ${CUR}:${after}" + PAY_OK_N=$((PAY_OK_N + 1)) + echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV" + metrics_report_coins "after-pay-${ptag}" || true + metrics_record_flow spent "$pamt" || true + else + pstatus="FAIL_SETTLE" + pnote="not settled order=$OID avail=${CUR}:${after}" + warn pay "settle $pamt" "$pnote" + echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV" + fi + return 0 +} + +ladder_wait_available() { + local wait_s="${1:-$LADDER_PAY_WAIT_AVAILABLE_S}" + local end bal + end=$(( $(date +%s) + wait_s )) + bal=$(wallet_avail) + while python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=0 else 1)' "$bal" 2>/dev/null; do + [ "$(date +%s)" -ge "$end" ] && break + wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true + sleep 2 + bal=$(wallet_avail) + done + printf '%s' "$bal" +} + +# Classic planned pays (PAY_LIST) +ladder_run_pay_classic() { + # shellcheck disable=SC2086 + set -- $PAY_LIST + local pay_n=$# prung=0 PAMT prange PNUM + if [ "$pay_n" -eq 0 ]; then + info pay "empty pay plan — skip" + return 0 + fi + info "pay rungs" "$pay_n · $*" + for PAMT in "$@"; do + prung=$((prung + 1)) + ladder_over && break + if [ "$prung" -eq 1 ]; then + prange="pin:0" + elif [ "$prung" -eq "$pay_n" ]; then + prange="pin:max" + elif [ "$prung" -eq $((pay_n - 1)) ] && [ "$pay_n" -ge 3 ]; then + prange="pin:max-1" + else + prange="random" + fi + section "ladder · pay rung $prung $PAMT · $(format_amount_alt "$PAMT")" + PNUM=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$PAMT") + if python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])==0 else 1)' "$PNUM" 2>/dev/null; then + echo -e "${prung}\t${prange}\t${PAMT}\tZERO_SKIP\t0\t0\t0\t0\t-\tzero" >>"$PAY_TSV" + PAY_OK_N=$((PAY_OK_N + 1)) + continue + fi + ladder_pay_attempt "$PAMT" "$prange" "$prung" + done +} + +# Max-search: highest payable ≤ available (uses max_search_next from withdraw module) +ladder_run_pay_max_search() { + local bal prung _next _anum prange _act _last _slo _shi + section "ladder · phase B · max-search pay (highest payable ≤ available)" + bal=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S") + MAX_PAY_LO="0" + MAX_PAY_HI="$bal" + MAX_PAY_BEST_AMT="" + if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$MAX_PAY_HI" 2>/dev/null; then + warn pay "max-search pay skipped" \ + "problem: wallet available is ${CUR}:0 — need spendable coins (pendingIncoming is not enough)" + return 0 + fi + info "max-search pay" "hi=available $(format_amount_alt "${CUR}:${MAX_PAY_HI}") · template=${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE}" + info "pay budget" "never order more than $(format_amount_alt "${CUR}:${MAX_PAY_HI}")" + prung=0 + _last="" + while [ "${LADDER_MAX_PROBES}" = "0" ] || [ "$prung" -lt "${LADDER_MAX_PROBES}" ]; do + ladder_over && break + # reuse withdraw amount picker with pay bounds + _slo=${MAX_LO:-0}; _shi=${MAX_HI:-0} + MAX_LO=$MAX_PAY_LO + MAX_HI=$MAX_PAY_HI + if ! type max_search_next >/dev/null 2>&1; then + warn pay "max_search_next missing — cannot run pay max-search" + MAX_LO=$_slo; MAX_HI=$_shi + return 0 + fi + _next=$(max_search_next) + MAX_LO=$_slo; MAX_HI=$_shi + _anum=$(printf '%s' "$_next" | cut -d'|' -f1) + prange=$(printf '%s' "$_next" | cut -d'|' -f2 | sed 's/^max:/pay:/') + _act=$(printf '%s' "$_next" | cut -d'|' -f3) + if [ "$_act" = "done" ]; then + info "max-search pay" "converged best=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")" + break + fi + if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=Decimal(sys.argv[2]) else 1)' "$_anum" "$MAX_PAY_HI" 2>/dev/null; then + _anum=$MAX_PAY_HI + fi + if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$_anum" 2>/dev/null; then + break + fi + PAMT="${CUR}:${_anum}" + if [ -n "$_last" ] && [ "$PAMT" = "$_last" ]; then + MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); print(max(Decimal(0), a-(1 if a>=1 else Decimal("0.000001"))))') + continue + fi + _last=$PAMT + prung=$((prung + 1)) + section "ladder · max pay probe $prung $PAMT · $(format_amount_alt "$PAMT") [lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")]" + ladder_pay_attempt "$PAMT" "$prange" "$prung" + _last_st=$(tail -1 "$PAY_TSV" 2>/dev/null | cut -f4) + case "$_last_st" in + OK) + MAX_PAY_LO=$_anum + MAX_PAY_BEST_AMT=$PAMT + info "max-search pay" "success bound lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}")" + ;; + SKIP_BALANCE|FAIL_ORDER|FAIL_HANDLE|FAIL_SETTLE) + MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); h=Decimal("'"$MAX_PAY_HI"'"); print(min(a,h))') + info "max-search pay" "too-high/fail hi:=$(format_amount_alt "${CUR}:${MAX_PAY_HI}") status=${_last_st}" + ;; + esac + if python3 -c 'from decimal import Decimal; import sys; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[2])<=Decimal(sys.argv[1]) else 1)' "$MAX_PAY_LO" "$MAX_PAY_HI" 2>/dev/null; then + info "max-search pay" "bounds crossed — stop" + break + fi + done + if [ -n "${MAX_PAY_BEST_AMT:-}" ]; then + ok "max-search best payable $(format_amount_alt "$MAX_PAY_BEST_AMT") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")" + else + warn pay "max-search no successful pay" "available was ${CUR}:$(wallet_avail)" + fi + { + echo "best_pay=${MAX_PAY_BEST_AMT:-}" + echo "best_pay_alt=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}" 2>/dev/null || true)" + echo "pay_lo=${MAX_PAY_LO}" + echo "pay_hi=${MAX_PAY_HI}" + } >>"${SCRATCH}/ladder-plan.txt" 2>/dev/null || true + printf '%s\n' "${MAX_PAY_BEST_AMT:-}" >"${SCRATCH}/ladder-pay-plan.txt" 2>/dev/null || true +} + +# Entry: glue from check_amount_ladder.sh after withdraw phase +ladder_run_pay_phase() { + PAY_OK_N=0 + PAY_FAIL_N=0 + if [ "${LADDER_PAY}" != "1" ]; then + info pay "LADDER_PAY=0 — pay phase skipped" + return 0 + fi + if [ "${FAIL_N_L:-0}" -gt 0 ]; then + warn pay "skipped pay ladder" "withdraw phase already failed" + return 0 + fi + + set_group pay + section "ladder · phase B · pay" + metrics_report_coins "before-pay-ladder" || true + ladder_pay_resolve_merchant + + if [ -z "${AUTH:-}" ] && ! { [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; }; then + warn pay "no merchant token and free template off — skip pay" + return 0 + fi + if [ -n "${AUTH:-}" ]; then + ok "merchant token" "instance ${INST}" + else + info pay "public free template ${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE} (no merchant token)" + fi + + wallet_prepare "main" + local bal0 + bal0=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S") + info "pay budget" "wallet available ${CUR}:${bal0} · $(format_amount_alt "${CUR}:${bal0}") — never order more than this" + + if [ "${LADDER_MODE}" = "max" ]; then + ladder_run_pay_max_search + else + ladder_run_pay_classic + fi + metrics_report_coins "after-pay-ladder" || true + info "pay summary" "ok=$PAY_OK_N fail=$PAY_FAIL_N tsv=$PAY_TSV" +} diff --git a/taler-monitoring.sh b/taler-monitoring.sh index b89fbaa..73f124a 100755 --- a/taler-monitoring.sh +++ b/taler-monitoring.sh @@ -94,7 +94,7 @@ Phases: server server-side only (SSH) e2e withdraw + pay (small amounts; remote aborts on login/KYC) ladder amount ladder (classic: 0→mids→max; any currency / stack ceiling) - max-ladder same script, LADDER_MODE=max — hunt highest withdrawable amount + max-ladder highest withdrawable + highest payable (free-amount template on GOA) auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance) aptdeploy koopa podman apt-src smoke: taler-merchant in koopa-taler-deploy-test-apt-src-trixie{,-testing}