hernani/taler-monitoring
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

release 1.22.0: modular ladder withdraw + pay max-search

Browse source 
Split pay into ladder/lib_pay.sh; force-select extract in
ladder/extract_rpubs.py. check_goa_ladder.sh is only a compat shim.

max-ladder now hunts highest withdrawable and highest payable
(never above wallet available), using public free-amount template
goa-free on hacktivism. Classic remains default 23-rung wd+pay.

Pay phase always runs when LADDER_PAY=1 (visible SKIP_BALANCE when
no spendable coins). Report includes best_pay / alt names.
This commit is contained in:
Hernâni Marques 2026-07-19 14:25:24 +02:00
parent 32ffb6cdb1
commit 76a253613a
No known key found for this signature in database
9 changed files with 449 additions and 274 deletions
Download patch file Download diff file Expand all files Collapse all files

6
README.md
View file

@ -277,7 +277,7 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC.
# Amount ladder (any currency): classic 0 → … → max, then optional pay # Amount ladder (any currency): classic 0 → … → max, then optional pay
./taler-monitoring.sh ladder # classic (current domain) ./taler-monitoring.sh ladder # classic (current domain)
./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # TESTPAYSAN maxima
# Max-search: start high, narrow toward highest withdrawable # Max-search: highest withdrawable + highest payable (≤ available; GOA free template goa-free)
./taler-monitoring.sh max-ladder ./taler-monitoring.sh max-ladder
# Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01). # Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01).
# Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=). # Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=).
@ -396,10 +396,12 @@ LADDER_LOAD=0 ./taler-monitoring.sh ladder
| `METRICS_LOAD` | `1` | `0` = skip all host/container load probes | | `METRICS_LOAD` | `1` | `0` = skip all host/container load probes |
| `LADDER_LOAD` | `1` | `0` = skip load in ladder only | | `LADDER_LOAD` | `1` | `0` = skip load in ladder only |
| `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws | | `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws |
| `LADDER_MODE` | `classic` | default **classic** 23-rung; `max` = hunt highest withdrawable (`max-ladder`) | | `LADDER_MODE` | `classic` | default **classic** 23-rung wd+pay; `max` = max-search wd+pay (`max-ladder`) |
| `LADDER_MAX_PROBES` | `32` | max-search probe budget; `0` = until converge / timeout | | `LADDER_MAX_PROBES` | `32` | max-search probe budget; `0` = until converge / timeout |
| `LADDER_HIGH_RUNGS` | `6` | max-search: first N probes random-high, then bisect | | `LADDER_HIGH_RUNGS` | `6` | max-search: first N probes random-high, then bisect |
| `LADDER_MAX_TOL_REL` | `0.001` | max-search relative lo/hi gap to stop | | `LADDER_MAX_TOL_REL` | `0.001` | max-search relative lo/hi gap to stop |
| `LADDER_FREE_TEMPLATE` | `goa-free` | public free-amount pay template (hacktivism) |
| `LADDER_USE_FREE_TEMPLATE` | `1` | create pays via template POST `{"amount":…}` |
| `LADDER_WITHDRAW_SCALE` | `1.5` | classic: withdraw mids / pay mids ratio (funding headroom) | | `LADDER_WITHDRAW_SCALE` | `1.5` | classic: withdraw mids / pay mids ratio (funding headroom) |
| `LADDER_STEPS` | `23` | classic: rungs for withdraw and pay (0 + mids + max−1 + max) | | `LADDER_STEPS` | `23` | classic: rungs for withdraw and pay (0 + mids + max−1 + max) |
| `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python | | `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python |

2
VERSION
View file

@ -1 +1 @@
1.21.0 1.22.0

3
VERSIONS.md
View file

@ -17,7 +17,8 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre
| Tag | Date (UTC) | Notes | | Tag | Date (UTC) | Notes |
|-----|------------|--------| |-----|------------|--------|
| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external `ladder_extract_rpubs.py` (no stdin SyntaxError); force-select multi-rpub + raw_decode; 409 tries next rpub; soft `CEILING_REJECT` on mid/high mint 5110/P0001; report python never aborts ladder; helper PATH python≥3.11; wallet-db not `*.json`. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. | | **v1.22.0** | 2026-07-19 | **Feature:** modular amount ladder — `ladder/lib_pay.sh` (pay) + withdraw in `check_amount_ladder.sh`; `ladder/extract_rpubs.py`. **max-ladder** hunts highest **withdrawable** and highest **payable** (≤ wallet available). GOA pays use public free-amount template **`goa-free`** (`LADDER_FREE_TEMPLATE`). Classic still default 23-rung wd+pay. `check_goa_ladder.sh` remains a thin compat shim only. |
| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external extract (no stdin SyntaxError); force-select multi-rpub; soft `CEILING_REJECT` on mint 5110/P0001; crash-proof report. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. |
| **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. *(Superseded notes folded into **v1.21.0** feature release.)* | | **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. *(Superseded notes folded into **v1.21.0** feature release.)* |
| **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. | | **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. |
| **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. | | **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. |

320
check_amount_ladder.sh
View file

@ -11,24 +11,26 @@
# 4) bank confirm when selected; settle = balance + transfer_done # 4) bank confirm when selected; settle = balance + transfer_done
# #
# Modes (LADDER_MODE): # Modes (LADDER_MODE):
# classic — random strictly increasing rungs + pins 0, max-1, max (default) # classic — random strictly increasing rungs + pins 0, max-1, max (default 23)
# max — free max-search: start randomly high, narrow toward highest # max — max-search: highest withdrawable AND highest payable
# amount that mint+accept+confirm (+ bank transfer_done) succeeds # (pay never exceeds wallet available; GOA uses public free-amount
# template goa-free by default)
# #
# Phase A: withdraw. Phase B: pay (classic only by default; off in max mode). # Phase A: withdraw. Phase B: pay (classic plan list, or max-search pay).
# #
# Stacks auto-tune via bank /config max_wire + exchange min denom when possible. # Stacks auto-tune via bank /config max_wire + exchange min denom when possible.
# #
# Usage: # Usage:
# ./taler-monitoring.sh ladder # ./taler-monitoring.sh ladder
# ./taler-monitoring.sh max-ladder # LADDER_MODE=max # ./taler-monitoring.sh max-ladder # LADDER_MODE=max (wd + pay hunt)
# ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder
# LADDER_MODE=max LADDER_MAX_PROBES=16 ./taler-monitoring.sh ladder # LADDER_MODE=max LADDER_MAX_PROBES=32 ./taler-monitoring.sh ladder
# LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder # LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder
# #
# Env: LADDER_MODE, LADDER_STEPS, LADDER_MAX_AMOUNT, LADDER_MIN_AMOUNT, # Env: LADDER_MODE, LADDER_STEPS, LADDER_MAX_AMOUNT, LADDER_MIN_AMOUNT,
# LADDER_MAX_PROBES, LADDER_MAX_TOL_REL, LADDER_HIGH_RUNGS, LADDER_STACK_AUTO, # LADDER_MAX_PROBES, LADDER_MAX_TOL_REL, LADDER_HIGH_RUNGS, LADDER_STACK_AUTO,
# LADDER_PAY, EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, … # LADDER_PAY, LADDER_FREE_TEMPLATE, LADDER_FREE_TEMPLATE_INSTANCE,
# EXP_PW / EXP_PW_FILE, CLI_JS, MERCHANT_INSTANCE, …
set -euo pipefail set -euo pipefail
ROOT=$(cd "$(dirname "$0")" && pwd) ROOT=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=lib.sh # shellcheck source=lib.sh
@ -71,7 +73,12 @@ elapsed_ms() {
# Public variable template for stage pays (optional; fixed templates used when amount maps) # Public variable template for stage pays (optional; fixed templates used when amount maps)
: "${LADDER_PAY_TEMPLATE:=}" : "${LADDER_PAY_TEMPLATE:=}"
: "${LADDER_PAY_TEMPLATE_INSTANCE:=fermes-des-collines}" : "${LADDER_PAY_TEMPLATE_INSTANCE:=fermes-des-collines}"
# classic | max (max = find highest withdrawable; phase alias max-ladder) # Free-amount public template (hacktivism: goa-free on goa-demo-cp4zqk — any GOA amount)
: "${LADDER_FREE_TEMPLATE:=goa-free}"
: "${LADDER_FREE_TEMPLATE_INSTANCE:=}"
# 1 = create pays via public free template POST (amount in body); 0 = private orders only
: "${LADDER_USE_FREE_TEMPLATE:=1}"
# classic | max (max = highest withdrawable + highest payable; phase alias max-ladder)
: "${LADDER_MODE:=classic}" : "${LADDER_MODE:=classic}"
# max-search: probe budget (0 = run until convergence / LADDER_TIMEOUT_S only) # max-search: probe budget (0 = run until convergence / LADDER_TIMEOUT_S only)
: "${LADDER_MAX_PROBES:=32}" : "${LADDER_MAX_PROBES:=32}"
@ -80,6 +87,8 @@ elapsed_ms() {
: "${LADDER_HIGH_RUNGS:=6}" : "${LADDER_HIGH_RUNGS:=6}"
# optional seed for reproducible max-search (empty = time-based) # optional seed for reproducible max-search (empty = time-based)
: "${LADDER_MAX_SEED:=}" : "${LADDER_MAX_SEED:=}"
# After withdraw, wait for spendable available>0 before pay (secs total)
: "${LADDER_PAY_WAIT_AVAILABLE_S:=60}"
# Historic libeufin-ish absolute ceiling used as default LADDER_MAX_AMOUNT on GOA # Historic libeufin-ish absolute ceiling used as default LADDER_MAX_AMOUNT on GOA
LADDER_ABS_CEILING="4503599627370496" LADDER_ABS_CEILING="4503599627370496"
@ -180,13 +189,10 @@ case "${LADDER_MODE}" in
LADDER_MODE=classic LADDER_MODE=classic
;; ;;
esac esac
# max mode: pay ladder off unless caller explicitly set LADDER_PAY=1 *and* LADDER_PAY_WITH_MAX=1 # max mode: withdraw + pay hunts (LADDER_PAY=0 still disables pay entirely)
if [ "$LADDER_MODE" = "max" ]; then : "${LADDER_FREE_TEMPLATE_INSTANCE:=${MERCHANT_INSTANCE}}"
if [ "${LADDER_PAY_WITH_MAX:-0}" != "1" ]; then
LADDER_PAY=0
fi
fi
export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_HIGH_RUNGS LADDER_MAX_SEED LADDER_PAY export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_HIGH_RUNGS LADDER_MAX_SEED LADDER_PAY
export LADDER_FREE_TEMPLATE LADDER_FREE_TEMPLATE_INSTANCE LADDER_USE_FREE_TEMPLATE LADDER_PAY_WAIT_AVAILABLE_S
SCRATCH=$(mktemp -d) SCRATCH=$(mktemp -d)
WDB="$SCRATCH/wallet.sqlite3" WDB="$SCRATCH/wallet.sqlite3"
@ -435,6 +441,9 @@ PY
# shellcheck source=metrics.sh # shellcheck source=metrics.sh
source "$ROOT/metrics.sh" source "$ROOT/metrics.sh"
# Payment module (classic + max-search payable)
# shellcheck source=ladder/lib_pay.sh
source "$ROOT/ladder/lib_pay.sh"
METRICS_DIR="$REPORT_DIR" METRICS_DIR="$REPORT_DIR"
ALT_UNITS_FILE="${REPORT_DIR}/alt_unit_names.json" ALT_UNITS_FILE="${REPORT_DIR}/alt_unit_names.json"
export METRICS_DIR CUR WDB CLI_JS ALT_UNITS_FILE export METRICS_DIR CUR WDB CLI_JS ALT_UNITS_FILE
@ -540,8 +549,9 @@ ok "wallet exchange + ToS (${ms_tos}ms) — cumulative DB for withdraw+pay (no r
: "${LADDER_MAX_RUNGS:=99}" : "${LADDER_MAX_RUNGS:=99}"
if [ "${LADDER_MODE}" = "max" ]; then if [ "${LADDER_MODE}" = "max" ]; then
info "ladder mode" "max-search (find highest withdrawable ${CUR})" info "ladder mode" "max-search (highest withdrawable + highest payable ${CUR})"
info "max-search params" "probes≤${LADDER_MAX_PROBES} high_rand=${LADDER_HIGH_RUNGS} tol_rel=${LADDER_MAX_TOL_REL} ceiling=${CUR}:${LADDER_MAX_AMOUNT}" info "max-search params" "probes≤${LADDER_MAX_PROBES} high_rand=${LADDER_HIGH_RUNGS} tol_rel=${LADDER_MAX_TOL_REL} ceiling=${CUR}:${LADDER_MAX_AMOUNT}"
info "max-search pay" "LADDER_PAY=${LADDER_PAY} free_template=${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE:-$INST}"
LADDER_LIST="" LADDER_LIST=""
PAY_LIST="" PAY_LIST=""
LADDER_N=0 LADDER_N=0
@ -757,8 +767,8 @@ ladder_withdraw_rung() {
# and regex on *both* accept and transactions output (reservePub lives in tx, not accept). # and regex on *both* accept and transactions output (reservePub lives in tx, not accept).
extract_rpubs_for_wid() { extract_rpubs_for_wid() {
# External module — never feed extract via bash heredoc (refactor tools # External module — never feed extract via bash heredoc (refactor tools
# previously turned Python continue into return → SyntaxError line 91). # lives in ladder/extract_rpubs.py).
local _ext="$ROOT/ladder_extract_rpubs.py" local _ext="$ROOT/ladder/extract_rpubs.py"
if [ ! -f "$_ext" ]; then if [ ! -f "$_ext" ]; then
warn bank "force-select extract missing" "problem: $_ext not found" warn bank "force-select extract missing" "problem: $_ext not found"
return 0 return 0
@ -1122,9 +1132,9 @@ print(min(a,h))
"problem: no OK/OK_BANK within ${rung} probes; hi=$(format_amount_alt "${CUR}:${MAX_HI}")" "problem: no OK/OK_BANK within ${rung} probes; hi=$(format_amount_alt "${CUR}:${MAX_HI}")"
fi fi
{ {
echo "# max-search mode" echo "# max-search mode (withdraw)"
echo "best=${MAX_BEST_AMT:-}" echo "best_wd=${MAX_BEST_AMT:-}"
echo "best_alt=$(format_amount_alt "${MAX_BEST_AMT:-${CUR}:0}")" echo "best_wd_alt=$(format_amount_alt "${MAX_BEST_AMT:-${CUR}:0}")"
echo "lo=${MAX_LO}" echo "lo=${MAX_LO}"
echo "lo_alt=$(format_amount_alt "${CUR}:${MAX_LO}")" echo "lo_alt=$(format_amount_alt "${CUR}:${MAX_LO}")"
echo "hi=${MAX_HI}" echo "hi=${MAX_HI}"
@ -1133,6 +1143,7 @@ print(min(a,h))
} >"$SCRATCH/ladder-plan.txt" } >"$SCRATCH/ladder-plan.txt"
printf '%s\n' "${MAX_BEST_AMT:-${CUR}:0}" >"$SCRATCH/ladder-wd-plan.txt" printf '%s\n' "${MAX_BEST_AMT:-${CUR}:0}" >"$SCRATCH/ladder-wd-plan.txt"
: >"$SCRATCH/ladder-pay-plan.txt" : >"$SCRATCH/ladder-pay-plan.txt"
# pay phase uses max-search (not classic PAY_LIST)
PAY_LIST="" PAY_LIST=""
else else
section "ladder · phase A · withdraw (${LADDER_N} rungs)" section "ladder · phase A · withdraw (${LADDER_N} rungs)"
@ -1163,255 +1174,15 @@ fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Phase B — payment ladder (same shape: 0 → low → … → max-1 → max) # Phase B — payment (ladder/lib_pay.sh)
# classic: planned pay list · max: highest payable ≤ available (goa-free)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
PAY_OK_N=0 PAY_OK_N=0
PAY_FAIL_N=0 PAY_FAIL_N=0
if [ "${LADDER_PAY}" = "1" ] && [ -n "${PAY_LIST:-}" ] && [ "$FAIL_N_L" -eq 0 ]; then MAX_PAY_BEST_AMT=""
set_group pay MAX_PAY_LO="0"
section "ladder · phase B · pay" MAX_PAY_HI="0"
metrics_report_coins "before-pay-ladder" || true ladder_run_pay_phase
# Merchant secret (same as e2e) — stage often has only public shop templates
MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}"
if [ -z "$MPW" ]; then
MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true)
fi
if [ -z "$MPW" ]; then
MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true)
fi
if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt")
fi
if [ -z "$MPW" ]; then
_remote_mer="${FRANCPAYSAN_REMOTE_MERCHANT_TOKEN:-}"
[ -z "$_remote_mer" ] && [ -n "${FRANCPAYSAN_REMOTE_SECRETS_ROOT:-}" ] && \
_remote_mer="${FRANCPAYSAN_REMOTE_SECRETS_ROOT}/merchant/secrets/default-instance-token.txt"
for host in ${INSIDE_SSH:+"$INSIDE_SSH"} ${FRANCPAYSAN_SSH:+"$FRANCPAYSAN_SSH"}; do
[ -n "$host" ] && [ -n "$_remote_mer" ] || continue
MPW=$(ssh -o BatchMode=yes -o ConnectTimeout=12 "$host" \
"tr -d '\\n\\r' <${_remote_mer} 2>/dev/null || true" \
2>/dev/null || true)
[ -n "$MPW" ] && break
done
unset _remote_mer
fi
fi
if [ -z "$MPW" ]; then
if [ "${CUR}" = "TESTPAYSAN" ]; then
# Stage: withdraw ladder is the main probe; pays need instance token or
# public templates for *exact* face values (not continuous ladder amounts).
info pay "no merchant token — skip pay ladder on TESTPAYSAN (withdraw maxima still covered)"
info pay "hint: set E2E_MERCHANT_TOKEN or use ./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e for shop templates"
else
warn pay "no merchant token — skip pay ladder (set E2E_MERCHANT_TOKEN or secrets)"
fi
else
ok "merchant token" "instance ${INST}"
case "$MPW" in
secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;;
*) AUTH="Authorization: Bearer secret-token:${MPW}" ;;
esac
wallet_prepare "main"
# shellcheck disable=SC2086
set -- $PAY_LIST
PAY_N=$#
_bal0=$(wallet_avail)
info "pay rungs" "$PAY_N · $*"
info "pay budget" "wallet available ${CUR}:${_bal0} · $(format_amount_alt "${CUR}:${_bal0}") — never order more than this"
prung=0
for PAMT in "$@"; do
prung=$((prung + 1))
if ladder_over; then
warn pay "time budget exhausted" "after ${PAY_OK_N} ok pays"
break
fi
section "ladder · pay rung $prung $PAMT · $(format_amount_alt "$PAMT")"
ptag=$(printf '%s' "$PAMT" | tr '.:' '__')
if [ "$prung" -eq 1 ]; then
prange="pin:0"
elif [ "$prung" -eq "$PAY_N" ]; then
prange="pin:max"
elif [ "$prung" -eq $((PAY_N - 1)) ] && [ "$PAY_N" -ge 3 ]; then
prange="pin:max-1"
else
prange="random"
fi
PNUM=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$PAMT")
IS_PZERO=0
python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])==0 else 1)' "$PNUM" 2>/dev/null && IS_PZERO=1
IS_PMAX=0
[ "$PNUM" = "${LADDER_MAX_AMOUNT}" ] && IS_PMAX=1
t_pay=$(now_ms)
ms_order=0 ms_handle=0 ms_psettle=0
pnote=""
pstatus="FAIL"
OID="-"
metrics_report_coins "before-pay-${ptag}" || true
bal=$(wallet_avail)
if [ "$IS_PZERO" = "1" ]; then
pstatus="ZERO_SKIP"
pnote="zero pay probe skipped"
warn pay "pay $PAMT skipped" "problem: zero amount order not useful. Ladder continues."
ms_total=$(elapsed_ms "$t_pay")
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
PAY_OK_N=$((PAY_OK_N + 1))
continue
fi
# Never spend more than wallet available (spendable coins only — not pendingIncoming).
# Soft-skip any over-budget rung; do not hard-fail the ladder for insufficient funds.
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>=Decimal(sys.argv[2]) else 1)' "$bal" "$PNUM" 2>/dev/null; then
pnote="skip: would spend more than available avail=${CUR}:${bal} need=${PAMT}"
ms_total=$(elapsed_ms "$t_pay")
pstatus="SKIP_BALANCE"
warn pay "pay $PAMT skipped (balance)" \
"problem: pay amount exceeds wallet available — never overspend. $(format_amount_alt "${CUR}:${bal}") free vs $(format_amount_alt "$PAMT"). detail: $pnote"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t0\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV"
continue
fi
t0=$(now_ms)
SUM_JSON=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "ladder pay ${PAMT}" 2>/dev/null || echo '"ladder pay"')
curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \
-H "$AUTH" -H 'Content-Type: application/json' \
-d "{\"order\":{\"summary\":${SUM_JSON},\"amount\":\"${PAMT}\",\"fulfillment_message\":\"ok\"},\"create_token\":true}" \
"${MER}/instances/${INST}/private/orders" 2>"$SCRATCH/ord-$ptag.err" || true
ms_order=$(elapsed_ms "$t0")
OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("order_id") or "")
except Exception:
m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("token") or "")
except Exception:
m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
if [ -z "$OID" ]; then
pnote="order create failed $(head -c 80 "$SCRATCH/ord-$ptag.json" 2>/dev/null | tr '\n\"' ' ')"
ms_total=$(elapsed_ms "$t_pay")
if [ "$IS_PMAX" = "1" ]; then
pstatus="CEILING_REJECT"
warn pay "pay $PAMT rejected (ceiling)" "$pnote"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV"
continue
fi
err pay "order $PAMT" "$pnote"
pstatus="FAIL_ORDER"
PAY_FAIL_N=$((PAY_FAIL_N + 1))
FAIL_N_L=$((FAIL_N_L + 1))
STOP_REASON="$pnote"
STOP_AMOUNT="$PAMT"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t-\t${pnote}" >>"$PAY_TSV"
break
fi
ok "order $OID ($PAMT) ${ms_order}ms"
curl -skS -m 12 -o "$SCRATCH/ord-det-$ptag.json" -H "$AUTH" \
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
PAYURI=$(python3 -c 'import json,sys
try: print(json.load(open(sys.argv[1])).get("taler_pay_uri") or "")
except Exception: print("")
' "$SCRATCH/ord-det-$ptag.json" 2>/dev/null || true)
if [ -z "$PAYURI" ] && [ -n "$OTOK" ]; then
MH=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"$MER"'").hostname or "taler.hacktivism.ch")' 2>/dev/null || echo "taler.hacktivism.ch")
PAYURI="taler://pay/${MH}/instances/${INST}/${OID}/?c=${OTOK}"
fi
PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g')
if [ -z "$PAYURI" ]; then
pnote="no pay URI for $OID"
ms_total=$(elapsed_ms "$t_pay")
err pay "uri $PAMT" "$pnote"
pstatus="FAIL_URI"
PAY_FAIL_N=$((PAY_FAIL_N + 1))
FAIL_N_L=$((FAIL_N_L + 1))
STOP_REASON="$pnote"
STOP_AMOUNT="$PAMT"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t0\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
break
fi
t0=$(now_ms)
if ! wcli handle-uri --yes "$PAYURI" >"$SCRATCH/pay-$ptag.out" 2>&1; then
ms_handle=$(elapsed_ms "$t0")
pnote="handle-uri failed $(tail -c 100 "$SCRATCH/pay-$ptag.out" | tr '\n\"' ' ')"
ms_total=$(elapsed_ms "$t_pay")
if [ "$IS_PMAX" = "1" ]; then
pstatus="CEILING_REJECT"
warn pay "pay $PAMT handle failed (ceiling)" "$pnote"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
continue
fi
err pay "handle $PAMT" "$pnote"
pstatus="FAIL_HANDLE"
PAY_FAIL_N=$((PAY_FAIL_N + 1))
FAIL_N_L=$((FAIL_N_L + 1))
STOP_REASON="$pnote"
STOP_AMOUNT="$PAMT"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
break
fi
ms_handle=$(elapsed_ms "$t0")
ok "handle-uri $PAMT ${ms_handle}ms"
# Short settle polls: merchant order + wallet tx only — never run-until-done
t0=$(now_ms)
settled=0
r=0
while [ "$r" -lt "${LADDER_PAY_SETTLE_ROUNDS}" ]; do
r=$((r + 1))
wcli transactions >"$SCRATCH/tx-$ptag.out" 2>&1 || true
curl -skS -m 8 -o "$SCRATCH/ord-paid-$ptag.json" -H "$AUTH" \
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-$ptag.out" 2>/dev/null \
|| python3 -c 'import json,sys
d=json.load(open(sys.argv[1]))
sys.exit(0 if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid" else 1)
' "$SCRATCH/ord-paid-$ptag.json" 2>/dev/null; then
settled=1
break
fi
sleep 0.5
done
ms_psettle=$(elapsed_ms "$t0")
ms_total=$(elapsed_ms "$t_pay")
after=$(wallet_avail)
if [ "$settled" = "1" ]; then
pstatus="OK"
pnote="avail=${CUR}:${after}"
ok "pay settled $PAMT → bal ${CUR}:${after} (total ${ms_total}ms)"
PAY_OK_N=$((PAY_OK_N + 1))
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
metrics_report_coins "after-pay-${ptag}" || true
metrics_record_flow spent "$PAMT" || true
else
pnote="not settled order=$OID avail=${CUR}:${after}"
if [ "$IS_PMAX" = "1" ]; then
pstatus="CEILING_REJECT"
warn pay "pay $PAMT not settled (ceiling)" "$pnote"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
continue
fi
err pay "settle $PAMT" "$pnote"
pstatus="FAIL_SETTLE"
PAY_FAIL_N=$((PAY_FAIL_N + 1))
FAIL_N_L=$((FAIL_N_L + 1))
STOP_REASON="$pnote"
STOP_AMOUNT="$PAMT"
echo -e "${prung}\t${prange}\t${PAMT}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t${ms_total}\t${OID}\t${pnote}" >>"$PAY_TSV"
metrics_report_coins "after-pay-fail-${ptag}" || true
break
fi
done
metrics_report_coins "after-pay-ladder" || true
info "pay summary" "ok=$PAY_OK_N fail=$PAY_FAIL_N tsv=$PAY_TSV"
fi
elif [ "${LADDER_PAY}" = "1" ] && [ "$FAIL_N_L" -gt 0 ]; then
warn pay "skipped pay ladder" "withdraw phase already failed"
fi
ms_phase=$(python3 -c 'import sys,time; print(int((time.time()-float(sys.argv[1]))*1000))' "$SECTION_T0") ms_phase=$(python3 -c 'import sys,time; print(int((time.time()-float(sys.argv[1]))*1000))' "$SECTION_T0")
@ -1433,6 +1204,10 @@ export LADDER_REPORT_STOP_AMOUNT="${STOP_AMOUNT:-}"
export LADDER_REPORT_STOP_REASON="${STOP_REASON:-}" export LADDER_REPORT_STOP_REASON="${STOP_REASON:-}"
export LADDER_REPORT_MODE="${LADDER_MODE:-classic}" export LADDER_REPORT_MODE="${LADDER_MODE:-classic}"
export LADDER_REPORT_MAX_BEST="${MAX_BEST_AMT:-}" export LADDER_REPORT_MAX_BEST="${MAX_BEST_AMT:-}"
export LADDER_REPORT_MAX_PAY_BEST="${MAX_PAY_BEST_AMT:-}"
export LADDER_REPORT_MAX_PAY_LO="${MAX_PAY_LO:-}"
export LADDER_REPORT_MAX_PAY_HI="${MAX_PAY_HI:-}"
export LADDER_REPORT_MAX_PAY_BEST_ALT="$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}" 2>/dev/null || true)"
export LADDER_REPORT_MAX_LO="${MAX_LO:-}" export LADDER_REPORT_MAX_LO="${MAX_LO:-}"
export LADDER_REPORT_MAX_HI="${MAX_HI:-}" export LADDER_REPORT_MAX_HI="${MAX_HI:-}"
# Human alt_unit_names (e.g. 8.15 Tera-GOA (GOA:…)) for report + final lines # Human alt_unit_names (e.g. 8.15 Tera-GOA (GOA:…)) for report + final lines
@ -1457,6 +1232,10 @@ try:
max_best_alt = os.environ.get("LADDER_REPORT_MAX_BEST_ALT") or None max_best_alt = os.environ.get("LADDER_REPORT_MAX_BEST_ALT") or None
max_lo_alt = os.environ.get("LADDER_REPORT_MAX_LO_ALT") or None max_lo_alt = os.environ.get("LADDER_REPORT_MAX_LO_ALT") or None
max_hi_alt = os.environ.get("LADDER_REPORT_MAX_HI_ALT") or None max_hi_alt = os.environ.get("LADDER_REPORT_MAX_HI_ALT") or None
max_pay_best = os.environ.get("LADDER_REPORT_MAX_PAY_BEST") or None
max_pay_lo = os.environ.get("LADDER_REPORT_MAX_PAY_LO") or None
max_pay_hi = os.environ.get("LADDER_REPORT_MAX_PAY_HI") or None
max_pay_best_alt = os.environ.get("LADDER_REPORT_MAX_PAY_BEST_ALT") or None
alt_path = os.environ.get("LADDER_REPORT_ALT_UNITS") or "" alt_path = os.environ.get("LADDER_REPORT_ALT_UNITS") or ""
alt = {} alt = {}
if alt_path: if alt_path:
@ -1578,6 +1357,7 @@ try:
_best = as_amt(max_best) if max_best else None _best = as_amt(max_best) if max_best else None
_lo = as_amt(max_lo) if max_lo is not None and str(max_lo) != "" else None _lo = as_amt(max_lo) if max_lo is not None and str(max_lo) != "" else None
_hi = as_amt(max_hi) if max_hi is not None and str(max_hi) != "" else None _hi = as_amt(max_hi) if max_hi is not None and str(max_hi) != "" else None
_pb = as_amt(max_pay_best) if max_pay_best else None
_max_block = { _max_block = {
"best": _best, "best": _best,
"best_alt": max_best_alt or (format_alt(_best) if _best else None), "best_alt": max_best_alt or (format_alt(_best) if _best else None),
@ -1585,6 +1365,10 @@ try:
"lo_alt": max_lo_alt or (format_alt(_lo) if _lo else None), "lo_alt": max_lo_alt or (format_alt(_lo) if _lo else None),
"hi": max_hi, "hi": max_hi,
"hi_alt": max_hi_alt or (format_alt(_hi) if _hi else None), "hi_alt": max_hi_alt or (format_alt(_hi) if _hi else None),
"best_pay": _pb,
"best_pay_alt": max_pay_best_alt or (format_alt(_pb) if _pb else None),
"pay_lo": max_pay_lo,
"pay_hi": max_pay_hi,
} }
report = { report = {
@ -1662,6 +1446,8 @@ try:
print(" lo_alt=%s" % la) print(" lo_alt=%s" % la)
print(" hi=%s" % (max_hi if max_hi is not None else "-")) print(" hi=%s" % (max_hi if max_hi is not None else "-"))
print(" hi_alt=%s" % ha) print(" hi_alt=%s" % ha)
print(" best_pay=%s" % (max_pay_best or "-"))
print(" best_pay_alt=%s" % (max_pay_best_alt or (format_alt(as_amt(max_pay_best) or "") if max_pay_best else "-")))
except Exception as e: except Exception as e:
print(" max-search print error: %s" % (e,)) print(" max-search print error: %s" % (e,))
if stop_amt: if stop_amt:
@ -1722,7 +1508,7 @@ if [ "$OK_N" -eq 0 ]; then
exit 1 exit 1
fi fi
if [ "${LADDER_MODE}" = "max" ] && [ -n "${MAX_BEST_AMT:-}" ]; then if [ "${LADDER_MODE}" = "max" ] && [ -n "${MAX_BEST_AMT:-}" ]; then
ok "ladder finished mode=max best=$(format_amount_alt "$MAX_BEST_AMT") · lo=$(format_amount_alt "${CUR}:${MAX_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_HI}") · withdraw_ok=$OK_N phase=${ms_phase}ms" ok "ladder finished mode=max wd_best=$(format_amount_alt "$MAX_BEST_AMT") · pay_best=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}") · withdraw_ok=$OK_N pay_ok=$PAY_OK_N phase=${ms_phase}ms"
else else
ok "ladder finished mode=${LADDER_MODE} withdraw_ok=$OK_N pay_ok=$PAY_OK_N phase=${ms_phase}ms" ok "ladder finished mode=${LADDER_MODE} withdraw_ok=$OK_N pay_ok=$PAY_OK_N phase=${ms_phase}ms"
fi fi

4
check_goa_ladder.sh
View file

@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Compat shim — use check_amount_ladder.sh (generic currency amount ladder). # Compat shim only — prefer: ./taler-monitoring.sh ladder | max-ladder
# Kept so older host-agent / docs invoking check_goa_ladder.sh still work. # Real implementation: check_amount_ladder.sh (+ ladder/ modules).
ROOT=$(cd "$(dirname "$0")" && pwd) ROOT=$(cd "$(dirname "$0")" && pwd)
exec bash "$ROOT/check_amount_ladder.sh" "$@" exec bash "$ROOT/check_amount_ladder.sh" "$@"

25
ladder/README.md Normal file
View file

@ -0,0 +1,25 @@
# Amount ladder modules
Glued by `../check_amount_ladder.sh` (phase entry: `ladder` / `max-ladder`).
| Path | Role |
|------|------|
| `extract_rpubs.py` | Force-select: scrape `reservePub` from wallet dumps (no bash heredoc) |
| `lib_pay.sh` | **Payment** ladder: free-amount template orders, classic pay list, max-search highest **payable** ≤ available |
| *(withdraw stays in `check_amount_ladder.sh`)* | **Withdraw** ladder: classic 23-rung plan + max-search highest **withdrawable** |
`check_goa_ladder.sh` at repo root is only a **compat shim** (`exec` → `check_amount_ladder.sh`). Not required for new scripts; phase alias `goa-ladder` already uses the amount ladder.
## Modes
- **classic** (default): withdraw plan 0→mids→max-1→max, then matching pay plan (never overspend).
- **max** (`max-ladder`): adaptive max withdraw, then adaptive max pay via public free template (hacktivism: `goa-free`).
## Env (pay)
| Variable | Default | Meaning |
|----------|---------|---------|
| `LADDER_PAY` | `1` | `0` = skip all pays |
| `LADDER_USE_FREE_TEMPLATE` | `1` | public POST template with `{"amount":…}` |
| `LADDER_FREE_TEMPLATE` | `goa-free` | template id (hacktivism) |
| `LADDER_FREE_TEMPLATE_INSTANCE` | merchant instance | e.g. `goa-demo-cp4zqk` |

0
ladder_extract_rpubs.py → ladder/extract_rpubs.py
View file

361
ladder/lib_pay.sh Normal file
View file

@ -0,0 +1,361 @@
# ladder/lib_pay.sh — payment side of amount ladder (sourced by check_amount_ladder.sh)
# Expects: lib.sh + metrics.sh already sourced; CUR BANK MER INST EX SCRATCH WDB
# wcli wallet_avail format_amount_alt ladder_over now_ms elapsed_ms
# LADDER_* env, AUTH optional, PAY_TSV PAY_OK_N PAY_FAIL_N
#
# Public free-amount template (hacktivism): goa-free @ goa-demo-cp4zqk
# Never spends more than wallet available.
: "${LADDER_FREE_TEMPLATE:=goa-free}"
: "${LADDER_FREE_TEMPLATE_INSTANCE:=}"
: "${LADDER_USE_FREE_TEMPLATE:=1}"
: "${LADDER_PAY_WAIT_AVAILABLE_S:=60}"
: "${LADDER_PAY_SETTLE_ROUNDS:=6}"
MAX_PAY_BEST_AMT="${MAX_PAY_BEST_AMT:-}"
MAX_PAY_LO="${MAX_PAY_LO:-0}"
MAX_PAY_HI="${MAX_PAY_HI:-0}"
ladder_pay_resolve_merchant() {
# Sets AUTH (may be empty if free template only)
MPW="${E2E_MERCHANT_TOKEN:-${MERCHANT_TOKEN:-}}"
if [ -z "$MPW" ]; then
MPW=$(read_secret "taler-merchant/merchant-${INST}-password.txt" 2>/dev/null || true)
fi
if [ -z "$MPW" ]; then
MPW=$(read_secret "taler-merchant/merchant-goa-demo-cp4zqk-password.txt" 2>/dev/null || true)
fi
if [ -z "$MPW" ] && [ "${CUR}" = "TESTPAYSAN" ]; then
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
MPW=$(tr -d '\n\r' <"${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt")
fi
fi
AUTH=""
if [ -n "$MPW" ]; then
case "$MPW" in
secret-token:*) AUTH="Authorization: Bearer ${MPW}" ;;
*) AUTH="Authorization: Bearer secret-token:${MPW}" ;;
esac
fi
: "${LADDER_FREE_TEMPLATE_INSTANCE:=${INST}}"
}
# Create order for PAMT → OID OTOK PAYURI. Prefers public free template.
ladder_create_order() {
local pamt="$1" ptag="$2"
local inst mh body tpl_https sum_json
OID=""; OTOK=""; PAYURI=""
inst="${LADDER_FREE_TEMPLATE_INSTANCE:-$INST}"
mh=$(python3 -c 'from urllib.parse import urlparse; print(urlparse("'"${MER}"'").hostname or "taler.hacktivism.ch")' 2>/dev/null || echo "taler.hacktivism.ch")
if [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; then
tpl_https="${MER}/instances/${inst}/templates/${LADDER_FREE_TEMPLATE}"
body=$(python3 -c 'import json,sys; print(json.dumps({"amount":sys.argv[1]}))' "$pamt" 2>/dev/null || printf '{"amount":"%s"}' "$pamt")
curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \
-H 'Content-Type: application/json' \
-d "$body" \
"$tpl_https" 2>"$SCRATCH/ord-$ptag.err" || true
OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("order_id") or "")
except Exception:
m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("token") or "")
except Exception:
m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
if [ -n "$OID" ] && [ -n "$OTOK" ]; then
PAYURI="taler://pay/${mh}/instances/${inst}/${OID}/?c=${OTOK}"
PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g')
return 0
fi
fi
[ -n "${AUTH:-}" ] || return 1
sum_json=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "ladder pay ${pamt}" 2>/dev/null || echo '"ladder pay"')
curl -skS -m 20 -o "$SCRATCH/ord-$ptag.json" -X POST \
-H "$AUTH" -H 'Content-Type: application/json' \
-d "{\"order\":{\"summary\":${sum_json},\"amount\":\"${pamt}\",\"fulfillment_message\":\"ok\"},\"create_token\":true}" \
"${MER}/instances/${INST}/private/orders" 2>"$SCRATCH/ord-$ptag.err" || true
OID=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("order_id") or "")
except Exception:
m=re.search(r"\"order_id\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
OTOK=$(python3 -c 'import json,re,sys;t=open(sys.argv[1]).read()
try: print(json.loads(t).get("token") or "")
except Exception:
m=re.search(r"\"token\"\s*:\s*\"([^\"]+)\"",t); print(m.group(1) if m else "")
' "$SCRATCH/ord-$ptag.json" 2>/dev/null || true)
[ -n "$OID" ] || return 1
curl -skS -m 12 -o "$SCRATCH/ord-det-$ptag.json" -H "$AUTH" \
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
PAYURI=$(python3 -c 'import json,sys
try: print(json.load(open(sys.argv[1])).get("taler_pay_uri") or "")
except Exception: print("")
' "$SCRATCH/ord-det-$ptag.json" 2>/dev/null || true)
if [ -z "$PAYURI" ] && [ -n "$OTOK" ]; then
PAYURI="taler://pay/${mh}/instances/${INST}/${OID}/?c=${OTOK}"
fi
PAYURI=$(printf '%s' "$PAYURI" | sed 's/:443\//\//g; s/:443?/?/g')
[ -n "$PAYURI" ]
}
# One pay attempt; never overspends. Always soft (no hard ladder stop).
ladder_pay_attempt() {
local pamt="$1" prange="$2" prung="$3"
local ptag pnum bal t_pay t0 ms_order ms_handle ms_psettle after settled r pstatus pnote
ptag=$(printf '%s' "$pamt" | tr '.:' '__')
pnum=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$pamt")
t_pay=$(now_ms)
ms_order=0; ms_handle=0; ms_psettle=0
pnote=""; pstatus="FAIL"; OID="-"
bal=$(wallet_avail)
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[1])>=Decimal(sys.argv[2]) else 1)' "$bal" "$pnum" 2>/dev/null; then
pstatus="SKIP_BALANCE"
pnote="skip: would spend more than available avail=${CUR}:${bal} need=${pamt}"
warn pay "pay $pamt skipped (balance)" \
"problem: never overspend. $(format_amount_alt "${CUR}:${bal}") free vs $(format_amount_alt "$pamt")"
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t0\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV"
return 0
fi
t0=$(now_ms)
if ! ladder_create_order "$pamt" "$ptag"; then
ms_order=$(elapsed_ms "$t0")
pstatus="FAIL_ORDER"
pnote="order create failed $(head -c 100 "$SCRATCH/ord-$ptag.json" 2>/dev/null | tr '\n\"' ' ')"
warn pay "order $pamt" "$pnote"
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t0\t0\t$(elapsed_ms "$t_pay")\t-\t${pnote}" >>"$PAY_TSV"
return 0
fi
ms_order=$(elapsed_ms "$t0")
ok "order $OID ($pamt) ${ms_order}ms · $(format_amount_alt "$pamt")"
t0=$(now_ms)
if ! wcli handle-uri --yes "$PAYURI" >"$SCRATCH/pay-$ptag.out" 2>&1; then
ms_handle=$(elapsed_ms "$t0")
pstatus="FAIL_HANDLE"
pnote="handle-uri failed $(tail -c 100 "$SCRATCH/pay-$ptag.out" | tr '\n\"' ' ')"
warn pay "handle $pamt" "$pnote"
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t0\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
return 0
fi
ms_handle=$(elapsed_ms "$t0")
ok "handle-uri $pamt ${ms_handle}ms"
t0=$(now_ms)
settled=0
r=0
while [ "$r" -lt "${LADDER_PAY_SETTLE_ROUNDS}" ]; do
r=$((r + 1))
wcli transactions >"$SCRATCH/tx-pay-$ptag.out" 2>&1 || true
if [ -n "${AUTH:-}" ]; then
curl -skS -m 8 -o "$SCRATCH/ord-paid-$ptag.json" -H "$AUTH" \
"${MER}/instances/${INST}/private/orders/${OID}" 2>/dev/null || true
fi
if grep -qiE 'payment|paid|Payment' "$SCRATCH/tx-pay-$ptag.out" 2>/dev/null \
|| python3 -c 'import json,sys
try:
d=json.load(open(sys.argv[1]))
sys.exit(0 if d.get("paid") is True or str(d.get("order_status","")).lower()=="paid" else 1)
except Exception:
sys.exit(1)
' "$SCRATCH/ord-paid-$ptag.json" 2>/dev/null; then
settled=1
break
fi
sleep 0.5
done
ms_psettle=$(elapsed_ms "$t0")
after=$(wallet_avail)
if [ "$settled" = "1" ]; then
pstatus="OK"
pnote="avail=${CUR}:${after}"
ok "pay settled $pamt · $(format_amount_alt "$pamt") → bal ${CUR}:${after}"
PAY_OK_N=$((PAY_OK_N + 1))
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
metrics_report_coins "after-pay-${ptag}" || true
metrics_record_flow spent "$pamt" || true
else
pstatus="FAIL_SETTLE"
pnote="not settled order=$OID avail=${CUR}:${after}"
warn pay "settle $pamt" "$pnote"
echo -e "${prung}\t${prange}\t${pamt}\t${pstatus}\t${ms_order}\t${ms_handle}\t${ms_psettle}\t$(elapsed_ms "$t_pay")\t${OID}\t${pnote}" >>"$PAY_TSV"
fi
return 0
}
ladder_wait_available() {
local wait_s="${1:-$LADDER_PAY_WAIT_AVAILABLE_S}"
local end bal
end=$(( $(date +%s) + wait_s ))
bal=$(wallet_avail)
while python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=0 else 1)' "$bal" 2>/dev/null; do
[ "$(date +%s)" -ge "$end" ] && break
wcli transactions >"$SCRATCH/tx-wait-pay.out" 2>&1 || true
sleep 2
bal=$(wallet_avail)
done
printf '%s' "$bal"
}
# Classic planned pays (PAY_LIST)
ladder_run_pay_classic() {
# shellcheck disable=SC2086
set -- $PAY_LIST
local pay_n=$# prung=0 PAMT prange PNUM
if [ "$pay_n" -eq 0 ]; then
info pay "empty pay plan — skip"
return 0
fi
info "pay rungs" "$pay_n · $*"
for PAMT in "$@"; do
prung=$((prung + 1))
ladder_over && break
if [ "$prung" -eq 1 ]; then
prange="pin:0"
elif [ "$prung" -eq "$pay_n" ]; then
prange="pin:max"
elif [ "$prung" -eq $((pay_n - 1)) ] && [ "$pay_n" -ge 3 ]; then
prange="pin:max-1"
else
prange="random"
fi
section "ladder · pay rung $prung $PAMT · $(format_amount_alt "$PAMT")"
PNUM=$(python3 -c 'import sys; print(sys.argv[1].split(":",1)[-1])' "$PAMT")
if python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])==0 else 1)' "$PNUM" 2>/dev/null; then
echo -e "${prung}\t${prange}\t${PAMT}\tZERO_SKIP\t0\t0\t0\t0\t-\tzero" >>"$PAY_TSV"
PAY_OK_N=$((PAY_OK_N + 1))
continue
fi
ladder_pay_attempt "$PAMT" "$prange" "$prung"
done
}
# Max-search: highest payable ≤ available (uses max_search_next from withdraw module)
ladder_run_pay_max_search() {
local bal prung _next _anum prange _act _last _slo _shi
section "ladder · phase B · max-search pay (highest payable ≤ available)"
bal=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S")
MAX_PAY_LO="0"
MAX_PAY_HI="$bal"
MAX_PAY_BEST_AMT=""
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$MAX_PAY_HI" 2>/dev/null; then
warn pay "max-search pay skipped" \
"problem: wallet available is ${CUR}:0 — need spendable coins (pendingIncoming is not enough)"
return 0
fi
info "max-search pay" "hi=available $(format_amount_alt "${CUR}:${MAX_PAY_HI}") · template=${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE}"
info "pay budget" "never order more than $(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
prung=0
_last=""
while [ "${LADDER_MAX_PROBES}" = "0" ] || [ "$prung" -lt "${LADDER_MAX_PROBES}" ]; do
ladder_over && break
# reuse withdraw amount picker with pay bounds
_slo=${MAX_LO:-0}; _shi=${MAX_HI:-0}
MAX_LO=$MAX_PAY_LO
MAX_HI=$MAX_PAY_HI
if ! type max_search_next >/dev/null 2>&1; then
warn pay "max_search_next missing — cannot run pay max-search"
MAX_LO=$_slo; MAX_HI=$_shi
return 0
fi
_next=$(max_search_next)
MAX_LO=$_slo; MAX_HI=$_shi
_anum=$(printf '%s' "$_next" | cut -d'|' -f1)
prange=$(printf '%s' "$_next" | cut -d'|' -f2 | sed 's/^max:/pay:/')
_act=$(printf '%s' "$_next" | cut -d'|' -f3)
if [ "$_act" = "done" ]; then
info "max-search pay" "converged best=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
break
fi
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])<=Decimal(sys.argv[2]) else 1)' "$_anum" "$MAX_PAY_HI" 2>/dev/null; then
_anum=$MAX_PAY_HI
fi
if ! python3 -c 'import sys; from decimal import Decimal; sys.exit(0 if Decimal(sys.argv[1])>0 else 1)' "$_anum" 2>/dev/null; then
break
fi
PAMT="${CUR}:${_anum}"
if [ -n "$_last" ] && [ "$PAMT" = "$_last" ]; then
MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); print(max(Decimal(0), a-(1 if a>=1 else Decimal("0.000001"))))')
continue
fi
_last=$PAMT
prung=$((prung + 1))
section "ladder · max pay probe $prung $PAMT · $(format_amount_alt "$PAMT") [lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")]"
ladder_pay_attempt "$PAMT" "$prange" "$prung"
_last_st=$(tail -1 "$PAY_TSV" 2>/dev/null | cut -f4)
case "$_last_st" in
OK)
MAX_PAY_LO=$_anum
MAX_PAY_BEST_AMT=$PAMT
info "max-search pay" "success bound lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}")"
;;
SKIP_BALANCE|FAIL_ORDER|FAIL_HANDLE|FAIL_SETTLE)
MAX_PAY_HI=$(python3 -c 'from decimal import Decimal; a=Decimal("'"$_anum"'"); h=Decimal("'"$MAX_PAY_HI"'"); print(min(a,h))')
info "max-search pay" "too-high/fail hi:=$(format_amount_alt "${CUR}:${MAX_PAY_HI}") status=${_last_st}"
;;
esac
if python3 -c 'from decimal import Decimal; import sys; sys.exit(0 if Decimal(sys.argv[1])>0 and Decimal(sys.argv[2])<=Decimal(sys.argv[1]) else 1)' "$MAX_PAY_LO" "$MAX_PAY_HI" 2>/dev/null; then
info "max-search pay" "bounds crossed — stop"
break
fi
done
if [ -n "${MAX_PAY_BEST_AMT:-}" ]; then
ok "max-search best payable $(format_amount_alt "$MAX_PAY_BEST_AMT") · lo=$(format_amount_alt "${CUR}:${MAX_PAY_LO}") · hi=$(format_amount_alt "${CUR}:${MAX_PAY_HI}")"
else
warn pay "max-search no successful pay" "available was ${CUR}:$(wallet_avail)"
fi
{
echo "best_pay=${MAX_PAY_BEST_AMT:-}"
echo "best_pay_alt=$(format_amount_alt "${MAX_PAY_BEST_AMT:-${CUR}:0}" 2>/dev/null || true)"
echo "pay_lo=${MAX_PAY_LO}"
echo "pay_hi=${MAX_PAY_HI}"
} >>"${SCRATCH}/ladder-plan.txt" 2>/dev/null || true
printf '%s\n' "${MAX_PAY_BEST_AMT:-}" >"${SCRATCH}/ladder-pay-plan.txt" 2>/dev/null || true
}
# Entry: glue from check_amount_ladder.sh after withdraw phase
ladder_run_pay_phase() {
PAY_OK_N=0
PAY_FAIL_N=0
if [ "${LADDER_PAY}" != "1" ]; then
info pay "LADDER_PAY=0 — pay phase skipped"
return 0
fi
if [ "${FAIL_N_L:-0}" -gt 0 ]; then
warn pay "skipped pay ladder" "withdraw phase already failed"
return 0
fi
set_group pay
section "ladder · phase B · pay"
metrics_report_coins "before-pay-ladder" || true
ladder_pay_resolve_merchant
if [ -z "${AUTH:-}" ] && ! { [ "${LADDER_USE_FREE_TEMPLATE}" = "1" ] && [ -n "${LADDER_FREE_TEMPLATE:-}" ]; }; then
warn pay "no merchant token and free template off — skip pay"
return 0
fi
if [ -n "${AUTH:-}" ]; then
ok "merchant token" "instance ${INST}"
else
info pay "public free template ${LADDER_FREE_TEMPLATE}@${LADDER_FREE_TEMPLATE_INSTANCE} (no merchant token)"
fi
wallet_prepare "main"
local bal0
bal0=$(ladder_wait_available "$LADDER_PAY_WAIT_AVAILABLE_S")
info "pay budget" "wallet available ${CUR}:${bal0} · $(format_amount_alt "${CUR}:${bal0}") — never order more than this"
if [ "${LADDER_MODE}" = "max" ]; then
ladder_run_pay_max_search
else
ladder_run_pay_classic
fi
metrics_report_coins "after-pay-ladder" || true
info "pay summary" "ok=$PAY_OK_N fail=$PAY_FAIL_N tsv=$PAY_TSV"
}

2
taler-monitoring.sh
View file

@ -94,7 +94,7 @@ Phases:
server server-side only (SSH) server server-side only (SSH)
e2e withdraw + pay (small amounts; remote aborts on login/KYC) e2e withdraw + pay (small amounts; remote aborts on login/KYC)
ladder amount ladder (classic: 0→mids→max; any currency / stack ceiling) ladder amount ladder (classic: 0→mids→max; any currency / stack ceiling)
max-ladder same script, LADDER_MODE=max — hunt highest withdrawable amount max-ladder highest withdrawable + highest payable (free-amount template on GOA)
auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance) auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
aptdeploy koopa podman apt-src smoke: taler-merchant in aptdeploy koopa podman apt-src smoke: taler-merchant in
koopa-taler-deploy-test-apt-src-trixie{,-testing} koopa-taler-deploy-test-apt-src-trixie{,-testing}