release 1.21.0: amount ladder classic + max-search

Generic currency ladder (check_amount_ladder.sh; goa-ladder shim).
Default remains classic 23-rung withdraw/pay. New max-ladder mode
hunts highest withdrawable amount with alt_unit_names in logs/report.

Fixes force-select extract (external ladder_extract_rpubs.py; no
stdin return SyntaxError), soft CEILING_REJECT on mint 5110/P0001,
force-select 409 tries next rpub, pay SKIP_BALANCE when over available,
and crash-proof report formatting. Verified live GOA: force-select 200,
confirm 204, OK_BANK, report finishes without Traceback.

CLI-AUTOMATION-NOTES.md

@@ -9,7 +9,7 @@ Companions:
 - Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md)
 - Git / suite tree → [`VERSIONS.md`](./VERSIONS.md)
 
-Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`.
+Scripts: `check_e2e.sh`, `check_amount_ladder.sh` (compat `check_goa_ladder.sh`), `taler-monitoring.sh`.
 
 Legend:
 
@@ -67,6 +67,16 @@ Legend:
 | **Workaround** | Use `LADDER_STEPS≥5` (or higher) so random mids exist; treat zero / absolute-max / max-1-5110 as probes (soft `CEILING_REJECT` in mon ≥1.20.2). |
 | **Wanted** | Ladder plan docs: pin vs mid semantics; CLI structured error for amount-too-large / zero. |
 
+### 3b. Higher withdraw amounts often fail at mint
+
+| | |
+|--|--|
+| **Seen** | Classic GOA ladder high/random mids and ceiling pins — e.g. `GOA:747827072844319` → `FAIL_MINT` / `STOPPED … mint HTTP 500 { code :5110, … P0001 }`; same class of error on max / max-1. |
+| **Area** | **bank** / **ops** |
+| **Problem** | **Larger amounts** frequently hit bank/libeufin limits (HTTP 500, Taler **5110**, SQL **P0001**). Not specific to pin:max only — mid-high ladder steps fail the same way. Do not treat as wallet/`reserve_pub` when mint never succeeded. |
+| **Workaround** | Expect failures above some live ceiling; use max-search / soft ceiling handling; keep mid-range rungs for real withdraw/confirm tests. |
+| **Wanted** | Clear amount-too-large from bank (not opaque SQL 500); mon reports ceiling separately from hard fails. |
+
 ---
 
 ## 4. Exchange not auto-known for bank deep links / withdraw URIs
@@ -277,6 +287,8 @@ If only a few changes land, these unlock the most automation:
 | ladder explorer | `read_secret` / EXP_PW_FILE |
 | stage maxima | bank `max_wire` + keys min denom |
 | force-select | multi-rpub try; soft-skip confirm; empty scrape → WARN not balance fail |
+| higher amounts | mint 5110/P0001 common above live ceiling (§3b); classic soft `CEILING_REJECT`; max-ladder bounds hi; report uses alt_unit_names (Tera-GOA …) |
+| pay vs balance | never order more than wallet **available** (soft `SKIP_BALANCE`); pendingIncoming does not count as spendable |
 | wallet-db path | Prefer `*.sqlite3` (never `*.json` as db path) |
 | helper / python | mon PATH must include helper + python≥3.11 |
 

README.md

@@ -274,12 +274,14 @@ Other domains: never SSH. Optional **e2e** aborts cleanly on login/KYC.
 ```bash
 ./taler-monitoring.sh e2e
 ./taler-monitoring.sh -d taler.net urls e2e
-# Amount ladder (withdraw 0 → … → max, then pay): same script for GOA + stage
+# Amount ladder (any currency): classic 0 → … → max, then optional pay
-./taler-monitoring.sh ladder                              # GOA / current domain
+./taler-monitoring.sh ladder                              # classic (current domain)
 ./taler-monitoring.sh -d stage.lefrancpaysan.ch ladder    # TESTPAYSAN maxima
+# Max-search: start high, narrow toward highest withdrawable
+./taler-monitoring.sh max-ladder
 # Stage auto-reads bank max_wire_transfer_amount (e.g. 2000) and min denom (0.01).
 # Explorer password: francpaysan-stage-user …/bank-explorer-password.txt (or EXP_PW=).
-# Override: LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 …
+# Override: LADDER_MODE=max LADDER_MAX_PROBES=16 LADDER_MAX_AMOUNT=100 LADDER_STEPS=7 LADDER_PAY=0 …
 # customize e2e:
 E2E_WITHDRAW_VALUES="20 50 100" E2E_PAY_VALUES="0.05 1 5" ./taler-monitoring.sh e2e
 E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e   # single fixed
@@ -304,7 +306,7 @@ E2E_VARIABLE=0 WITHDRAW_AMT=GOA:50 PAY_AMT=GOA:1 ./taler-monitoring.sh e2e   # s
 | **versions** | `deb.taler.net` reachable (InRelease/Packages/pool `.deb`); containers can reach it + have apt source; installed Taler packages vs **trixie** |
 | **sanity** | bank · exchange · merchant sections (public + server) |
 | **e2e** | account → credit → withdraw → wallet → confirm → order → pay |
-| **ladder** | GOA withdraw: fixed **0** + low bands + **random high ranges** + fixed **max** |
+| **ladder** | amount ladder: **classic** (0 + mids + max) or **max-search** (`max-ladder` / `LADDER_MODE=max`) |
 
 ```bash
 # package suite (default trixie on deb.taler.net)
@@ -394,8 +396,12 @@ LADDER_LOAD=0 ./taler-monitoring.sh ladder
 | `METRICS_LOAD` | `1` | `0` = skip all host/container load probes |
 | `LADDER_LOAD` | `1` | `0` = skip load in ladder only |
 | `LADDER_PAY` | `1` | `0` = skip payment ladder after withdraws |
-| `LADDER_WITHDRAW_SCALE` | `1.5` | withdraw mids / pay mids ratio (funding headroom) |
+| `LADDER_MODE` | `classic` | default **classic** 23-rung; `max` = hunt highest withdrawable (`max-ladder`) |
-| `LADDER_STEPS` | `23` | rungs for both withdraw and pay (0 + mids + max−1 + max) |
+| `LADDER_MAX_PROBES` | `32` | max-search probe budget; `0` = until converge / timeout |
+| `LADDER_HIGH_RUNGS` | `6` | max-search: first N probes random-high, then bisect |
+| `LADDER_MAX_TOL_REL` | `0.001` | max-search relative lo/hi gap to stop |
+| `LADDER_WITHDRAW_SCALE` | `1.5` | classic: withdraw mids / pay mids ratio (funding headroom) |
+| `LADDER_STEPS` | `23` | classic: rungs for withdraw and pay (0 + mids + max−1 + max) |
 | `METRICS_LOAD_SSH_TIMEOUT` | `90` | seconds for remote load python |
 
 ## Needs

TESTS.md

@@ -25,7 +25,7 @@ Every check line has a **global** run number and a **grouped** id:
 | **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
 | **server** | `check_server.sh` | (flat `server-NN` or host groups) |
 | **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `bankwd` `settle` `pay` `shop` `paivana` `dig` `report` |
-| **ladder** | `check_goa_ladder.sh` | `plan` `load` `withdraw` `pay` `report` |
+| **ladder** | `check_amount_ladder.sh` (alias `check_goa_ladder.sh`) | `plan` `load` `withdraw` `pay` `report` — modes classic / max (`max-ladder`) |
 
 **Why groups:** flat `www-001`…`www-080` was hard to talk about.  
 `www.bank-04` / `e2e.bankwd-02` pin the failure to a logical block.

VERSION

@@ -1 +1 @@
-1.20.2
+1.21.0

VERSIONS.md

@@ -17,8 +17,8 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre
 
 | Tag | Date (UTC) | Notes |
 |-----|------------|--------|
-| **v1.20.2** | 2026-07-19 | **Bugfix:** ladder force-select empty `reserve_pub` scrape — wallet dumps mix log lines + JSON so `json.loads(rest_of_file)` always failed; extract now uses `JSONDecoder.raw_decode` and regex on **both** accept + transactions (`reservePub` is in tx). Prefer Homebrew python on macOS for helper; `find_wallet_cli` prefers ts-core 1.6.x over stale system wrappers. Soft-skip max-1 mint 5110/P0001 as ceiling. Verified: mid rungs `force-select HTTP 200` + `confirm 204` → `OK_BANK`. |
+| **v1.21.0** | 2026-07-19 | **Feature:** generic **amount ladder** (`check_amount_ladder.sh`; compat shim `check_goa_ladder.sh`). Default **classic** 23-rung withdraw (+ pay when enabled); **max-search** (`max-ladder` / `LADDER_MODE=max`) hunts highest mintable amount (probes default 32, `0`=until timeout). **Pay:** never overspend — soft `SKIP_BALANCE` if amount > wallet available. **Alt units:** report/logs show human names (e.g. `8.15 Tera-GOA (GOA:…)`) + raw. **Fixes bundled:** external `ladder_extract_rpubs.py` (no stdin SyntaxError); force-select multi-rpub + raw_decode; 409 tries next rpub; soft `CEILING_REJECT` on mid/high mint 5110/P0001; report python never aborts ladder; helper PATH python≥3.11; wallet-db not `*.json`. CLI-AUTOMATION-NOTES §3/3a/3b/§15–18. |
-| **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. |
+| **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. *(Superseded notes folded into **v1.21.0** feature release.)* |
 | **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. |
 | **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. |
 | **v1.19.0** | 2026-07-19 | **Feature:** domains.conf **`lang`** column (en\|fr) is the global UI language default per stack; i18n + HTML read `TALER_DOMAIN_LANG` / conf (FP profiles `lang=fr`). |

ladder_extract_rpubs.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+"""Extract reserve_pub candidates from wallet accept + transactions dumps.
+
+Used by check_amount_ladder.sh force-select. Kept as a file (not a bash
+heredoc) so shell refactors cannot turn Python ``continue`` into ``return``.
+
+Usage:
+  ladder_extract_rpubs.py WID AMT accept.out tx.json used-rpubs.txt
+Prints one reserve pub per line, preferred first.
+"""
+from __future__ import annotations
+
+import json
+import re
+import sys
+from pathlib import Path
+
+
+def main(argv: list[str]) -> int:
+    if len(argv) < 6:
+        print(
+            "usage: ladder_extract_rpubs.py WID AMT accept.out tx.json used-rpubs.txt",
+            file=sys.stderr,
+        )
+        return 2
+
+    wid = argv[1]
+    amt = argv[2]
+    paths = argv[3:5]
+    used_path = argv[5]
+    used: set[str] = set()
+    if Path(used_path).is_file():
+        used = {ln.strip() for ln in open(used_path) if ln.strip()}
+
+    amt_num = amt.split(":", 1)[-1] if amt else ""
+    dec = json.JSONDecoder()
+
+    def walk_collect(o, bag, ctx=None):
+        ctx = dict(ctx or {})
+        if isinstance(o, dict):
+            for k, v in o.items():
+                if not isinstance(v, str):
+                    continue
+                kl = str(k).lower().replace("-", "_")
+                if kl in (
+                    "withdrawal_id",
+                    "withdraw_id",
+                    "wopid",
+                    "id",
+                    "transactionid",
+                    "transaction_id",
+                ):
+                    ctx["id"] = v
+                elif kl in (
+                    "amount",
+                    "rawamount",
+                    "instructedamount",
+                    "amounteffective",
+                    "amountraw",
+                    "transferamount",
+                ):
+                    ctx["amount"] = v
+                elif kl in (
+                    "taler_withdraw_uri",
+                    "talerwithdrawuri",
+                    "uri",
+                    "bankconfirmationurl",
+                    "confirmtransferurl",
+                ):
+                    ctx["uri"] = v
+            for k, v in o.items():
+                kl = str(k).lower().replace("-", "_")
+                if kl in ("reserve_pub", "reservepub") and isinstance(v, str) and len(v) >= 40:
+                    bag.append((v, dict(ctx)))
+                elif not isinstance(v, (str, int, float, bool)) and v is not None:
+                    walk_collect(v, bag, ctx)
+        elif isinstance(o, list):
+            for i in o:
+                walk_collect(i, bag, ctx)
+
+    def score_ctx(pub, ctx, window=""):
+        score = 0
+        cid = str(ctx.get("id") or "")
+        camt = str(ctx.get("amount") or "")
+        curi = str(ctx.get("uri") or "")
+        if wid and wid in curi:
+            score += 120
+        if wid and wid in cid:
+            score += 100
+        if wid and wid in window:
+            score += 80
+        if amt and (camt == amt or camt.endswith(amt_num)):
+            score += 50
+        if amt and amt in window:
+            score += 30
+        if pub in used:
+            score -= 200
+        return score
+
+    def iter_json_objects(text: str):
+        i, n = 0, len(text)
+        while i < n:
+            if text[i] == "{":
+                try:
+                    o, end = dec.raw_decode(text, i)
+                    yield o
+                    i = end
+                    continue
+                except Exception:
+                    pass
+            i += 1
+
+    raw_pubs: list[str] = []
+    scored: list[tuple[int, str]] = []
+
+    for p in paths:
+        try:
+            t = open(p, errors="replace").read()
+        except Exception:
+            continue
+        for o in iter_json_objects(t):
+            bag = []
+            walk_collect(o, bag)
+            for pub, ctx in bag:
+                raw_pubs.append(pub)
+                scored.append((score_ctx(pub, ctx), pub))
+
+    rpub_pats = (
+        r"\"reservePub\"\s*:\s*\"([A-Z0-9]{40,})\"",
+        r"\"reserve_pub\"\s*:\s*\"([A-Z0-9]{40,})\"",
+        r"reserve[_ ]?pub[\"\s:=]+([A-Z0-9]{40,})",
+    )
+    for p in paths:
+        try:
+            blob = open(p, errors="replace").read()
+        except Exception:
+            continue
+        for pat in rpub_pats:
+            for m in re.finditer(pat, blob, re.I):
+                pub = m.group(1)
+                raw_pubs.append(pub)
+                window = blob[max(0, m.start() - 600) : m.end() + 600]
+                scored.append((score_ctx(pub, {}, window) + 10, pub))
+
+    order: list[str] = []
+    seen: set[str] = set()
+    for score, pub in sorted(scored, key=lambda x: x[0], reverse=True):
+        if pub in seen or pub in used:
+            continue
+        seen.add(pub)
+        order.append(pub)
+    for pub in reversed(raw_pubs):
+        if pub in seen or pub in used:
+            continue
+        seen.add(pub)
+        order.append(pub)
+    for pub in order:
+        print(pub)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))

metrics.sh

@@ -70,11 +70,15 @@ path = sys.argv[2] if len(sys.argv) > 2 else ""
 alt = {}
 if path:
     try:
-        alt = json.load(open(path))
+        _loaded = json.load(open(path))
+        if isinstance(_loaded, dict):
+            alt = {str(k): str(v) for k, v in _loaded.items()}
     except Exception:
         alt = {}
-if not alt:
+if not isinstance(alt, dict) or not alt:
     alt = {"0": "GOA"}
+elif "0" not in alt:
+    alt["0"] = "GOA"
 
 def parse(s):
     if ":" in s:

taler-monitoring.sh

@@ -93,7 +93,8 @@ Phases:
   sanity   public + optional server
   server   server-side only (SSH)
   e2e      withdraw + pay (small amounts; remote aborts on login/KYC)
-  ladder   withdraw/pay amount ladder (GOA ceiling or stage TESTPAYSAN max_wire)
+  ladder   amount ladder (classic: 0→mids→max; any currency / stack ceiling)
+  max-ladder  same script, LADDER_MODE=max — hunt highest withdrawable amount
   auth401  merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
   aptdeploy  koopa podman apt-src smoke: taler-merchant in
              koopa-taler-deploy-test-apt-src-trixie{,-testing}
@@ -265,7 +266,7 @@ while [ $# -gt 0 ]; do
       CURRENCY_OVERRIDE="$2"; shift 2
       ;;
     --no-probe) NO_PROBE=1; shift ;;
-    urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;;
+    urls|inside|versions|sanity|server|e2e|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;;
     *)
       # bare domain shorthand: ./taler-monitoring.sh taler.net
       if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then
@@ -316,7 +317,7 @@ export INSIDE_PROFILE INSIDE_SSH
 export INSIDE_BANK_CTR INSIDE_EXCHANGE_CTR INSIDE_MERCHANT_CTR
 export INSIDE_BANK_PORT INSIDE_EXCHANGE_PORT INSIDE_MERCHANT_PORT
 export INSIDE_DNS_BANK INSIDE_DNS_EXCHANGE INSIDE_DNS_MERCHANT
-# Ladder: withdraw then pay — 0 + random mids + max-1 + max (see check_goa_ladder.sh).
+# Amount ladder — classic or max-search (see check_amount_ladder.sh; goa-ladder alias).
 # Defaults so set -u export is safe when vars were never set by caller.
 : "${LADDER_STEPS:=23}"
 : "${LADDER_MIN_AMOUNT:=0.000001}"
@@ -330,16 +331,23 @@ export INSIDE_DNS_BANK INSIDE_DNS_EXCHANGE INSIDE_DNS_MERCHANT
 : "${LADDER_INCLUDE_ZERO:=1}"
 : "${LADDER_INCLUDE_MAX:=1}"
 : "${LADDER_HIGH_FROM:=1000000}"
-: "${LADDER_HIGH_RUNGS:=12}"
+: "${LADDER_HIGH_RUNGS:=4}"
 : "${LADDER_LOAD:=1}"
 : "${LADDER_PAY:=1}"
 : "${LADDER_WITHDRAW_SCALE:=1.5}"
 : "${LADDER_PAY_SETTLE_ROUNDS:=6}"
+: "${LADDER_MODE:=classic}"
+: "${LADDER_MAX_PROBES:=32}"
+: "${LADDER_MAX_TOL_REL:=0.001}"
+: "${LADDER_MAX_SEED:=}"
+: "${LADDER_PAY_WITH_MAX:=0}"
+: "${LADDER_HIGH_RUNGS:=6}"
 export LADDER_STEPS LADDER_MIN_AMOUNT LADDER_CONFIRM_POLLS LADDER_MAX_RUNGS LADDER_TIMEOUT_S LADDER_REPORT_DIR
 export LADDER_SETTLE_ROUNDS LADDER_SETTLE_SLEEP EXP_PW_FILE EXP_USER
 export LADDER_MAX_AMOUNT LADDER_INCLUDE_ZERO LADDER_INCLUDE_MAX
 export LADDER_HIGH_FROM LADDER_HIGH_RUNGS LADDER_LOAD
 export LADDER_PAY LADDER_WITHDRAW_SCALE LADDER_PAY_SETTLE_ROUNDS
+export LADDER_MODE LADDER_MAX_PROBES LADDER_MAX_TOL_REL LADDER_MAX_SEED LADDER_PAY_WITH_MAX
 export TALER_DOMAIN_APPLIED=1
   i18n_init 2>/dev/null || true
 # Optional: auth401 and other phases may honor CONTINUE_ON_ERROR
@@ -435,7 +443,7 @@ _progress_estimate_phase() {
     server)   n=20 ;;
     # e2e: bank withdraw ladder emits many INFO lines (coins before/after each note)
     e2e)      n=240 ;;
-    ladder|goa-ladder) n=120 ;;
+    ladder|goa-ladder|max-ladder|maxladder) n=120 ;;
     auth401)  n=70 ;;
     aptdeploy) n=40 ;;
     surface|ecosystem) n=120 ;;
@@ -611,7 +619,15 @@ while [ "$_phase_idx" -lt "$_n_phases" ]; do
     sanity)   run_phase sanity   "$ROOT/check_sanity.sh"   || ec=1 ;;
     server)   run_phase server   "$ROOT/check_server.sh"   || ec=1 ;;
     e2e)      run_phase e2e      "$ROOT/check_e2e.sh"      || ec=1 ;;
-    ladder|goa-ladder) run_phase ladder "$ROOT/check_goa_ladder.sh" || ec=1 ;;
+    ladder|goa-ladder)
+      run_phase ladder "$ROOT/check_amount_ladder.sh" || ec=1
+      ;;
+    max-ladder|maxladder)
+      # Feature: hunt highest mint+confirm withdrawable amount (any currency)
+      LADDER_MODE=max
+      export LADDER_MODE
+      run_phase ladder "$ROOT/check_amount_ladder.sh" || ec=1
+      ;;
     auth401)  run_phase auth401  "$ROOT/check_auth401.sh"  || ec=1 ;;
     aptdeploy|apt-deploy|apt_src) run_phase aptdeploy "$ROOT/check_apt_deploy.sh" || ec=1 ;;
     surface|ecosystem) run_phase surface "$ROOT/check_surface.sh" || ec=1 ;;