docs: CLI-AUTOMATION-NOTES force-select, wallet-db, helper python (1.20.1)

Document magikoopa ladder repro: empty reserve_pub scrape, pin-only
plans, --wallet-db *.json rejection, taler-helper-sqlite3 needs
Python >=3.11, portable wall-clock, GOA withdraw checklist.

CLI-AUTOMATION-NOTES.md

@@ -47,15 +47,25 @@ Legend:
 
 ---
 
-## 3. `reserve_pub` / force-select is fragile (5114)
+## 3. `reserve_pub` / force-select is fragile (5114 + empty scrape)
 
 | | |
 |--|--|
-| **Seen** | ladder force-select, e2e confirm path, Android spinner issues |
+| **Seen** | ladder force-select, e2e confirm path, Android spinner issues; **repro 2026-07-19 magikoopa** GOA ladder: `force-select skipped · no reserve_pub for this withdraw (WID=…); wallet may not have selected yet` |
-| **Area** | **cli** / **core** / **bank** |
+| **Area** | **cli** / **core** / **bank** / **ops** |
-| **Problem** | Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). Automation scrapes accept/tx dumps for candidates. |
+| **Problem** | (a) Cumulative wallets re-emit old reserves; binding the wrong `reserve_pub` yields **HTTP 409 / code 5114** (“already bound”). (b) Automation scrapes accept/tx dumps for candidates — when the dump has **no** pub at all, ladder WARNs and skips force-select (soft), then often **SKIP_CONFIRM**. (c) Empty scrape is **not** the same as “bank out of money”; do not treat as balance failure. |
-| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. |
+| **Workaround** | Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”. Refresh `transactions` before each force-select try. Soft-skip confirm if still pending after polls. Fix wallet DB / helper first if accept/tx only print FATAL (see §15–16) — otherwise scrape can never find a pub. |
-| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. |
+| **Wanted** | CLI returns **the** `reserve_pub` for the just-accepted withdraw in structured JSON; `force-select` / bank-side select API documented with idempotency. Optional: bank integration already exposes `selected_reserve_pub` when wallet selected — prefer that over scraping wallet logs. |
+
+### 3a. Ladder pin amounts hide mid-range behaviour
+
+| | |
+|--|--|
+| **Seen** | `LADDER_STEPS=3` plan collapses to pins only: `GOA:0`, `max-1`, `max` (no random mid rungs) |
+| **Area** | **ops** / **bank** / **cli** |
+| **Problem** | **Zero** withdraw often never produces a usable `reserve_pub` / coins (wallet 7006 or no denoms). **Absolute max** often **mint HTTP 500 / code 5110** SQL `P0001` (ceiling probe — expected soft on max pin). Reproducing “real” force-select needs a **mid** amount (e.g. `GOA:1` … exchange min denom scale), not only pins. |
+| **Workaround** | Use `LADDER_STEPS≥5` (or higher) so random mids exist; treat zero / absolute-max as probes. Soft-skip zero (7006) and max ceiling (5110). |
+| **Wanted** | Ladder plan docs: pin vs mid semantics; CLI structured error for amount-too-large / zero. |
 
 ---
 
@@ -191,6 +201,59 @@ Legend:
 
 ---
 
+## 15. `--wallet-db=*.json` is rejected (“memory backend not supported”)
+
+| | |
+|--|--|
+| **Seen** | **2026-07-19 magikoopa** wallet-cli **1.6.8** / **1.6.11** bundle: any command with `--wallet-db=/tmp/foo.json` fails immediately with `Error: memory backend not supported` |
+| **Area** | **cli** / **ops** |
+| **Problem** | Current node host treats storage paths ending in **`.json` as unsupported memory backend** and throws before opening sqlite. Old notes / local tests that used `WALLET_DB=….json` break. Accept/transactions then never emit `reserve_pub` → ladder §3 empty-scrape WARNs. |
+| **Workaround** | Use a **non-`.json` path**, e.g. `--wallet-db=$SCRATCH/wallet.sqlite3` (ladder already prefers `wallet.sqlite3`). Never pass `….json` as wallet-db. |
+| **Wanted** | Clearer error: `wallet-db path must not end in .json; use a directory or .sqlite3 path`. Document legal path forms in `--help`. |
+
+---
+
+## 16. `taler-helper-sqlite3` requires Python ≥ 3.11 on PATH
+
+| | |
+|--|--|
+| **Seen** | magikoopa: helper is a **Python** script (`#!/usr/bin/env python3`); with Apple `/usr/bin/python3` **3.9.6** → `FATAL: python version >=3.11 required but running on 3.9.6` in accept/tx logs; wallet may still exit 0 on some paths while stdout is FATAL-only |
+| **Area** | **ops** / **cli** |
+| **Problem** | Wallet spawns `taler-helper-sqlite3` from **PATH**. Wrong `python3` ⇒ sqlite backend fails ⇒ no real wallet state ⇒ no `reserve_pub` for force-select. |
+| **Workaround** | Put **Python ≥3.11** (or 3.12+) first on PATH for mon jobs; ensure `command -v taler-helper-sqlite3` works; smoke: `taler-helper-sqlite3` starts without FATAL. Prefer Homebrew / user `python3.11+` over macOS system 3.9 for agent PATH. |
+| **Wanted** | Helper shebang or wrapper that finds a suitable python; wallet preflight error if helper HELLO fails (not silent empty logs). |
+
+---
+
+## 17. Portable wall-clock for scripts (no GNU `timeout` assumed)
+
+| | |
+|--|--|
+| **Seen** | macOS agent shells: `timeout: command not found`; e2e already tries `timeout` then `gtimeout` then `perl -e alarm` |
+| **Area** | **ops** / **doc** |
+| **Problem** | GNU coreutils `timeout` is not on stock macOS. Hard-wiring it breaks portable repro and CI on Darwin. |
+| **Workaround** | Prefer **python3** `subprocess.Popen(…).wait(timeout=…)` for outer budgets; for wallet-cli inner calls keep e2e order: `timeout` → `gtimeout` → perl alarm. Document both. |
+| **Wanted** | One suite helper `run_with_budget SECS cmd…` in `lib.sh` used by ladder/e2e/smoke. |
+
+---
+
+## 18. How to withdraw (operator checklist, GOA laptop)
+
+Portable path that works when §§15–16 are fixed:
+
+1. **PATH:** `taler-helper-sqlite3` + **python3 ≥ 3.11** ahead of system 3.9.  
+2. **Wallet DB:** `WDB=$TMP/wallet.sqlite3` — **not** `*.json`.  
+3. **Exchange:** `exchanges add` → `update -f` → `accept-tos` for `https://exchange.hacktivism.ch/`.  
+4. **Bank mint (explorer):** token with explorer password from `SECRETS_ROOT` / `EXP_PW`; `POST …/accounts/explorer/withdrawals` with e.g. `{"amount":"GOA:1"}` (avoid pure **0** and absolute **max** pins for happy-path tests).  
+5. **Strip** `:443` from `taler_withdraw_uri`.  
+6. **`withdraw accept-uri --exchange URL URI`**.  
+7. **Bank:** poll `…/taler-integration/withdrawal-operation/{WID}` → when `selected`, confirm; if stuck `pending`, force-select with scraped `reserve_pub` (§3).  
+8. **Settle:** poll wallet balance / bank `transfer_done` — **no** `run-until-done` (§1).
+
+If step 6’s stdout is only `FATAL: python version…` or `memory backend not supported`, **stop** and fix §15–16 before debugging force-select.
+
+---
+
 ## Priority shortlist for `taler-wallet-cli` / core
 
 If only a few changes land, these unlock the most automation:
@@ -200,6 +263,7 @@ If only a few changes land, these unlock the most automation:
 3. **Auto-add exchange + ToS on accept-uri** (§4, §8)  
 4. **Canonical URIs without :443** (§5)  
 5. **Emit reserve_pub for last withdraw** (§3)  
+6. **Clear wallet-db path rules + helper preflight** (§15, §16)  
 
 ---
 
@@ -212,7 +276,9 @@ If only a few changes land, these unlock the most automation:
 | exchange | Explicit add/update/accept-tos |
 | ladder explorer | `read_secret` / EXP_PW_FILE |
 | stage maxima | bank `max_wire` + keys min denom |
-| force-select | multi-rpub try; soft-skip confirm |
+| force-select | multi-rpub try; soft-skip confirm; empty scrape → WARN not balance fail |
+| wallet-db path | Prefer `*.sqlite3` (never `*.json` as db path) |
+| helper / python | mon PATH must include helper + python≥3.11 |
 
 ---
 

VERSION

@@ -1 +1 @@
-1.20.0
+1.20.1

VERSIONS.md

@@ -17,6 +17,7 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre
 
 | Tag | Date (UTC) | Notes |
 |-----|------------|--------|
+| **v1.20.1** | 2026-07-19 | **Docs:** CLI-AUTOMATION-NOTES — force-select empty scrape + ladder pins (§3/3a); wallet-db must not be `*.json` (§15); `taler-helper-sqlite3` needs Python ≥3.11 (§16); portable wall-clock (§17); GOA withdraw checklist (§18). No code change. |
 | **v1.20.0** | 2026-07-19 | **Feature:** domains.conf **`locale`** (l10n) — FP **`fr-CH`**, others **`de-CH`** (aliases ch-FR/ch-DE); UI **`lang=de`** supported (sticky + console tags) but **never** a stock profile default; env `TALER_MON_LOCALE` / `TALER_DOMAIN_LOCALE`. |
 | **v1.19.1** | 2026-07-19 | **Bugfix / docs:** DEPENDENCIES — e2e requires `taler-helper-sqlite3` on mon PATH + preflight; README domains.conf documents **`lang`** column. |
 | **v1.19.0** | 2026-07-19 | **Feature:** domains.conf **`lang`** column (en\|fr) is the global UI language default per stack; i18n + HTML read `TALER_DOMAIN_LANG` / conf (FP profiles `lang=fr`). |