taler-monitoring/CLI-AUTOMATION-NOTES.md

CLI automation notes (taler-wallet-cli + monitoring)

Living list of recurring issues seen while driving GOA (hacktivism) and stage TESTPAYSAN via CLI (taler-wallet-cli, bank/merchant HTTP, monitoring e2e/ladder). Ordered roughly by how often they hurt automation and how useful a fix in wallet-cli / wallet-core (or clearer APIs) would be.

Companions:

• Android GUI → android-test/GUI-AUTOMATION-NOTES.md
• Git / suite tree → VERSIONS.md

Scripts: check_e2e.sh, check_goa_ladder.sh, taler-monitoring.sh.

Legend:

Tag	Meaning
cli	wallet-cli UX / flags / hang behaviour
core	wallet-core protocol / state machine
bank	libeufin-bank / integration API
ops	secrets, paths, multi-host
doc	missing or wrong documentation

---

1. run-until-done is unusable for automation

Seen	e2e, ladder, Android notes, macOS comments: hangs / banned in monitoring
Area	cli / core
Problem	taler-wallet-cli run-until-done (or long shepherd runs) often never returns or blocks CI-length timeouts. Monitoring forbids it and polls balance + bank transfer_done instead.
Workaround	advanced serve + socket (WSOCK) or pure poll loops (LADDER_SETTLE_*, e2e settle wait).
Wanted for wallet-cli	Bounded wait: e.g. `run-until-done --timeout=Ns --exit-on=withdrawn

---

2. Withdraw completion is not observable from CLI alone

Seen	e2e “coins missing after withdraw”; ladder OK_BANK vs wallet avail
Area	cli / core / bank
Problem	After accept-uri + bank confirm, coins may lag (wirewatch). CLI has no single “withdraw settled” command with clear success/failure; automation reimplements status from bank integration JSON + balance polls.
Workaround	Poll balance + …/withdrawal-operation/{id} (status, transfer_done, selected_reserve_pub).
Wanted	wallet-cli withdrawals wait --id=… --timeout= or JSON event stream: selected → confirmed → coins-available.

---

3. reserve_pub / force-select is fragile (5114)

Seen	ladder force-select, e2e confirm path, Android spinner issues
Area	cli / core / bank
Problem	Cumulative wallets re-emit old reserves; binding the wrong reserve_pub yields HTTP 409 / code 5114 (“already bound”). Automation scrapes accept/tx dumps for candidates.
Workaround	Score pubs by WID/amount; try several; treat 5114 as “stale reserve” not “out of money”.
Wanted	CLI returns the reserve_pub for the just-accepted withdraw in structured JSON; force-select / bank-side select API documented with idempotency.

---

4. Exchange not auto-known for bank deep links / withdraw URIs

Seen	Android endless spinner; GOA bank.hacktivism.ch withdraw; fixed in app branch fix/bank-withdraw-auto-exchange
Area	core / cli
Problem	Withdraw URI names an exchange the wallet has never added → stuck loading / empty exchange list. CLI requires explicit exchanges add + update + accept-tos every fresh DB.
Workaround	e2e always adds exchange before bank withdraw ladder; Android needs ensureExchange-style fix.
Wanted	CLI/core: on accept-uri withdraw, auto-add exchange from URI/bank details (allowCompletion), then surface ToS if needed in one command.

---

5. Default ports in URIs (:443 / :80) break round-trips

Seen	ladder/e2e strip ports; QR checks; mint URIs with :443
Area	cli / bank / doc
Problem	Some bank/wallet outputs include taler://…host:443/…. QR validators and some clients reject or double-normalize inconsistently.
Workaround	sed 's/:443\//\//g' everywhere in automation.
Wanted	Canonical form without default ports in all wallet-cli and bank integration outputs; document as invariant.

---

6. No first-class “demo withdraw” / explorer mint in wallet-cli

Seen	e2e uses bank admin + bank withdraw create; ladder/gui-chain reimplement explorer mint in Python
Area	cli / ops
Problem	Automation always hand-rolls Basic auth → token → POST …/withdrawals → taler_withdraw_uri. Easy to get wrong (auth header, amount currency).
Workaround	Shared Python in goa-chain / ladder / e2e; secrets files.
Wanted	Optional helper: wallet-cli testing mint-withdraw --bank=URL --user=explorer --password-file=… --amount=GOA:10 (or bank-side only tool in libeufin) emitting a clean URI.

---

7. Long-lived wallet process required for pay/withdraw progress

Seen	e2e advanced serve / shepherd socket
Area	cli
Problem	One-shot CLI invocations do not keep background work alive; without serve/shepherd, withdraw/pay may stall mid-flight.
Workaround	Start serve once per e2e run; route wcli through the socket.
Wanted	Documented, supported “session mode”: start/stop serve; or each mutating command optionally --background-until=… with timeout.

---

8. ToS accept is a separate step that fails silently or blocks

Seen	e2e exchanges accept-tos; ladder after add; Android GUI ToS screens
Area	cli / core
Problem	Fresh exchange → operations need ToS; CLI must call accept-tos explicitly. Failures are easy to miss in multi-step scripts.
Workaround	Always accept-tos after update in e2e/ladder.
Wanted	accept-uri --accept-tos or auto-prompt with non-interactive --yes that covers exchange ToS for that URI’s exchange.

---

9. Pay template / public order path is merchant-shaped, not CLI-shaped

Seen	e2e shop templates, stage farmer shops, ladder private orders
Area	cli / doc
Problem	Wallet-cli pays via handle-uri on taler://pay/…; creating the order is always custom curl (template POST or private order + token). No unified “pay this amount to instance” for public templates.
Workaround	Monitoring builds URI externally then handle-uri --yes.
Wanted	Documented recipe only, or wallet-cli testing pay-template --base=… --instance=… --id=… for demos.

---

10. Amount / currency parsing edge cases

Seen	ladder min denom 0.01 vs GOA 1e-6; zero withdraw rejected (HTTP 409 / amount too low); max wire ceilings
Area	cli / bank
Problem	CURRENCY:0 and sub-min amounts fail differently per stack; CLI error strings are not machine-stable.
Workaround	Soft-skip zero rung; clamp ladder min to exchange denoms / bank max_wire.
Wanted	Structured errors (AMOUNT_too_small, currency_unknown) in JSON mode; wallet-cli amount validate --exchange=.

---

11. Finding the right taler-wallet-cli binary

Seen	find_wallet_cli in lib.sh; hardcoded laptop paths; wrapper vs .mjs
Area	ops / cli
Problem	Debian package is a shell wrapper; some tools need node …/taler-wallet-cli.mjs. Hardcoded /Users/… paths break other hosts.
Workaround	find_wallet_cli search list; WALLET_CLI= override.
Wanted	Single install story: wallet-cli --version JSON with path + libversion; no need to pass .mjs to node by hand.

---

12. Secrets and multi-stack confusion

Seen	explorer vs admin password; GOA secrets used on stage; ladder EXP_PW
Area	ops
Problem	CLI does not know “which stack”; wrong password → 401 mid-ladder. Not a wallet-cli bug, but every CLI automation hits it.
Workaround	SECRETS_ROOT, stage SSH, STACK= profiles.
Wanted	Optional ~/.config/taler/stacks.d/goa.env convention documented next to wallet-cli; still no secrets in repo.

---

13. No stable machine-readable “step result” for scripts

Seen	e2e greps accept output; ladder scrapes JSON from mixed stdout
Area	cli
Problem	Human logs + occasional JSON blobs; hard to parse reliably.
Workaround	Python scrapers, temp files, tee.
Wanted	Global --json / --ndjson for all commands; one object per completed operation with ok, op, ids, amounts.

---

14. Pay settlement wait is symmetric to withdraw pain

Seen	e2e pay settle loops; ladder pay settle rounds
Area	cli / core
Problem	After handle-uri pay, success is “order paid” / balance drop / merchant order status — not one CLI wait.
Workaround	Short poll loops; never run-until-done.
Wanted	Same as §1–2: bounded wait on transaction id / order id.

---

Priority shortlist for taler-wallet-cli / core

If only a few changes land, these unlock the most automation:

1. Bounded run-until-done / wait-for-state (§1, §2, §14)
2. Structured JSON on every command (§13)
3. Auto-add exchange + ToS on accept-uri (§4, §8)
4. Canonical URIs without :443 (§5)
5. Emit reserve_pub for last withdraw (§3)

---

How we work around today (monitoring)

Issue	Monitoring behaviour
run-until-done	Disabled; balance + bank poll
serve	Optional long-lived socket in e2e
exchange	Explicit add/update/accept-tos
ladder explorer	read_secret / EXP_PW_FILE
stage maxima	bank max_wire + keys min denom
force-select	multi-rpub try; soft-skip confirm

---

How to add an issue

### N. short title
| | |
|--|--|
| **Seen** | where / which stack |
| **Area** | cli / core / bank / ops / doc |
| **Problem** | … |
| **Workaround** | … |
| **Wanted** | concrete CLI/core behaviour |