docs: §3b analysis — healthy load but we do not manage to withdraw
Replace probe dumps with outcome analysis: stack load is fine, mint 5110 / SKIP_CONFIRM / pendingIncoming with available=0; sparse what-worked table and pay board example (tsv_ok=2, wallet_wd=14, done=0).
This commit is contained in:
Hernâni Marques
parent
a534ef8d1c
commit
0ae4325d4b
1 changed files with 41 additions and 100 deletions
|
|
@ -67,121 +67,62 @@ Legend:
|
|||
| **Workaround** | Use `LADDER_STEPS≥5` (or higher) so random mids exist; treat zero / absolute-max / max-1-5110 as probes (soft `CEILING_REJECT` in mon ≥1.20.2). |
|
||||
| **Wanted** | Ladder plan docs: pin vs mid semantics; CLI structured error for amount-too-large / zero. |
|
||||
|
||||
### 3b. Mint fails with bank SQL `5110` / `P0001` (server-side; recurring)
|
||||
### 3b. We do **not** manage to withdraw (GOA) — bank path broken, not load
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| **Seen** | **Several times** on GOA (`bank.hacktivism.ch`), classic ladder high/random mids + ceiling pins **and** max-search bisect — not a one-off. Same class of error across 2026-07 runs (magikoopa / mon ladder). |
|
||||
| **Area** | **bank** (libeufin / taler-corebank + **Postgres**) / **ops** — **not** mon SQL, **not** wallet-cli at mint time |
|
||||
| **Call site** | mon `POST ${BANK}/accounts/explorer/withdrawals` with `{"amount":"GOA:…"}` only. Wallet/exchange not involved until mint returns 200 + URI. |
|
||||
| **Response** | HTTP **500**, body e.g. `{ code: 5110, hint: "Unexpected sql error with state P0001" }` |
|
||||
| **Codes** | **`5110` = `BANK_UNMANAGED_EXCEPTION`** (“no known exception types captured the exception”). **`P0001`** = PostgreSQL SQLSTATE for application `RAISE EXCEPTION` (function/trigger), not a mon bug and not max_wire config alone. |
|
||||
| **Config check** | Public `/config` still allows huge wires (`max_wire_transfer_amount` ≈ `GOA:4503599627370496.99999999`, `default_debit_threshold` same order). Failing mints are often **far below** that (e.g. ~Kilo-GOA). |
|
||||
| **API mismatch** | Corebank docs: insufficient funds should be HTTP **409**. Getting **500/5110** means the bank is **leaking** an unmapped SQL error instead of a clean client error. |
|
||||
| **Problem** | Live mint “ceiling” is **opaque and unstable**. Do **not** treat 5110 as wallet/`reserve_pub` failure when mint never succeeded. Do **not** assume “only absolute pin:max fails” — mid and max-search probes hit the same error. |
|
||||
| **Workaround (mon ≥1.21)** | Soft `CEILING_REJECT` on 5110/P0001 (classic mids + max pins); **max-ladder** treats mint reject as too-high upper bound and bisects. Keep mid-range rungs for real withdraw/confirm tests. |
|
||||
| **Wanted (bank)** | Map the SQL condition to a structured error (insufficient funds / amount too large / debit limit) — **not** `BANK_UNMANAGED_EXCEPTION`. Log full `RAISE` text server-side. |
|
||||
| **Bottom line** | On live **hacktivism GOA**, automation **does not reliably complete a withdraw to spendable coins**. Max-search may report a “best” amount; pay then sees **`available=GOA:0`**. This is **recurring (several runs, 2026-07-19)**. |
|
||||
| **Area** | **bank** + wire/wallet settle — **not** mon overload, **not** “laptop too slow” |
|
||||
| **Not the cause: ecosystem load** | Outside-in check **2026-07-19 ~14:56 CEST**: bank/exchange/merchant all **HTTP 200**, config ~**50–150 ms**, exchange `/keys` ~**0.5 s**. Container **loadavg ~1.5–1.7**, RSS bank **~737 MiB** / exchange **~670 MiB**, bank probes **47 ms**, **`wirewatch_running: true`**. Stack is **healthy under moderate load** — failures are **application/SQL/protocol**, not “host on fire.” |
|
||||
| **Call site (mint)** | `POST ${BANK}/accounts/explorer/withdrawals` → often HTTP **500** `{ code: 5110, hint: "Unexpected sql error with state P0001" }` (**`BANK_UNMANAGED_EXCEPTION`**; Postgres **`P0001`** = `RAISE EXCEPTION`). Should often be clean **409** (insufficient funds); instead opaque **500**. |
|
||||
| **Config** | `/config` still allows ~**2⁵²** wire max; live mint fails **far below** that. Explorer **balance_explorer** on landing still **~4.5 Peta-GOA**. |
|
||||
|
||||
#### Full max-search cascade (primary repro log, 2026-07-19, magikoopa / GOA)
|
||||
#### Analysis — where withdraw dies
|
||||
|
||||
Mon phase **max-ladder** withdraw hunt. Every reject below is the **same** bank body:
|
||||
| Stage | What we see | Manage to withdraw? |
|
||||
|-------|-------------|---------------------|
|
||||
| **Mint (high amounts)** | 5110/P0001 while hi falls Tera→Giga→Mega→Kilo with **lo=0** | **No** — withdrawal never created |
|
||||
| **Mint (some mid amounts)** | occasional **HTTP 200** + URI | Mint only — not done |
|
||||
| **Confirm** | force-select 200 then stuck **`selected`**, confirm **HTTP 500** → mon **`SKIP_CONFIRM`** (bounds unchanged) | **No** — bank never wires; **not** out-of-money (§3 / 5114 class) |
|
||||
| **Bank settle** | rare full path: confirm 204 + `transfer_done` | Bank side “OK_BANK” |
|
||||
| **Wallet coins** | almost always **`available=0`**, large **`pendingIncoming`**, board **`done=0`** | **No spendable coins** → pay **SKIP_BALANCE** / skip max-pay |
|
||||
|
||||
**Pay-wait board (same cumulative wallet), example:**
|
||||
|
||||
```text
|
||||
mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }
|
||||
summary: tsv_ok=2 · wallet_wd=14 (pendingIncoming=14 done=0)
|
||||
timer 0s left — available still GOA:0
|
||||
pendingIncoming=GOA:335776
|
||||
```
|
||||
|
||||
Mon treats each as soft too-high (`CEILING_REJECT` / `hi:=amount`). **`lo` stays 0** until the first successful mint.
|
||||
Reading: only **2** TSV rows are full OK/OK_BANK, but the wallet holds **14** withdrawal txs still **pendingIncoming** and **none done**. Max-search leaves many **mint-OK / confirm-failed** (and lagging) withdraws in the DB; they **inflate pendingIncoming** without ever becoming **available**. After **90 s** wait + short `run-pending` nudge: **still no spendable balance** → **we do not manage to withdraw** in the sense that matters (pay).
|
||||
|
||||
**Phase A — pure descent (no success yet): hi falls Tera → Giga → Mega while lo=0**
|
||||
#### What *did* work (sparse) — for contrast
|
||||
|
||||
| Probe | Amount (alt) | Raw `GOA:…` | Result |
|
||||
|------:|--------------|-------------|--------|
|
||||
| (earlier) | hi already **467.409 Tera-GOA** | `467409181353380` | 5110 (prior probe) |
|
||||
| 4 | **154.93 Tera-GOA** | `154929915073879` | 5110 → hi:= |
|
||||
| 5 | **16.20 Tera-GOA** | `16198385073981` | 5110 → hi:= |
|
||||
| 6 | **1.417 Tera-GOA** | `1416945306636` | 5110 → hi:= |
|
||||
| 7 | **282.49 Giga-GOA** | `282485970397` | 5110 → hi:= |
|
||||
| 8 | **132.18 Giga-GOA** | `132183255296` | 5110 → hi:= |
|
||||
| 9 | **7.417 Giga-GOA** | `7416937654` | 5110 → hi:= |
|
||||
| 10 | **711.38 Mega-GOA** | `711383774` | 5110 → hi:= |
|
||||
| 11 | **395.88 Mega-GOA** | `395879334` | 5110 → hi:= |
|
||||
| 12 | **119.26 Mega-GOA** | `119263564` | 5110 → hi:= |
|
||||
| 13 | **5.992 Mega-GOA** | `5992111` | 5110 → hi:= |
|
||||
| When / run | What worked | Amounts |
|
||||
|------------|-------------|---------|
|
||||
| Landing **2026-07-19 14:19–14:29 CEST** | Explorer singles **recorded** on bank intro stats | **~1.5–2 Giga-GOA**, **~145–202 Mega-GOA**, **~1.8–3.4 Mega-GOA** |
|
||||
| Max-search run A (same day) | Full bank path (mint→confirm→`transfer_done`) for **two** amounts only; mon best | **`GOA:424114`**, **`GOA:822256`** (~822 Kilo best) |
|
||||
| Max-search run B (later) | Even fewer full successes; many mint OK + **SKIP_CONFIRM**; mon best collapsed | e.g. full success around **`GOA:12276`** / **`GOA:17257`** (~**17 Kilo** best) |
|
||||
| Pay after either | Spendable **available** | **`GOA:0`** (fail) |
|
||||
|
||||
**Phase B — first success raises lo; still 5110 just above**
|
||||
So: **landing/history prove large withdraws were possible earlier the same day**; **automation now does not manage a usable withdraw** (best amount drops run-to-run; coins never spendable).
|
||||
|
||||
| Probe | Amount (alt) | Raw | Result |
|
||||
|------:|--------------|-----|--------|
|
||||
| **14** | **424.114 Kilo-GOA** | **`424114`** | **OK mint** → accept → force-select `PNK5YRKT…` → confirm 204 → settle `transfer_done` (wallet **avail still 0**) · **lo:=424114** |
|
||||
| 15 | **1.594 Mega-GOA** | `1594158` | 5110 → hi:= |
|
||||
| **16** | **822.256 Kilo-GOA** | **`822256`** | **OK mint** → force-select `SVKV4E9G…` → confirm → settle · **lo:=822256** (best so far) |
|
||||
#### Mon behaviour (workaround only)
|
||||
|
||||
**Phase C — fine bisect above best: every step 5110 until convergence**
|
||||
- 5110 → soft **`CEILING_REJECT`** / max-search **`hi:=`**
|
||||
- Full OK only raises **`lo`**; **SKIP_CONFIRM** does not
|
||||
- Pay waits up to **`LADDER_PAY_WAIT_AVAILABLE_S`** (default 90) with withdraw board; then **SKIP_BALANCE** if still 0
|
||||
- **Does not fix** bank SQL mapping, stuck confirm, or wirewatch→coins lag
|
||||
|
||||
| Probe | Amount (alt) | Raw | Result |
|
||||
|------:|--------------|-----|--------|
|
||||
| 17 | 1.145 Mega-GOA | `1144904` | 5110 |
|
||||
| 18 | 970.26 Kilo-GOA | `970260` | 5110 |
|
||||
| 19 | 893.198 Kilo-GOA | `893198` | 5110 |
|
||||
| 20 | 856.993 Kilo-GOA | `856993` | 5110 |
|
||||
| 21 | 839.445 Kilo-GOA | `839445` | 5110 |
|
||||
| 22 | 830.806 Kilo-GOA | `830806` | 5110 |
|
||||
| 23 | 826.52 Kilo-GOA | `826520` | 5110 |
|
||||
| 24 | 824.385 Kilo-GOA | `824385` | 5110 |
|
||||
| 25 | 823.32 Kilo-GOA | `823320` | 5110 |
|
||||
| 26 | 822.788 Kilo-GOA | `822788` | 5110 → hi:= |
|
||||
#### Wanted (bank / exchange ops)
|
||||
|
||||
**Converged (mon):**
|
||||
1. Map mint **P0001** to structured errors (not **5110**).
|
||||
2. Confirm after force-select must not stick **selected** + HTTP 500.
|
||||
3. After `transfer_done`, coins must become wallet **available** (or document delay + API to wait).
|
||||
4. File bugs with: max-search “what worked” table + landing `recent_withdraws`/`balance_explorer` + load note (healthy) + pay board `tsv_ok` vs `wallet_wd`/`pendingIncoming`/`done`.
|
||||
|
||||
```text
|
||||
max-search best withdrawable 822.256 Kilo-GOA (GOA:822256)
|
||||
· lo=822.256 Kilo-GOA · hi=822.788 Kilo-GOA · probes=26
|
||||
```
|
||||
|
||||
**Reading this log**
|
||||
|
||||
1. **Same error from ~467 Tera-GOA down to ~823 Kilo-GOA** — not a single “absurd pin:max” case; the bank returns **5110/P0001** for a **wide continuous range** of amounts.
|
||||
2. **Successes are real but sparse:** only **`GOA:424114`** and **`GOA:822256`** minted in this hunt (both full bank confirm + `transfer_done`). Everything strictly larger than **822256** in the fine band failed at **mint**.
|
||||
3. **lo=0 for probes 4–13** means max-search had **no** successful withdraw yet while already rejecting Tera/Giga/Mega — so 5110 is **not** “only after pool drained by this run.”
|
||||
4. Settle notes **wallet avail=GOA:0** after OK bank path — coins lag / `pendingIncoming` (pay-wait board, § pay wait UX); separate from mint 5110.
|
||||
5. Landing `recent_withdraws` at **14:42 CEST** lists the same two OK amounts (`424114`, `822256`) with reserves matching force-select prefixes above.
|
||||
|
||||
#### Landing page: much larger withdraws **same day, earlier** (still on explorer)
|
||||
|
||||
Public bank intro: `https://bank.hacktivism.ch/intro/stats.json` (HTML landing uses this). Snapshot **2026-07-19 ~14:47 CEST**.
|
||||
|
||||
**Pool still looks full in the UI:**
|
||||
|
||||
| Field | Snapshot |
|
||||
|-------|----------|
|
||||
| `balance_explorer` | **4.5036 Peta-GOA** (`GOA:4503599627250215.32499924`) |
|
||||
| `withdraws.total_amount` | **4.5036 Peta-GOA** (292 ops) |
|
||||
| `withdraws.last_24h` | **3.2366 Peta-GOA** (45 ops) |
|
||||
|
||||
**Dated explorer withdraws that succeeded *before* this max-search tight band** (`recent_withdraws`, account=`explorer`):
|
||||
|
||||
| When (CEST) | Amount (alt) | Raw | Notes |
|
||||
|-------------|--------------|-----|--------|
|
||||
| **2026-07-19 14:19** | **1.9761 Giga-GOA** | `GOA:1976113563` | ~2h before max-search best |
|
||||
| **2026-07-19 14:19** | **1.4495 Giga-GOA** | `GOA:1449489732` | same minute |
|
||||
| **2026-07-19 14:26** | **201.636 Mega-GOA** | `GOA:201635991` | |
|
||||
| **2026-07-19 14:26** | **144.53 Mega-GOA** | `GOA:144530029` | |
|
||||
| **2026-07-19 14:29** | **3.4099 Mega-GOA** | `GOA:3409891` | |
|
||||
| **2026-07-19 14:29** | **1.8009 Mega-GOA** | `GOA:1800875` | |
|
||||
| **2026-07-19 14:42** | 822.256 Kilo-GOA | `GOA:822256` | = max-search **best** (probe 16) |
|
||||
| **2026-07-19 14:42** | 424.114 Kilo-GOA | `GOA:424114` | = max-search first OK (probe 14) |
|
||||
|
||||
**Contrast:** earlier the **same day**, explorer minted **~1.5–2 Giga-GOA** singles; later the same API returns **5110** for **~823 Kilo-GOA** and for all of Tera→Mega probes in the cascade. Landing **balance_explorer** still shows **~Peta-GOA**. So:
|
||||
|
||||
- not “config max_wire forbids Kilo-GOA”;
|
||||
- not “landing says explorer is empty”;
|
||||
- **live mintability collapsed / became amount-sensitive** relative to history — bank SQL path is wrong or mis-mapped.
|
||||
|
||||
**Ops checklist when filing bank bug:** attach (1) this max-search probe table, (2) landing `recent_withdraws` + `balance_explorer` at the same timestamps, (3) bank journal / Postgres `P0001` RAISE text for one failing mint.
|
||||
|
||||
**Also seen (same error class):** classic ladder e.g. `GOA:747827072844319` → 500/5110/P0001; absolute max / max-1 ceiling pins → soft `CEILING_REJECT`.
|
||||
|
||||
**Exchange intro (not mint path):** `https://exchange.hacktivism.ch/intro/stats.json` — wire-in can still be Peta-scale; use **bank** landing for explorer mint history.
|
||||
Sources: `https://bank.hacktivism.ch/intro/stats.json`, exchange/merchant intro stats, mon max-ladder logs 2026-07-19.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -393,9 +334,9 @@ If only a few changes land, these unlock the most automation:
|
|||
| ladder explorer | `read_secret` / EXP_PW_FILE |
|
||||
| stage maxima | bank `max_wire` + keys min denom |
|
||||
| force-select | multi-rpub try; soft-skip confirm; empty scrape → WARN not balance fail |
|
||||
| higher amounts | mint **5110/P0001** recurring server-side (§3b); landing shows much larger **dated** explorer withdraws (Giga/Mega same day); soft `CEILING_REJECT` + max-ladder hi; alt units in report |
|
||||
| higher amounts / withdraw fail | **§3b: we do not manage to withdraw** (spendable coins) on GOA despite healthy load; mint 5110/P0001 + SKIP_CONFIRM + pendingIncoming pile-up; landing had Giga/Mega earlier same day |
|
||||
| pay vs balance | never order more than wallet **available** (soft `SKIP_BALANCE`); pendingIncoming does not count as spendable |
|
||||
| pay wait UX | after withdraw, board lists OK/OK_BANK in **start order** with live wallet `pendingIncoming`/`done` + timer (`LADDER_PAY_WAIT_AVAILABLE_S`, default 90s); same wallet DB as withdraw |
|
||||
| pay wait UX | board: OK/OK_BANK start order + live pendingIncoming/`done`; often `tsv_ok≪wallet_wd` and **done=0** after 90s → skip pay |
|
||||
| wallet-db path | Prefer `*.sqlite3` (never `*.json` as db path) |
|
||||
| helper / python | mon PATH must include helper + python≥3.11 |
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue