From 84388ae6a9ffd607f4051ea35958231342470bb4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hern=C3=A2ni=20Marques?= Date: Fri, 10 Jul 2026 11:33:09 +0200 Subject: [PATCH] ops: castopod/bonfire notes; amount-ladder bench --- README.md | 51 ++- benchmarks/README.md | 11 + benchmarks/amount-ladder/README.md | 90 +++++ benchmarks/amount-ladder/run.sh | 375 ++++++++++++++++++ .../amount-ladder/sample-load-inside.sh | 26 ++ scripts/castopod/README.md | 30 ++ scripts/castopod/lib.sh | 116 ++++++ scripts/castopod/status.sh | 21 + scripts/castopod/up.sh | 32 ++ 9 files changed, 749 insertions(+), 3 deletions(-) create mode 100644 benchmarks/README.md create mode 100644 benchmarks/amount-ladder/README.md create mode 100755 benchmarks/amount-ladder/run.sh create mode 100755 benchmarks/amount-ladder/sample-load-inside.sh create mode 100644 scripts/castopod/README.md create mode 100644 scripts/castopod/lib.sh create mode 100644 scripts/castopod/status.sh create mode 100644 scripts/castopod/up.sh diff --git a/README.md b/README.md index 0b6cf5a..d6dabb8 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,52 @@ # koopa-admin-log -Ops log and config mirror for host **koopa**. +Ops log and config mirror for host **koopa** (openSUSE Tumbleweed). -**Day logs:** `2026/` (major only, same-day). +**Day logs:** `2026/` (major only, written same day). Topic notes: `YYYY-MM-DD--topic.md`. -**Secrets:** see `SECRETS.md` / `koopa-admin-secrets`. +**Secrets:** not in this repo — see **`SECRETS.md`** and sibling **`koopa-admin-secrets`**. + +## Layout + +``` +host/ # openSUSE host-level config + overview + overview/services.md # graphical service map + systemd/ firewalld/ caddy/ network/ +configs/ # mirrored app configs (caddy, taler, …) + taler-merchant/ # overrides + nginx (same tree style as exchange) + taler-exchange/ # overrides + coins +scripts/ # merchant, exchange, monitoring helpers + taler-merchant/ # merchant-model start scripts + taler-exchange/ # same model; archive/ = greenfield bootstrap +2026/ # day logs + dated topic notes +``` + +## Live containers (podman) + +| Name | Image / role | Host port | +|------|----------------|-----------| +| `taler-hacktivism` | merchant | **9010** | +| `taler-exchange-hacktivism` | exchange | **9011** | +| `taler-bank-hacktivism` | libeufin-bank (GOA, no IBAN) | **9012** | +| `koopa-castopod` (+ mariadb/redis) | Castopod Fediverse podcast | **9020** | + +Start model (Taler): **root** `/root/start_base_services_*` → shell as service user → `/usr/local/bin/start_*.sh [--restart]`. +Start model (Castopod): `hernani` → `cd ~/koopa-castopod && podman-compose up -d`. + +Public hosts: `taler.` / `exchange.` / `bank.` / **`castopod.hacktivism.ch`** (Caddy → 9010/9011/9012/**9020**). + +Service **details**: `configs/taler-*`, `scripts/*`. +Host-wide picture: **`host/overview/services.md`**. +Router: **`../vecigate-admin-log`**. + +**Benchmarks** (drive from the local machine, measure on koopa): **`benchmarks/`** +→ amount ladder: `benchmarks/amount-ladder/`. + +**Exchange snapshot/restore (GOA, local images only):** +→ `2026/2026-07-09--exchange-snapshot-and-restore.md` + +**Exchange `/keys` + Wire + bank account (diagram):** +`configs/taler-exchange/README.md` (section "/keys, Wire and Account"). + +**Castopod pilot content (FOSS Airwaves, user ngi, free licenses):** +`2026/2026-07-09--castopod-content.md` → https://castopod.hacktivism.ch/@foss diff --git a/benchmarks/README.md b/benchmarks/README.md new file mode 100644 index 0000000..127ad0f --- /dev/null +++ b/benchmarks/README.md @@ -0,0 +1,11 @@ +# Benchmarks (koopa Taler stack) + +Run **from the local machine** against public endpoints. +**Measure load inside** the three Podman containers on koopa via `ssh hernani@koopa` + `podman exec` (no root/screen required for metrics). + +| Benchmark | What it does | +|-----------|----------------| +| [`amount-ladder/`](amount-ladder/) | Free-amount template payments across a wide GOA ladder; wall times + host/container CPU/RSS samples | + +Secrets: sibling repo **`koopa-admin-secrets`** (never committed here). +Ephemeral run output: **`koopa-admin-log/.tmp/`** (gitignored). diff --git a/benchmarks/amount-ladder/README.md b/benchmarks/amount-ladder/README.md new file mode 100644 index 0000000..737a196 --- /dev/null +++ b/benchmarks/amount-ladder/README.md @@ -0,0 +1,90 @@ +# Amount-ladder payment benchmark + +## Model + +| Side | Machine | Role | +|------|---------|------| +| **Drive (external)** | Local machine (this tree) | `taler-wallet-cli`, orchestration, CSV/RESULTS | +| **SUT (internal)** | koopa containers | merchant / exchange / bank under load | +| **Measure (internal)** | `ssh hernani@koopa` | `loadavg`, `ps` in each container | + +``` +[ Local-machine wallet-cli ] --HTTPS--> taler.hacktivism.ch / exchange / bank + | ^ + | ssh sample_load | pasta :9010/:9011/:9012 + v | +[ hernani@koopa ] --podman exec--> process CPU/RSS snapshots +``` + +## Prerequisites + +1. Stack healthy (`check_*-health.sh` all green; public `/config` + exchange `/keys` 200). +2. Local wallet CLI (default path in script, override `MONO=`). +3. Secrets in **`../koopa-admin-secrets/koopa/host-root/`**: + - `taler-bank/bank-admin-password.txt` + - `taler-bank/bank-bench-fat-password.txt` (created if missing) + - `taler-merchant/merchant-goa-demo-cp4zqk-password.txt` +4. SSH: `hernani@koopa` BatchMode, `podman` without su. + +## Amount ladder (~20 payments: small / medium / large) + +| Band | Amounts (GOA) | +|------|----------------| +| **small** | `0.000001` `0.00001` `0.0001` `0.001` `0.01` `0.05` `0.1` `0.5` | +| **medium** | `1` `2` `5` `10` `25` `50` `100` | +| **large** | `250` `500` `1000` `2500` `5000` | + +Default list (20 steps): + +```text +GOA:0.000001 GOA:0.00001 GOA:0.0001 GOA:0.001 GOA:0.01 GOA:0.05 GOA:0.1 GOA:0.5 +GOA:1 GOA:2 GOA:5 GOA:10 GOA:25 GOA:50 GOA:100 +GOA:250 GOA:500 GOA:1000 GOA:2500 GOA:5000 +``` + +Σ ladder ≈ **GOA:9433** → default withdraw **GOA:50000**, admin credit **GOA:200000**. + +Override: `AMOUNTS='GOA:1 GOA:10' ./run.sh` + +## Pay-template URI (important) + +Do **not** put a trailing slash after the template id (wallet then 404s). + +```text +taler://pay-template/taler.hacktivism.ch/instances/goa-demo-cp4zqk/goa-free?amount=GOA:1 +``` + +- Template `goa-free` is free-amount with fixed summary `"pay"` → pass **only** `amount=` (no `summary=` or merchant 409). +- HTTP GET probe: + `https://taler.hacktivism.ch/instances/goa-demo-cp4zqk/templates/goa-free` → 200. + +## Run + +```bash +cd koopa-admin-log/benchmarks/amount-ladder +./run.sh +# short: +AMOUNTS='GOA:0.05 GOA:1 GOA:10' WITHDRAW_AMT=GOA:100 ./run.sh +``` + +Outputs under `../../.tmp/amount-bench-/`: + +| File | Content | +|------|---------| +| `payments.csv` | per-amount timings + status | +| `RESULTS.md` | table summary | +| `koopa-load-*.out` | **inside** process samples | +| `run.log` | timeline | + +## Columns (RESULTS) + +- **handle**: wall time of `wallet handle-uri` +- **run-until-done**: wallet background after handle +- **total**: sum +- **loadavg**: koopa 1‑min load after payment +- **status**: `paid` / `error` / `insufficient` / … + +## Related + +- Partial earlier run (wrong URI): `2026/2026-07-09--amount-ladder-partial-results.md` +- Instance notes: `configs/taler-merchant/demo-instance-goa-demo-cp4zqk.md` diff --git a/benchmarks/amount-ladder/run.sh b/benchmarks/amount-ladder/run.sh new file mode 100755 index 0000000..2b7d599 --- /dev/null +++ b/benchmarks/amount-ladder/run.sh @@ -0,0 +1,375 @@ +#!/bin/bash +# Amount-ladder benchmark — drive from the local machine, measure on koopa (inside containers). +# See README.md in this directory. +set -eu + +BENCH_DIR=$(cd "$(dirname "$0")" && pwd) +ROOT=$(cd "$BENCH_DIR/../.." && pwd) +SECRETS="${SECRETS_DIR:-}" +if [ -z "$SECRETS" ]; then + if [ -d "$ROOT/../koopa-admin-secrets" ]; then + SECRETS=$(cd "$ROOT/../koopa-admin-secrets" && pwd) + else + SECRETS="$ROOT/../koopa-admin-secrets" + fi +fi + +SCRATCH="${ROOT}/.tmp/amount-bench-$(date -u +%Y%m%d-%H%M%S)" +mkdir -p "$SCRATCH" +chmod 700 "$SCRATCH" + +MONO="${MONO:-/Users/newkamek/src/taler/taler-typescript-core/packages/taler-wallet-cli/bin/taler-wallet-cli.mjs}" +WDB="$SCRATCH/wallet.sqlite3" +EX="${EXCHANGE_URL:-https://exchange.hacktivism.ch/}" +KOOPA="${KOOPA_HOST:-koopa}" +INST="${MERCHANT_INSTANCE:-goa-demo-cp4zqk}" +TPL="${TEMPLATE_ID:-goa-free}" +MER_HOST="${MER_HOST:-taler.hacktivism.ch}" +FAT_USER="${FAT_USER:-bench-fat}" +# ~20 payments: small → medium → large (atomic/tiny through multi-kilo) +# Override: AMOUNTS='GOA:1 GOA:10' ./run.sh +AMOUNTS="${AMOUNTS:-GOA:0.000001 GOA:0.00001 GOA:0.0001 GOA:0.001 GOA:0.01 GOA:0.05 GOA:0.1 GOA:0.5 GOA:1 GOA:2 GOA:5 GOA:10 GOA:25 GOA:50 GOA:100 GOA:250 GOA:500 GOA:1000 GOA:2500 GOA:5000}" +# sum of default ladder ≈ GOA:9433 → withdraw headroom for multi-coin + fees +WITHDRAW_AMT="${WITHDRAW_AMT:-GOA:50000}" +CREDIT_AMT="${CREDIT_AMT:-GOA:200000}" + +BANK_ADMIN_PW_FILE="${BANK_ADMIN_PW_FILE:-$SECRETS/koopa/host-root/taler-bank/bank-admin-password.txt}" +BANK_FAT_PW_FILE="${BANK_FAT_PW_FILE:-$SECRETS/koopa/host-root/taler-bank/bank-bench-fat-password.txt}" +MER_PW_FILE="${MER_PW_FILE:-$SECRETS/koopa/host-root/taler-merchant/merchant-${INST}-password.txt}" + +wcli() { node "$MONO" --wallet-db="$WDB" --no-throttle --skip-defaults "$@"; } +log() { echo "$(date -u +%H:%M:%S) $*" | tee -a "$SCRATCH/run.log"; } +now() { python3 -c 'import time;print(time.time())'; } +die() { log "FAIL $*"; exit 1; } + +[ -f "$MONO" ] || die "wallet cli missing: $MONO" +[ -f "$BANK_ADMIN_PW_FILE" ] || die "missing $BANK_ADMIN_PW_FILE" +[ -f "$MER_PW_FILE" ] || die "missing $MER_PW_FILE" + +ADMIN_PW=$(tr -d '\n' <"$BANK_ADMIN_PW_FILE") +MER_PW=$(tr -d '\n' <"$MER_PW_FILE") +if [ -f "$BANK_FAT_PW_FILE" ]; then + FAT_PW=$(tr -d '\n' <"$BANK_FAT_PW_FILE") +else + FAT_PW=$(openssl rand -hex 12) + mkdir -p "$(dirname "$BANK_FAT_PW_FILE")" + printf '%s\n' "$FAT_PW" >"$BANK_FAT_PW_FILE" + chmod 600 "$BANK_FAT_PW_FILE" + log "generated fat password -> $BANK_FAT_PW_FILE (secrets repo)" +fi + +# --- remote: run short script as hernani, capture stdout --- +run_koopa() { + local id="$1" + shift + local script_local="$SCRATCH/remote-$id.sh" + local mark="BENCH_${id}_OK" + { + echo '#!/bin/bash' + echo 'set +e' + printf '%s\n' "$@" + echo "echo $mark" + } >"$script_local" + scp -o BatchMode=yes -q "$script_local" "${KOOPA}:/tmp/bench-run-$id.sh" + ssh -o BatchMode=yes "$KOOPA" "bash /tmp/bench-run-$id.sh; rm -f /tmp/bench-run-$id.sh" \ + >"$SCRATCH/koopa-$id.out" 2>&1 || true + if ! grep -q "$mark" "$SCRATCH/koopa-$id.out" 2>/dev/null; then + log "WARN run_koopa $id incomplete (see koopa-$id.out)" + return 1 + fi + return 0 +} + +# Measure **inside** containers (and host loadavg) +sample_load() { + local tag="$1" + scp -o BatchMode=yes -q "$BENCH_DIR/sample-load-inside.sh" "${KOOPA}:/tmp/sample-load-inside.sh" + ssh -o BatchMode=yes "$KOOPA" "bash /tmp/sample-load-inside.sh $(printf %q "$tag"); rm -f /tmp/sample-load-inside.sh" \ + >"$SCRATCH/koopa-load-$tag.out" 2>&1 || true +} + +log "scratch=$SCRATCH" +log "AMOUNTS=$AMOUNTS" + +# --- preflight (external, public HTTPS) --- +for url in \ + "https://taler.hacktivism.ch/config" \ + "https://exchange.hacktivism.ch/keys" \ + "https://taler.hacktivism.ch/instances/${INST}/templates/${TPL}" +do + code=$(curl -sk -o /dev/null -w '%{http_code}' -m 10 "$url" || echo err) + log "preflight $url -> $code" + [ "$code" = "200" ] || die "preflight failed $url" +done + +# --- bank setup on koopa (API localhost, orchestrated from the local machine) --- +log "=== bank setup $FAT_USER credit=$CREDIT_AMT withdraw=$WITHDRAW_AMT ===" +{ + echo '#!/bin/bash' + echo 'set -euo pipefail' + echo "export ADMIN_PW=$(printf %q "$ADMIN_PW")" + echo "export FAT_PW=$(printf %q "$FAT_PW")" + echo "export FAT_USER=$(printf %q "$FAT_USER")" + echo "export CREDIT_AMT=$(printf %q "$CREDIT_AMT")" + echo "export WITHDRAW_AMT=$(printf %q "$WITHDRAW_AMT")" + cat <<'EOS' +BANK=http://127.0.0.1:9012 +AT=$(curl -sS -u "admin:${ADMIN_PW}" -H "Content-Type: application/json" -d '{"scope":"readwrite"}' \ + "$BANK/accounts/admin/token" | python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])") +echo admin_tok_len=${#AT} +curl -sS -o /tmp/acc.json -w "create:%{http_code}\n" -X POST -H "Authorization: Bearer $AT" \ + -H "Content-Type: application/json" \ + -d "{\"username\":\"${FAT_USER}\",\"password\":\"${FAT_PW}\",\"name\":\"Bench Fat\",\"is_public\":false,\"is_taler_exchange\":false,\"debit_threshold\":\"GOA:0\"}" \ + "$BANK/accounts" || true +echo create_body=$(head -c 160 /tmp/acc.json) +python3 - <<'PY' +import json, os +A="0123456789ABCDEFGHJKMNPQRSTVWXYZ" +def c32(b): + v=bits=0; o=[] + for x in b: + v=(v<<8)|x; bits+=8 + while bits>=5: + bits-=5; o.append(A[(v>>bits)&31]) + if bits: o.append(A[(v<<(5-bits))&31]) + return "".join(o) +uid=c32(os.urandom(32)) +user=os.environ["FAT_USER"] +amt=os.environ["CREDIT_AMT"] +payto=f"payto://x-taler-bank/bank.hacktivism.ch/{user}?receiver-name={user}&message=bench-credit" +json.dump({"payto_uri":payto,"amount":amt,"request_uid":uid}, open("/tmp/tx.json","w")) +print("credit", amt, "->", user, "uid", uid) +PY +curl -sS -o /tmp/cred.json -w "credit:%{http_code}\n" -X POST -H "Authorization: Bearer $AT" \ + -H "Content-Type: application/json" -d @/tmp/tx.json "$BANK/accounts/admin/transactions" +echo credit_body=$(head -c 200 /tmp/cred.json) +BT=$(curl -sS -u "${FAT_USER}:${FAT_PW}" -H "Content-Type: application/json" -d '{"scope":"readwrite"}' \ + "$BANK/accounts/${FAT_USER}/token" | python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])") +echo fat_tok_len=${#BT} +curl -sS -H "Authorization: Bearer $BT" "$BANK/accounts/${FAT_USER}" -o /tmp/fatbal.json +echo fat_account=$(head -c 280 /tmp/fatbal.json) +curl -sS -o /tmp/wd.json -w "wd:%{http_code}\n" -X POST -H "Authorization: Bearer $BT" \ + -H "Content-Type: application/json" -d "{\"amount\":\"${WITHDRAW_AMT}\"}" \ + "$BANK/accounts/${FAT_USER}/withdrawals" +python3 - <<'PY' +import json +d=json.load(open("/tmp/wd.json")) +print("wd_keys", list(d.keys())[:20]) +uri=d.get("taler_withdraw_uri") or d.get("withdrawal_uri") or "" +wid=d.get("withdrawal_id") or d.get("id") or "" +if not uri: + for v in d.values(): + if isinstance(v, str) and v.startswith("taler"): + uri=v +open("/tmp/bench-wuri.txt","w").write(uri) +open("/tmp/bench-wid.txt","w").write(wid) +print("WURI", uri) +print("WID", wid) +assert uri and wid, d +PY +# cleanup setup temps on host (not durable) +rm -f /tmp/acc.json /tmp/tx.json /tmp/cred.json /tmp/fatbal.json /tmp/wd.json +echo BENCH_banksetup_OK +EOS +} >"$SCRATCH/remote-banksetup.sh" +scp -o BatchMode=yes -q "$SCRATCH/remote-banksetup.sh" "${KOOPA}:/tmp/bench-run-banksetup.sh" +ssh -o BatchMode=yes "$KOOPA" 'bash /tmp/bench-run-banksetup.sh; rm -f /tmp/bench-run-banksetup.sh' \ + | tee "$SCRATCH/koopa-banksetup.out" +grep -q BENCH_banksetup_OK "$SCRATCH/koopa-banksetup.out" || die "bank setup failed" +scp -o BatchMode=yes -q "${KOOPA}:/tmp/bench-wuri.txt" "${KOOPA}:/tmp/bench-wid.txt" "$SCRATCH/" +ssh -o BatchMode=yes "$KOOPA" 'rm -f /tmp/bench-wuri.txt /tmp/bench-wid.txt' || true +WURI=$(tr -d '\n' <"$SCRATCH/bench-wuri.txt") +WID=$(tr -d '\n' <"$SCRATCH/bench-wid.txt") +[ -n "$WURI" ] || die "missing withdraw URI" +log "WURI=$WURI WID=$WID" + +log "=== load baseline (internal) ===" +sample_load baseline + +log "=== wallet: exchange + withdraw (external) ===" +wcli exchanges add "$EX" 2>&1 | tee "$SCRATCH/ex-add.out" || true +wcli exchanges update "$EX" 2>&1 | tee "$SCRATCH/ex-upd.out" || true +wcli exchanges tos "$EX" 2>&1 | tee "$SCRATCH/ex-tos.out" || true +wcli exchanges accept-tos "$EX" 2>&1 | tee "$SCRATCH/ex-accept.out" || true + +TW0=$(now) +# 1) Wallet selects exchange (status → selected). Must happen BEFORE bank confirm. +wcli withdraw accept-uri --exchange "$EX" "$WURI" 2>&1 | tee "$SCRATCH/wd-accept.out" || true + +# 2) Wait until bank sees selection, then confirm once (libeufin rejects confirm if still unselected). +log "=== bank: wait selected → confirm ===" +run_koopa wdconfirm \ + "export FAT_USER=$(printf %q "$FAT_USER")" \ + "export FAT_PW=$(printf %q "$FAT_PW")" \ + "export WID=$(printf %q "$WID")" \ + 'BANK=http://127.0.0.1:9012' \ + 'BT=$(curl -sS -u "${FAT_USER}:${FAT_PW}" -H "Content-Type: application/json" -d "{\"scope\":\"readwrite\"}" "$BANK/accounts/${FAT_USER}/token" | python3 -c "import sys,json;print(json.load(sys.stdin).get(\"access_token\",\"\"))")' \ + 'echo WID=$WID tok_len=${#BT}' \ + 'st=""' \ + 'for i in $(seq 1 60); do + curl -sS "$BANK/taler-integration/withdrawal-operation/${WID}" -o /tmp/wdst.json || true + st=$(python3 -c "import json;print(json.load(open(\"/tmp/wdst.json\")).get(\"status\",\"\"))" 2>/dev/null || true) + echo "wait_select #$i status=$st" + case "$st" in selected|confirmed) break ;; esac + sleep 1 + done' \ + 'case "$st" in + selected) + echo confirming... + curl -sS -o /tmp/wdconf.out -w "confirm_http:%{http_code}\n" -X POST \ + -H "Authorization: Bearer $BT" -H "Content-Type: application/json" -d "{}" \ + "$BANK/accounts/${FAT_USER}/withdrawals/${WID}/confirm" + head -c 120 /tmp/wdconf.out; echo + ;; + confirmed) echo already_confirmed ;; + *) echo "WARN never selected (last=$st)" ;; + esac' \ + 'for i in $(seq 1 30); do + curl -sS "$BANK/taler-integration/withdrawal-operation/${WID}" -o /tmp/wdst.json || true + python3 -c "import json;d=json.load(open(\"/tmp/wdst.json\"));print(\"post\",d.get(\"status\"),\"transfer\",d.get(\"transfer_done\"))" + st=$(python3 -c "import json;print(json.load(open(\"/tmp/wdst.json\")).get(\"status\",\"\"))" 2>/dev/null || true) + td=$(python3 -c "import json;print(json.load(open(\"/tmp/wdst.json\")).get(\"transfer_done\"))" 2>/dev/null || true) + case "$st" in confirmed) break ;; esac + [ "$td" = "True" ] || [ "$td" = "true" ] && break + sleep 1 + done + rm -f /tmp/wdst.json /tmp/wdconf.out' || true + +# 3) Wallet finishes planchets (may take minutes for large withdraws) +log "=== wallet: run-until-done after confirm ===" +for round in $(seq 1 30); do + log "withdraw run-until-done round $round" + # timeout each round so we do not hang forever on long-poll + set +e + # 120s max per round + perl -e 'alarm shift; exec @ARGV' 120 \ + node "$MONO" --wallet-db="$WDB" --no-throttle --skip-defaults run-until-done \ + 2>&1 | tee -a "$SCRATCH/wd-rud.out" + set -e + wcli balance 2>&1 | tee "$SCRATCH/bal-wd-$round.out" + if python3 - "$SCRATCH/bal-wd-$round.out" <<'PY' +import re,sys +t=open(sys.argv[1]).read() +vals=[float(x) for x in re.findall(r"GOA:([0-9.]+)", t)] +sys.exit(0 if vals and max(vals) >= 100 else 1) +PY + then + log "withdraw OK" + break + fi + sleep 1 +done +TW1=$(now) +python3 -c "print('withdraw_wall_s', round(float('$TW1')-float('$TW0'), 3))" | tee "$SCRATCH/withdraw-time.txt" +wcli balance 2>&1 | tee "$SCRATCH/balance-before-pays.out" +sample_load after-withdraw + +CSV="$SCRATCH/payments.csv" +echo "idx,amount,handle_s,rud_s,total_s,status,order_id,loadavg,notes" >"$CSV" + +idx=0 +for AMT in $AMOUNTS; do + idx=$((idx + 1)) + # no trailing slash; amount only (template has fixed summary) + URI="taler://pay-template/${MER_HOST}/instances/${INST}/${TPL}?amount=${AMT}" + echo "$URI" >"$SCRATCH/uri-$idx.txt" + log "=== pay #$idx $AMT ===" + log "uri=$URI" + + load_before=$(ssh -o BatchMode=yes "$KOOPA" 'cut -d" " -f1 /proc/loadavg' 2>/dev/null || echo "?") + sample_load "pay$idx" & + SPID=$! + + T0=$(now) + set +e + wcli handle-uri --yes --non-interactive "$URI" 2>&1 | tee "$SCRATCH/pay-$idx-handle.out" + T1=$(now) + wcli run-until-done 2>&1 | tee "$SCRATCH/pay-$idx-rud.out" + T2=$(now) + set -e + + handle_s=$(python3 -c "print(round(float('$T1')-float('$T0'),3))") + rud_s=$(python3 -c "print(round(float('$T2')-float('$T1'),3))") + total_s=$(python3 -c "print(round(float('$T2')-float('$T0'),3))") + wait "$SPID" 2>/dev/null || true + load_after=$(ssh -o BatchMode=yes "$KOOPA" 'cut -d" " -f1 /proc/loadavg' 2>/dev/null || echo "?") + + run_koopa "ord$idx" \ + "export MPW=$(printf %q "$MER_PW")" \ + "export INST=$(printf %q "$INST")" \ + "export AMT=$(printf %q "$AMT")" \ + 'AUTH="Authorization: Bearer secret-token:${MPW}"' \ + 'curl -sk -H "$AUTH" "https://127.0.0.1:9010/instances/${INST}/private/orders?paid=YES&delta=-30" -o /tmp/ol.json' \ + 'python3 - </dev/null || true)' \ + 'if [ -n "$OID" ]; then + curl -sk -H "$AUTH" "https://127.0.0.1:9010/instances/${INST}/private/orders/${OID}" -o /tmp/od.json + python3 -c "import json;d=json.load(open(\"/tmp/od.json\"));print(\"DETAIL paid\",d.get(\"paid\"),\"status\",d.get(\"order_status\"),\"amount\",(d.get(\"contract_terms\")or{}).get(\"amount\"))" + fi + rm -f /tmp/ol.json /tmp/od.json /tmp/oid.txt' || true + + status=unknown + if grep -qE 'DETAIL paid True|DETAIL paid true' "$SCRATCH/koopa-ord$idx.out" 2>/dev/null; then + status=paid + elif grep -qiE 'insufficient balance' "$SCRATCH/pay-$idx-handle.out" 2>/dev/null; then + status=insufficient + elif grep -qiE 'error|fail|invalid|404|409|Unexpected HTTP' "$SCRATCH/pay-$idx-handle.out" 2>/dev/null; then + status=error + elif grep -qi 'paid' "$SCRATCH/pay-$idx-rud.out" 2>/dev/null; then + status=maybe + fi + oid=$(awk '/^ORDER /{print $2; exit}' "$SCRATCH/koopa-ord$idx.out" 2>/dev/null || true) + notes="load ${load_before}->${load_after}" + echo "$idx,$AMT,$handle_s,$rud_s,$total_s,$status,$oid,$load_after,$notes" >>"$CSV" + log "pay #$idx $AMT total=${total_s}s status=$status oid=$oid load=$load_after" + wcli balance 2>&1 | tee "$SCRATCH/bal-after-$idx.out" || true +done + +sample_load final +wcli balance 2>&1 | tee "$SCRATCH/balance-final.out" || true + +python3 - "$CSV" "$SCRATCH" <<'PY' | tee "$SCRATCH/RESULTS.md" +import csv, sys +csv_path, scratch = sys.argv[1], sys.argv[2] +rows = list(csv.DictReader(open(csv_path))) +print("# Amount ladder payment benchmark") +print() +print(f"- Scratch: `{scratch}`") +print(f"- Drive: Local-machine wallet-cli (external) · Measure: koopa podman (internal)") +print(f"- Template: `goa-free` on `goa-demo-cp4zqk`") +print(f"- URI: `taler://pay-template/…/instances/…/goa-free?amount=…` (no trailing slash, no summary)") +print() +print("| # | Amount | handle (s) | run-until-done (s) | **total (s)** | status | order | loadavg |") +print("|--:|--------|----------:|-------------------:|--------------:|--------|-------|--------:|") +for r in rows: + print(f"| {r['idx']} | `{r['amount']}` | {r['handle_s']} | {r['rud_s']} | **{r['total_s']}** | {r['status']} | `{r.get('order_id','')}` | {r.get('loadavg','')} |") +print() +paid = [r for r in rows if r["status"] == "paid"] +print(f"Paid **{len(paid)}/{len(rows)}**.") +if paid: + totals = [float(r["total_s"]) for r in paid if r["total_s"]] + if totals: + print(f"**Min** {min(totals):.3f}s · **Max** {max(totals):.3f}s · **Avg** {sum(totals)/len(totals):.3f}s") +print() +print("Inside samples: `koopa-load-*.out` in scratch.") +PY + +# durable copy of last RESULTS into benchmarks tree (no secrets) +cp "$SCRATCH/RESULTS.md" "$BENCH_DIR/LAST_RESULTS.md" +cp "$CSV" "$BENCH_DIR/LAST_payments.csv" +log "DONE -> $SCRATCH/RESULTS.md (also benchmarks/amount-ladder/LAST_*)" +cat "$SCRATCH/RESULTS.md" diff --git a/benchmarks/amount-ladder/sample-load-inside.sh b/benchmarks/amount-ladder/sample-load-inside.sh new file mode 100755 index 0000000..2b335fd --- /dev/null +++ b/benchmarks/amount-ladder/sample-load-inside.sh @@ -0,0 +1,26 @@ +#!/bin/bash +# Sample host + three-container CPU/RSS from koopa (run as hernani via ssh). +# Usage (local): ssh hernani@koopa 'bash -s' < sample-load-inside.sh +# Or: ./sample-load-inside.sh # if executed ON koopa as hernani +set -eu +tag="${1:-manual}" +echo "=== sample_load tag=$tag $(date -u +%FT%TZ) ===" +echo "=== loadavg ===" +cat /proc/loadavg +echo "=== mem ===" +free -h 2>/dev/null || true +echo "=== top host cpu ===" +ps -eo pid,pcpu,pmem,rss,etime,comm --sort=-pcpu 2>/dev/null | head -15 +echo "=== exchange (taler-exchange-hacktivism) ===" +podman exec taler-exchange-hacktivism \ + ps -eo pid,pcpu,pmem,rss,etime,args 2>/dev/null \ + | grep -E 'taler-exchange|postgres -D|PID' | head -25 || true +echo "=== merchant (taler-hacktivism) ===" +podman exec taler-hacktivism \ + ps -eo pid,pcpu,pmem,rss,etime,args 2>/dev/null \ + | grep -E 'taler-merchant|nginx|postgres -D' | head -25 || true +echo "=== bank (taler-bank-hacktivism) ===" +podman exec taler-bank-hacktivism \ + ps -eo pid,pcpu,pmem,rss,etime,args 2>/dev/null \ + | grep -E 'java|libeufin|postgres -D' | head -15 || true +echo "=== sample_load done ===" diff --git a/scripts/castopod/README.md b/scripts/castopod/README.md new file mode 100644 index 0000000..3b911c0 --- /dev/null +++ b/scripts/castopod/README.md @@ -0,0 +1,30 @@ +# Castopod ops scripts (koopa) + +Live stack: `/home/hernani/koopa-castopod/` (podman-compose). +These scripts live in admin-log and should be **copied** to the host when changed. + +## Hang-avoidance rules (learned 2026-07-09) + +| Failure mode | What hung | Fix in scripts | +|--------------|-----------|----------------| +| `php spark shield:user activate` | interactive `[y,n]` without TTY → **TypeError / hang** | SQL: `UPDATE cp_users SET active=1` via `cp_activate_users` | +| bare `curl` / image pulls | no deadline → agent waits forever | `curl --connect-timeout` + `--max-time`; `timeout(1)` around `podman-compose pull` | +| health loops | infinite `sleep` | capped `CP_HEALTH_TRIES` + fail | +| Wikimedia/random downloads | slow/blocked CDN | local/known files only; always `--max-time` | +| empty optional file fields | Castopod validates empty transcript/chapters | send real `.srt` / `.json` **or** omit fields carefully | + +## Usage (on koopa as `hernani`) + +```bash +# copy once +mkdir -p ~/koopa-castopod/bin +cp /path/to/koopa-admin-log/scripts/castopod/{lib.sh,status.sh,up.sh} ~/koopa-castopod/bin/ +chmod +x ~/koopa-castopod/bin/*.sh + +~/koopa-castopod/bin/status.sh +~/koopa-castopod/bin/up.sh +``` + +Env overrides: `CP_MAX_TIME`, `CP_PULL_TIMEOUT`, `CP_HEALTH_TRIES`, `CP_BASEURL`. + +Passwords stay in `~/koopa-castopod/.env` and `users.env` (mode 600) — never in admin-log. diff --git a/scripts/castopod/lib.sh b/scripts/castopod/lib.sh new file mode 100644 index 0000000..4c64f32 --- /dev/null +++ b/scripts/castopod/lib.sh @@ -0,0 +1,116 @@ +# shellcheck shell=bash +# Shared helpers for Castopod ops on koopa — fail fast, never hang forever. + +set -euo pipefail + +# --- defaults (override via env) --- +: "${CP_BASEURL:=https://castopod.hacktivism.ch}" +: "${CP_COMPOSE_DIR:=/home/hernani/koopa-castopod}" +: "${CP_CONNECT_TIMEOUT:=10}" # seconds — TCP connect +: "${CP_MAX_TIME:=120}" # seconds — whole HTTP transfer +: "${CP_PULL_TIMEOUT:=600}" # seconds — image pull +: "${CP_HEALTH_TRIES:=30}" # health poll attempts +: "${CP_HEALTH_SLEEP:=5}" # seconds between polls + +# curl that cannot hang forever +cp_curl() { + curl -sS \ + --connect-timeout "${CP_CONNECT_TIMEOUT}" \ + --max-time "${CP_MAX_TIME}" \ + --retry 2 \ + --retry-delay 2 \ + --retry-connrefused \ + "$@" +} + +# HTTP status only (no hang) +cp_http_code() { + local url=$1 + cp_curl -o /dev/null -w '%{http_code}' "$url" || echo "000" +} + +# Run command with hard wall-clock limit (GNU coreutils timeout) +cp_timeout() { + local secs=$1 + shift + if command -v timeout >/dev/null 2>&1; then + timeout --signal=TERM --kill-after=15s "${secs}" "$@" + else + # fallback: no timeout binary — still run, but warn + echo "WARN: timeout(1) missing; running without wall limit: $*" >&2 + "$@" + fi +} + +# Non-interactive answer stream for spark / CLI that prompts [y,n] or passwords. +# Usage: cp_yes | podman exec -i … php spark … +# Prefer SQL for activate when possible (spark prompts hang without TTY). +cp_yes() { + # enough y's for a few prompts; never block waiting for input + yes y 2>/dev/null | head -n 20 +} + +# Poll until URL returns expected code or give up +cp_wait_http() { + local url=$1 + local want=${2:-200} + local i code + for ((i = 1; i <= CP_HEALTH_TRIES; i++)); do + code=$(cp_http_code "$url") + echo "health try $i/${CP_HEALTH_TRIES}: $url → $code" + if [[ "$code" == "$want" || "$code" =~ ^[23] ]]; then + return 0 + fi + sleep "${CP_HEALTH_SLEEP}" + done + echo "ERROR: $url still not healthy after ${CP_HEALTH_TRIES} tries (last=$code)" >&2 + return 1 +} + +# MariaDB password from compose .env (no hang if missing) +cp_mysql_password() { + local envf="${CP_COMPOSE_DIR}/.env" + [[ -f "$envf" ]] || { echo "ERROR: missing $envf" >&2; return 1; } + # shellcheck disable=SC1090 + grep -E '^MYSQL_PASSWORD=' "$envf" | head -1 | cut -d= -f2- +} + +cp_mysql() { + local pw + pw=$(cp_mysql_password) + podman exec koopa-castopod-mariadb \ + mariadb -ucastopod -p"$pw" castopod "$@" +} + +# Activate shield users without spark interactive prompt +cp_activate_users() { + cp_mysql -e "UPDATE cp_users SET active=1 WHERE username IN ('admin','ngi');" +} + +# Session login for admin UI automation (uses users.env) +cp_admin_login() { + local cookie=${1:-/tmp/cp-cj} + local envf="${CP_COMPOSE_DIR}/users.env" + [[ -f "$envf" ]] || { echo "ERROR: missing $envf" >&2; return 1; } + # shellcheck disable=SC1090 + source "$envf" + [[ -n "${ADMIN_PW:-}" ]] || { echo "ERROR: ADMIN_PW empty" >&2; return 1; } + + rm -f "$cookie" + cp_curl -c "$cookie" -b "$cookie" "${CP_BASEURL}/cp-auth/login" -o /tmp/cp-login.html + local csrf + csrf=$(sed -n 's/.*name="csrf_test_name" value="\([^"]*\)".*/\1/p' /tmp/cp-login.html | head -1) + [[ -n "$csrf" ]] || { echo "ERROR: no CSRF on login page" >&2; return 1; } + + local code + code=$(cp_curl -c "$cookie" -b "$cookie" -o /dev/null -w '%{http_code}' \ + -X POST "${CP_BASEURL}/cp-auth/login" \ + --data-urlencode "csrf_test_name=${csrf}" \ + --data-urlencode "email=admin@castopod.hacktivism.ch" \ + --data-urlencode "password=${ADMIN_PW}") + # 303 see other is success + if [[ "$code" != "303" && "$code" != "302" && "$code" != "200" ]]; then + echo "ERROR: login HTTP $code" >&2 + return 1 + fi +} diff --git a/scripts/castopod/status.sh b/scripts/castopod/status.sh new file mode 100644 index 0000000..2a234df --- /dev/null +++ b/scripts/castopod/status.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# Quick Castopod health — bounded network, no hangs. +set -euo pipefail +ROOT="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "${ROOT}/lib.sh" + +echo "== podman ==" +podman ps --filter name=koopa-castopod --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' || true + +echo "== local backend :9020 ==" +code=$(cp_http_code "http://127.0.0.1:9020/health" || true) +echo "local health → ${code:-000} (307 redirect to public is OK)" + +echo "== public ==" +# Note: do not put bare @url in curl -w; @ means "read file" in some curl contexts. +for path in "/" "/@foss" "/@foss/feed.xml" "/@foss/episodes/four-freedoms"; do + url="${CP_BASEURL}${path}" + code=$(cp_http_code "$url") + printf ' %s → %s\n' "$url" "$code" +done diff --git a/scripts/castopod/up.sh b/scripts/castopod/up.sh new file mode 100644 index 0000000..dc911ef --- /dev/null +++ b/scripts/castopod/up.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# Start Castopod stack with hard timeouts on pull/up (no infinite hang). +set -euo pipefail +ROOT="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "${ROOT}/lib.sh" + +cd "${CP_COMPOSE_DIR}" + +echo "== pull (max ${CP_PULL_TIMEOUT}s) ==" +cp_timeout "${CP_PULL_TIMEOUT}" podman-compose pull + +echo "== up ==" +cp_timeout 180 podman-compose up -d + +echo "== wait for app on :9020 ==" +# Castopod often 307 from /health to https — accept 2xx/3xx +ok=0 +for ((i = 1; i <= CP_HEALTH_TRIES; i++)); do + code=$(cp_http_code "http://127.0.0.1:9020/" || true) + echo "try $i: local / → $code" + if [[ "$code" =~ ^[23] ]]; then + ok=1 + break + fi + sleep "${CP_HEALTH_SLEEP}" +done +[[ "$ok" -eq 1 ]] || { echo "ERROR: app not answering on 9020"; podman-compose ps; exit 1; } + +echo "== public via Caddy ==" +cp_wait_http "${CP_BASEURL}/" || true +echo "done."