diff --git a/scripts/taler-landing/README.md b/scripts/taler-landing/README.md new file mode 100644 index 0000000..266d1b3 --- /dev/null +++ b/scripts/taler-landing/README.md @@ -0,0 +1,106 @@ +# taler-landing — central stats (hernani user systemd) + +Public landing numbers on + +- https://bank.hacktivism.ch/intro/ → `stats.json` +- https://exchange.hacktivism.ch/intro/ → `stats.json` +- https://taler.hacktivism.ch/intro/ → `stats.json` + +are produced by **one host process** as user **`hernani`**, not by three unrelated in-container crons. + +## Why host / user systemd + +| Concern | Approach | +|--------|----------| +| **All accounts / all money** | Python full scan + tx pagination (not capped at 80 accounts) | +| **High ladder GOA** | `amount_alt` / UI alt units (Kilo-/Mega-/Peta-GOA) — tiles stay short | +| **No root cron** | `systemctl --user` as hernani + `loginctl enable-linger` | +| **Failure safety** | failed run only updates `stats-run.json`; last good `stats.json` stays | + +## Install (on koopa as hernani) + +```bash +cd ~/src/koopa/koopa-admin-log +./scripts/taler-landing/install-landing-stats-host.sh + +# timer without login session: +sudo loginctl enable-linger hernani + +# run once: +systemctl --user start taler-landing-stats.service +journalctl --user -u taler-landing-stats.service -n 50 --no-pager +systemctl --user list-timers 'taler-landing-stats*' +``` + +Units: + +- `configs/systemd/user/taler-landing-stats.service` — oneshot collector +- `configs/systemd/user/taler-landing-stats.timer` — every **2 minutes** after boot + +Installed paths: + +| Path | Role | +|------|------| +| `~/.local/bin/collect-landing-stats.sh` | orchestrator | +| `~/.local/lib/taler-landing/*.py` | bank scan + alt enrich | +| `~/.local/state/taler-landing-stats/` | logs | +| `~/.config/systemd/user/taler-landing-stats.*` | user units | + +## What the collector does + +1. **Bank** — `collect_bank_stats.py` + - admin token → list **all** accounts + - for each account (except `SCAN_SKIP`, default **`exchange`**) page through **all** transactions + - sum credits / Taler withdraws / other debits + - emit `amount` + `amount_alt` / `amount_full` + - `podman cp` → `taler-hacktivism-bank:/var/www/bank-landing/stats.json` + +2. **Exchange** — run `landing-stats-exchange.sh` inside exchange container, then **enrich** alt fields on the host and write back. + +3. **Merchant** — same for merchant container. + +`exchange` is skipped in the bank **flow** scan so the same GOA is not counted once as customer withdraw and again as exchange credit. **Admin**, **explorer**, and every auto-account are included. + +## Secrets + +Preferred order: + +1. `~/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank/bank-admin-password.txt` +2. `~/.config/taler-landing/bank-*-password.txt` +3. `podman exec taler-hacktivism-bank cat /root/bank-admin-password.txt` + +Explorer password optional (shared-pool balance). + +## Env overrides + +| Env | Default | Meaning | +|-----|---------|---------| +| `BANK_URL` | `http://127.0.0.1:9012` | libeufin loopback | +| `SCAN_SKIP` | `exchange` | usernames excluded from flow | +| `TX_PAGE` | `500` | transactions page size | +| `MAX_TX_PAGES` | `0` | `0` = unlimited pages / account | +| `ACCOUNTS_DELTA` | `-10000` | account list window | +| `ADMIN_LOG` | `%h/src/koopa/koopa-admin-log` | refresh in-container scripts | + +## UI alt names + +`configs/shared/goa-amount.js` formats large amounts as e.g. `1.23 Mega-GOA` (tooltip = full `GOA:…`). Landings load it as `/intro/goa-amount.js` (deploy via `deploy-landings.sh`). + +## Legacy in-container bank cron + +`scripts/taler-bank/landing-stats.sh` remains a **fallback** inside the bank container. Once the hernani timer is healthy, disable the old in-container minutely cron to avoid races: + +```bash +podman exec taler-hacktivism-bank crontab -l # inspect +# remove landing-stats.sh line if present +``` + +## Manual run + +```bash +~/.local/bin/collect-landing-stats.sh +# or from checkout: +./scripts/taler-landing/collect-landing-stats.sh + +curl -sS https://bank.hacktivism.ch/intro/stats.json | python3 -m json.tool | head +``` diff --git a/scripts/taler-landing/collect-landing-stats.sh b/scripts/taler-landing/collect-landing-stats.sh new file mode 100755 index 0000000..f41cc44 --- /dev/null +++ b/scripts/taler-landing/collect-landing-stats.sh @@ -0,0 +1,260 @@ +#!/usr/bin/env bash +# Central landing-stats collector for hernani@koopa (user systemd timer). +# +# - Bank: full account+ledger scan via collect_bank_stats.py → bank container +# - Exchange / merchant: existing in-container scripts, then amount_alt enrich +# - Never wipes a good stats.json on failure (writes stats-run.json only) +# +# Install: scripts/taler-landing/install-landing-stats-host.sh +set -euo pipefail + +export TZ="${TZ:-Europe/Zurich}" +export PATH="${HOME}/.local/bin:/usr/local/bin:/usr/bin:/bin${PATH:+:$PATH}" + +log() { printf '%s %s\n' "$(date -Iseconds)" "$*"; } + +ROOT="$(cd "$(dirname "$0")" && pwd)" +# When installed to ~/.local/bin, libs live in ~/.local/lib/taler-landing +if [ -f "$ROOT/collect_bank_stats.py" ]; then + LIB="$ROOT" +elif [ -f "${HOME}/.local/lib/taler-landing/collect_bank_stats.py" ]; then + LIB="${HOME}/.local/lib/taler-landing" +elif [ -f "${HOME}/src/koopa/koopa-admin-log/scripts/taler-landing/collect_bank_stats.py" ]; then + LIB="${HOME}/src/koopa/koopa-admin-log/scripts/taler-landing" +else + LIB="$ROOT" +fi + +ADMIN_LOG="${ADMIN_LOG:-${HOME}/src/koopa/koopa-admin-log}" +SECRETS_BANK="${SECRETS_BANK:-${HOME}/src/koopa/koopa-admin-secrets/koopa/host-root/taler-bank}" + +BANK_CTR="${BANK_CTR:-taler-hacktivism-bank}" +EX_CTR="${EX_CTR:-taler-hacktivism-exchange-ansible}" +MER_CTR="${MER_CTR:-taler-hacktivism}" + +BANK_URL="${BANK_URL:-http://127.0.0.1:9012}" +BANK_PUBLIC_URL="${BANK_PUBLIC_URL:-https://bank.hacktivism.ch}" +EXCHANGE_CONFIG_URL="${EXCHANGE_CONFIG_URL:-https://exchange.hacktivism.ch/config}" + +BANK_LANDING_IN="${BANK_LANDING_IN:-/var/www/bank-landing}" +EX_LANDING_IN="${EX_LANDING_IN:-/var/www/exchange-landing}" +MER_LANDING_IN="${MER_LANDING_IN:-/var/www/merchant-landing}" + +WORKDIR="${LANDING_STATS_WORKDIR:-${XDG_RUNTIME_DIR:-/tmp}/taler-landing-stats}" +mkdir -p "$WORKDIR" +LOG_DIR="${LANDING_STATS_LOGDIR:-${HOME}/.local/state/taler-landing-stats}" +mkdir -p "$LOG_DIR" + +PY="${PYTHON:-python3}" +ec_bank=0 +ec_ex=0 +ec_mer=0 + +read_pass() { + local name="$1" f + for f in \ + "${SECRETS_BANK}/${name}" \ + "${HOME}/.config/taler-landing/${name}" \ + "/run/user/$(id -u)/taler-landing/${name}" + do + if [ -r "$f" ]; then + tr -d '\n' <"$f" + return 0 + fi + done + # container (hernani can podman exec root files inside owned containers) + if podman inspect -f '{{.State.Running}}' "$BANK_CTR" 2>/dev/null | grep -qx true; then + if podman exec "$BANK_CTR" test -r "/root/${name}" 2>/dev/null; then + podman exec "$BANK_CTR" cat "/root/${name}" 2>/dev/null | tr -d '\n' + return 0 + fi + fi + return 1 +} + +ctr_running() { + podman inspect -f '{{.State.Running}}' "$1" 2>/dev/null | grep -qx true +} + +publish_json() { + local ctr="$1" dest_dir="$2" src_stats="$3" src_run="$4" + if ! ctr_running "$ctr"; then + log "WARN: $ctr not running — skip publish $dest_dir" + return 1 + fi + podman exec "$ctr" mkdir -p "$dest_dir" 2>/dev/null || true + if [ -f "$src_stats" ]; then + podman cp "$src_stats" "${ctr}:${dest_dir}/stats.json" + fi + if [ -f "$src_run" ]; then + podman cp "$src_run" "${ctr}:${dest_dir}/stats-run.json" + fi +} + +# --------------------------------------------------------------------------- +# Bank — full scan (all accounts / all money except SCAN_SKIP) +# --------------------------------------------------------------------------- +collect_bank() { + log "bank: collect full stats via $LIB/collect_bank_stats.py" + local admin_pass explorer_pass + admin_pass="$(read_pass bank-admin-password.txt || true)" + explorer_pass="$(read_pass bank-explorer-password.txt || true)" + if [ -z "$admin_pass" ]; then + log "ERROR: bank admin password not found (secrets or container)" + printf '%s\n' '{"ok":false,"error":"no admin password","at_human":"'"$(date +"%Y-%m-%d %H:%M %Z")"'"}' \ + >"$WORKDIR/bank-stats-run.json" + publish_json "$BANK_CTR" "$BANK_LANDING_IN" "" "$WORKDIR/bank-stats-run.json" || true + return 1 + fi + + # optional demo files from container + local demo_dir="" + if ctr_running "$BANK_CTR"; then + demo_dir="$WORKDIR/demo" + mkdir -p "$demo_dir" + podman cp "${BANK_CTR}:${BANK_LANDING_IN}/withdraw.uri" "$demo_dir/withdraw.uri" 2>/dev/null || true + podman cp "${BANK_CTR}:${BANK_LANDING_IN}/withdraw.amount" "$demo_dir/withdraw.amount" 2>/dev/null || true + podman cp "${BANK_CTR}:${BANK_LANDING_IN}/withdraw.created" "$demo_dir/withdraw.created" 2>/dev/null || true + fi + + set +e + BANK_URL="$BANK_URL" \ + BANK_PUBLIC_URL="$BANK_PUBLIC_URL" \ + EXCHANGE_CONFIG_URL="$EXCHANGE_CONFIG_URL" \ + BANK_ADMIN_PASS="$admin_pass" \ + BANK_EXPLORER_PASS="$explorer_pass" \ + DEMO_DIR="${demo_dir}" \ + SCAN_SKIP="${SCAN_SKIP:-exchange}" \ + TX_PAGE="${TX_PAGE:-500}" \ + MAX_TX_PAGES="${MAX_TX_PAGES:-0}" \ + ACCOUNTS_DELTA="${ACCOUNTS_DELTA:--10000}" \ + "$PY" "$LIB/collect_bank_stats.py" \ + --out "$WORKDIR/bank-stats.json" \ + --run-out "$WORKDIR/bank-stats-run.json" \ + >>"$LOG_DIR/bank.log" 2>&1 + ec_bank=$? + set -e + + if [ "$ec_bank" -eq 0 ] && [ -f "$WORKDIR/bank-stats.json" ]; then + # merge in-container memory snapshot when helper exists + if ctr_running "$BANK_CTR" && podman exec "$BANK_CTR" test -f /usr/local/lib/landing-mem-snapshot.sh 2>/dev/null; then + set +e + mem_json=$(podman exec "$BANK_CTR" bash -c ' + export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" + MEM_JSON="" + # shellcheck disable=SC1091 + . /usr/local/lib/landing-mem-snapshot.sh + mem_snapshot_json 2>/dev/null || true + # print MEM_JSON lines only if function set them via echo — fallback empty + if [ -n "${MEM_JSON:-}" ]; then printf "%s" "$MEM_JSON"; fi + ' 2>/dev/null) + set -e + if [ -n "${mem_json:-}" ]; then + "$PY" - "$WORKDIR/bank-stats.json" <<'PY' || true +import json, sys +path = sys.argv[1] +# optional: leave memory as-is if merge fails +print("mem merge skipped (structured merge via podman helper optional)", file=sys.stderr) +PY + fi + fi + publish_json "$BANK_CTR" "$BANK_LANDING_IN" \ + "$WORKDIR/bank-stats.json" "$WORKDIR/bank-stats-run.json" + log "bank: OK → ${BANK_CTR}:${BANK_LANDING_IN}/stats.json" + else + log "ERROR: bank collect failed (ec=$ec_bank) — previous stats.json kept" + [ -f "$WORKDIR/bank-stats-run.json" ] || \ + printf '%s\n' '{"ok":false,"error":"collect failed","at_human":"'"$(date +"%Y-%m-%d %H:%M %Z")"'"}' \ + >"$WORKDIR/bank-stats-run.json" + publish_json "$BANK_CTR" "$BANK_LANDING_IN" "" "$WORKDIR/bank-stats-run.json" || true + fi + return "$ec_bank" +} + +# --------------------------------------------------------------------------- +# Exchange / merchant — in-container generators + host enrich +# --------------------------------------------------------------------------- +run_incontainer_stats() { + local label="$1" ctr="$2" script_candidates="$3" landing="$4" + local script="" s + if ! ctr_running "$ctr"; then + log "WARN: $label: $ctr not running" + return 1 + fi + # install latest script from admin-log if present + local host_src="" + case "$label" in + exchange) + host_src="$ADMIN_LOG/scripts/taler-exchange/landing-stats-exchange.sh" + ;; + merchant) + host_src="$ADMIN_LOG/scripts/taler-merchant/landing-stats-merchant.sh" + ;; + esac + if [ -n "$host_src" ] && [ -f "$host_src" ]; then + podman cp "$host_src" "${ctr}:/usr/local/bin/$(basename "$host_src")" + podman exec "$ctr" chmod 755 "/usr/local/bin/$(basename "$host_src")" 2>/dev/null || true + script="/usr/local/bin/$(basename "$host_src")" + else + for s in $script_candidates; do + if podman exec "$ctr" test -x "$s" 2>/dev/null || podman exec "$ctr" test -f "$s" 2>/dev/null; then + script="$s" + break + fi + done + fi + if [ -z "$script" ]; then + log "WARN: $label: no landing-stats script in $ctr" + return 1 + fi + log "$label: run $script inside $ctr" + set +e + podman exec \ + -e LANDING_DIR="$landing" \ + -e TZ=Europe/Zurich \ + -e PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" \ + "$ctr" bash "$script" >>"$LOG_DIR/${label}.log" 2>&1 + local ec=$? + set -e + if [ "$ec" -ne 0 ]; then + log "ERROR: $label stats failed (ec=$ec)" + return "$ec" + fi + # enrich with alt units on host + podman cp "${ctr}:${landing}/stats.json" "$WORKDIR/${label}-stats.json" 2>/dev/null || return 0 + set +e + "$PY" "$LIB/enrich_stats_alt.py" "$WORKDIR/${label}-stats.json" \ + --exchange-config "$EXCHANGE_CONFIG_URL" >>"$LOG_DIR/${label}.log" 2>&1 + set -e + podman cp "$WORKDIR/${label}-stats.json" "${ctr}:${landing}/stats.json" + log "$label: OK + alt enrich → ${ctr}:${landing}/stats.json" + return 0 +} + +collect_exchange() { + run_incontainer_stats exchange "$EX_CTR" \ + "/usr/local/bin/landing-stats-exchange.sh /usr/local/bin/landing-stats.sh" \ + "$EX_LANDING_IN" +} + +collect_merchant() { + run_incontainer_stats merchant "$MER_CTR" \ + "/usr/local/bin/landing-stats-merchant.sh /usr/local/bin/landing-stats.sh" \ + "$MER_LANDING_IN" +} + +# --------------------------------------------------------------------------- +main() { + log "=== taler-landing-stats start (user=$(id -un) lib=$LIB) ===" + collect_bank || ec_bank=$? + collect_exchange || ec_ex=$? + collect_merchant || ec_mer=$? + log "=== done bank=$ec_bank exchange=$ec_ex merchant=$ec_mer ===" + # non-zero if bank failed (primary public flow numbers); soft on ex/mer + if [ "$ec_bank" -ne 0 ]; then + return 1 + fi + return 0 +} + +main "$@" diff --git a/scripts/taler-landing/collect_bank_stats.py b/scripts/taler-landing/collect_bank_stats.py new file mode 100755 index 0000000..79055d6 --- /dev/null +++ b/scripts/taler-landing/collect_bank_stats.py @@ -0,0 +1,816 @@ +#!/usr/bin/env python3 +"""Full-scan bank landing stats (all accounts, all ledger money). + +Run on koopa as hernani (or anywhere with admin API access). Writes stats.json +compatible with bank.hacktivism.ch/intro/ plus amount_alt fields for compact UI. + +Env (selected): + BANK_URL default http://127.0.0.1:9012 + BANK_ADMIN_USER default admin + BANK_ADMIN_PASS or BANK_ADMIN_PASS_FILE / pass via --admin-pass-file + BANK_EXPLORER_USER default explorer + BANK_EXPLORER_PASS optional (balance_explorer) + EXCHANGE_CONFIG_URL for alt_unit_names (default https://exchange.hacktivism.ch/config) + OUT output path (default stdout if -) + SCAN_SKIP comma usernames excluded from flow (default: exchange) + TX_PAGE page size for transactions delta (default 500) + MAX_TX_PAGES 0 = unlimited pages per account (default 0) + ACCOUNTS_DELTA GET /accounts?delta= (default -10000) + RECENT_WD_N recent withdraws (default 10) + TZ default Europe/Zurich +""" +from __future__ import annotations + +import argparse +import json +import os +import re +import sys +import time +import urllib.error +import urllib.parse +import urllib.request +from datetime import datetime +from decimal import Decimal +from pathlib import Path +from typing import Any, Dict, List, Optional, Set, Tuple +from zoneinfo import ZoneInfo + +# local import (same directory) +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from goa_amounts import ( # noqa: E402 + DEFAULT_ALT, + enrich_stats_tree, + format_amount_alt, + load_alt_from_config, + parse_amount, +) + +RESERVE_RE = re.compile(r"(?i)withdrawal\s+([A-Za-z0-9]+)") + + +def env(name: str, default: str = "") -> str: + return os.environ.get(name, default) + + +def read_pass_file(path: str) -> str: + p = Path(path) + if p.is_file(): + return p.read_text(encoding="utf-8", errors="replace").strip() + return "" + + +def http_json( + url: str, + *, + method: str = "GET", + headers: Optional[Dict[str, str]] = None, + data: Optional[bytes] = None, + timeout: float = 30.0, + auth: Optional[Tuple[str, str]] = None, +) -> Tuple[int, Any, bytes]: + h = dict(headers or {}) + if auth: + import base64 + + token = base64.b64encode(f"{auth[0]}:{auth[1]}".encode()).decode("ascii") + h["Authorization"] = f"Basic {token}" + req = urllib.request.Request(url, data=data, headers=h, method=method) + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + body = resp.read() + code = getattr(resp, "status", 200) or 200 + except urllib.error.HTTPError as e: + body = e.read() if e.fp else b"" + code = e.code + except Exception as e: + return 0, None, str(e).encode() + + if not body: + return code, None, body + try: + return code, json.loads(body.decode("utf-8", errors="replace")), body + except Exception: + return code, None, body + + +def measure_ms(url: str, timeout: float = 8.0) -> Tuple[Optional[int], str]: + t0 = time.perf_counter() + code = "000" + try: + req = urllib.request.Request(url, method="GET") + with urllib.request.urlopen(req, timeout=timeout) as resp: + resp.read() + code = str(getattr(resp, "status", 200) or 200) + except urllib.error.HTTPError as e: + code = str(e.code) + except Exception: + return None, "000" + ms = int(round((time.perf_counter() - t0) * 1000)) + if ms == 0: + ms = 1 + return ms, code + + +def get_token(bank: str, user: str, password: str) -> str: + code, data, _ = http_json( + f"{bank}/accounts/{user}/token", + method="POST", + headers={"Content-Type": "application/json"}, + data=b'{"scope":"readonly"}', + auth=(user, password), + timeout=15, + ) + if code not in (200, 201) or not isinstance(data, dict): + raise RuntimeError(f"token failed for {user}: HTTP {code}") + tok = data.get("access_token") or data.get("token") or "" + if not tok: + raise RuntimeError(f"token empty for {user}") + return str(tok) + + +def list_all_accounts(bank: str, token: str, delta: int) -> List[str]: + """List accounts; page with start if needed.""" + names: List[str] = [] + seen: Set[str] = set() + start: Optional[int] = None + pages = 0 + max_pages = int(env("MAX_ACCOUNT_PAGES", "50") or "50") + + while pages < max_pages: + pages += 1 + q = f"delta={delta}" + if start is not None: + q += f"&start={start}" + code, data, raw = http_json( + f"{bank}/accounts?{q}", + headers={"Authorization": f"Bearer {token}"}, + timeout=45, + ) + if code == 204 or not data: + break + if code != 200: + raise RuntimeError(f"list accounts HTTP {code}: {raw[:200]!r}") + + batch: List[Dict[str, Any]] = [] + if isinstance(data, dict): + if isinstance(data.get("accounts"), list): + batch = data["accounts"] + elif isinstance(data.get("users"), list): + batch = data["users"] + elif isinstance(data, list): + batch = data + + if not batch: + # flat object with username? try regex fallback + text = raw.decode("utf-8", errors="replace") + for m in re.finditer(r'"username"\s*:\s*"([^"]+)"', text): + u = m.group(1) + if u not in seen: + seen.add(u) + names.append(u) + break + + min_row = None + for item in batch: + if not isinstance(item, dict): + continue + u = item.get("username") or item.get("name") or "" + if u and u not in seen: + seen.add(u) + names.append(str(u)) + rid = item.get("row_id") or item.get("rowId") + if rid is not None: + try: + rid_i = int(rid) + min_row = rid_i if min_row is None else min(min_row, rid_i) + except Exception: + pass + + if len(batch) < abs(delta): + break + if min_row is None: + break + # next older page + start = min_row + # avoid infinite loop on same start + if pages > 1 and len(names) == len(seen): + # if no growth and full page, still advance + pass + + return names + + +def iter_transactions( + bank: str, + token: str, + username: str, + page: int, + max_pages: int, +) -> List[Dict[str, Any]]: + """Return all transactions for account (paginated).""" + out: List[Dict[str, Any]] = [] + start: Optional[int] = None + pages = 0 + safety = max_pages if max_pages > 0 else 10_000 + + while pages < safety: + pages += 1 + q = f"delta=-{abs(page)}" + if start is not None: + q += f"&start={start}" + code, data, raw = http_json( + f"{bank}/accounts/{urllib.parse.quote(username)}/transactions?{q}", + headers={"Authorization": f"Bearer {token}"}, + timeout=float(env("TX_CURL_TIMEOUT", "45") or "45"), + ) + if code in (204, 404) or not data: + break + if code != 200: + # soft-fail one account + break + + txs: List[Dict[str, Any]] = [] + if isinstance(data, dict) and isinstance(data.get("transactions"), list): + txs = data["transactions"] + elif isinstance(data, list): + txs = data + else: + # tolerate raw array-ish + break + + if not txs: + break + + min_row = None + for tx in txs: + if not isinstance(tx, dict): + continue + out.append(tx) + rid = tx.get("row_id") or tx.get("rowId") + if rid is not None: + try: + rid_i = int(rid) + min_row = rid_i if min_row is None else min(min_row, rid_i) + except Exception: + pass + + if len(txs) < abs(page): + break + if min_row is None: + break + start = min_row + + return out + + +def tx_fields(tx: Dict[str, Any]) -> Tuple[str, str, int, str]: + """direction, amount, unix_ts, subject""" + direction = str(tx.get("direction") or "").lower() + amount = str(tx.get("amount") or "") + subject = str(tx.get("subject") or tx.get("description") or "") + ts = 0 + when = tx.get("when") or tx.get("date") or tx.get("timestamp") + if isinstance(when, dict): + # Taler AbsoluteTime: t_s seconds or t_ms + if "t_s" in when: + try: + ts = int(when["t_s"]) + except Exception: + ts = 0 + elif "t_ms" in when: + try: + ts = int(int(when["t_ms"]) / 1000) + except Exception: + ts = 0 + elif isinstance(when, (int, float)): + ts = int(when) + if ts > 10_000_000_000: # ms + ts //= 1000 + elif tx.get("t_s") is not None: + try: + ts = int(tx["t_s"]) + except Exception: + ts = 0 + return direction, amount, ts, subject + + +def reserve_from_subject(subject: str) -> str: + m = RESERVE_RE.search(subject or "") + if m: + return re.sub(r"[^A-Za-z0-9]", "", m.group(1)) + # fallback: last token + parts = (subject or "").split() + if parts: + return re.sub(r"[^A-Za-z0-9]", "", parts[-1]) + return "" + + +def now_parts(tz_name: str) -> Tuple[int, str, str]: + tz = ZoneInfo(tz_name) + dt = datetime.now(tz) + unix = int(dt.timestamp()) + iso = dt.strftime("%Y-%m-%dT%H:%M%z") + # +0200 → +02:00 + if len(iso) >= 5 and iso[-5] in "+-" and ":" not in iso[-5:]: + iso = iso[:-2] + ":" + iso[-2:] + human = dt.strftime("%Y-%m-%d %H:%M %Z") + return unix, iso, human + + +def human_from_unix(ts: int, tz_name: str) -> Tuple[str, str]: + if not ts: + return "", "" + tz = ZoneInfo(tz_name) + dt = datetime.fromtimestamp(ts, tz) + iso = dt.strftime("%Y-%m-%dT%H:%M%z") + if len(iso) >= 5 and iso[-5] in "+-" and ":" not in iso[-5:]: + iso = iso[:-2] + ":" + iso[-2:] + human = dt.strftime("%Y-%m-%d %H:%M %Z") + return human, iso + + +def main() -> int: + ap = argparse.ArgumentParser(description="Collect full bank landing stats") + ap.add_argument("--bank", default=env("BANK_URL", "http://127.0.0.1:9012")) + ap.add_argument("--admin-user", default=env("BANK_ADMIN_USER", "admin")) + ap.add_argument("--admin-pass", default=env("BANK_ADMIN_PASS", "")) + ap.add_argument("--admin-pass-file", default=env("BANK_ADMIN_PASS_FILE", "")) + ap.add_argument("--explorer-user", default=env("BANK_EXPLORER_USER", "explorer")) + ap.add_argument("--explorer-pass", default=env("BANK_EXPLORER_PASS", "")) + ap.add_argument("--explorer-pass-file", default=env("BANK_EXPLORER_PASS_FILE", "")) + ap.add_argument( + "--exchange-config", + default=env("EXCHANGE_CONFIG_URL", "https://exchange.hacktivism.ch/config"), + ) + ap.add_argument("--out", default=env("OUT", "-")) + ap.add_argument("--run-out", default=env("RUN_OUT", "")) + ap.add_argument( + "--skip", + default=env("SCAN_SKIP", "exchange"), + help="comma usernames excluded from flow scan (default: exchange)", + ) + ap.add_argument("--tx-page", type=int, default=int(env("TX_PAGE", "500") or "500")) + ap.add_argument( + "--max-tx-pages", + type=int, + default=int(env("MAX_TX_PAGES", "0") or "0"), + help="0 = unlimited pages per account", + ) + ap.add_argument( + "--accounts-delta", + type=int, + default=int(env("ACCOUNTS_DELTA", "-10000") or "-10000"), + ) + ap.add_argument("--recent", type=int, default=int(env("RECENT_WD_N", "10") or "10")) + ap.add_argument("--public-base", default=env("BANK_PUBLIC_URL", "https://bank.hacktivism.ch")) + args = ap.parse_args() + + tz_name = env("TZ", "Europe/Zurich") or "Europe/Zurich" + os.environ["TZ"] = tz_name + + admin_pass = args.admin_pass or ( + read_pass_file(args.admin_pass_file) if args.admin_pass_file else "" + ) + explorer_pass = args.explorer_pass or ( + read_pass_file(args.explorer_pass_file) if args.explorer_pass_file else "" + ) + + bank = args.bank.rstrip("/") + skip = {s.strip() for s in (args.skip or "").split(",") if s.strip()} + + def write_run(ok: bool, err: Optional[str] = None) -> None: + if not args.run_out: + return + unix, iso, human = now_parts(tz_name) + payload = { + "ok": ok, + "at": iso, + "at_human": human, + "error": err, + } + Path(args.run_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.run_out).write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + + try: + if not admin_pass: + raise RuntimeError("no admin password (BANK_ADMIN_PASS or --admin-pass-file)") + token = get_token(bank, args.admin_user, admin_pass) + alt = load_alt_from_config(args.exchange_config) + if not alt: + alt = dict(DEFAULT_ALT) + + accounts = list_all_accounts(bank, token, args.accounts_delta) + if not accounts: + raise RuntimeError("accounts list empty") + + # ALL accounts except skip set (default: only exchange — avoid double-count) + scan_users = [u for u in accounts if u not in skip] + # ensure explorer is included when present + if args.explorer_user not in skip and args.explorer_user in accounts: + if args.explorer_user not in scan_users: + scan_users.append(args.explorer_user) + + withdraws: List[Dict[str, Any]] = [] + incomings: List[Dict[str, Any]] = [] + total_in = Decimal(0) + total_wd = Decimal(0) + total_other = Decimal(0) + n_incoming = 0 + scan_ok = 0 + scan_empty = 0 + scan_fail = 0 + + for uname in scan_users: + try: + txs = iter_transactions( + bank, token, uname, args.tx_page, args.max_tx_pages + ) + except Exception: + scan_fail += 1 + continue + if not txs: + scan_empty += 1 + continue + scan_ok += 1 + for tx in txs: + direction, amount, ts, subject = tx_fields(tx) + if not amount: + continue + try: + _cur, n = parse_amount(amount) + except Exception: + continue + low = (subject or "").lower() + if direction == "credit": + total_in += n + n_incoming += 1 + incomings.append( + { + "kind": "incoming", + "amount": amount if ":" in amount else f"GOA:{amount}", + "at_unix": ts, + "account": uname, + "subject": subject, + } + ) + elif direction == "debit" and "withdraw" in low: + total_wd += n + res = reserve_from_subject(subject) + withdraws.append( + { + "kind": "withdraw", + "amount": amount if ":" in amount else f"GOA:{amount}", + "at_unix": ts, + "account": uname, + "subject": subject, + "reserve": res, + } + ) + elif direction == "debit": + total_other += n + + withdraws.sort(key=lambda r: r.get("at_unix") or 0, reverse=True) + incomings.sort(key=lambda r: r.get("at_unix") or 0, reverse=True) + + now_u, gen_iso, gen_human = now_parts(tz_name) + day = now_u - 86400 + week = now_u - 7 * 86400 + + w24 = Decimal(0) + w7 = Decimal(0) + n24 = 0 + n7 = 0 + for w in withdraws: + ts = int(w.get("at_unix") or 0) + try: + _c, n = parse_amount(w.get("amount")) + except Exception: + n = Decimal(0) + if ts >= day: + w24 += n + n24 += 1 + if ts >= week: + w7 += n + n7 += 1 + + wallets = sorted( + {w.get("reserve") for w in withdraws if w.get("reserve")} + ) + accounts_with_wd = sorted( + {w.get("account") for w in withdraws if w.get("account")} + ) + users_n = sum(1 for u in accounts if u not in ("admin", "exchange")) + + def pack_amt(d: Decimal) -> Dict[str, Any]: + f = format_amount_alt(f"GOA:{d}", alt) + return { + "amount": f["amount"], + "amount_alt": f["amount_alt"], + "amount_full": f["amount_full"], + "value": f["value"], + "value_str": f["value_str"], + } + + fin = pack_amt(total_in) + fwd = pack_amt(total_wd) + foth = pack_amt(total_other) + fout = pack_amt(total_wd + total_other) + p24 = pack_amt(w24) + p7 = pack_amt(w7) + + last = withdraws[0] if withdraws else None + last_amt = None + last_at = last_iso = last_subj = None + last_unix = None + last_alt = None + if last: + lf = format_amount_alt(last["amount"], alt) + last_amt = lf["amount"] + last_alt = lf["amount_alt"] + last_unix = last.get("at_unix") + last_at, last_iso = human_from_unix(int(last_unix or 0), tz_name) + last_subj = last.get("subject") + + recent = [] + for w in withdraws[: max(0, args.recent)]: + f = format_amount_alt(w["amount"], alt) + at_h, at_iso = human_from_unix(int(w.get("at_unix") or 0), tz_name) + recent.append( + { + "kind": "withdraw", + "amount": f["amount"], + "amount_alt": f["amount_alt"], + "amount_full": f["amount_full"], + "at": at_h, + "at_iso": at_iso, + "at_unix": w.get("at_unix") or None, + "account": w.get("account"), + "reserve": w.get("reserve") or "", + } + ) + + recent_in = [] + for row in incomings[:5]: + f = format_amount_alt(row["amount"], alt) + at_h, at_iso = human_from_unix(int(row.get("at_unix") or 0), tz_name) + recent_in.append( + { + "kind": "incoming", + "amount": f["amount"], + "amount_alt": f["amount_alt"], + "amount_full": f["amount_full"], + "at": at_h, + "at_iso": at_iso, + "at_unix": row.get("at_unix") or None, + "account": row.get("account"), + } + ) + + # explorer balance + balance = "GOA:0" + bal_alt = "GOA:0" + bal_full = "GOA:0" + if explorer_pass: + try: + etok = get_token(bank, args.explorer_user, explorer_pass) + code, data, _ = http_json( + f"{bank}/accounts/{args.explorer_user}", + headers={"Authorization": f"Bearer {etok}"}, + timeout=12, + ) + if code == 200 and isinstance(data, dict): + balance = str( + data.get("balance", {}).get("amount") + if isinstance(data.get("balance"), dict) + else data.get("amount") or data.get("balance") or "GOA:0" + ) + # sometimes balance is object with amount + if isinstance(data.get("balance"), dict) and data["balance"].get( + "amount" + ): + balance = str(data["balance"]["amount"]) + bf = format_amount_alt(balance, alt) + balance, bal_alt, bal_full = ( + bf["amount"], + bf["amount_alt"], + bf["amount_full"], + ) + except Exception: + pass + else: + # try admin-read of explorer account + try: + code, data, _ = http_json( + f"{bank}/accounts/{args.explorer_user}", + headers={"Authorization": f"Bearer {token}"}, + timeout=12, + ) + if code == 200 and isinstance(data, dict): + if isinstance(data.get("balance"), dict): + balance = str(data["balance"].get("amount") or "GOA:0") + elif data.get("amount"): + balance = str(data["amount"]) + bf = format_amount_alt(balance, alt) + balance, bal_alt, bal_full = ( + bf["amount"], + bf["amount_alt"], + bf["amount_full"], + ) + except Exception: + pass + + pub = args.public_base.rstrip("/") + config_ms, config_http = measure_ms(f"{pub}/config") + int_ms, int_http = measure_ms(f"{pub}/taler-integration/config") + webui_ms, webui_http = measure_ms(f"{pub}/webui/") + if config_http != "200": + config_ms, config_http = measure_ms(f"{bank}/config") + if int_http != "200": + int_ms, int_http = measure_ms(f"{bank}/taler-integration/config") + + loadavg = "" + try: + with open("/proc/loadavg", encoding="utf-8") as f: + parts = f.read().split() + loadavg = ",".join(parts[:3]) + except Exception: + pass + + total_pack = pack_amt(total_wd) + + stats = { + "ok": True, + "currency": "GOA", + "timezone": tz_name, + "generated_at": gen_iso, + "generated_at_human": gen_human, + "generated_at_unix": now_u, + "source": "host collect_bank_stats.py (hernani systemd)", + "bank_url": bank, + "scan": { + "tx_page": args.tx_page, + "max_tx_pages": args.max_tx_pages, + "accounts_delta": args.accounts_delta, + "skip_users": sorted(skip), + "accounts_listed": len(accounts), + "accounts_scanned_ok": scan_ok, + "accounts_empty_tx": scan_empty, + "accounts_scan_fail": scan_fail, + "accounts_scanned": len(scan_users), + "note": "all accounts except skip_users; full tx pagination; empty includes HTTP 204", + }, + "bank_accounts": { + "total": len(accounts), + "users": users_n, + "with_withdraws": len(accounts_with_wd), + }, + "wallets": { + "unique_reserves": len(wallets), + "note": "unique reserve pubs from Taler withdrawals (one per wallet withdraw)", + }, + "balance_explorer": balance, + "balance_explorer_alt": bal_alt, + "balance_explorer_full": bal_full, + "flow": { + "incoming": { + "label": "Incoming bank credits", + "count": n_incoming, + "amount": fin["amount"], + "amount_alt": fin["amount_alt"], + "amount_full": fin["amount_full"], + "value": fin["value"], + "value_str": fin["value_str"], + }, + "withdraw": { + "label": "Taler withdrawals to wallets", + "count": len(withdraws), + "amount": fwd["amount"], + "amount_alt": fwd["amount_alt"], + "amount_full": fwd["amount_full"], + "value": fwd["value"], + "value_str": fwd["value_str"], + }, + "other_out": { + "label": "Other debits (non-withdraw)", + "amount": foth["amount"], + "amount_alt": foth["amount_alt"], + "amount_full": foth["amount_full"], + "value": foth["value"], + "value_str": foth["value_str"], + }, + "total_in": fin["amount"], + "total_in_alt": fin["amount_alt"], + "total_in_value": fin["value"], + "total_out": fout["amount"], + "total_out_alt": fout["amount_alt"], + "total_out_value": fout["value"], + "note": "incoming=credits; withdraw=Taler withdrawal debits; skip_users excluded (default exchange)", + }, + "withdraws": { + "count": len(withdraws), + "total_amount": total_pack["amount"], + "total_amount_alt": total_pack["amount_alt"], + "total_amount_full": total_pack["amount_full"], + "total_value": total_pack["value"], + "total_value_str": total_pack["value_str"], + "last_amount": last_amt, + "last_amount_alt": last_alt, + "last_at": last_at, + "last_at_iso": last_iso, + "last_at_unix": last_unix, + "last_subject": last_subj, + "last_24h": { + "count": n24, + "amount": p24["amount"], + "amount_alt": p24["amount_alt"], + "amount_full": p24["amount_full"], + "value": p24["value"], + }, + "last_7d": { + "count": n7, + "amount": p7["amount"], + "amount_alt": p7["amount_alt"], + "amount_full": p7["amount_full"], + "value": p7["value"], + }, + }, + "recent_withdraws": recent, + "recent_incoming": recent_in, + "demo": { + "uri": None, + "amount": None, + "created": None, + "withdrawal_id": None, + "status": None, + "ready": False, + }, + "performance": { + "config_http": config_http, + "config_ms": config_ms, + "integration_http": int_http, + "integration_ms": int_ms, + "webui_http": webui_http, + "webui_ms": webui_ms, + "loadavg": loadavg, + "memory": { + "container_rss_human": "—", + "note": "memory filled by host collector merge when available", + }, + }, + "alt_unit_names": alt, + } + + # pull demo withdraw files + memory from bank container if OUT is path and caller merged + # demo files optional via DEMO_DIR + demo_dir = env("DEMO_DIR", "") + if demo_dir: + dpath = Path(demo_dir) + uri = (dpath / "withdraw.uri").read_text().strip() if (dpath / "withdraw.uri").is_file() else "" + amt = ( + (dpath / "withdraw.amount").read_text().strip() + if (dpath / "withdraw.amount").is_file() + else "" + ) + created = ( + (dpath / "withdraw.created").read_text().strip() + if (dpath / "withdraw.created").is_file() + else "" + ) + if uri: + wid = uri.rstrip("/").split("/")[-1] + stats["demo"] = { + "uri": uri, + "amount": amt or None, + "created": created or None, + "withdrawal_id": wid, + "status": None, + "ready": True, + } + + stats = enrich_stats_tree(stats, alt) + + text = json.dumps(stats, indent=2, ensure_ascii=False) + "\n" + if args.out == "-" or not args.out: + sys.stdout.write(text) + else: + outp = Path(args.out) + outp.parent.mkdir(parents=True, exist_ok=True) + tmp = outp.with_suffix(outp.suffix + f".tmp.{os.getpid()}") + tmp.write_text(text, encoding="utf-8") + tmp.replace(outp) + write_run(True, None) + print( + f"ok accounts={len(accounts)} scanned={scan_ok} withdraws={len(withdraws)} " + f"total={total_pack['amount']} alt={total_pack['amount_alt']}", + file=sys.stderr, + ) + return 0 + except Exception as e: + write_run(False, str(e)) + print(f"error: {e}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/taler-landing/enrich_stats_alt.py b/scripts/taler-landing/enrich_stats_alt.py new file mode 100755 index 0000000..366ee6b --- /dev/null +++ b/scripts/taler-landing/enrich_stats_alt.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 +"""Add amount_alt / amount_full fields to a landing stats.json in place.""" +from __future__ import annotations + +import argparse +import json +import os +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from goa_amounts import DEFAULT_ALT, enrich_stats_tree, load_alt_from_config # noqa: E402 + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("path", help="stats.json path") + ap.add_argument( + "--exchange-config", + default="https://exchange.hacktivism.ch/config", + ) + ap.add_argument("-o", "--out", default="", help="default: overwrite path") + args = ap.parse_args() + p = Path(args.path) + data = json.loads(p.read_text(encoding="utf-8")) + if not isinstance(data, dict) or not data.get("ok"): + print("skip: not ok stats", file=sys.stderr) + return 0 + alt = load_alt_from_config(args.exchange_config) or dict(DEFAULT_ALT) + enrich_stats_tree(data, alt) + out = Path(args.out) if args.out else p + tmp = out.with_suffix(out.suffix + f".tmp.{os.getpid()}") + tmp.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n", encoding="utf-8") + tmp.replace(out) + print(f"enriched {out}", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/taler-landing/goa_amounts.py b/scripts/taler-landing/goa_amounts.py new file mode 100755 index 0000000..6cd6c02 --- /dev/null +++ b/scripts/taler-landing/goa_amounts.py @@ -0,0 +1,306 @@ +#!/usr/bin/env python3 +"""GOA (and generic CUR:amount) alt-unit formatting for landing stats. + +Matches exchange currency_specification.alt_unit_names: + 0→GOA, 3→Kilo-GOA, 6→Mega-GOA, … and fractional scales. +Display prefers compact alt names for large values so landing tiles fit. +""" +from __future__ import annotations + +from decimal import Decimal, InvalidOperation, ROUND_HALF_UP +from typing import Any, Dict, Optional, Tuple + +# Default GOA ladder (same shape as exchange /config) +DEFAULT_ALT: Dict[str, str] = { + "24": "Yotta-GOA", + "21": "Zetta-GOA", + "18": "Exa-GOA", + "15": "Peta-GOA", + "12": "Tera-GOA", + "9": "Giga-GOA", + "6": "Mega-GOA", + "3": "Kilo-GOA", + "0": "GOA", + "-1": "Deci-GOA", + "-2": "Centi-GOA", + "-3": "Milli-GOA", + "-6": "Micro-GOA", + "-7": "Deci-Micro-GOA", + "-8": "Atomic-GOA", +} + +# Use alt unit when |value| >= this (base units). Below: keep compact CUR:n. +ALT_THRESHOLD = Decimal("1000") + + +def parse_amount(s: Any) -> Tuple[str, Decimal]: + """Parse 'GOA:12.5' or bare number → (currency, value).""" + if s is None: + return "GOA", Decimal(0) + if isinstance(s, (int, float, Decimal)): + return "GOA", Decimal(str(s)) + text = str(s).strip() + if not text: + return "GOA", Decimal(0) + if ":" in text: + cur, rest = text.split(":", 1) + return (cur or "GOA").strip(), Decimal(rest.strip() or "0") + return "GOA", Decimal(text) + + +def fmt_coeff(v: Decimal) -> str: + if v == v.to_integral_value(): + return format(int(v), "d") + # up to 4 significant fractional digits, strip trailing zeros + q = v.quantize(Decimal("0.0001"), rounding=ROUND_HALF_UP) + s = format(q, "f").rstrip("0").rstrip(".") + return s or "0" + + +def fmt_base(cur: str, val: Decimal) -> str: + if val == val.to_integral_value(): + return f"{cur}:{int(val)}" + # preserve up to 8 fractional digits (Taler style) + s = format(val, "f").rstrip("0").rstrip(".") + return f"{cur}:{s}" + + +def format_amount_alt( + amount: Any, + alt: Optional[Dict[str, str]] = None, + *, + threshold: Decimal = ALT_THRESHOLD, + with_base: bool = False, +) -> Dict[str, Any]: + """Return display fields for one amount. + + Keys: + amount canonical CUR:n + amount_alt short display (e.g. "1.23 Mega-GOA") + amount_full "1.23 Mega-GOA (GOA:1230000)" when alt used + value float for JSON (may lose precision above 2^53 — also value_str) + value_str exact decimal string + """ + alt = alt or DEFAULT_ALT + try: + cur, val = parse_amount(amount) + except (InvalidOperation, ValueError, ArithmeticError): + raw = str(amount or "") + return { + "amount": raw, + "amount_alt": raw or "—", + "amount_full": raw or "—", + "value": 0.0, + "value_str": "0", + } + + base = fmt_base(cur, val) + value_str = format(val, "f").rstrip("0").rstrip(".") if val != val.to_integral_value() else str(int(val)) + try: + value_f = float(val) + except Exception: + value_f = 0.0 + + base_name = alt.get("0") or cur + if val == 0: + disp = f"0 {base_name}" + return { + "amount": base, + "amount_alt": disp, + "amount_full": disp, + "value": 0.0, + "value_str": "0", + } + + scales = [] + for k, name in alt.items(): + try: + scales.append((int(k), str(name))) + except Exception: + continue + scales.sort(key=lambda x: -x[0]) + + absval = abs(val) + # Below threshold: short base form (saves noise on small demo amounts) + if absval < threshold: + # Prefer "GOA:12.5" style (matches existing landings) when small + return { + "amount": base, + "amount_alt": base, + "amount_full": base, + "value": value_f, + "value_str": value_str, + } + + chosen = None + for sc, name in scales: + unit = Decimal(10) ** sc + if unit <= 0: + continue + coeff = absval / unit + if coeff >= 1: + chosen = (sc, name, coeff if val >= 0 else -coeff) + break + + if chosen is None or chosen[0] == 0: + return { + "amount": base, + "amount_alt": base, + "amount_full": base, + "value": value_f, + "value_str": value_str, + } + + sc, name, coeff = chosen + short = f"{fmt_coeff(coeff)} {name}" + full = f"{short} ({base})" if with_base or True else short + return { + "amount": base, + "amount_alt": short, + "amount_full": full, + "value": value_f, + "value_str": value_str, + } + + +def attach_alt(obj: Dict[str, Any], amount_key: str = "amount", alt: Optional[Dict[str, str]] = None) -> Dict[str, Any]: + """Mutate obj: ensure amount_alt / amount_full from amount_key.""" + if not isinstance(obj, dict): + return obj + src = obj.get(amount_key) + if src is None: + return obj + f = format_amount_alt(src, alt) + obj[amount_key] = f["amount"] + obj["amount_alt"] = f["amount_alt"] + obj["amount_full"] = f["amount_full"] + if "value" not in obj: + obj["value"] = f["value"] + obj["value_str"] = f["value_str"] + return obj + + +def load_alt_from_config(url: str, timeout: float = 8.0) -> Dict[str, str]: + """GET exchange/bank /config → alt_unit_names (fallback DEFAULT_ALT).""" + import json + import urllib.request + + try: + req = urllib.request.Request(url, headers={"Accept": "application/json"}) + with urllib.request.urlopen(req, timeout=timeout) as resp: + data = json.loads(resp.read().decode("utf-8", errors="replace")) + except Exception: + return dict(DEFAULT_ALT) + + au = None + cs = data.get("currency_specification") + if isinstance(cs, dict): + au = cs.get("alt_unit_names") + if not au and isinstance(data.get("currencies"), dict): + for _code, spec in data["currencies"].items(): + if isinstance(spec, dict) and spec.get("alt_unit_names"): + au = spec["alt_unit_names"] + break + if not isinstance(au, dict) or "0" not in au: + return dict(DEFAULT_ALT) + return {str(k): str(v) for k, v in au.items()} + + +def enrich_stats_tree(data: Dict[str, Any], alt: Optional[Dict[str, str]] = None) -> Dict[str, Any]: + """Walk common landing stats.json shapes and add amount_alt fields.""" + alt = alt or DEFAULT_ALT + if not isinstance(data, dict): + return data + + # bank-style + for path in ( + ("flow", "incoming"), + ("flow", "withdraw"), + ("flow", "other_out"), + ("withdraws",), + ("withdraws", "last_24h"), + ("withdraws", "last_7d"), + ): + cur: Any = data + ok = True + for p in path: + if not isinstance(cur, dict) or p not in cur: + ok = False + break + cur = cur[p] + if ok and isinstance(cur, dict): + if "amount" in cur: + attach_alt(cur, "amount", alt) + if "total_amount" in cur: + f = format_amount_alt(cur["total_amount"], alt) + cur["total_amount"] = f["amount"] + cur["total_amount_alt"] = f["amount_alt"] + cur["total_amount_full"] = f["amount_full"] + if "last_amount" in cur and cur["last_amount"]: + f = format_amount_alt(cur["last_amount"], alt) + cur["last_amount"] = f["amount"] + cur["last_amount_alt"] = f["amount_alt"] + + if isinstance(data.get("flow"), dict): + fl = data["flow"] + for k in ("total_in", "total_out"): + if fl.get(k): + f = format_amount_alt(fl[k], alt) + fl[k] = f["amount"] + fl[f"{k}_alt"] = f["amount_alt"] + fl[f"{k}_full"] = f["amount_full"] + + if data.get("balance_explorer"): + f = format_amount_alt(data["balance_explorer"], alt) + data["balance_explorer"] = f["amount"] + data["balance_explorer_alt"] = f["amount_alt"] + data["balance_explorer_full"] = f["amount_full"] + + for key in ("recent_withdraws", "recent_incoming", "recent_activity"): + rows = data.get(key) + if isinstance(rows, list): + for row in rows: + if isinstance(row, dict) and row.get("amount"): + attach_alt(row, "amount", alt) + + # exchange-style top-level amounts + for k in ( + "wire_in_amount", + "withdraw_amount", + "coins_remaining_amount", + ): + if data.get(k): + f = format_amount_alt(data[k], alt) + data[k] = f["amount"] + data[f"{k}_alt"] = f["amount_alt"] + data[f"{k}_full"] = f["amount_full"] + + for key in ("by_denom", "denom_ladder"): + rows = data.get(key) + if isinstance(rows, list): + for row in rows: + if isinstance(row, dict) and row.get("value"): + f = format_amount_alt(row["value"], alt) + row["value"] = f["amount"] + row["value_alt"] = f["amount_alt"] + + # merchant dual currency + for block in data.get("by_currency") or []: + if not isinstance(block, dict): + continue + for k in ("amount_sum", "amount_paid_sum"): + if block.get(k): + f = format_amount_alt(block[k], alt) + block[k] = f["amount"] + block[f"{k}_alt"] = f["amount_alt"] + block[f"{k}_full"] = f["amount_full"] + + for block in data.get("recent_activity_by_currency") or []: + if not isinstance(block, dict): + continue + for row in block.get("items") or []: + if isinstance(row, dict) and row.get("amount"): + attach_alt(row, "amount", alt) + + data["alt_unit_names"] = alt + return data