fix(monitoring): withdraw URI without :443 and show perf ms
This commit is contained in:
Hernâni Marques
parent
4ec9f07276
commit
594714bbfd
5 changed files with 183 additions and 48 deletions
|
|
@ -94,6 +94,22 @@ E2E maps failures to blockers, e.g.:
|
|||
- `Alarm clock` during pay → usually missing `handle-uri --yes` or wrong pay URI (must be `…/instances/{inst}/{oid}/?c={token}` from merchant `taler_pay_uri`)
|
||||
- `insufficient balance` → withdraw incomplete
|
||||
|
||||
## Performance (urls phase)
|
||||
|
||||
Outside-in HTTPS RTT for bank / exchange / merchant critical paths. Each probe prints
|
||||
`HTTP … · N ms` on the `[OK]`/`[WARN]`/`[ERROR]` line; end of the block prints
|
||||
`perf summary` (n / min / p50 / avg / max).
|
||||
|
||||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `PERF_WARN_MS` | `8000` | WARN if latency ≥ this |
|
||||
| `PERF_FAIL_MS` | `20000` | ERROR if latency ≥ this |
|
||||
| `PERF_CURL_TIMEOUT` | `25` | curl max seconds per probe |
|
||||
|
||||
```bash
|
||||
PERF_WARN_MS=3000 PERF_FAIL_MS=10000 ./taler-monitoring.sh urls
|
||||
```
|
||||
|
||||
## Needs
|
||||
|
||||
- SSH `koopa` (inside/sanity server bits)
|
||||
|
|
|
|||
|
|
@ -46,15 +46,15 @@ IDs are assigned **in run order** within the area (`set_area` resets the counter
|
|||
| www-… | **landing exposed links** (bank / merchant / exchange): parse each `/intro/` HTML, probe every own-stack `https://` + root-relative `href`/`src`/`content`, soft-check external stores/docs |
|
||||
| www-… | landing static: `qrcode.min.js`, `og-goa-shop.png`, `qr-logo.png`, shop-pay.js/css |
|
||||
| www-… | cross-links between bank ↔ merchant ↔ exchange intros (local stack) |
|
||||
| www-… | **bank `/intro/demo-withdraw.json`** → `taler://withdraw/HOST:PORT/taler-integration/…` + integration op HTTP 200 |
|
||||
| www-… | **bank `/intro/demo-withdraw.json`** → `taler://withdraw/HOST/taler-integration/…` (no default `:443`/`:80`; non-default port OK) + integration op HTTP 200 |
|
||||
| www-… | bank `/intro/auto-account.json` (earlier) → same withdraw shape, **no payto_uri**, login at `/webui/` |
|
||||
| www-… | **performance** (outside-in): public HTTPS RTT for bank `/config`, `/taler-integration/config`, `/webui/`, `/intro/`, `stats.json`; exchange `/config`, `/keys`, `/intro/`; merchant `/config`, `/webui/`, `/intro/` — report ms; WARN ≥ `PERF_WARN_MS` (default 8000); **ERROR ≥ `PERF_FAIL_MS` (default 20000)** |
|
||||
|
||||
**Legal docs rule:** HTTP 200, non-empty body, not plain `not configured`, not merchant API JSON `code:21`. On local stack, optional content needle (terms/privacy/FADP/GOA…).
|
||||
|
||||
**Performance rule:** Measured from the **monitoring runner** (public URLs via Caddy), not container loopback. HTTP must match expect (usually 200); latency is reported on the OK line. Slow ≥ `PERF_WARN_MS` → WARN only (no ERROR on slowness alone).
|
||||
**Performance rule:** Measured from the **monitoring runner** (public URLs via Caddy), not container loopback. HTTP must match expect (usually 200); **latency (ms) is on every perf OK/WARN/ERROR line**, plus a **perf summary** (n / min / p50 / avg / max). Slow ≥ `PERF_WARN_MS` (default 8000) → WARN; ≥ `PERF_FAIL_MS` (default 20000) → ERROR.
|
||||
|
||||
**Landing links rule:** Own-stack (bank/exchange/taler.\* + page host) must be HTTP 200 (or redirect→200). External (App Store, Play, F-Droid, wallet.taler.net, docs/git.taler.net, …) soft WARN if down. Auto-account wallet link must be `taler://withdraw/…:port/taler-integration/…`, never payto.
|
||||
**Landing links rule:** Own-stack (bank/exchange/taler.\* + page host) must be HTTP 200 (or redirect→200). External (App Store, Play, F-Droid, wallet.taler.net, docs/git.taler.net, …) soft WARN if down. Auto-account wallet link must be `taler://withdraw/HOST/taler-integration/…` (default ports stripped for mobile wallets), never payto.
|
||||
|
||||
**alt_unit_names rule:** wallet codec requires a non-empty map including scale key `"0"`. For multi-currency merchant, also follow every entry in `exchanges[]` and check that exchange’s public `/config`.
|
||||
|
||||
|
|
|
|||
|
|
@ -98,6 +98,77 @@ check_legal_doc() {
|
|||
rm -f "$f"
|
||||
}
|
||||
|
||||
# Shared shape for bank mint JSON (demo-withdraw + auto-account).
|
||||
# Args: $1=json file $2=auto|demo
|
||||
# stdout (ok): one detail line; for demo, optional "\twithdrawal_id" suffix.
|
||||
# stderr (fail): short reason. Exit 0/1.
|
||||
validate_bank_withdraw_json() {
|
||||
python3 - "$1" "$2" <<'PY'
|
||||
import json, re, sys
|
||||
from urllib.parse import urlparse
|
||||
|
||||
def check_withdraw_uri(wuri: str):
|
||||
"""HOST or HOST:non-default-port; default :443/:80 must be stripped (wallet fix)."""
|
||||
m = re.match(
|
||||
r"^taler://withdraw/([^/]+)/taler-integration/([0-9a-fA-F-]+)$",
|
||||
wuri or "",
|
||||
)
|
||||
if not m:
|
||||
print(
|
||||
"need taler://withdraw/HOST/taler-integration/ID:",
|
||||
(wuri or "")[:120],
|
||||
file=sys.stderr,
|
||||
)
|
||||
return None
|
||||
host = m.group(1)
|
||||
if not host or host.startswith(":"):
|
||||
print("bad withdraw host:", host, file=sys.stderr)
|
||||
return None
|
||||
if ":" in host:
|
||||
h, _, p = host.rpartition(":")
|
||||
if not h or not p.isdigit():
|
||||
print("bad withdraw host:port:", host, file=sys.stderr)
|
||||
return None
|
||||
if p in ("443", "80"):
|
||||
print("withdraw must strip default port :%s:" % p, host, file=sys.stderr)
|
||||
return None
|
||||
return host, m.group(2), wuri
|
||||
|
||||
path, mode = sys.argv[1], sys.argv[2]
|
||||
d = json.load(open(path))
|
||||
wuri = d.get("taler_withdraw_uri") or (
|
||||
d.get("qr_payload") if mode == "auto" else ""
|
||||
) or ""
|
||||
|
||||
if mode == "auto":
|
||||
if not d.get("ok"):
|
||||
print("ok!=true", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
if d.get("payto_uri"):
|
||||
print("payto_uri must not be present", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
if not check_withdraw_uri(wuri):
|
||||
sys.exit(1)
|
||||
webui = d.get("login_url") or d.get("webui") or d.get("account_url") or ""
|
||||
u = urlparse(webui)
|
||||
if u.scheme not in ("http", "https") or "webui" not in (u.path or ""):
|
||||
print("login webui missing:", webui[:80], file=sys.stderr)
|
||||
sys.exit(1)
|
||||
print("%s · %s" % (d.get("username", ""), wuri[:72]))
|
||||
elif mode == "demo":
|
||||
if not d.get("ok", True) and "taler_withdraw_uri" not in d:
|
||||
print("not ok", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
if not check_withdraw_uri(wuri):
|
||||
sys.exit(1)
|
||||
print("%s\t%s" % (wuri[:80], d.get("withdrawal_id") or ""))
|
||||
else:
|
||||
print("bad mode:", mode, file=sys.stderr)
|
||||
sys.exit(2)
|
||||
sys.exit(0)
|
||||
PY
|
||||
}
|
||||
|
||||
# --- exchange (core; always required) --- www-001 …
|
||||
check_url "exchange /config" 200 "$EXCHANGE_PUBLIC/config"
|
||||
code=$(http_body "$EXCHANGE_PUBLIC/config" "$tmp/ec.json")
|
||||
|
|
@ -150,6 +221,8 @@ section "www · performance · public HTTPS latency (outside-in)"
|
|||
# ≥ PERF_WARN_MS → WARN. ≥ PERF_FAIL_MS → ERROR (suite fails).
|
||||
PERF_WARN_MS="${PERF_WARN_MS:-8000}"
|
||||
PERF_FAIL_MS="${PERF_FAIL_MS:-20000}"
|
||||
PERF_TSV="$tmp/perf.tsv"
|
||||
: >"$PERF_TSV"
|
||||
|
||||
# Measure one URL: require HTTP expect (default 200), report time_total in ms.
|
||||
# $1=label $2=url $3=optional expected codes (default 200)
|
||||
|
|
@ -165,6 +238,8 @@ check_perf() {
|
|||
if (ms>0 && ms<1) ms=1
|
||||
printf "%d", int(ms+0.5)
|
||||
}')
|
||||
# Always record sample for rollup (label, ms, http, url)
|
||||
printf '%s\t%s\t%s\t%s\n' "$label" "$ms" "$code" "$url" >>"$PERF_TSV"
|
||||
case ",$expect," in
|
||||
*",$code,"*)
|
||||
if [ "$ms" -ge "${PERF_FAIL_MS}" ] 2>/dev/null; then
|
||||
|
|
@ -202,7 +277,76 @@ check_perf "perf merchant /config" "$MERCHANT_PUBLIC/config"
|
|||
check_perf "perf merchant /webui/" "$MERCHANT_PUBLIC/webui/" 200,301,302
|
||||
check_perf "perf merchant /intro/" "$MERCHANT_PUBLIC/intro/"
|
||||
|
||||
info "perf note" "measured from this host (outside-in); not container loopback"
|
||||
# Rollup + optional JSON for metrics_print_overall
|
||||
if [ -s "$PERF_TSV" ]; then
|
||||
PERF_JSON="$tmp/perf-summary.json"
|
||||
perf_line=$(python3 - "$PERF_TSV" "$PERF_JSON" "$PERF_WARN_MS" "$PERF_FAIL_MS" <<'PY'
|
||||
import json, sys
|
||||
rows = []
|
||||
for line in open(sys.argv[1]):
|
||||
parts = line.rstrip("\n").split("\t")
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
label, ms_s, code = parts[0], parts[1], parts[2]
|
||||
try:
|
||||
ms = int(float(ms_s))
|
||||
except Exception:
|
||||
continue
|
||||
rows.append({"label": label, "ms": ms, "http": code})
|
||||
vals = sorted(r["ms"] for r in rows)
|
||||
n = len(vals)
|
||||
if n == 0:
|
||||
print("n=0")
|
||||
json.dump({"n": 0}, open(sys.argv[2], "w"))
|
||||
raise SystemExit(0)
|
||||
def pct(p):
|
||||
if n == 1:
|
||||
return vals[0]
|
||||
i = min(n - 1, max(0, int(round((p / 100.0) * (n - 1)))))
|
||||
return vals[i]
|
||||
avg = int(round(sum(vals) / n))
|
||||
warn_ms = int(sys.argv[3])
|
||||
fail_ms = int(sys.argv[4])
|
||||
slow = [r for r in rows if r["ms"] >= warn_ms]
|
||||
rep = {
|
||||
"n": n,
|
||||
"min_ms": vals[0],
|
||||
"p50_ms": pct(50),
|
||||
"avg_ms": avg,
|
||||
"max_ms": vals[-1],
|
||||
"warn_ms": warn_ms,
|
||||
"fail_ms": fail_ms,
|
||||
"slow": [{"label": r["label"], "ms": r["ms"]} for r in slow],
|
||||
"samples": rows,
|
||||
}
|
||||
# metrics_print_overall expects named buckets with n/min/p50/avg/max
|
||||
out = {
|
||||
"www_public_https": {
|
||||
"n": n,
|
||||
"min_ms": vals[0],
|
||||
"p50_ms": pct(50),
|
||||
"avg_ms": avg,
|
||||
"max_ms": vals[-1],
|
||||
},
|
||||
**{r["label"].replace(" ", "_"): {"n": 1, "min_ms": r["ms"], "p50_ms": r["ms"], "avg_ms": r["ms"], "max_ms": r["ms"]} for r in rows},
|
||||
}
|
||||
json.dump(out, open(sys.argv[2], "w"), indent=2)
|
||||
extra = ""
|
||||
if slow:
|
||||
extra = " slow: " + ", ".join("%s=%dms" % (r["label"].replace("perf ", ""), r["ms"]) for r in slow)
|
||||
print(
|
||||
"n=%d min=%dms p50=%dms avg=%dms max=%dms (warn≥%dms fail≥%dms)%s"
|
||||
% (n, vals[0], pct(50), avg, vals[-1], warn_ms, fail_ms, extra)
|
||||
)
|
||||
PY
|
||||
)
|
||||
info "perf summary" "$perf_line"
|
||||
# Keep a copy if METRICS_DIR is set (e2e/ladder overall stats)
|
||||
if [ -n "${METRICS_DIR:-}" ] && [ -d "${METRICS_DIR}" ]; then
|
||||
cp -f "$PERF_JSON" "${METRICS_DIR}/perf-summary.json" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
info "perf note" "measured from this host (outside-in); thresholds PERF_WARN_MS=${PERF_WARN_MS} PERF_FAIL_MS=${PERF_FAIL_MS}"
|
||||
|
||||
|
||||
# Terms + privacy (legal docs)
|
||||
|
|
@ -247,31 +391,10 @@ if [ "$code" = "200" ]; then
|
|||
aa_code=$(http_body "$BANK_PUBLIC/intro/auto-account.json" "$tmp/aa.json")
|
||||
case "$aa_code" in
|
||||
200)
|
||||
if python3 - "$tmp/aa.json" <<'PY'
|
||||
import json, re, sys
|
||||
from urllib.parse import urlparse
|
||||
d = json.load(open(sys.argv[1]))
|
||||
if not d.get("ok"):
|
||||
print("ok!=true"); sys.exit(1)
|
||||
if "payto_uri" in d and d.get("payto_uri"):
|
||||
print("payto_uri must not be present"); sys.exit(1)
|
||||
wuri = d.get("taler_withdraw_uri") or d.get("qr_payload") or ""
|
||||
wm = re.match(r"^taler://withdraw/([^/]+)/taler-integration/([0-9a-fA-F-]+)$", wuri)
|
||||
if not wm:
|
||||
print("need taler://withdraw/HOST:PORT/taler-integration/ID:", wuri[:120]); sys.exit(1)
|
||||
if ":" not in wm.group(1):
|
||||
print("withdraw missing port:", wm.group(1)); sys.exit(1)
|
||||
webui = d.get("login_url") or d.get("webui") or d.get("account_url") or ""
|
||||
u = urlparse(webui)
|
||||
if u.scheme not in ("http", "https") or "webui" not in (u.path or ""):
|
||||
print("login webui missing:", webui[:80]); sys.exit(1)
|
||||
print("user=%s withdraw=%s login=%s" % (d.get("username"), wm.group(1), webui))
|
||||
sys.exit(0)
|
||||
PY
|
||||
then
|
||||
ok "bank /intro/auto-account.json" "$(python3 -c 'import json;d=json.load(open("'"$tmp/aa.json"'"));print(d.get("username",""),"·",(d.get("taler_withdraw_uri") or "")[:72])' 2>/dev/null || true)"
|
||||
if aa_detail=$(validate_bank_withdraw_json "$tmp/aa.json" auto 2>"$tmp/aa-val"); then
|
||||
ok "bank /intro/auto-account.json" "$aa_detail"
|
||||
else
|
||||
fail "bank /intro/auto-account.json" "invalid withdraw/login (HTTP body bad)"
|
||||
fail "bank /intro/auto-account.json" "invalid withdraw/login ($(tr '\n' ' ' <"$tmp/aa-val" | sed 's/[[:space:]]*$//'))"
|
||||
fi
|
||||
;;
|
||||
405|501|404|502|503|000)
|
||||
|
|
@ -591,29 +714,17 @@ if [ "${LOCAL_STACK:-1}" = "1" ] || [ -n "${BANK_PUBLIC:-}" ]; then
|
|||
dw_code=$(http_body "$BANK_PUBLIC/intro/demo-withdraw.json" "$tmp/dw.json")
|
||||
case "$dw_code" in
|
||||
200)
|
||||
if python3 - "$tmp/dw.json" <<'PY'
|
||||
import json, re, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
if not d.get("ok", True) and "taler_withdraw_uri" not in d:
|
||||
print("not ok"); sys.exit(1)
|
||||
u = d.get("taler_withdraw_uri") or ""
|
||||
m = re.match(r"^taler://withdraw/([^/]+)/taler-integration/([0-9a-fA-F-]+)$", u)
|
||||
if not m:
|
||||
print("bad uri:", u[:120]); sys.exit(1)
|
||||
if ":" not in m.group(1):
|
||||
print("missing port:", m.group(1)); sys.exit(1)
|
||||
print(u[:88])
|
||||
sys.exit(0)
|
||||
PY
|
||||
then
|
||||
ok "bank /intro/demo-withdraw.json" "$(python3 -c 'import json;print(json.load(open("'"$tmp/dw.json"'")).get("taler_withdraw_uri","")[:80])' 2>/dev/null || true)"
|
||||
wid=$(python3 -c 'import json;print(json.load(open("'"$tmp/dw.json"'")).get("withdrawal_id",""))' 2>/dev/null || true)
|
||||
if dw_out=$(validate_bank_withdraw_json "$tmp/dw.json" demo 2>"$tmp/dw-val"); then
|
||||
dw_detail=${dw_out%%$'\t'*}
|
||||
wid=${dw_out#*$'\t'}
|
||||
[ "$wid" = "$dw_out" ] && wid=
|
||||
ok "bank /intro/demo-withdraw.json" "$dw_detail"
|
||||
if [ -n "$wid" ]; then
|
||||
check_landing_asset "bank taler-integration withdraw op" \
|
||||
"$BANK_PUBLIC/taler-integration/withdrawal-operation/${wid}"
|
||||
fi
|
||||
else
|
||||
fail "bank /intro/demo-withdraw.json" "invalid taler://withdraw shape"
|
||||
fail "bank /intro/demo-withdraw.json" "invalid taler://withdraw ($(tr '\n' ' ' <"$tmp/dw-val" | sed 's/[[:space:]]*$//'))"
|
||||
fi
|
||||
;;
|
||||
405|501|404|502|503|000)
|
||||
|
|
|
|||
|
|
@ -282,9 +282,16 @@ _fmt_tid() {
|
|||
}
|
||||
|
||||
ok() {
|
||||
local label="$1"
|
||||
# Forms (same idea as info/warn):
|
||||
# ok "what passed"
|
||||
# ok "what passed" "detail / ms / bytes / …"
|
||||
local label="$1" detail="${2:-}"
|
||||
_take_tid
|
||||
if [ -n "$detail" ]; then
|
||||
printf '%s[OK]%s %s%s — %s\n' "$G" "$N" "$(_fmt_tid)" "$label" "$detail"
|
||||
else
|
||||
printf '%s[OK]%s %s%s\n' "$G" "$N" "$(_fmt_tid)" "$label"
|
||||
fi
|
||||
PASS_N=$((PASS_N + 1))
|
||||
}
|
||||
# component-scoped error: err bank "libeufin down" "detail"
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ Examples:
|
|||
Env (same meaning):
|
||||
TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC EXPECT_CURRENCY
|
||||
SKIP_SSH=1 NO_COLOR=1
|
||||
PERF_WARN_MS PERF_FAIL_MS (urls latency; default 8000 / 20000)
|
||||
EOF
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue