Compare commits
2 commits
llm-lightw
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fb68a1b011 | ||
|
|
6d5adeef9c |
34 changed files with 741 additions and 7484 deletions
43
README.md
43
README.md
|
|
@ -1,8 +1,5 @@
|
||||||
# taler-monitoring
|
# taler-monitoring
|
||||||
|
|
||||||
**Product name:** **LLM** = **Lightweight Monitoring** (Taler-stack watcher UI/daemon).
|
|
||||||
Git/CLI tree remains `taler-monitoring` / `taler-monitord`. Live stacks page: https://hacktivism.ch/stacks/
|
|
||||||
|
|
||||||
Checks for **GNU Taler** stacks (bank, exchange, merchant; optional surface inventory).
|
Checks for **GNU Taler** stacks (bank, exchange, merchant; optional surface inventory).
|
||||||
|
|
||||||
| Mode | What |
|
| Mode | What |
|
||||||
|
|
@ -15,7 +12,7 @@ Outside-only example: `./taler-monitoring.sh -d demo.taler.net urls`
|
||||||
Surface (ecosystem + mail + mattermost + versions): `./taler-monitoring.sh surface mattermost mail versions`
|
Surface (ecosystem + mail + mattermost + versions): `./taler-monitoring.sh surface mattermost mail versions`
|
||||||
Full local GOA example: `./taler-monitoring.sh -d hacktivism.ch urls inside versions`
|
Full local GOA example: `./taler-monitoring.sh -d hacktivism.ch urls inside versions`
|
||||||
|
|
||||||
Repo: https://git.hacktivism.ch/hernani/taler-monitoring · tree: `~/src/taler/taler-monitoring`
|
Repo: https://git.hacktivism.ch/hernani/taler-monitoring · tree: `~/src/taler-monitoring`
|
||||||
Version: `./taler-monitoring.sh --ver` (or `--version` / `-V`)
|
Version: `./taler-monitoring.sh --ver` (or `--version` / `-V`)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -178,31 +175,10 @@ STATS_STALE_SECS=900 STATS_FAIL_SECS=3600 ./taler-monitoring.sh -d stage.lefranc
|
||||||
|
|
||||||
# --- Full load (GOA / hacktivism) ---
|
# --- Full load (GOA / hacktivism) ---
|
||||||
# all = urls + inside + versions + sanity + e2e
|
# all = urls + inside + versions + sanity + e2e
|
||||||
# full = all + server + ladder + auth401 (long; needs secrets / SECRETS_ROOT)
|
# full = all + server + ladder + auth401 (long; needs secrets.env)
|
||||||
./taler-monitoring.sh -d hacktivism.ch full
|
./taler-monitoring.sh -d hacktivism.ch full
|
||||||
./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401
|
./taler-monitoring.sh -d hacktivism.ch urls inside versions sanity server e2e ladder auth401
|
||||||
|
|
||||||
# --- GOA payment / withdraw tests (hacktivism) ---
|
|
||||||
# Core e2e: bank credit → withdraw ladder → merchant pays (+ optional paivana GOA:4200)
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch e2e
|
|
||||||
# Only two small bank withdraws (diagnose confirm path), tiny pay:
|
|
||||||
E2E_VARIABLE=1 E2E_WITHDRAW_VALUES='20 20' E2E_PAY_VALUES='0.01' \
|
|
||||||
E2E_BANK_WITHDRAW_MAX=2 E2E_TOKEN_ENABLE=0 RUN_TIMEOUT=0 \
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch e2e
|
|
||||||
# Moderated classic ladder (withdraw+pay, timings in log; not full 23-rung):
|
|
||||||
LADDER_STEPS=8 LADDER_MAX_AMOUNT=5000 LADDER_MIN_AMOUNT=1 RUN_TIMEOUT=0 \
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch ladder
|
|
||||||
# Same ladder under legacy phase name:
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch goa-ladder
|
|
||||||
# Hunt highest withdrawable / payable (slow; set RUN_TIMEOUT=0):
|
|
||||||
RUN_TIMEOUT=0 ./taler-monitoring.sh -d hacktivism.ch max-ladder
|
|
||||||
# Merchant Basic-auth matrix (may create throwaway instance):
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch auth401
|
|
||||||
# Public URL + latency only (no wallet):
|
|
||||||
./taler-monitoring.sh -d hacktivism.ch urls
|
|
||||||
# Newer wallet-cli (optional):
|
|
||||||
# WALLET_CLI=$HOME/…/taler-wallet-cli.mjs ./taler-monitoring.sh -d hacktivism.ch e2e
|
|
||||||
|
|
||||||
# SPA pin after selfbuild-webui.sh (container):
|
# SPA pin after selfbuild-webui.sh (container):
|
||||||
EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \
|
EXPECT_WEBUI_VERSION=1.6.11 EXPECT_WEBUI_OVERLAY=selfbuild-v1.6.11 \
|
||||||
WEBUI_OVERLAY_DENY='c778af04|master-11340' \
|
WEBUI_OVERLAY_DENY='c778af04|master-11340' \
|
||||||
|
|
@ -249,12 +225,12 @@ stage.lefrancpaysan.ch stage.bank.… stage.exchange.… stage.monnaie.… T
|
||||||
| Field | Meaning |
|
| Field | Meaning |
|
||||||
|-------|---------|
|
|-------|---------|
|
||||||
| **name** | `-d NAME` / `TALER_DOMAIN` match (add alias lines as needed) |
|
| **name** | `-d NAME` / `TALER_DOMAIN` match (add alias lines as needed) |
|
||||||
| **bank** | libeufin / `x-taler-bank` host, or `-` / `ebics` when IBAN/EBICS only (no public `bank.*`) |
|
| **bank** | libeufin/bank public host or `https://…` |
|
||||||
| **exchange** | exchange public host or URL |
|
| **exchange** | exchange public host or URL |
|
||||||
| **merchant** | merchant **backend** (SPA/API), e.g. `my.taler-ops.ch` |
|
| **merchant** | merchant **backend** (SPA/API), e.g. `my.taler-ops.ch` |
|
||||||
| **currency** | `GOA` / `KUDOS` / `CHF` / … |
|
| **currency** | `GOA` / `KUDOS` / `CHF` / … |
|
||||||
| **local** | `1` = koopa SSH stack; `0` = public only |
|
| **local** | `1` = koopa SSH stack; `0` = public only |
|
||||||
| **landing** | `1` = custom `/intro` (GOA + FrancPaysan/TESTPAYSAN stage only); `0` = no intro |
|
| **landing** | `1` = check `/intro` landings + assets; `0` = skip (TOPS / mytops) |
|
||||||
| **lang** | UI language **`en`** / **`fr`** / **`de`** when `TALER_MON_LANG_SET≠1`. Stock profiles use **en** or **fr** only — **`de` is never a default** (opt-in via `--lang de` or an explicit profile edit). |
|
| **lang** | UI language **`en`** / **`fr`** / **`de`** when `TALER_MON_LANG_SET≠1`. Stock profiles use **en** or **fr** only — **`de` is never a default** (opt-in via `--lang de` or an explicit profile edit). |
|
||||||
| **locale** | Regional l10n (BCP 47): **`fr-CH`** for FrancPaysan (Suisse romande); **`de-CH`** for other Swiss stacks. Aliases `ch-FR` / `ch-DE` accepted. Not the same as UI `lang`. |
|
| **locale** | Regional l10n (BCP 47): **`fr-CH`** for FrancPaysan (Suisse romande); **`de-CH`** for other Swiss stacks. Aliases `ch-FR` / `ch-DE` accepted. Not the same as UI `lang`. |
|
||||||
| **canonical** | optional label after alias (e.g. `my.taler-ops.ch` → `taler-ops.ch`) |
|
| **canonical** | optional label after alias (e.g. `my.taler-ops.ch` → `taler-ops.ch`) |
|
||||||
|
|
@ -283,18 +259,15 @@ Helpers in `lib.sh`: `set_taler_stack`, `load_domain_profile`, `apply_taler_doma
|
||||||
| Domain / alias | Bank | Exchange | Merchant | Currency |
|
| Domain / alias | Bank | Exchange | Merchant | Currency |
|
||||||
|----------------|------|----------|----------|----------|
|
|----------------|------|----------|----------|----------|
|
||||||
| `hacktivism.ch` / `koopa` | bank.hacktivism.ch | exchange.hacktivism.ch | taler.hacktivism.ch | GOA |
|
| `hacktivism.ch` / `koopa` | bank.hacktivism.ch | exchange.hacktivism.ch | taler.hacktivism.ch | GOA |
|
||||||
| `taler.net` / `demo.taler.net` | bank.demo.taler.net | exchange.demo.taler.net | backend.demo.taler.net | KUDOS · **no custom /intro** |
|
| `taler.net` / `demo.taler.net` | bank.demo.taler.net | exchange.demo.taler.net | backend.demo.taler.net | KUDOS |
|
||||||
| `test.taler.net` | bank.test… | exchange.test… | backend.test… | **TESTKUDOS** · no landings |
|
| `test.taler.net` | bank.test… | exchange.test… | backend.test… | **TESTKUDOS** · no landings |
|
||||||
| `lefrancpaysan.ch` | bank.… | exchange.… | **monnaie.…** (merchant) | report-only · no landings |
|
| `lefrancpaysan.ch` | bank.… | exchange.… | **monnaie.…** (merchant) | report-only · no landings |
|
||||||
| **`stage.lefrancpaysan.ch`** | stage.bank.… | stage.exchange.… | **stage.monnaie.…** | **TESTPAYSAN** · landings · **inside via stagepaysan** |
|
| **`stage.lefrancpaysan.ch`** | stage.bank.… | stage.exchange.… | **stage.monnaie.…** | **TESTPAYSAN** · landings · **inside via stagepaysan** |
|
||||||
| `taler-ops.ch` / `my.taler-ops.ch` | *(EBICS — no public bank.\*)* | exchange.taler-ops.ch | **my.taler-ops.ch** | CHF · **no landings** · auto `CHECK_BANK=0` |
|
| `taler-ops.ch` / `my.taler-ops.ch` | bank.taler-ops.ch | exchange.taler-ops.ch | **my.taler-ops.ch** | CHF · **no landings** |
|
||||||
| `stage.taler-ops.ch` | *(EBICS)* | exchange.stage… | **my.stage…** | CHF · **no landings** · auto `CHECK_BANK=0` |
|
| `stage.taler-ops.ch` | bank.stage… | exchange.stage… | **my.stage…** | CHF · **no landings** |
|
||||||
| unknown | bank.DOMAIN | exchange.DOMAIN | my.DOMAIN then probe | any · no landings |
|
| unknown | bank.DOMAIN | exchange.DOMAIN | my.DOMAIN then probe | any · no landings |
|
||||||
|
|
||||||
**Custom /intro landings:** only **GOA** (hacktivism) and **FrancPaysan / TESTPAYSAN stage**. Demo, test, regio, TOPS: no custom intro (`CHECK_LANDING=0`).
|
**Landings:** GOA, stage LFP (`TESTPAYSAN`), and optionally demo use public `/intro` pages. **taler-ops.ch does not** — `CHECK_LANDING=0` skips intro crawl.
|
||||||
|
|
||||||
**Bank backend:** after domain apply, the suite reads exchange `/keys` `payto://` methods.
|
|
||||||
`x-taler-bank` → public bank checks on. **IBAN-only (EBICS / TOPS)** → `TALER_BANK_BACKEND=ebics`, `CHECK_BANK=0` (urls/sanity skip bank; e2e/ladder exit early). Override: `CHECK_BANK_SET=1` / `TALER_BANK_BACKEND_SET=1`.
|
|
||||||
|
|
||||||
**SSH:**
|
**SSH:**
|
||||||
| Stack | Host alias | User | Scope |
|
| Stack | Host alias | User | Scope |
|
||||||
|
|
|
||||||
2
TESTS.md
2
TESTS.md
|
|
@ -56,7 +56,7 @@ Numbering follows **executed** checks (early skip may shift later NN inside the
|
||||||
| **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three |
|
| **www.stats-** | `/intro/stats.json` **reachable**; **timestamps** (`generated_at_unix` + human/ISO); **freshness** vs wall clock (`STATS_STALE_SECS` WARN, `STATS_FAIL_SECS` ERROR); **display fields** (bank-shape withdraws/accounts, or exchange-db / merchant-db keys); optional performance block; shared-feed equal `gen_unix` when bank collector is published to all three |
|
||||||
| **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` |
|
| **www.bank-** | `/config`, currency, alt_unit_names; integration/webui/intro; **auto-account.json** (required GOA/local; skip/soft off-GOA e.g. TESTPAYSAN); `/terms`, `/privacy` |
|
||||||
| **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** |
|
| **www.merchant-** | `/config` currency + currencies alt_unit_names; listed exchanges alt; webui/intro; **`/terms`**, **`/privacy`** |
|
||||||
| **www.webui-** | SPA fingerprints: **`/webui/version.txt`**, **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` |
|
| **www.webui-** | SPA fingerprints: **`/webui/version.txt`** (INFO if missing — earlier merchant SPA had it; bank still has; current merchant often omits), **`/webui/version-overlay.txt`**, `index.html` / `index.js`; optional pin via `EXPECT_WEBUI_VERSION` / `EXPECT_WEBUI_OVERLAY` / `WEBUI_OVERLAY_DENY` |
|
||||||
| **www.paivana-** | local GOA paywall front (redirect to template) |
|
| **www.paivana-** | local GOA paywall front (redirect to template) |
|
||||||
| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local |
|
| **www.landing-** | own-stack intro links; static assets (`qrcode.min.js` hard; `og-goa-shop.png` hard only GOA/local); **demo-withdraw.json** GOA-only; shop-pay soft; stage merchant shop: `shop-ui.js` + `shops.css` + `qrcode.min.js`; cross-links local |
|
||||||
| **www.taler-uri-ports-** | **no default `:443`/`:80`** in public `taler://` URIs — bank/exchange/merchant `/intro/` HTML, `demo-withdraw.json`, `auto-account.json`, bank `stats.json` (ERROR on GOA/local). Unit: `tests/test_normalize_taler_uri.sh` |
|
| **www.taler-uri-ports-** | **no default `:443`/`:80`** in public `taler://` URIs — bank/exchange/merchant `/intro/` HTML, `demo-withdraw.json`, `auto-account.json`, bank `stats.json` (ERROR on GOA/local). Unit: `tests/test_normalize_taler_uri.sh` |
|
||||||
|
|
|
||||||
18
TODO-OPEN-ngi0-landing.md
Normal file
18
TODO-OPEN-ngi0-landing.md
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
# OPEN — ngi0-landing bereinigen (**für mich**, nicht Agent-Queue)
|
||||||
|
|
||||||
|
Stand: 2026-09-05 · **persönliche** Merkhilfe Hernâni · **unvollständiger Stand**
|
||||||
|
|
||||||
|
## Ist (Teil)
|
||||||
|
Domains für Dossiers-Testseiten (ngi0-landing) angelegt:
|
||||||
|
|
||||||
|
- `dossiers.ngi.hacktivism.ch`
|
||||||
|
- `dossiers.2.ngi.hacktivism.ch`
|
||||||
|
- `dossiers.3.ngi.hacktivism.ch`
|
||||||
|
|
||||||
|
## Offen / unvollständig
|
||||||
|
- Caddy static vhosts + Docroots auf koopa (sudo) — Snippet besprochen, **nicht** eingespielt
|
||||||
|
- `ngi0-landing` Repo **bereinigen**
|
||||||
|
- Inhalt der Testseiten / Deploy-Pfad klären
|
||||||
|
|
||||||
|
## Kanonisch
|
||||||
|
`PATH[llm-configs]/TODO-AGENT.md` → **HOLD-ngi0-landing** (außerhalb 1–11; Agents nur auf expliziten Auftrag).
|
||||||
26
TODO-OPEN-stack-status.md
Normal file
26
TODO-OPEN-stack-status.md
Normal file
|
|
@ -0,0 +1,26 @@
|
||||||
|
# OPEN — LLM Lightweight Monitoring / stack status
|
||||||
|
|
||||||
|
Stand: 2026-09-05 · tip `e293ff4` · prefer `/compact`
|
||||||
|
|
||||||
|
## Product
|
||||||
|
**LLM** = **Lightweight Monitoring**. Suite tree still `taler-monitoring`.
|
||||||
|
|
||||||
|
## Live
|
||||||
|
- https://hacktivism.ch/stacks/
|
||||||
|
- Wiki: `~/git/www-hacktivism-wiki` (4h)
|
||||||
|
|
||||||
|
## IDs
|
||||||
|
**TASK106–114 stay fixed.** New work → next free id.
|
||||||
|
|
||||||
|
## Done
|
||||||
|
- TASK107–111+113 · TASK114 catalog ok
|
||||||
|
- TASK112: trixie×4 + **ubuntu noble** apt-src on koopa · `ANSIBLE_SMOKE_REQUIRE_UBUNTU=1` → **OVERALL=PASS**
|
||||||
|
- Branch `llm-lightweight-monitoring` **pushed** · tag **v1.27.0** @ `e8f2820` · tip `e293ff4`
|
||||||
|
- Admin-log `2026-09-05--taler-apt-src-ubuntu.md` → Forgejo `a2300b6`
|
||||||
|
|
||||||
|
## Next
|
||||||
|
**Keine zweite Liste.** Kanonisch: `PATH[llm-configs]/TODO-AGENT.md` § **Arbeitsliste 1–11**.
|
||||||
|
|
||||||
|
**#8 TASK116 (2026-09-05):** Domain **monitoring.hacktivism.ch** (CNAME→taler→WAN). koopa `taler-monitord.service` **active**; local HTML at `/var/www/monitoring-sites/monitoring.hacktivism.ch/index.html`. **Blocker:** Caddy vhost+cert (sudo) — snippet `host-agent/caddy-monitoring.hacktivism.ch.snippet`. Clementine #5 still blocked.
|
||||||
|
|
||||||
|
**HOLD-ngi0-landing (für mich, nicht Agent):** **unvollständiger Stand** — Domains da, Caddy/Repo-Bereinigung offen — `TODO-OPEN-ngi0-landing.md` / `TODO-AGENT.md`.
|
||||||
14
TODO-TASK114-catalog-pass.md
Normal file
14
TODO-TASK114-catalog-pass.md
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
# TASK114 catalog pass
|
||||||
|
|
||||||
|
Stand: 2026-09-05 · Grok + Osaurus `foundation` (hallucinated; discarded) · Jan GP hung 0 tools (killed)
|
||||||
|
|
||||||
|
STATUS: ok (partial notes only)
|
||||||
|
|
||||||
|
FINDING:
|
||||||
|
- services-catalog ↔ ports-whitelist ↔ mail-catalog: coherent for pixel/firefly/vecigate (git/mm/buildbot + WAN 212.51.151.254 present).
|
||||||
|
- domains.conf holds Taler stacks; services-catalog correctly stays non-Taler/company — no duplicate stacks.
|
||||||
|
- mail-catalog planes (firefly / mail.anastasis.lu / pixel) match ports whitelist mail hosts.
|
||||||
|
- Soft gap: no dedicated `tortuna` services row (only WAN DNAT ports on vecigate) — intentional per comments.
|
||||||
|
- Soft gap: apex `taler-ops.ch` is mail+domains; services has `www.taler-ops.ch` company web only.
|
||||||
|
|
||||||
|
NEXT: no catalog edit required; optional later: document tortuna-as-DNAT-only in services header comment.
|
||||||
2
VERSION
2
VERSION
|
|
@ -1 +1 @@
|
||||||
1.27.0
|
1.26.2
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,7 @@ Git tags: `vMAJOR.FEATURE.FIX` (e.g. `v1.8.0`). File `VERSION` omits the `v` pre
|
||||||
|
|
||||||
| Tag | Date (UTC) | Notes |
|
| Tag | Date (UTC) | Notes |
|
||||||
|-----|------------|--------|
|
|-----|------------|--------|
|
||||||
| **v1.27.0** | 2026-09-05 | **Feature / LLM:** `taler-monitord` + stack-status HTML/JSON; services/ports catalogs; EBICS vs x-taler-bank detect + check skips; regio FLODOS ladder/e2e; ansible-smoke podman tracks; mail ipv6/rdns + taler-ops catalog. |
|
| **v1.26.2** | 2026-09-06 | **Bugfix / urls:** missing merchant `/webui/version.txt` is **INFO** (earlier merchant SPA had stamp; current packages often omit; bank `/webui/version.txt` still present). |
|
||||||
| **v1.26.1** | 2026-08-31 | **Bugfix:** public order settle accepts `StatusPaidResponse` (`refunded`/`refund_taken`); `paid`/`order_status` not on public GET. |
|
| **v1.26.1** | 2026-08-31 | **Bugfix:** public order settle accepts `StatusPaidResponse` (`refunded`/`refund_taken`); `paid`/`order_status` not on public GET. |
|
||||||
| **v1.26.0** | 2026-08-30 | **Feature:** `--wallet-db PATH` / `--wallet-migrate` (taler-wallet-cli / taler-harness names) for e2e, ladder, zero-pay. TESTPAYSAN: one Quadlet `stage-testpaysan`; e2e token-pay via env `E2E_TOKEN_*` / `--choice-index`. Stack READMEs `README_testpaysan.md` / `README_mytops.md`. |
|
| **v1.26.0** | 2026-08-30 | **Feature:** `--wallet-db PATH` / `--wallet-migrate` (taler-wallet-cli / taler-harness names) for e2e, ladder, zero-pay. TESTPAYSAN: one Quadlet `stage-testpaysan`; e2e token-pay via env `E2E_TOKEN_*` / `--choice-index`. Stack READMEs `README_testpaysan.md` / `README_mytops.md`. |
|
||||||
| **v1.24.0** | 2026-08-30 | **Feature / zero-pay:** public-template pays as phase `zero-pay` (`check_zero_pay.sh`). Default merchant **stage.my.taler-ops.ch** (CHF:0 stickers). Wrapper `pay_talersticker_zero.sh` COUNT=100. TESTPAYSAN: empty POST `{}` + auto-confirm of bank wires (host `auto-confirm-withdrawals`). Not in default 4h PHASES (use COUNT=1). |
|
| **v1.24.0** | 2026-08-30 | **Feature / zero-pay:** public-template pays as phase `zero-pay` (`check_zero_pay.sh`). Default merchant **stage.my.taler-ops.ch** (CHF:0 stickers). Wrapper `pay_talersticker_zero.sh` COUNT=100. TESTPAYSAN: empty POST `{}` + auto-confirm of bank wires (host `auto-confirm-withdrawals`). Not in default 4h PHASES (use COUNT=1). |
|
||||||
|
|
|
||||||
|
|
@ -41,11 +41,6 @@ load_monitoring_secrets_env 2>/dev/null || true
|
||||||
# Area ladder.* — withdraw/pay amount ladder (any EXPECT_CURRENCY / stack)
|
# Area ladder.* — withdraw/pay amount ladder (any EXPECT_CURRENCY / stack)
|
||||||
# Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report
|
# Groups: ladder.plan / ladder.load / ladder.withdraw / ladder.pay / ladder.report
|
||||||
set_area ladder
|
set_area ladder
|
||||||
if [ "${CHECK_BANK:-1}" != "1" ] || [ "${TALER_BANK_BACKEND:-}" = "ebics" ]; then
|
|
||||||
set_group plan
|
|
||||||
info "ladder" "skipped — backend=${TALER_BANK_BACKEND:-ebics} (needs public x-taler-bank)"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
set_group plan
|
set_group plan
|
||||||
SECTION_T0=$(date +%s)
|
SECTION_T0=$(date +%s)
|
||||||
now_ms() { python3 -c 'import time; print(int(time.time()*1000))'; }
|
now_ms() { python3 -c 'import time; print(int(time.time()*1000))'; }
|
||||||
|
|
@ -133,16 +128,6 @@ INST="${MERCHANT_INSTANCE}"
|
||||||
# Uses bank max_wire_transfer_amount and exchange min denom when still on GOA defaults.
|
# Uses bank max_wire_transfer_amount and exchange min denom when still on GOA defaults.
|
||||||
apply_ladder_stack_defaults() {
|
apply_ladder_stack_defaults() {
|
||||||
[ "${LADDER_STACK_AUTO}" = "1" ] || return 0
|
[ "${LADDER_STACK_AUTO}" = "1" ] || return 0
|
||||||
if [ "${CUR}" = "FLODOS" ] || [[ "${TALER_DOMAIN:-}" == *regio.taler.net* ]] || [ "${TALER_DOMAIN:-}" = "regio" ]; then
|
|
||||||
CUR="FLODOS"
|
|
||||||
# demo bank: keep ladder inside a small funded pool (override with LADDER_MAX_AMOUNT=)
|
|
||||||
if [ "${LADDER_MAX_AMOUNT}" = "4503599627370496" ] || [ "${LADDER_MAX_AMOUNT}" = "${GOA_LADDER_CEILING:-}" ]; then
|
|
||||||
LADDER_MAX_AMOUNT="${REGIO_LADDER_MAX:-20}"
|
|
||||||
fi
|
|
||||||
LADDER_MIN_AMOUNT="${LADDER_MIN_AMOUNT:-0.02}"
|
|
||||||
info "ladder stack" "FLODOS/regio · max=${CUR}:${LADDER_MAX_AMOUNT} min=${CUR}:${LADDER_MIN_AMOUNT}"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
if [ "${CUR}" != "TESTPAYSAN" ]; then
|
if [ "${CUR}" != "TESTPAYSAN" ]; then
|
||||||
case "${TALER_DOMAIN:-}" in
|
case "${TALER_DOMAIN:-}" in
|
||||||
stage.lefrancpaysan.ch|stage.bank.lefrancpaysan.ch|stage.exchange.lefrancpaysan.ch|stage.monnaie.lefrancpaysan.ch)
|
stage.lefrancpaysan.ch|stage.bank.lefrancpaysan.ch|stage.exchange.lefrancpaysan.ch|stage.monnaie.lefrancpaysan.ch)
|
||||||
|
|
@ -274,28 +259,6 @@ resolve_explorer_pw() {
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# regio.taler.net (FLODOS) — dedicated pool user, not admin (override default EXP_USER=explorer)
|
|
||||||
if [ "${CUR:-${EXPECT_CURRENCY:-}}" = "FLODOS" ] \
|
|
||||||
|| [[ "${TALER_DOMAIN:-}" == *regio.taler.net* ]] || [ "${TALER_DOMAIN:-}" = "regio" ]; then
|
|
||||||
if [ -n "${REGIO_BANK_EXPLORER_USER_FILE:-}" ] && [ -f "${REGIO_BANK_EXPLORER_USER_FILE}" ]; then
|
|
||||||
EXP_USER=$(tr -d '\n\r' <"${REGIO_BANK_EXPLORER_USER_FILE}")
|
|
||||||
elif [ -n "${TALER_DEPLOYMENT_SECRETS:-}" ] && [ -f "${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-user.txt" ]; then
|
|
||||||
EXP_USER=$(tr -d '\n\r' <"${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-user.txt")
|
|
||||||
else
|
|
||||||
EXP_USER="${EXP_USER:-lena.v92}"
|
|
||||||
[ "$EXP_USER" = "explorer" ] && EXP_USER=lena.v92
|
|
||||||
fi
|
|
||||||
export EXP_USER
|
|
||||||
if [ -n "${REGIO_BANK_EXPLORER_SECRET:-}" ] && [ -f "${REGIO_BANK_EXPLORER_SECRET}" ]; then
|
|
||||||
tr -d '\n\r' <"${REGIO_BANK_EXPLORER_SECRET}"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
if [ -n "${TALER_DEPLOYMENT_SECRETS:-}" ] && [ -f "${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-password.txt" ]; then
|
|
||||||
tr -d '\n\r' <"${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-password.txt"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Stage TESTPAYSAN — never use GOA koopa-admin-secrets explorer pw first
|
# Stage TESTPAYSAN — never use GOA koopa-admin-secrets explorer pw first
|
||||||
if [ "${CUR}" = "TESTPAYSAN" ] \
|
if [ "${CUR}" = "TESTPAYSAN" ] \
|
||||||
|| [[ "${TALER_DOMAIN:-}" == stage.*lefrancpaysan* ]] \
|
|| [[ "${TALER_DOMAIN:-}" == stage.*lefrancpaysan* ]] \
|
||||||
|
|
@ -569,18 +532,6 @@ metrics_report_load "$LOAD_BEFORE" "ladder-start" || true
|
||||||
# Fresh wallet per rung — baseline empty (or last-rung DB if re-used later)
|
# Fresh wallet per rung — baseline empty (or last-rung DB if re-used later)
|
||||||
metrics_report_coins "ladder-start" || true
|
metrics_report_coins "ladder-start" || true
|
||||||
|
|
||||||
# EXP_USER must be set in this shell (resolve_explorer_pw runs in $(…) subshell)
|
|
||||||
if [ "${CUR:-${EXPECT_CURRENCY:-}}" = "FLODOS" ] \
|
|
||||||
|| [[ "${TALER_DOMAIN:-}" == *regio.taler.net* ]] || [ "${TALER_DOMAIN:-}" = "regio" ]; then
|
|
||||||
if [ -n "${REGIO_BANK_EXPLORER_USER_FILE:-}" ] && [ -f "${REGIO_BANK_EXPLORER_USER_FILE}" ]; then
|
|
||||||
EXP_USER=$(tr -d '\n\r' <"${REGIO_BANK_EXPLORER_USER_FILE}")
|
|
||||||
elif [ -n "${TALER_DEPLOYMENT_SECRETS:-}" ] && [ -f "${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-user.txt" ]; then
|
|
||||||
EXP_USER=$(tr -d '\n\r' <"${TALER_DEPLOYMENT_SECRETS}/regio/bank-explorer-user.txt")
|
|
||||||
elif [ -z "${EXP_USER:-}" ] || [ "${EXP_USER}" = "explorer" ]; then
|
|
||||||
EXP_USER=lena.v92
|
|
||||||
fi
|
|
||||||
export EXP_USER
|
|
||||||
fi
|
|
||||||
if ! EXP_PW=$(resolve_explorer_pw); then
|
if ! EXP_PW=$(resolve_explorer_pw); then
|
||||||
err bank "explorer password missing" \
|
err bank "explorer password missing" \
|
||||||
"set EXP_PW / EXP_PW_FILE or SECRETS_ROOT=…/koopa/host-root (taler-bank/bank-explorer-password.txt)${SECRETS_ROOT:+ · SECRETS_ROOT=${SECRETS_ROOT}}"
|
"set EXP_PW / EXP_PW_FILE or SECRETS_ROOT=…/koopa/host-root (taler-bank/bank-explorer-password.txt)${SECRETS_ROOT:+ · SECRETS_ROOT=${SECRETS_ROOT}}"
|
||||||
|
|
@ -588,9 +539,9 @@ if ! EXP_PW=$(resolve_explorer_pw); then
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
if [ -n "${SECRETS_ROOT:-}" ]; then
|
if [ -n "${SECRETS_ROOT:-}" ]; then
|
||||||
info "explorer secret" "resolved user=${EXP_USER} (SECRETS_ROOT=${SECRETS_ROOT})"
|
info "explorer secret" "resolved (SECRETS_ROOT=${SECRETS_ROOT} · stage uses stagepaysan secret first when CUR=TESTPAYSAN)"
|
||||||
else
|
else
|
||||||
info "explorer secret" "resolved user=${EXP_USER} via EXP_PW / file / stage SSH / koopa"
|
info "explorer secret" "resolved via EXP_PW / EXP_PW_FILE / stage SSH / koopa"
|
||||||
fi
|
fi
|
||||||
if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then
|
if [ -n "${CLI_JS:-}" ] && [ -f "${CLI_JS}" ]; then
|
||||||
ok "wallet-cli" "$CLI_JS"
|
ok "wallet-cli" "$CLI_JS"
|
||||||
|
|
@ -605,66 +556,20 @@ if ! wcli balance >"$SCRATCH/wcli-smoke.out" 2>&1; then
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# --- personal account (intro auto-account, else open registration) ---
|
# --- auto-account ---
|
||||||
t0=$(now_ms)
|
t0=$(now_ms)
|
||||||
_acct_ok=0
|
if ! curl -sS -m 30 -o "$SCRATCH/auto-account.json" "${BANK}/intro/auto-account.json"; then
|
||||||
if curl -sS -m 30 -o "$SCRATCH/auto-account.json" "${BANK}/intro/auto-account.json" \
|
err bank "auto-account.json unreachable"
|
||||||
&& python3 -c 'import json; d=json.load(open("'"$SCRATCH"'/auto-account.json")); assert d.get("ok") or d.get("username")' 2>/dev/null; then
|
|
||||||
_acct_ok=1
|
|
||||||
elif [ "${CUR:-}" = "FLODOS" ] || [[ "${TALER_DOMAIN:-}" == *regio.taler.net* ]]; then
|
|
||||||
# regio has no /intro landings — register a discreet one-shot user (disposable mail list)
|
|
||||||
_mail_file="${REGIO_DISPOSABLE_EMAILS:-${TALER_DEPLOYMENT_SECRETS:+$TALER_DEPLOYMENT_SECRETS/regio/disposable-emails.txt}}"
|
|
||||||
_email=""
|
|
||||||
if [ -n "${_mail_file:-}" ] && [ -f "$_mail_file" ]; then
|
|
||||||
_email=$(awk -F'\t' '$2=="UNUSED" && $1 !~ /^#/ {print $1; exit}' "$_mail_file")
|
|
||||||
fi
|
|
||||||
: "${_email:=sofia.keller4@grr.la}"
|
|
||||||
_user=$(printf '%s' "$_email" | python3 -c 'import sys; e=sys.stdin.read().strip().split("@")[0]; print((e.replace(".","")[:10])+str(abs(hash(e))%100))')
|
|
||||||
_pw=$(openssl rand -base64 18 | tr -d '/+=' | head -c 20)
|
|
||||||
_name=$(printf '%s' "$_email" | python3 -c 'import sys; e=sys.stdin.read().strip().split("@")[0].replace("."," ").title(); print(e)')
|
|
||||||
_code=$(curl -sS -m 20 -o "$SCRATCH/auto-account.raw" -w '%{http_code}' -X POST \
|
|
||||||
-H 'Content-Type: application/json' \
|
|
||||||
-d "{\"username\":\"${_user}\",\"password\":\"${_pw}\",\"name\":\"${_name}\",\"email\":\"${_email}\"}" \
|
|
||||||
"${BANK}/accounts")
|
|
||||||
if [ "$_code" = "200" ] || [ "$_code" = "201" ]; then
|
|
||||||
python3 - "$_user" "$_pw" "$_name" "$_email" "$SCRATCH/auto-account.json" <<'PY'
|
|
||||||
import json,sys
|
|
||||||
u,p,n,e,out=sys.argv[1:6]
|
|
||||||
json.dump({"ok":True,"created_for_you":True,"username":u,"password":p,"name":n,"email":e,"balance":"FLODOS:0"}, open(out,"w"))
|
|
||||||
PY
|
|
||||||
if [ -n "${_mail_file:-}" ] && [ -f "$_mail_file" ]; then
|
|
||||||
python3 - "$_mail_file" "$_email" "$_user" <<'PY'
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
p,email,user=Path(sys.argv[1]),sys.argv[2],sys.argv[3]
|
|
||||||
lines=[]
|
|
||||||
for line in p.read_text().splitlines():
|
|
||||||
if line.startswith(email+"\t") or line.startswith(email+" "):
|
|
||||||
parts=line.split("\t")
|
|
||||||
while len(parts)<3: parts.append("")
|
|
||||||
parts[1]="USED"
|
|
||||||
parts[2]=f"personal {user}"
|
|
||||||
line="\t".join(parts)
|
|
||||||
lines.append(line)
|
|
||||||
p.write_text("\n".join(lines)+"\n")
|
|
||||||
PY
|
|
||||||
fi
|
|
||||||
_acct_ok=1
|
|
||||||
info "personal account" "open-register ${_user} (no /intro; disposable mail)"
|
|
||||||
else
|
|
||||||
err bank "open register failed HTTP ${_code}" "$(head -c 120 "$SCRATCH/auto-account.raw" | tr '\n' ' ')"
|
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
fi
|
|
||||||
ms_auto=$(elapsed_ms "$t0")
|
ms_auto=$(elapsed_ms "$t0")
|
||||||
if [ "$_acct_ok" != "1" ]; then
|
if ! python3 -c 'import json; d=json.load(open("'"$SCRATCH"'/auto-account.json")); assert d.get("ok") or d.get("username")' 2>/dev/null; then
|
||||||
err bank "auto-account create failed" "$(head -c 120 "$SCRATCH/auto-account.json" 2>/dev/null | tr '\n' ' ')"
|
err bank "auto-account create failed" "$(head -c 120 "$SCRATCH/auto-account.json" | tr '\n' ' ')"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
ACCT_USER=$(python3 -c 'import json; print(json.load(open("'"$SCRATCH"'/auto-account.json"))["username"])')
|
ACCT_USER=$(python3 -c 'import json; print(json.load(open("'"$SCRATCH"'/auto-account.json"))["username"])')
|
||||||
ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal ${CUR}:0; pool=${EXP_USER:-explorer}"
|
ok "auto-account ${ACCT_USER} (${ms_auto}ms) — personal ${CUR}:0; pool=explorer"
|
||||||
info "auto-account password" "(see $SCRATCH/auto-account.json — not logged)"
|
info "auto-account password" "(see $SCRATCH/auto-account.json — not logged)"
|
||||||
unset _acct_ok _mail_file _email _user _pw _name _code
|
|
||||||
|
|
||||||
# --- explorer token ---
|
# --- explorer token ---
|
||||||
t0=$(now_ms)
|
t0=$(now_ms)
|
||||||
|
|
|
||||||
|
|
@ -1,182 +0,0 @@
|
||||||
#!/usr/bin/env bash
|
|
||||||
# check_ansible_smoke.sh — taler-deployment ansible / apt full-stack smoke (TASK112)
|
|
||||||
#
|
|
||||||
# Goals (from REF106/TASK112):
|
|
||||||
# Container try-install against Taler apt for: trixie · trixie-testing · Ubuntu
|
|
||||||
# PASS/FAIL artifact. Source-build path is out of scope (open-work later).
|
|
||||||
#
|
|
||||||
# This phase is opt-in (not in default PHASES). Prefer running on a host with
|
|
||||||
# podman (koopa). On laptops without containers it still syntax-checks ansible
|
|
||||||
# when TALER_DEPLOYMENT is set.
|
|
||||||
#
|
|
||||||
# Env:
|
|
||||||
# TALER_DEPLOYMENT path to taler-deployment.git checkout
|
|
||||||
# ANSIBLE_SMOKE_SKIP=1
|
|
||||||
# ANSIBLE_SMOKE_PODMAN=1 inventory podman apt-src tracks (default 0 on non-koopa)
|
|
||||||
# ANSIBLE_SMOKE_ARTIFACT output dir (default: $HTML_OUT or /tmp/taler-ansible-smoke)
|
|
||||||
# ANSIBLE_SMOKE_UBUNTU_CONTAINER name (default koopa-taler-deploy-test-apt-src-ubuntu)
|
|
||||||
# ANSIBLE_SMOKE_REQUIRE_UBUNTU=1 hard-fail when ubuntu container missing
|
|
||||||
#
|
|
||||||
set -euo pipefail
|
|
||||||
ROOT=$(cd "$(dirname "$0")" && pwd)
|
|
||||||
# shellcheck source=lib.sh
|
|
||||||
source "$ROOT/lib.sh"
|
|
||||||
|
|
||||||
if [ "${ANSIBLE_SMOKE_SKIP:-0}" = "1" ]; then
|
|
||||||
echo "[INFO] skip phase ansible-smoke (ANSIBLE_SMOKE_SKIP=1)"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
set_area ansible-smoke
|
|
||||||
section "ansible-smoke · taler-deployment full-stack (trixie / trixie-testing / Ubuntu)"
|
|
||||||
|
|
||||||
DEPLOY="${TALER_DEPLOYMENT:-${TALER_DEPLOYMENT_ROOT:-}}"
|
|
||||||
if [ -z "$DEPLOY" ]; then
|
|
||||||
for c in \
|
|
||||||
"${HOME}/src/taler-deployment" \
|
|
||||||
"${HOME}/git/taler-deployment" \
|
|
||||||
"${SUITE_DIR:+$SUITE_DIR/../taler-deployment}"; do
|
|
||||||
[ -n "$c" ] && [ -d "$c/regional-currency-ng" ] && DEPLOY=$c && break
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
ART="${ANSIBLE_SMOKE_ARTIFACT:-${HTML_OUT:-/tmp}/taler-ansible-smoke}"
|
|
||||||
mkdir -p "$ART"
|
|
||||||
STAMP=$(date -u +%Y%m%dT%H%M%SZ)
|
|
||||||
REPORT="$ART/report-$STAMP.txt"
|
|
||||||
: >"$REPORT"
|
|
||||||
log() { printf '%s\n' "$*" | tee -a "$REPORT"; }
|
|
||||||
|
|
||||||
log "host=$(hostname 2>/dev/null || echo unknown) ts=$STAMP"
|
|
||||||
log "deploy=${DEPLOY:-MISSING}"
|
|
||||||
|
|
||||||
fail=0
|
|
||||||
# On koopa, taler-deployment.git is often absent; podman track inventory still useful.
|
|
||||||
if [ -z "$DEPLOY" ] || [ ! -d "$DEPLOY/regional-currency-ng" ]; then
|
|
||||||
if [ "${ANSIBLE_SMOKE_PODMAN:-0}" = "1" ]; then
|
|
||||||
info "deploy-tree" "taler-deployment missing — syntax-check skipped; continuing podman track inventory"
|
|
||||||
log "SKIP deploy-tree (set TALER_DEPLOYMENT for syntax-check)"
|
|
||||||
else
|
|
||||||
err "deploy-tree" "taler-deployment regional-currency-ng not found (set TALER_DEPLOYMENT)"
|
|
||||||
fail=1
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
ok "deploy-tree" "$DEPLOY/regional-currency-ng"
|
|
||||||
RC_NG="$DEPLOY/regional-currency-ng"
|
|
||||||
if [ -f "$RC_NG/site.yml" ]; then
|
|
||||||
ok "site.yml" "present"
|
|
||||||
else
|
|
||||||
err "site.yml" "missing"
|
|
||||||
fail=1
|
|
||||||
fi
|
|
||||||
if command -v ansible-playbook >/dev/null 2>&1; then
|
|
||||||
ok "ansible-playbook" "$(ansible-playbook --version 2>/dev/null | head -1)"
|
|
||||||
if ansible-playbook --syntax-check -i "$RC_NG/inventory.example.yml" "$RC_NG/site.yml" \
|
|
||||||
>>"$REPORT" 2>&1; then
|
|
||||||
ok "syntax-check" "site.yml + inventory.example.yml"
|
|
||||||
log "PASS syntax-check"
|
|
||||||
else
|
|
||||||
err "syntax-check" "ansible-playbook --syntax-check failed (see $REPORT)"
|
|
||||||
fail=1
|
|
||||||
log "FAIL syntax-check"
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
info "ansible-playbook" "not installed — syntax-check skipped"
|
|
||||||
log "SKIP syntax-check (no ansible-playbook)"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Container apt tracks: reuse aptdeploy naming; Ubuntu track is additive.
|
|
||||||
# Heavy merchant install stays in check_apt_deploy.sh; here we inventory tracks.
|
|
||||||
TARGETS="trixie trixie-testing ubuntu"
|
|
||||||
log "targets=$TARGETS"
|
|
||||||
UBUNTU_NAME="${ANSIBLE_SMOKE_UBUNTU_CONTAINER:-koopa-taler-deploy-test-apt-src-ubuntu}"
|
|
||||||
REQUIRE_UBUNTU="${ANSIBLE_SMOKE_REQUIRE_UBUNTU:-0}"
|
|
||||||
|
|
||||||
_podman_track_state() {
|
|
||||||
# stdout: running|exited|missing
|
|
||||||
local bin=$1 name=$2
|
|
||||||
if "$bin" inspect -f '{{.State.Status}}' "$name" >/dev/null 2>&1; then
|
|
||||||
"$bin" inspect -f '{{.State.Status}}' "$name" 2>/dev/null | tr -d '\r'
|
|
||||||
else
|
|
||||||
printf '%s\n' missing
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ "${ANSIBLE_SMOKE_PODMAN:-0}" = "1" ]; then
|
|
||||||
PODMAN_BIN="${APT_DEPLOY_PODMAN:-podman}"
|
|
||||||
if ! command -v "$PODMAN_BIN" >/dev/null 2>&1; then
|
|
||||||
err "podman" "ANSIBLE_SMOKE_PODMAN=1 but podman missing"
|
|
||||||
fail=1
|
|
||||||
else
|
|
||||||
ok "podman" "$("$PODMAN_BIN" --version 2>/dev/null | head -1)"
|
|
||||||
log "NOTE: full container install remains check_apt_deploy.sh (trixie tracks)."
|
|
||||||
|
|
||||||
for suite in trixie trixie-testing; do
|
|
||||||
for mode in "" "-upgrade"; do
|
|
||||||
cname="koopa-taler-deploy-test-apt-src-${suite}${mode}"
|
|
||||||
st=$(_podman_track_state "$PODMAN_BIN" "$cname")
|
|
||||||
case "$st" in
|
|
||||||
running|created)
|
|
||||||
ok "track-$suite${mode:-}" "$cname state=$st"
|
|
||||||
log "PASS track $cname state=$st"
|
|
||||||
;;
|
|
||||||
exited|stopped)
|
|
||||||
info "track-$suite${mode:-}" "$cname state=$st (present, not running)"
|
|
||||||
log "WARN track $cname state=$st"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
info "track-$suite${mode:-}" "$cname missing"
|
|
||||||
log "MISS track $cname"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
||||||
# Ubuntu track (additive; create image/container only with Freigabe on koopa)
|
|
||||||
ust=$(_podman_track_state "$PODMAN_BIN" "$UBUNTU_NAME")
|
|
||||||
case "$ust" in
|
|
||||||
running|created)
|
|
||||||
ok "track-ubuntu" "$UBUNTU_NAME state=$ust"
|
|
||||||
log "PASS track $UBUNTU_NAME state=$ust"
|
|
||||||
;;
|
|
||||||
exited|stopped)
|
|
||||||
info "track-ubuntu" "$UBUNTU_NAME state=$ust (present, not running)"
|
|
||||||
log "WARN track $UBUNTU_NAME state=$ust"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
if [ "$REQUIRE_UBUNTU" = "1" ]; then
|
|
||||||
err "track-ubuntu" "$UBUNTU_NAME missing (ANSIBLE_SMOKE_REQUIRE_UBUNTU=1)"
|
|
||||||
fail=1
|
|
||||||
log "FAIL track $UBUNTU_NAME missing"
|
|
||||||
else
|
|
||||||
info "track-ubuntu" "$UBUNTU_NAME missing — Freigabe needed to create ubuntu apt-src image/container"
|
|
||||||
log "TODO ubuntu: create $UBUNTU_NAME (Freigabe); soft-miss until then"
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
if "$PODMAN_BIN" image exists "docker.io/library/ubuntu:24.04" 2>/dev/null \
|
|
||||||
|| "$PODMAN_BIN" image exists "ubuntu:24.04" 2>/dev/null; then
|
|
||||||
ok "ubuntu-base-image" "ubuntu:24.04 present"
|
|
||||||
log "PASS ubuntu-base-image"
|
|
||||||
else
|
|
||||||
info "ubuntu-base-image" "ubuntu:24.04 not pulled yet (optional)"
|
|
||||||
log "MISS ubuntu-base-image"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
info "podman-smoke" "skipped (set ANSIBLE_SMOKE_PODMAN=1 on koopa)"
|
|
||||||
log "SKIP podman-smoke"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$fail" -eq 0 ]; then
|
|
||||||
log "OVERALL=PASS"
|
|
||||||
ok "ansible-smoke" "PASS · artifact $REPORT"
|
|
||||||
else
|
|
||||||
log "OVERALL=FAIL"
|
|
||||||
err "ansible-smoke" "FAIL · artifact $REPORT"
|
|
||||||
fi
|
|
||||||
info "artifact" "$REPORT"
|
|
||||||
summary
|
|
||||||
exit "$fail"
|
|
||||||
|
|
@ -314,7 +314,7 @@ if [ "${AUTH401_FULL}" = "1" ]; then
|
||||||
if [ "$vcode" = "200" ] && [ -s "$tmp/spa-ver.txt" ]; then
|
if [ "$vcode" = "200" ] && [ -s "$tmp/spa-ver.txt" ]; then
|
||||||
ok "webui version.txt" "$(tr -d '\n\r' <"$tmp/spa-ver.txt")"
|
ok "webui version.txt" "$(tr -d '\n\r' <"$tmp/spa-ver.txt")"
|
||||||
else
|
else
|
||||||
warn "webui version.txt" "HTTP ${vcode:-000}"
|
info "webui version.txt" "HTTP ${vcode:-000} · earlier merchant SPA versions had this stamp; current merchant packages often omit it · bank /webui/version.txt still present on current bank builds"
|
||||||
fi
|
fi
|
||||||
# Download a slice of index.js — look for lowercasing mitigation (#11340)
|
# Download a slice of index.js — look for lowercasing mitigation (#11340)
|
||||||
jcode=$(curl -skS -m 60 -o "$tmp/spa-index.js" -w '%{http_code}' "$BASE/webui/index.js" 2>/dev/null || echo 000)
|
jcode=$(curl -skS -m 60 -o "$tmp/spa-index.js" -w '%{http_code}' "$BASE/webui/index.js" 2>/dev/null || echo 000)
|
||||||
|
|
|
||||||
17
check_e2e.sh
17
check_e2e.sh
|
|
@ -489,17 +489,6 @@ if [ "$E2E_REMOTE" = "1" ]; then
|
||||||
fi
|
fi
|
||||||
unset _fp_try _fp_host _remote_bank_sec
|
unset _fp_try _fp_host _remote_bank_sec
|
||||||
fi
|
fi
|
||||||
# regio.taler.net (FLODOS): path from taler-deployment-secrets via env
|
|
||||||
if [ -z "$ADMIN_PASS" ] && [ "${CUR:-${EXPECT_CURRENCY:-}}" = "FLODOS" ]; then
|
|
||||||
_regio_sec="${REGIO_BANK_ADMIN_SECRET:-}"
|
|
||||||
[ -z "$_regio_sec" ] && [ -n "${TALER_DEPLOYMENT_SECRETS:-}" ] && \
|
|
||||||
_regio_sec="${TALER_DEPLOYMENT_SECRETS}/regio/bank-admin-password.txt"
|
|
||||||
if [ -n "$_regio_sec" ] && [ -f "$_regio_sec" ]; then
|
|
||||||
ADMIN_PASS=$(tr -d '\n\r' <"$_regio_sec")
|
|
||||||
info "bank admin secret" "from $_regio_sec"
|
|
||||||
fi
|
|
||||||
unset _regio_sec
|
|
||||||
fi
|
|
||||||
# Stage merchant: public templates need no token; optional default-instance token
|
# Stage merchant: public templates need no token; optional default-instance token
|
||||||
if [ -z "$MPW" ] && [ "${CUR:-${EXPECT_CURRENCY:-}}" = "TESTPAYSAN" ]; then
|
if [ -z "$MPW" ] && [ "${CUR:-${EXPECT_CURRENCY:-}}" = "TESTPAYSAN" ]; then
|
||||||
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
|
if [ -n "${FRANCPAYSAN_SECRETS:-}" ] && [ -f "${FRANCPAYSAN_SECRETS}/stage/default-instance-token.txt" ]; then
|
||||||
|
|
@ -563,12 +552,6 @@ merchant_auth_header() {
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
# EBICS / no public bank.* — withdraw path needs x-taler-bank (libeufin)
|
|
||||||
if [ "${CHECK_BANK:-1}" != "1" ] || [ "${TALER_BANK_BACKEND:-}" = "ebics" ]; then
|
|
||||||
info "e2e" "skipped — backend=${TALER_BANK_BACKEND:-ebics} (no public bank.* / x-taler-bank)"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Public reachability gates (remote: soft-skip if bank/merchant absent)
|
# Public reachability gates (remote: soft-skip if bank/merchant absent)
|
||||||
for pair in \
|
for pair in \
|
||||||
"bank|$BANK_PUBLIC/config" \
|
"bank|$BANK_PUBLIC/config" \
|
||||||
|
|
|
||||||
|
|
@ -213,44 +213,12 @@ while IFS= read -r line || [ -n "$line" ]; do
|
||||||
h=$(echo "$h" | tr '[:upper:]' '[:lower:]' | sed 's/\.$//;s/^[[:space:]]*//;s/[[:space:]]*$//')
|
h=$(echo "$h" | tr '[:upper:]' '[:lower:]' | sed 's/\.$//;s/^[[:space:]]*//;s/[[:space:]]*$//')
|
||||||
[ -z "$h" ] && continue
|
[ -z "$h" ] && continue
|
||||||
|
|
||||||
mapfile -t _a4 < <(dns_a "$h" | sed '/^$/d')
|
addrs=$(dns_a "$h" | tr '\n' ' ')
|
||||||
addrs=$(printf '%s ' "${_a4[@]}")
|
|
||||||
if [ -z "${addrs// }" ]; then
|
if [ -z "${addrs// }" ]; then
|
||||||
fail "dns" "$h does not resolve (A/AAAA)"
|
fail "dns" "$h does not resolve (A/AAAA)"
|
||||||
continue
|
continue
|
||||||
fi
|
fi
|
||||||
ok "dns" "$h → $addrs"
|
ok "dns" "$h → $addrs"
|
||||||
# Explicit IPv6 signal
|
|
||||||
_has_v4=0
|
|
||||||
_has_v6=0
|
|
||||||
for _ip in "${_a4[@]}"; do
|
|
||||||
case "$_ip" in
|
|
||||||
*:*) _has_v6=1 ;;
|
|
||||||
*) _has_v4=1 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
if [ "$_has_v4" = "1" ] && [ "$_has_v6" = "0" ]; then
|
|
||||||
warn "ipv6" "$h has A but no AAAA"
|
|
||||||
elif [ "$_has_v6" = "1" ]; then
|
|
||||||
ok "ipv6" "$h has AAAA"
|
|
||||||
fi
|
|
||||||
# Reverse DNS (PTR) on first IPv4
|
|
||||||
for _ip in "${_a4[@]}"; do
|
|
||||||
case "$_ip" in
|
|
||||||
*:*) continue ;;
|
|
||||||
esac
|
|
||||||
if command -v dig >/dev/null 2>&1; then
|
|
||||||
ptr=$(dig +short +time=3 +tries=2 -x "$_ip" 2>/dev/null | sed 's/\.$//' | tr '\n' ' ')
|
|
||||||
else
|
|
||||||
ptr=$(host "$_ip" 2>/dev/null | awk '/domain name pointer/{print $NF}' | sed 's/\.$//' | tr '\n' ' ')
|
|
||||||
fi
|
|
||||||
if [ -n "${ptr// }" ]; then
|
|
||||||
ok "rdns" "$_ip PTR → $ptr"
|
|
||||||
else
|
|
||||||
warn "rdns" "$_ip has no PTR"
|
|
||||||
fi
|
|
||||||
break
|
|
||||||
done
|
|
||||||
|
|
||||||
# skip full protocol re-probe if already done
|
# skip full protocol re-probe if already done
|
||||||
if [ "${HOST_DONE[$h]:-}" = "1" ]; then
|
if [ "${HOST_DONE[$h]:-}" = "1" ]; then
|
||||||
|
|
|
||||||
332
check_mail_deliverability.sh
Executable file
332
check_mail_deliverability.sh
Executable file
|
|
@ -0,0 +1,332 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# check_mail_deliverability.sh — Taler mail-plane deliverability (ops-checklist)
|
||||||
|
#
|
||||||
|
# SUT domains (NOT cyon): taler.net, gnunet.org, firefly.gnunet.org,
|
||||||
|
# taler-systems.com, anastasis.lu, taler-ops.ch (from mail-catalog.conf)
|
||||||
|
#
|
||||||
|
# Based on admin-log/firefly/ops-checklist.md (2026-03 MT/ED scores):
|
||||||
|
# dig SPF + DMARC + DKIM selector(s); SMTP RCPT smoke via monitord helper;
|
||||||
|
# optional IMAP sink poll (mon001–003@hacktivism.ch on cyon = sink only).
|
||||||
|
#
|
||||||
|
# Explicit phase (not in all/full):
|
||||||
|
# ./taler-monitoring.sh mail-deliverability
|
||||||
|
# ./taler-monitoring.sh maildeliv
|
||||||
|
#
|
||||||
|
# Env:
|
||||||
|
# MAIL_CATALOG default mail-catalog.conf
|
||||||
|
# MAIL_DKIM_SELECTORS default "mail-2026-3 mail" (space-separated)
|
||||||
|
# MAIL_DELIV_IMAP=1 enable sink IMAP login+INBOX check (needs pass)
|
||||||
|
# MAIL_DELIV_IMAP_USERS default mon001@hacktivism.ch mon002@… mon003@…
|
||||||
|
# MAIL_DELIV_IMAP_HOST default mail.cyon.ch (sink host — not SUT)
|
||||||
|
# MAIL_DELIV_IMAP_PASS shared password for mon* (prefer secrets.env)
|
||||||
|
# MAIL_DELIV_SEND=0 1 = authenticated send (Freigabe; needs SMTP_*)
|
||||||
|
# MAIL_DELIV_ALERT=0 1 = on ERROR, send alert via sink SMTP (Freigabe)
|
||||||
|
# MAIL_DELIV_ALERT_FROM default mon001@hacktivism.ch
|
||||||
|
# MAIL_DELIV_ALERT_TO default hernani+mon@taler-systems.com
|
||||||
|
# hernani+mon@vecirex.net vecirex+mon@icloud.com
|
||||||
|
# fdold+mon@taler.net
|
||||||
|
# MAIL_DELIV_ALERT_SMTP default mail.cyon.ch:587 (sink submission — not SUT)
|
||||||
|
# MAIL_DELIV_ALERT_PASS default MAIL_DELIV_IMAP_PASS
|
||||||
|
# MAIL_DELIV_ALERT_COOLDOWN_S default 21600 (6h); stamp under monitord state
|
||||||
|
# TALER_MONITORD_DELIVER_* see site-gen smtp_deliverability_probe
|
||||||
|
#
|
||||||
|
set -euo pipefail
|
||||||
|
ROOT=$(cd "$(dirname "$0")" && pwd)
|
||||||
|
# shellcheck source=lib.sh
|
||||||
|
source "$ROOT/lib.sh"
|
||||||
|
|
||||||
|
set_area maildeliv
|
||||||
|
section "maildeliv · Taler mail domains (SPF/DMARC/DKIM + SMTP RCPT; cyon=sink only)"
|
||||||
|
|
||||||
|
CATALOG="${MAIL_CATALOG:-$ROOT/mail-catalog.conf}"
|
||||||
|
SELECTORS="${MAIL_DKIM_SELECTORS:-mail-2026-3 mail}"
|
||||||
|
: "${MAIL_DELIV_IMAP:=0}"
|
||||||
|
: "${MAIL_DELIV_SEND:=0}"
|
||||||
|
: "${MAIL_DELIV_ALERT:=0}"
|
||||||
|
: "${MAIL_DELIV_PROBE:=0}"
|
||||||
|
IMAP_HOST="${MAIL_DELIV_IMAP_HOST:-mail.cyon.ch}"
|
||||||
|
IMAP_USERS="${MAIL_DELIV_IMAP_USERS:-mon001@hacktivism.ch mon002@hacktivism.ch mon003@hacktivism.ch}"
|
||||||
|
ALERT_FROM="${MAIL_DELIV_ALERT_FROM:-mon001@hacktivism.ch}"
|
||||||
|
ALERT_TO="${MAIL_DELIV_ALERT_TO:-hernani+mon@taler-systems.com hernani+mon@vecirex.net vecirex+mon@icloud.com fdold+mon@taler.net}"
|
||||||
|
ALERT_SMTP="${MAIL_DELIV_ALERT_SMTP:-mail.cyon.ch:587}"
|
||||||
|
ALERT_PASS="${MAIL_DELIV_ALERT_PASS:-${MAIL_DELIV_IMAP_PASS:-}}"
|
||||||
|
ALERT_COOLDOWN_S="${MAIL_DELIV_ALERT_COOLDOWN_S:-21600}"
|
||||||
|
STATE_DIR="${TALER_MONITORD_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/taler-monitord}"
|
||||||
|
ALERT_STAMP="${STATE_DIR}/mail-deliv-alert.stamp"
|
||||||
|
|
||||||
|
if [ ! -f "$CATALOG" ]; then
|
||||||
|
fail "catalog" "missing $CATALOG"
|
||||||
|
summary
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "scope" "SUT = catalog Taler domains · sink IMAP host=$IMAP_HOST (not under test)"
|
||||||
|
|
||||||
|
dns_txt() {
|
||||||
|
local name="$1"
|
||||||
|
if command -v dig >/dev/null 2>&1; then
|
||||||
|
dig +short +time=3 +tries=2 TXT "$name" 2>/dev/null | tr -d '"'
|
||||||
|
else
|
||||||
|
host -t TXT "$name" 2>/dev/null | sed 's/.*"\(.*\)"/\1/'
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
dns_mx() {
|
||||||
|
local d="$1"
|
||||||
|
dig +short +time=3 +tries=2 MX "$d" 2>/dev/null | awk '{print tolower($2)}' | sed 's/\.$//'
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- per catalog domain ---
|
||||||
|
while IFS= read -r line || [ -n "$line" ]; do
|
||||||
|
line=${line%%#*}
|
||||||
|
line=$(echo "$line" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
|
||||||
|
[ -z "$line" ] && continue
|
||||||
|
domain=$(echo "$line" | awk '{print $1}')
|
||||||
|
expect_mx=$(echo "$line" | awk '{print $2}')
|
||||||
|
label=$(echo "$line" | awk '{print $6}')
|
||||||
|
[ -n "$domain" ] || continue
|
||||||
|
|
||||||
|
set_group "${label:-$domain}"
|
||||||
|
|
||||||
|
# SPF
|
||||||
|
spf=$(dns_txt "$domain" | tr '\n' ' ' | grep -oE 'v=spf1[^"]*' | head -1 || true)
|
||||||
|
if [ -n "$spf" ]; then
|
||||||
|
ok "spf" "$domain has SPF" "$spf"
|
||||||
|
else
|
||||||
|
fail "spf" "$domain missing SPF TXT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# DMARC
|
||||||
|
dmarc=$(dns_txt "_dmarc.$domain" | tr '\n' ' ' | grep -oE 'v=DMARC1.*' | head -1 || true)
|
||||||
|
if [ -n "$dmarc" ]; then
|
||||||
|
case "$dmarc" in
|
||||||
|
*p=none*)
|
||||||
|
if [ "$domain" = "taler-ops.ch" ] || [ "$domain" = "firefly.gnunet.org" ]; then
|
||||||
|
warn "dmarc" "$domain DMARC p=none (known weak · ops-checklist)" "$dmarc"
|
||||||
|
else
|
||||||
|
warn "dmarc" "$domain DMARC p=none" "$dmarc"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
ok "dmarc" "$domain DMARC present" "$dmarc"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
else
|
||||||
|
fail "dmarc" "$domain missing _dmarc TXT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# DKIM selectors (ops-checklist / 2026-03)
|
||||||
|
dkim_ok=0
|
||||||
|
dkim_hit=""
|
||||||
|
for sel in $SELECTORS; do
|
||||||
|
[ -n "$sel" ] || continue
|
||||||
|
txt=$(dns_txt "${sel}._domainkey.${domain}" | tr '\n' ' ')
|
||||||
|
if echo "$txt" | grep -qiE 'v=DKIM1|p='; then
|
||||||
|
dkim_ok=1
|
||||||
|
dkim_hit="$sel"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$dkim_ok" = "1" ]; then
|
||||||
|
ok "dkim" "$domain DKIM selector $dkim_hit"
|
||||||
|
else
|
||||||
|
warn "dkim" "$domain no DKIM TXT for selectors [$SELECTORS] (mail-tester often -1)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# MX matches catalog expectation (first token)
|
||||||
|
mx_got=$(dns_mx "$domain" | head -1)
|
||||||
|
exp_first=${expect_mx%%,*}
|
||||||
|
exp_first=${exp_first%% *}
|
||||||
|
if [ -n "$mx_got" ]; then
|
||||||
|
if [ -z "$exp_first" ] || [ "$exp_first" = "-" ] || [ "$mx_got" = "$exp_first" ]; then
|
||||||
|
ok "mx" "$domain MX=$mx_got"
|
||||||
|
else
|
||||||
|
# allow alternate listed in expect_mx comma list
|
||||||
|
case ",${expect_mx}," in
|
||||||
|
*",${mx_got},"*) ok "mx" "$domain MX=$mx_got (listed)" ;;
|
||||||
|
*) warn "mx" "$domain MX=$mx_got want~$exp_first" ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
fail "mx" "$domain no MX"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# SMTP RCPT smoke (no DATA) — reuse site-gen helper
|
||||||
|
if [ -f "$ROOT/site-gen/stack_status_lib.py" ]; then
|
||||||
|
out=$(
|
||||||
|
ROOT="$ROOT" DOMAIN="$domain" MX="$mx_got" python3 - <<'PY' 2>/dev/null || echo "err=import"
|
||||||
|
import os, sys
|
||||||
|
sys.path.insert(0, os.path.join(os.environ["ROOT"], "site-gen"))
|
||||||
|
from stack_status_lib import smtp_deliverability_probe
|
||||||
|
r = smtp_deliverability_probe(os.environ["MX"], domain=os.environ["DOMAIN"])
|
||||||
|
print(f"ok={r.get('ok')} light={r.get('light')} reason={r.get('reason')} rcpt={r.get('rcpt')}")
|
||||||
|
PY
|
||||||
|
)
|
||||||
|
case "$out" in
|
||||||
|
ok=True*)
|
||||||
|
ok "smtp-rcpt" "$domain $out"
|
||||||
|
;;
|
||||||
|
ok=False*light=yellow*|ok=False*rcpt_reject*|ok=False*reason=rcpt_reject*)
|
||||||
|
# Many MTAs reject postmaster@ from random clients; still useful signal (ops-checklist uses mail-tester for full score).
|
||||||
|
warn "smtp-rcpt" "$domain $out"
|
||||||
|
;;
|
||||||
|
ok=False*)
|
||||||
|
fail "smtp-rcpt" "$domain $out"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
warn "smtp-rcpt" "$domain probe unavailable · $out"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
else
|
||||||
|
info "smtp-rcpt" "skipped (no site-gen/stack_status_lib.py)"
|
||||||
|
fi
|
||||||
|
done <"$CATALOG"
|
||||||
|
|
||||||
|
# --- optional IMAP sinks (cyon hosts mon*; not SUT) ---
|
||||||
|
if [ "$MAIL_DELIV_IMAP" = "1" ]; then
|
||||||
|
set_group sink-imap
|
||||||
|
if [ -z "${MAIL_DELIV_IMAP_PASS:-}" ]; then
|
||||||
|
warn "sink-imap" "MAIL_DELIV_IMAP=1 but MAIL_DELIV_IMAP_PASS unset — skip"
|
||||||
|
else
|
||||||
|
info "sink-imap" "probing sinks on $IMAP_HOST (infrastructure only · not Taler MTA)"
|
||||||
|
for user in $IMAP_USERS; do
|
||||||
|
res=$(
|
||||||
|
MAIL_DELIV_IMAP_HOST="$IMAP_HOST" MAIL_DELIV_IMAP_USER="$user" \
|
||||||
|
MAIL_DELIV_IMAP_PASS="$MAIL_DELIV_IMAP_PASS" python3 - <<'PY' 2>/dev/null || echo FAIL
|
||||||
|
import imaplib, os, ssl
|
||||||
|
host=os.environ["MAIL_DELIV_IMAP_HOST"]
|
||||||
|
user=os.environ["MAIL_DELIV_IMAP_USER"]
|
||||||
|
pw=os.environ["MAIL_DELIV_IMAP_PASS"]
|
||||||
|
ctx=ssl.create_default_context()
|
||||||
|
M=imaplib.IMAP4_SSL(host, 993, ssl_context=ctx)
|
||||||
|
M.login(user, pw)
|
||||||
|
typ, data=M.select("INBOX")
|
||||||
|
n=data[0].decode() if data and data[0] else "?"
|
||||||
|
# also see junk/spam exists
|
||||||
|
boxes=[]
|
||||||
|
t, bl=M.list()
|
||||||
|
for b in bl or []:
|
||||||
|
s=b.decode(errors="replace")
|
||||||
|
if "Junk" in s or "spam" in s.lower():
|
||||||
|
boxes.append("junk/spam")
|
||||||
|
M.logout()
|
||||||
|
print(f"OK inbox={n} extra={','.join(boxes) or 'none'}")
|
||||||
|
PY
|
||||||
|
)
|
||||||
|
case "$res" in
|
||||||
|
OK*) ok "sink" "$user · $res" ;;
|
||||||
|
*) fail "sink" "$user · $res" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
info "sink-imap" "skipped (MAIL_DELIV_IMAP=0) · sinks mon001–003 for Freigabe send tests"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$MAIL_DELIV_SEND" = "1" ]; then
|
||||||
|
warn "send" "MAIL_DELIV_SEND=1 deprecated here — use MAIL_DELIV_PROBE=1 (inbox vs spam)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$MAIL_DELIV_PROBE" = "1" ]; then
|
||||||
|
set_group probe
|
||||||
|
if [ -z "${MAIL_DELIV_IMAP_PASS:-}${MAIL_DELIV_ALERT_PASS:-}" ]; then
|
||||||
|
warn "probe" "MAIL_DELIV_PROBE=1 but no IMAP pass"
|
||||||
|
else
|
||||||
|
info "probe" "mon* send + IMAP inbox-vs-spam (regular path · host-agent timer 6h)"
|
||||||
|
if out=$(python3 "$ROOT/mail_deliv_probe.py" 2>&1); then
|
||||||
|
ok "probe" "all sinks inbox"
|
||||||
|
info "probe-detail" "$(echo "$out" | tr '\n' ' ' | head -c 240)"
|
||||||
|
else
|
||||||
|
ec=$?
|
||||||
|
echo "$out" | while IFS= read -r line; do
|
||||||
|
case "$line" in
|
||||||
|
*result=spam*|*result=both*)
|
||||||
|
fail "probe-spam" "$line"
|
||||||
|
;;
|
||||||
|
*result=missing*|*IMAP_FAIL*|*SEND_FAIL*)
|
||||||
|
fail "probe-miss" "$line"
|
||||||
|
;;
|
||||||
|
SENT*|ALERT*)
|
||||||
|
info "probe" "$line"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
info "probe" "$line"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[ "$ec" -eq 0 ] || true
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
info "probe" "skipped (MAIL_DELIV_PROBE=0) · enable for inbox-vs-spam; timer: taler-monitoring-mail-deliv.timer"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- alert on ERROR via sink SMTP (cyon submission; recipients = hernani+mon / icloud) ---
|
||||||
|
maildeliv_alert_if_needed() {
|
||||||
|
set_group alert
|
||||||
|
if [ "$MAIL_DELIV_ALERT" != "1" ]; then
|
||||||
|
info "alert" "skipped (MAIL_DELIV_ALERT=0) · would To: $ALERT_TO"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if [ "${#ERRORS[@]}" -eq 0 ]; then
|
||||||
|
ok "alert" "no ERRORs — no alert mail"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if [ -z "$ALERT_PASS" ]; then
|
||||||
|
warn "alert" "MAIL_DELIV_ALERT=1 but no MAIL_DELIV_ALERT_PASS / MAIL_DELIV_IMAP_PASS"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if ! command -v swaks >/dev/null 2>&1; then
|
||||||
|
warn "alert" "swaks missing — apt/brew install swaks"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
mkdir -p "$STATE_DIR" 2>/dev/null || true
|
||||||
|
now=$(date +%s)
|
||||||
|
if [ -f "$ALERT_STAMP" ]; then
|
||||||
|
prev=$(cat "$ALERT_STAMP" 2>/dev/null || echo 0)
|
||||||
|
if [ "$prev" -gt 0 ] 2>/dev/null; then
|
||||||
|
age=$((now - prev))
|
||||||
|
if [ "$age" -lt "$ALERT_COOLDOWN_S" ]; then
|
||||||
|
info "alert" "cooldown ${age}s < ${ALERT_COOLDOWN_S}s — skip send"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
smtp_host=${ALERT_SMTP%%:*}
|
||||||
|
smtp_port=${ALERT_SMTP##*:}
|
||||||
|
[ "$smtp_port" = "$ALERT_SMTP" ] && smtp_port=587
|
||||||
|
body=$(mktemp)
|
||||||
|
{
|
||||||
|
echo "taler-monitoring mail-deliverability ALERT"
|
||||||
|
echo "host=$(hostname 2>/dev/null || echo ?)"
|
||||||
|
echo "when=$(date -Iseconds 2>/dev/null || date)"
|
||||||
|
echo "errors=${#ERRORS[@]}"
|
||||||
|
echo ""
|
||||||
|
for e in "${ERRORS[@]}"; do
|
||||||
|
echo "- $e"
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
echo "SUT = Taler mail domains (not cyon). From=$ALERT_FROM via $ALERT_SMTP"
|
||||||
|
} >"$body"
|
||||||
|
subj="[taler-mon maildeliv] ${#ERRORS[@]} ERROR(s)"
|
||||||
|
send_fail=0
|
||||||
|
for to in $ALERT_TO; do
|
||||||
|
if swaks --server "$smtp_host" --port "$smtp_port" \
|
||||||
|
--auth LOGIN --auth-user "$ALERT_FROM" --auth-password "$ALERT_PASS" \
|
||||||
|
--tls --from "$ALERT_FROM" --to "$to" \
|
||||||
|
--header "Subject: $subj" --body "$body" >/dev/null 2>&1; then
|
||||||
|
ok "alert" "sent → $to"
|
||||||
|
else
|
||||||
|
warn "alert" "send failed → $to"
|
||||||
|
send_fail=1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
rm -f "$body"
|
||||||
|
if [ "$send_fail" = "0" ]; then
|
||||||
|
echo "$now" >"$ALERT_STAMP" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
maildeliv_alert_if_needed
|
||||||
|
|
||||||
|
summary
|
||||||
|
|
@ -31,9 +31,6 @@ set_area sanity
|
||||||
set_group bank
|
set_group bank
|
||||||
section "sanity · bank"
|
section "sanity · bank"
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
if [ "${CHECK_BANK:-1}" != "1" ] || [ "${TALER_BANK_BACKEND:-}" = "ebics" ]; then
|
|
||||||
info "bank" "IBAN/EBICS (CHECK_BANK=${CHECK_BANK:-0} · backend=${TALER_BANK_BACKEND:-ebics} — no public bank.*)"
|
|
||||||
else
|
|
||||||
expect_code "bank public /config" 200 "$BANK_PUBLIC/config"
|
expect_code "bank public /config" 200 "$BANK_PUBLIC/config"
|
||||||
expect_code "bank public /taler-integration/config" 200 "$BANK_PUBLIC/taler-integration/config"
|
expect_code "bank public /taler-integration/config" 200 "$BANK_PUBLIC/taler-integration/config"
|
||||||
expect_code "bank public /webui/" 200 "$BANK_PUBLIC/webui/"
|
expect_code "bank public /webui/" 200 "$BANK_PUBLIC/webui/"
|
||||||
|
|
@ -101,7 +98,6 @@ REMOTE
|
||||||
else
|
else
|
||||||
warn "bank server-side" "ssh ${KOOPA_SSH} unavailable"
|
warn "bank server-side" "ssh ${KOOPA_SSH} unavailable"
|
||||||
fi
|
fi
|
||||||
fi
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
set_group exchange
|
set_group exchange
|
||||||
|
|
|
||||||
|
|
@ -879,11 +879,9 @@ PY
|
||||||
info "merchant webui version" "${got_ver} (set EXPECT_WEBUI_VERSION= to pin)"
|
info "merchant webui version" "${got_ver} (set EXPECT_WEBUI_VERSION= to pin)"
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
if [ "${LOCAL_STACK:-1}" = "1" ]; then
|
# Newer merchant-webui packages omit version.txt (GOA / FP stage / prod).
|
||||||
fail "merchant /webui/version.txt" "HTTP ${vcode:-000} (SPA missing version stamp)"
|
# INFO only — earlier merchant SPA builds shipped this stamp; bank webui still has it.
|
||||||
else
|
info "merchant /webui/version.txt" "HTTP ${vcode:-000} · earlier merchant SPA versions had this stamp; current merchant packages often omit it · bank /webui/version.txt still present on current bank builds"
|
||||||
warn "merchant /webui/version.txt" "HTTP ${vcode:-000}"
|
|
||||||
fi
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
ocode=$(http_body "$MERCHANT_PUBLIC/webui/version-overlay.txt" "$tmp/webui-overlay.txt")
|
ocode=$(http_body "$MERCHANT_PUBLIC/webui/version-overlay.txt" "$tmp/webui-overlay.txt")
|
||||||
|
|
|
||||||
27
domains.conf
27
domains.conf
|
|
@ -5,13 +5,9 @@
|
||||||
#
|
#
|
||||||
# - name: what -d / TALER_DOMAIN matches (aliases allowed as extra lines)
|
# - name: what -d / TALER_DOMAIN matches (aliases allowed as extra lines)
|
||||||
# - bank / exchange / merchant: hostname or full https:// URL
|
# - bank / exchange / merchant: hostname or full https:// URL
|
||||||
# bank may be "-" / ebics / none when the stack is IBAN/EBICS (no public
|
|
||||||
# libeufin bank.*). monitord also auto-detects from exchange /keys payto://
|
|
||||||
# (x-taler-bank → expect bank.*; iban-only → ebics_no_bank, blue).
|
|
||||||
# - currency: GOA | KUDOS | CHF | … (empty or "-" = report only)
|
# - currency: GOA | KUDOS | CHF | … (empty or "-" = report only)
|
||||||
# - local: 1 = koopa SSH/inside/e2e stack; 0 = public-only
|
# - local: 1 = koopa SSH/inside/e2e stack; 0 = public-only
|
||||||
# - landing: 1 = custom /intro landings (GOA + FrancPaysan/TESTPAYSAN stage only).
|
# - landing: 1 = check /intro landings + assets; 0 = skip (no GOA-style landings)
|
||||||
# 0 = no /intro (demo/test/regio/TOPS — not the custom landing stack)
|
|
||||||
# - lang: UI language for mon console/HTML (en|fr|de). Source of truth when
|
# - lang: UI language for mon console/HTML (en|fr|de). Source of truth when
|
||||||
# TALER_MON_LANG_SET≠1. Workstation TALER_MON_LANG=en does not override.
|
# TALER_MON_LANG_SET≠1. Workstation TALER_MON_LANG=en does not override.
|
||||||
# **de is supported but must not be used as a profile default** (no row
|
# **de is supported but must not be used as a profile default** (no row
|
||||||
|
|
@ -25,37 +21,26 @@
|
||||||
# CLI: --lang en|fr|de · TALER_MON_LANG_SET=1 locks UI lang
|
# CLI: --lang en|fr|de · TALER_MON_LANG_SET=1 locks UI lang
|
||||||
# Alternate file: TALER_DOMAINS_CONF=/path/to/domains.conf
|
# Alternate file: TALER_DOMAINS_CONF=/path/to/domains.conf
|
||||||
|
|
||||||
# Local GOA (koopa) — merchant host taler.hacktivism.ch · landings · UI en · locale de-CH
|
# Local GOA (koopa) — landings matter · UI en · locale de-CH
|
||||||
# monitord includes local=1 by default (stack id hacktivism.ch).
|
|
||||||
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
koopa bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
||||||
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
hacktivism bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
||||||
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH
|
hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH
|
||||||
taler.hacktivism.ch bank.hacktivism.ch exchange.hacktivism.ch taler.hacktivism.ch GOA 1 1 en de-CH hacktivism.ch
|
|
||||||
|
|
||||||
# Public demo (KUDOS) — no custom /intro landings
|
# Public demo (KUDOS)
|
||||||
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 0 en de-CH demo.taler.net
|
taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 en de-CH demo.taler.net
|
||||||
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 0 en de-CH
|
demo.taler.net bank.demo.taler.net exchange.demo.taler.net backend.demo.taler.net KUDOS 0 1 en de-CH
|
||||||
|
|
||||||
# Public test stack (TESTKUDOS)
|
# Public test stack (TESTKUDOS)
|
||||||
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0 en de-CH
|
test.taler.net bank.test.taler.net exchange.test.taler.net backend.test.taler.net TESTKUDOS 0 0 en de-CH
|
||||||
|
|
||||||
# TOPS CHF — EBICS/IBAN wire (no public bank.*); monitord softens bank via /keys
|
# TOPS CHF
|
||||||
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH
|
taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH
|
||||||
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH taler-ops.ch
|
my.taler-ops.ch bank.taler-ops.ch exchange.taler-ops.ch my.taler-ops.ch CHF 0 0 en de-CH taler-ops.ch
|
||||||
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH
|
stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH
|
||||||
stage.my.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
stage.my.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
||||||
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
my.stage.taler-ops.ch bank.stage.taler-ops.ch exchange.stage.taler-ops.ch stage.my.taler-ops.ch CHF 0 0 en de-CH stage.taler-ops.ch
|
||||||
|
|
||||||
# Public regional-currency demo (FLODOS) — taler-deployment / regio.taler.net
|
|
||||||
regio.taler.net bank.regio.taler.net exchange.regio.taler.net backend.regio.taler.net FLODOS 0 0 en de-CH
|
|
||||||
bank.regio.taler.net bank.regio.taler.net exchange.regio.taler.net backend.regio.taler.net FLODOS 0 0 en de-CH regio.taler.net
|
|
||||||
exchange.regio.taler.net bank.regio.taler.net exchange.regio.taler.net backend.regio.taler.net FLODOS 0 0 en de-CH regio.taler.net
|
|
||||||
backend.regio.taler.net bank.regio.taler.net exchange.regio.taler.net backend.regio.taler.net FLODOS 0 0 en de-CH regio.taler.net
|
|
||||||
regio bank.regio.taler.net exchange.regio.taler.net backend.regio.taler.net FLODOS 0 0 en de-CH regio.taler.net
|
|
||||||
|
|
||||||
# Franc Paysan — UI fr · regional locale fr-CH (Suisse romande)
|
# Franc Paysan — UI fr · regional locale fr-CH (Suisse romande)
|
||||||
# currency "-" = no Taler prod stack yet (static site only; monitord: static_pending).
|
|
||||||
# Stage TESTPAYSAN below is the live Taler stack.
|
|
||||||
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH
|
lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH
|
||||||
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
monnaie.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
||||||
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
bank.lefrancpaysan.ch bank.lefrancpaysan.ch exchange.lefrancpaysan.ch monnaie.lefrancpaysan.ch - 0 0 fr fr-CH lefrancpaysan.ch
|
||||||
|
|
|
||||||
|
|
@ -51,14 +51,12 @@ Needs explorer secrets (`SECRETS_ROOT` / bank-explorer password). Wall ~3h (`RUN
|
||||||
| `koopa-taler-deploy-test-apt-src-trixie-testing` | **fresh** for `trixie-testing` |
|
| `koopa-taler-deploy-test-apt-src-trixie-testing` | **fresh** for `trixie-testing` |
|
||||||
| `koopa-taler-deploy-test-apt-src-trixie-upgrade` | **upgrade**: initial install, then mytops-style `apt install` + `upgrade` |
|
| `koopa-taler-deploy-test-apt-src-trixie-upgrade` | **upgrade**: initial install, then mytops-style `apt install` + `upgrade` |
|
||||||
| `koopa-taler-deploy-test-apt-src-trixie-testing-upgrade` | **upgrade** for testing suite |
|
| `koopa-taler-deploy-test-apt-src-trixie-testing-upgrade` | **upgrade** for testing suite |
|
||||||
| `koopa-taler-deploy-test-apt-src-ubuntu` | **fresh** Ubuntu 24.04 (`noble` @ deb.taler.net/apt/ubuntu) |
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# from ops workstation → triggers on koopa-external
|
# from ops workstation → triggers on koopa-external
|
||||||
./host-agent/run-aptdeploy.sh
|
./host-agent/run-aptdeploy.sh
|
||||||
./host-agent/run-aptdeploy.sh upgrade
|
./host-agent/run-aptdeploy.sh upgrade
|
||||||
./host-agent/run-aptdeploy.sh fresh
|
./host-agent/run-aptdeploy.sh fresh
|
||||||
./host-agent/run-aptdeploy.sh ubuntu
|
|
||||||
./host-agent/run-aptdeploy.sh check-only
|
./host-agent/run-aptdeploy.sh check-only
|
||||||
|
|
||||||
# on koopa
|
# on koopa
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,11 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# ensure-apt-deploy-test-containers.sh
|
# ensure-apt-deploy-test-containers.sh
|
||||||
#
|
#
|
||||||
# Rootless podman containers on koopa (no nginx / no HTTPS):
|
# Four rootless podman containers on koopa (no nginx / no HTTPS):
|
||||||
#
|
#
|
||||||
# Fresh rebuild when repo fingerprint changes:
|
# Fresh rebuild when repo fingerprint changes:
|
||||||
# koopa-taler-deploy-test-apt-src-trixie
|
# koopa-taler-deploy-test-apt-src-trixie
|
||||||
# koopa-taler-deploy-test-apt-src-trixie-testing
|
# koopa-taler-deploy-test-apt-src-trixie-testing
|
||||||
# koopa-taler-deploy-test-apt-src-ubuntu (noble / ubuntu:24.04; mode ubuntu|all)
|
|
||||||
#
|
#
|
||||||
# Persistent upgrade path (mytops upgrade.sh style):
|
# Persistent upgrade path (mytops upgrade.sh style):
|
||||||
# koopa-taler-deploy-test-apt-src-trixie-upgrade
|
# koopa-taler-deploy-test-apt-src-trixie-upgrade
|
||||||
|
|
@ -25,13 +24,12 @@ mkdir -p "$STATE_DIR"
|
||||||
# Packages we track for "new version → rebuild fresh containers"
|
# Packages we track for "new version → rebuild fresh containers"
|
||||||
TRACK_PKGS="${APT_DEPLOY_TRACK_PKGS:-taler-merchant libtalermerchant libtalerexchange libdonau}"
|
TRACK_PKGS="${APT_DEPLOY_TRACK_PKGS:-taler-merchant libtalermerchant libtalerexchange libdonau}"
|
||||||
|
|
||||||
# Canonical containers (keep). Everything else matching legacy names is removed.
|
# Canonical four containers (keep). Everything else matching legacy names is removed.
|
||||||
CANONICAL_CONTAINERS=(
|
CANONICAL_CONTAINERS=(
|
||||||
koopa-taler-deploy-test-apt-src-trixie
|
koopa-taler-deploy-test-apt-src-trixie
|
||||||
koopa-taler-deploy-test-apt-src-trixie-testing
|
koopa-taler-deploy-test-apt-src-trixie-testing
|
||||||
koopa-taler-deploy-test-apt-src-trixie-upgrade
|
koopa-taler-deploy-test-apt-src-trixie-upgrade
|
||||||
koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
|
koopa-taler-deploy-test-apt-src-trixie-testing-upgrade
|
||||||
koopa-taler-deploy-test-apt-src-ubuntu
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Legacy / typo names from earlier iterations — always delete if present
|
# Legacy / typo names from earlier iterations — always delete if present
|
||||||
|
|
@ -168,13 +166,14 @@ Types: deb
|
||||||
URIs: ${URI_BASE}
|
URIs: ${URI_BASE}
|
||||||
EOF
|
EOF
|
||||||
apt-get update -qq
|
apt-get update -qq
|
||||||
# noble (ubuntu) has no taler-merchant-webui; install if present
|
apt-get install -y \
|
||||||
pkgs='taler-merchant taler-merchant-typst taler-terms-generator postgresql s-nail jq'
|
taler-merchant \
|
||||||
if apt-cache show taler-merchant-webui >/dev/null 2>&1; then
|
taler-merchant-typst \
|
||||||
pkgs=\"\$pkgs taler-merchant-webui\"
|
taler-merchant-webui \
|
||||||
fi
|
taler-terms-generator \
|
||||||
# shellcheck disable=SC2086
|
postgresql \
|
||||||
apt-get install -y \$pkgs
|
s-nail \
|
||||||
|
jq
|
||||||
"
|
"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -349,14 +348,7 @@ ensure_upgrade() {
|
||||||
upgrade_inside "$name"
|
upgrade_inside "$name"
|
||||||
}
|
}
|
||||||
|
|
||||||
MODE="${1:-all}" # all | fresh | upgrade | ubuntu
|
MODE="${1:-all}" # all | fresh | upgrade
|
||||||
|
|
||||||
ensure_ubuntu_noble() {
|
|
||||||
# Ubuntu 24.04 (noble) apt-src track — Freigabe 2026-09-05
|
|
||||||
IMG="${APT_DEPLOY_UBUNTU_IMAGE:-docker.io/library/ubuntu:24.04}" \
|
|
||||||
URI_BASE="${APT_DEPLOY_UBUNTU_URI:-https://deb.taler.net/apt/ubuntu}" \
|
|
||||||
ensure_fresh koopa-taler-deploy-test-apt-src-ubuntu noble taler.sources
|
|
||||||
}
|
|
||||||
|
|
||||||
case "$MODE" in
|
case "$MODE" in
|
||||||
all|fresh)
|
all|fresh)
|
||||||
|
|
@ -366,12 +358,9 @@ case "$MODE" in
|
||||||
all|upgrade)
|
all|upgrade)
|
||||||
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-upgrade trixie taler.sources
|
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-upgrade trixie taler.sources
|
||||||
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-testing-upgrade trixie-testing taler-testing.sources
|
ensure_upgrade koopa-taler-deploy-test-apt-src-trixie-testing-upgrade trixie-testing taler-testing.sources
|
||||||
;;&
|
|
||||||
all|ubuntu)
|
|
||||||
ensure_ubuntu_noble
|
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
echo "usage: $0 [all|fresh|upgrade|ubuntu]" >&2
|
echo "usage: $0 [all|fresh|upgrade]" >&2
|
||||||
exit 2
|
exit 2
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
|
||||||
2
host-agent/env/mytops-stage.env.example
vendored
2
host-agent/env/mytops-stage.env.example
vendored
|
|
@ -27,9 +27,7 @@ TALER_MON_LANG=en
|
||||||
AGENT_LABEL=mytops-stage-host-agent
|
AGENT_LABEL=mytops-stage-host-agent
|
||||||
STATE_NAME=taler-monitoring-mytops-stage
|
STATE_NAME=taler-monitoring-mytops-stage
|
||||||
SUITE_UPDATE_STRICT=1
|
SUITE_UPDATE_STRICT=1
|
||||||
# EBICS/IBAN wire — no public bank.* (also auto-detected from exchange /keys)
|
|
||||||
CHECK_BANK=0
|
CHECK_BANK=0
|
||||||
CHECK_BANK_SET=1
|
|
||||||
MERCHANT_REQUIRED=1
|
MERCHANT_REQUIRED=1
|
||||||
CHECK_LANDING=0
|
CHECK_LANDING=0
|
||||||
# rusty devtesting (SSH Host taler-rusty-devtesting → devtesting@rusty.taler-ops.ch)
|
# rusty devtesting (SSH Host taler-rusty-devtesting → devtesting@rusty.taler-ops.ch)
|
||||||
|
|
|
||||||
40
host-agent/run-mail-deliv-probe.sh
Executable file
40
host-agent/run-mail-deliv-probe.sh
Executable file
|
|
@ -0,0 +1,40 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Regular mon* inbox-vs-spam probe (Taler deliverability sinks).
|
||||||
|
# Timer: taler-monitoring-mail-deliv.timer (default every 6h).
|
||||||
|
#
|
||||||
|
# Loads secrets.env if present. Does NOT print passwords.
|
||||||
|
set -euo pipefail
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
if [[ -d /home/hernani/src/taler-monitoring ]]; then
|
||||||
|
ROOT=/home/hernani/src/taler-monitoring
|
||||||
|
fi
|
||||||
|
cd "$ROOT"
|
||||||
|
|
||||||
|
# secrets (gitignored)
|
||||||
|
for f in \
|
||||||
|
"${TALER_MONITORING_ENV:-}" \
|
||||||
|
"$HOME/.config/taler-monitoring/env" \
|
||||||
|
"$ROOT/secrets.env" \
|
||||||
|
"$HOME/src/taler-monitoring-env/secrets.env"
|
||||||
|
do
|
||||||
|
[[ -n "$f" && -f "$f" ]] || continue
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
source "$f"
|
||||||
|
set +a
|
||||||
|
break
|
||||||
|
done
|
||||||
|
|
||||||
|
export MAIL_DELIV_PROBE_FROM="${MAIL_DELIV_PROBE_FROM:-mon001@hacktivism.ch}"
|
||||||
|
export MAIL_DELIV_PROBE_TO="${MAIL_DELIV_PROBE_TO:-mon001@hacktivism.ch mon002@hacktivism.ch mon003@hacktivism.ch}"
|
||||||
|
export MAIL_DELIV_PROBE_SMTP="${MAIL_DELIV_PROBE_SMTP:-mail.cyon.ch:587}"
|
||||||
|
export MAIL_DELIV_IMAP_HOST="${MAIL_DELIV_IMAP_HOST:-mail.cyon.ch}"
|
||||||
|
export MAIL_DELIV_PROBE_WAIT_S="${MAIL_DELIV_PROBE_WAIT_S:-45}"
|
||||||
|
export MAIL_DELIV_PROBE_ALERT_TO="${MAIL_DELIV_PROBE_ALERT_TO:-hernani+mon@taler-systems.com hernani+mon@vecirex.net vecirex+mon@icloud.com fdold+mon@taler.net}"
|
||||||
|
|
||||||
|
if [[ -z "${MAIL_DELIV_IMAP_PASS:-}${MAIL_DELIV_ALERT_PASS:-}" ]]; then
|
||||||
|
echo "run-mail-deliv-probe: no MAIL_DELIV_IMAP_PASS in env/secrets.env" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
exec python3 "$ROOT/mail_deliv_probe.py"
|
||||||
17
host-agent/taler-monitoring-mail-deliv.service
Normal file
17
host-agent/taler-monitoring-mail-deliv.service
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
[Unit]
|
||||||
|
Description=taler-monitoring mon* inbox-vs-spam mail probe
|
||||||
|
Documentation=file:%h/src/taler-monitoring/check_mail_deliverability.sh
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
WorkingDirectory=%h/src/taler-monitoring
|
||||||
|
Environment=PYTHONUNBUFFERED=1
|
||||||
|
# Prefer secrets via EnvironmentFile if present (create on host; never commit)
|
||||||
|
EnvironmentFile=-%h/.config/taler-monitoring/env
|
||||||
|
EnvironmentFile=-%h/src/taler-monitoring/secrets.env
|
||||||
|
ExecStart=%h/src/taler-monitoring/host-agent/run-mail-deliv-probe.sh
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
12
host-agent/taler-monitoring-mail-deliv.timer
Normal file
12
host-agent/taler-monitoring-mail-deliv.timer
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
[Unit]
|
||||||
|
Description=taler-monitoring mon* mail probe every 6h
|
||||||
|
Documentation=file:%h/src/taler-monitoring/host-agent/run-mail-deliv-probe.sh
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnBootSec=20min
|
||||||
|
OnUnitActiveSec=6h
|
||||||
|
Persistent=true
|
||||||
|
RandomizedDelaySec=10min
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
87
lib.sh
87
lib.sh
|
|
@ -79,12 +79,6 @@ fi
|
||||||
: "${EXCHANGE_LOCAL:=http://127.0.0.1:9011}"
|
: "${EXCHANGE_LOCAL:=http://127.0.0.1:9011}"
|
||||||
: "${MERCHANT_LOCAL:=https://127.0.0.1:9010}"
|
: "${MERCHANT_LOCAL:=https://127.0.0.1:9010}"
|
||||||
: "${LANDING_LOCAL:=http://127.0.0.1:9013}"
|
: "${LANDING_LOCAL:=http://127.0.0.1:9013}"
|
||||||
# Public libeufin bank.* checks (urls/sanity/e2e). Auto 0 for EBICS/IBAN wire.
|
|
||||||
: "${CHECK_BANK:=1}"
|
|
||||||
: "${CHECK_BANK_SET:=0}"
|
|
||||||
# x-taler-bank | ebics | unknown — filled by resolve_taler_bank_backend
|
|
||||||
: "${TALER_BANK_BACKEND:=}"
|
|
||||||
: "${TALER_BANK_BACKEND_SET:=0}"
|
|
||||||
# Hostnames/paths: set via ~/.config/taler-monitoring/env (no hardcoded machine values).
|
# Hostnames/paths: set via ~/.config/taler-monitoring/env (no hardcoded machine values).
|
||||||
: "${KOOPA_SSH:=}"
|
: "${KOOPA_SSH:=}"
|
||||||
: "${KOOPA_SSH_FALLBACKS:=}"
|
: "${KOOPA_SSH_FALLBACKS:=}"
|
||||||
|
|
@ -155,77 +149,16 @@ MERCHANT_PUBLIC=${MERCHANT_PUBLIC%/}
|
||||||
_MONITOR_LIB_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
_MONITOR_LIB_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||||
: "${TALER_DOMAINS_CONF:=${_MONITOR_LIB_DIR}/domains.conf}"
|
: "${TALER_DOMAINS_CONF:=${_MONITOR_LIB_DIR}/domains.conf}"
|
||||||
|
|
||||||
# host_or_url → https://… (no trailing slash). Empty for EBICS skip tokens.
|
# host_or_url → https://… (no trailing slash)
|
||||||
_to_https_base() {
|
_to_https_base() {
|
||||||
local x="${1%/}"
|
local x="${1%/}"
|
||||||
case "$(printf '%s' "$x" | tr '[:upper:]' '[:lower:]')" in
|
case "$x" in
|
||||||
""|"-"|"."|"none"|"n/a"|"na"|"ebics"|"iban"|"skip"|"off")
|
|
||||||
printf ''
|
|
||||||
;;
|
|
||||||
https://*|http://*) printf '%s' "$x" ;;
|
https://*|http://*) printf '%s' "$x" ;;
|
||||||
|
"") printf '' ;;
|
||||||
*) printf 'https://%s' "$x" ;;
|
*) printf 'https://%s' "$x" ;;
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
# True when domains.conf bank column means "no public bank.*"
|
|
||||||
bank_host_configured() {
|
|
||||||
local b
|
|
||||||
b=$(printf '%s' "${1:-}" | tr '[:upper:]' '[:lower:]')
|
|
||||||
case "$b" in
|
|
||||||
""|"-"|"."|"none"|"n/a"|"na"|"ebics"|"iban"|"skip"|"off") return 1 ;;
|
|
||||||
*) return 0 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
# Fetch exchange /keys and classify → x-taler-bank|ebics|unknown
|
|
||||||
# (python fetch avoids SIGPIPE/pipefail on huge /keys bodies)
|
|
||||||
detect_taler_bank_backend_from_keys() {
|
|
||||||
local ex="${EXCHANGE_PUBLIC%/}"
|
|
||||||
[ -n "$ex" ] || { printf 'unknown'; return 0; }
|
|
||||||
python3 -c '
|
|
||||||
import re, ssl, sys, urllib.error, urllib.request
|
|
||||||
ex = sys.argv[1].rstrip("/")
|
|
||||||
ctx = ssl.create_default_context()
|
|
||||||
ctx.check_hostname = False
|
|
||||||
ctx.verify_mode = ssl.CERT_NONE
|
|
||||||
req = urllib.request.Request(
|
|
||||||
ex + "/keys",
|
|
||||||
headers={"User-Agent": "taler-monitoring/bank-backend"},
|
|
||||||
method="GET",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req, timeout=10, context=ctx) as resp:
|
|
||||||
text = resp.read(600000).decode("utf-8", "replace")
|
|
||||||
except Exception:
|
|
||||||
print("unknown")
|
|
||||||
raise SystemExit(0)
|
|
||||||
methods = {m.lower() for m in re.findall(r"payto://([a-z0-9-]+)/", text, flags=re.I)}
|
|
||||||
if "x-taler-bank" in methods:
|
|
||||||
print("x-taler-bank")
|
|
||||||
elif "iban" in methods:
|
|
||||||
print("ebics")
|
|
||||||
else:
|
|
||||||
print("unknown")
|
|
||||||
' "$ex" 2>/dev/null || printf 'unknown'
|
|
||||||
}
|
|
||||||
|
|
||||||
# Set TALER_BANK_BACKEND + auto CHECK_BANK=0 for EBICS (unless CHECK_BANK_SET=1).
|
|
||||||
resolve_taler_bank_backend() {
|
|
||||||
local backend="${TALER_BANK_BACKEND:-}"
|
|
||||||
if [ "${TALER_BANK_BACKEND_SET:-0}" = "1" ] && [ -n "$backend" ]; then
|
|
||||||
:
|
|
||||||
elif [ -z "${BANK_PUBLIC}" ]; then
|
|
||||||
backend=ebics
|
|
||||||
else
|
|
||||||
backend=$(detect_taler_bank_backend_from_keys)
|
|
||||||
fi
|
|
||||||
TALER_BANK_BACKEND="$backend"
|
|
||||||
if [ "$TALER_BANK_BACKEND" = "ebics" ] && [ "${CHECK_BANK_SET:-0}" != "1" ]; then
|
|
||||||
CHECK_BANK=0
|
|
||||||
fi
|
|
||||||
export TALER_BANK_BACKEND CHECK_BANK CHECK_BANK_SET TALER_BANK_BACKEND_SET
|
|
||||||
}
|
|
||||||
|
|
||||||
# 1 = check GOA-style /intro landing pages (assets, link crawl, demo-withdraw).
|
# 1 = check GOA-style /intro landing pages (assets, link crawl, demo-withdraw).
|
||||||
# 0 = skip (e.g. taler-ops.ch / mytops — no public landings).
|
# 0 = skip (e.g. taler-ops.ch / mytops — no public landings).
|
||||||
: "${CHECK_LANDING:=1}"
|
: "${CHECK_LANDING:=1}"
|
||||||
|
|
@ -233,19 +166,12 @@ resolve_taler_bank_backend() {
|
||||||
# Declare bank + exchange + merchant (+ currency, local stack, landing, domain id).
|
# Declare bank + exchange + merchant (+ currency, local stack, landing, domain id).
|
||||||
# Usage:
|
# Usage:
|
||||||
# set_taler_stack BANK EXCHANGE MERCHANT [CURRENCY] [LOCAL 0|1] [LANDING 0|1] [TALER_DOMAIN]
|
# set_taler_stack BANK EXCHANGE MERCHANT [CURRENCY] [LOCAL 0|1] [LANDING 0|1] [TALER_DOMAIN]
|
||||||
# BANK/EXCHANGE/MERCHANT: hostname or full URL; bank may be "-" / ebics (no public bank.*)
|
# BANK/EXCHANGE/MERCHANT: hostname or full URL
|
||||||
set_taler_stack() {
|
set_taler_stack() {
|
||||||
local bank="${1:-}" exchange="${2:-}" merchant="${3:-}"
|
local bank="${1:-}" exchange="${2:-}" merchant="${3:-}"
|
||||||
local currency="${4:-}" local_stack="${5:-0}" landing="${6:-}" domain_id="${7:-}"
|
local currency="${4:-}" local_stack="${5:-0}" landing="${6:-}" domain_id="${7:-}"
|
||||||
|
|
||||||
if bank_host_configured "$bank"; then
|
|
||||||
BANK_PUBLIC=$(_to_https_base "$bank")
|
BANK_PUBLIC=$(_to_https_base "$bank")
|
||||||
else
|
|
||||||
BANK_PUBLIC=""
|
|
||||||
if [ "${TALER_BANK_BACKEND_SET:-0}" != "1" ]; then
|
|
||||||
TALER_BANK_BACKEND=ebics
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
EXCHANGE_PUBLIC=$(_to_https_base "$exchange")
|
EXCHANGE_PUBLIC=$(_to_https_base "$exchange")
|
||||||
MERCHANT_PUBLIC=$(_to_https_base "$merchant")
|
MERCHANT_PUBLIC=$(_to_https_base "$merchant")
|
||||||
# currency "-" or empty → report-only (do not force GOA)
|
# currency "-" or empty → report-only (do not force GOA)
|
||||||
|
|
@ -526,11 +452,6 @@ apply_taler_domain() {
|
||||||
if [ "${LOCAL_STACK}" = "1" ]; then
|
if [ "${LOCAL_STACK}" = "1" ]; then
|
||||||
INSIDE_PROFILE="${INSIDE_PROFILE:-koopa}"
|
INSIDE_PROFILE="${INSIDE_PROFILE:-koopa}"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# EBICS/IBAN (TOPS) → no public bank.*; x-taler-bank → keep CHECK_BANK
|
|
||||||
if [ "${TALER_SKIP_BANK_BACKEND_DETECT:-0}" != "1" ]; then
|
|
||||||
resolve_taler_bank_backend
|
|
||||||
fi
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# Apply TALER_DOMAIN from env once (CLI exports TALER_DOMAIN_APPLIED=1 after overrides).
|
# Apply TALER_DOMAIN from env once (CLI exports TALER_DOMAIN_APPLIED=1 after overrides).
|
||||||
|
|
|
||||||
|
|
@ -18,4 +18,3 @@ gnunet.org firefly.gnunet.org firefly.gnunet.org
|
||||||
# Port 587 often closed/filtered on this plane → WARN only (see check_mail.sh).
|
# Port 587 often closed/filtered on this plane → WARN only (see check_mail.sh).
|
||||||
taler-systems.com mail.anastasis.lu mail.anastasis.lu,pixel.taler-systems.com 25,465,587 143,993 tsa-anastasis
|
taler-systems.com mail.anastasis.lu mail.anastasis.lu,pixel.taler-systems.com 25,465,587 143,993 tsa-anastasis
|
||||||
anastasis.lu mail.anastasis.lu mail.anastasis.lu 25,465,587 143,993 anastasis-lu
|
anastasis.lu mail.anastasis.lu mail.anastasis.lu 25,465,587 143,993 anastasis-lu
|
||||||
taler-ops.ch pixel.taler-systems.com mail.anastasis.lu,pixel.taler-systems.com 25,465,587 143,993 tops-mail
|
|
||||||
|
|
|
||||||
232
mail_deliv_probe.py
Executable file
232
mail_deliv_probe.py
Executable file
|
|
@ -0,0 +1,232 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Send a tagged probe mail via sink SMTP, then IMAP-search INBOX vs Junk/spam.
|
||||||
|
|
||||||
|
SUT remains Taler mail domains elsewhere; this only exercises mon* sinks
|
||||||
|
(and optional g0v) for inbox-vs-spam visibility.
|
||||||
|
|
||||||
|
Env:
|
||||||
|
MAIL_DELIV_IMAP_PASS / MAIL_DELIV_ALERT_PASS required (shared default)
|
||||||
|
MAIL_DELIV_PASS_mon001 / _mon002 / _mon003 optional per-mailbox override
|
||||||
|
MAIL_DELIV_PROBE_FROM default mon001@hacktivism.ch
|
||||||
|
MAIL_DELIV_PROBE_TO default mon001 mon002 mon003 (keep all — odd-looking addrs)
|
||||||
|
MAIL_DELIV_PROBE_SMTP default mail.cyon.ch:587
|
||||||
|
MAIL_DELIV_IMAP_HOST default mail.cyon.ch
|
||||||
|
MAIL_DELIV_PROBE_WAIT_S default 25
|
||||||
|
MAIL_DELIV_PROBE_ALERT_TO optional extra alert recipients on spam/miss
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import imaplib
|
||||||
|
import os
|
||||||
|
import smtplib
|
||||||
|
import ssl
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def _env(name: str, default: str = "") -> str:
|
||||||
|
return (os.environ.get(name) or default).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _pass_for(addr: str) -> str:
|
||||||
|
"""Shared pass, or MAIL_DELIV_PASS_<localpart> (e.g. MAIL_DELIV_PASS_mon002)."""
|
||||||
|
local = addr.split("@", 1)[0].replace(".", "_").replace("+", "_")
|
||||||
|
specific = _env(f"MAIL_DELIV_PASS_{local}") or _env(f"MAIL_DELIV_IMAP_PASS_{local}")
|
||||||
|
if specific:
|
||||||
|
return specific
|
||||||
|
return _env("MAIL_DELIV_IMAP_PASS") or _env("MAIL_DELIV_ALERT_PASS")
|
||||||
|
|
||||||
|
|
||||||
|
def smtp_send(host: str, port: int, user: str, pw: str, to: str, subj: str, body: str, msgid: str) -> None:
|
||||||
|
msg = EmailMessage()
|
||||||
|
msg["From"] = user
|
||||||
|
msg["To"] = to
|
||||||
|
msg["Subject"] = subj
|
||||||
|
msg["Message-ID"] = msgid
|
||||||
|
msg["X-Taler-Mon-Probe"] = msgid.strip("<>")
|
||||||
|
msg.set_content(body)
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
with smtplib.SMTP(host, port, timeout=45) as s:
|
||||||
|
s.ehlo()
|
||||||
|
s.starttls(context=ctx)
|
||||||
|
s.ehlo()
|
||||||
|
s.login(user, pw)
|
||||||
|
s.send_message(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def imap_find(host: str, user: str, pw: str, token: str) -> dict:
|
||||||
|
"""Return {folder: count} for folders where token appears in Message-ID/Subject/body."""
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
M = imaplib.IMAP4_SSL(host, 993, ssl_context=ctx)
|
||||||
|
M.login(user, pw)
|
||||||
|
typ, bl = M.list()
|
||||||
|
folders = []
|
||||||
|
for raw in bl or []:
|
||||||
|
line = raw.decode(errors="replace")
|
||||||
|
# LIST (\HasNoChildren) "." INBOX.Junk
|
||||||
|
if '"/"' in line or ' "." ' in line or ' "."' in line:
|
||||||
|
name = line.split(' "." ')[-1].strip().strip('"')
|
||||||
|
elif ' "/" ' in line:
|
||||||
|
name = line.split(' "/" ')[-1].strip().strip('"')
|
||||||
|
else:
|
||||||
|
parts = line.rsplit(" ", 1)
|
||||||
|
name = parts[-1].strip().strip('"') if parts else ""
|
||||||
|
if name:
|
||||||
|
folders.append(name)
|
||||||
|
# ensure common names
|
||||||
|
for n in ("INBOX", "INBOX.Junk", "INBOX.spam", "Junk", "Spam"):
|
||||||
|
if n not in folders:
|
||||||
|
folders.append(n)
|
||||||
|
|
||||||
|
hits: dict[str, int] = {}
|
||||||
|
needle = token.encode()
|
||||||
|
for folder in folders:
|
||||||
|
try:
|
||||||
|
typ, _ = M.select(folder, readonly=True)
|
||||||
|
if typ != "OK":
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
# SEARCH TEXT is broad; prefer HEADER Message-ID
|
||||||
|
found = 0
|
||||||
|
for crit in (f'(HEADER Message-ID "{token}")', f'(HEADER X-Taler-Mon-Probe "{token}")', f'(SUBJECT "{token}")'):
|
||||||
|
try:
|
||||||
|
typ, data = M.search(None, crit)
|
||||||
|
if typ == "OK" and data and data[0]:
|
||||||
|
ids = data[0].split()
|
||||||
|
found = max(found, len(ids))
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if found:
|
||||||
|
hits[folder] = found
|
||||||
|
M.logout()
|
||||||
|
return hits
|
||||||
|
|
||||||
|
|
||||||
|
def classify(hits: dict[str, int]) -> str:
|
||||||
|
if not hits:
|
||||||
|
return "missing"
|
||||||
|
# prefer INBOX without junk
|
||||||
|
junkish = []
|
||||||
|
inboxish = []
|
||||||
|
for f, n in hits.items():
|
||||||
|
fl = f.lower()
|
||||||
|
if "junk" in fl or "spam" in fl:
|
||||||
|
junkish.append(f)
|
||||||
|
elif fl == "inbox" or fl.endswith(".inbox") or fl == "inbox.inbox":
|
||||||
|
inboxish.append(f)
|
||||||
|
elif "inbox" in fl and "junk" not in fl and "spam" not in fl and "trash" not in fl:
|
||||||
|
inboxish.append(f)
|
||||||
|
if inboxish and not junkish:
|
||||||
|
return "inbox"
|
||||||
|
if junkish and not inboxish:
|
||||||
|
return "spam"
|
||||||
|
if inboxish and junkish:
|
||||||
|
return "both"
|
||||||
|
return "other:" + ",".join(hits.keys())
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
frm = _env("MAIL_DELIV_PROBE_FROM", "mon001@hacktivism.ch")
|
||||||
|
# Keep mon001–003 even if one address looks odd / one pass differs.
|
||||||
|
tos = _env(
|
||||||
|
"MAIL_DELIV_PROBE_TO",
|
||||||
|
"mon001@hacktivism.ch mon002@hacktivism.ch mon003@hacktivism.ch",
|
||||||
|
).split()
|
||||||
|
pw_from = _pass_for(frm)
|
||||||
|
if not pw_from:
|
||||||
|
print("FAIL missing MAIL_DELIV_IMAP_PASS (or per-mailbox MAIL_DELIV_PASS_*)", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
smtp = _env("MAIL_DELIV_PROBE_SMTP", "mail.cyon.ch:587")
|
||||||
|
imap_host = _env("MAIL_DELIV_IMAP_HOST", "mail.cyon.ch")
|
||||||
|
wait_s = float(_env("MAIL_DELIV_PROBE_WAIT_S", "25") or "25")
|
||||||
|
host, _, port_s = smtp.partition(":")
|
||||||
|
port = int(port_s or "587")
|
||||||
|
|
||||||
|
token = uuid.uuid4().hex
|
||||||
|
msgid = f"<{token}@taler-monitoring.probe>"
|
||||||
|
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||||
|
subj = f"[taler-mon probe] {token[:12]}"
|
||||||
|
body = (
|
||||||
|
f"taler-monitoring regular mon* probe\n"
|
||||||
|
f"token={token}\n"
|
||||||
|
f"when={now}\n"
|
||||||
|
f"Hallo Hernâni, hallo Florian — Inbox-vs-Spam check.\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
results = []
|
||||||
|
sent_ok = []
|
||||||
|
for to in tos:
|
||||||
|
try:
|
||||||
|
smtp_send(host, port, frm, pw_from, to, subj, body, msgid)
|
||||||
|
print(f"SENT to={to} token={token[:12]}")
|
||||||
|
sent_ok.append(to)
|
||||||
|
except Exception as e:
|
||||||
|
print(f"SEND_FAIL to={to} {type(e).__name__}: {e}")
|
||||||
|
results.append((to, "send_fail", {}))
|
||||||
|
|
||||||
|
time.sleep(max(5.0, wait_s))
|
||||||
|
|
||||||
|
rc = 0
|
||||||
|
for to in tos:
|
||||||
|
pw = _pass_for(to)
|
||||||
|
if not pw:
|
||||||
|
print(f"IMAP_SKIP to={to} no password (set MAIL_DELIV_PASS_{to.split('@')[0]})")
|
||||||
|
results.append((to, "no_pass", {}))
|
||||||
|
rc = 1
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
hits = imap_find(imap_host, to, pw, token)
|
||||||
|
kind = classify(hits)
|
||||||
|
print(f"IMAP to={to} result={kind} hits={hits}")
|
||||||
|
results.append((to, kind, hits))
|
||||||
|
if kind != "inbox":
|
||||||
|
rc = 1
|
||||||
|
except Exception as e:
|
||||||
|
# Keep name in rotation; soft note — odd addr / wrong pass still listed
|
||||||
|
print(f"IMAP_FAIL to={to} {type(e).__name__}: {e}")
|
||||||
|
results.append((to, "imap_fail", {}))
|
||||||
|
rc = 1
|
||||||
|
|
||||||
|
# optional notify alert recipients if any non-inbox
|
||||||
|
alert_to = _env("MAIL_DELIV_PROBE_ALERT_TO") or _env("MAIL_DELIV_ALERT_TO")
|
||||||
|
bad = [r for r in results if r[1] not in ("inbox",)]
|
||||||
|
if alert_to and bad:
|
||||||
|
summary = "\n".join(f"- {t}: {k} {h}" for t, k, h in results)
|
||||||
|
for ato in alert_to.split():
|
||||||
|
try:
|
||||||
|
smtp_send(
|
||||||
|
host,
|
||||||
|
port,
|
||||||
|
frm,
|
||||||
|
pw_from,
|
||||||
|
ato,
|
||||||
|
f"[taler-mon probe] spam/miss {token[:12]}",
|
||||||
|
f"Probe issues:\n{summary}\n",
|
||||||
|
f"<{uuid.uuid4().hex}@taler-monitoring.alert>",
|
||||||
|
)
|
||||||
|
print(f"ALERT → {ato}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"ALERT_FAIL → {ato} {e}")
|
||||||
|
|
||||||
|
# stamp state
|
||||||
|
state = Path(
|
||||||
|
_env(
|
||||||
|
"TALER_MONITORD_STATE_DIR",
|
||||||
|
str(Path.home() / ".local/state/taler-monitord"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
state.mkdir(parents=True, exist_ok=True)
|
||||||
|
(state / "mail-deliv-probe.last").write_text(
|
||||||
|
f"ts={now}\ntoken={token}\nrc={rc}\n" + "\n".join(f"{t}\t{k}\t{h}" for t, k, h in results) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return rc
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
|
@ -1,48 +0,0 @@
|
||||||
# ports-whitelist.conf — operator override for *intentionally* open public ports
|
|
||||||
#
|
|
||||||
# Monitoring is outside-in only (TCP connect from the internet). Anything odd that
|
|
||||||
# is still open and not listed here stays suspicious — many weird ports ⇒ look closer.
|
|
||||||
# This file does not invent internal topology; it only silences known-good public opens.
|
|
||||||
#
|
|
||||||
# Format: host_or_pattern port[,port…] [# note]
|
|
||||||
# Patterns: exact FQDN | *.example.com | * (any host)
|
|
||||||
# Baseline already allows 80/443 on all hosts, and mail ports on mail-catalog hosts.
|
|
||||||
# Override path: TALER_PORTS_WHITELIST=/path/to/file
|
|
||||||
#
|
|
||||||
# Whitelisted ports must NOT raise orange/red on the stack card.
|
|
||||||
|
|
||||||
# --- hacktivism public edge (observed / intentional WAN services) ---
|
|
||||||
# 22 ssh · 8080 Tor OR · 9200 Forgejo git-SSH (public; not treated as Elasticsearch here)
|
|
||||||
*.hacktivism.ch 22,8080,9200
|
|
||||||
hacktivism.ch 22,8080,9200
|
|
||||||
|
|
||||||
# Common intentional SSH on public edges:
|
|
||||||
*.taler.net 22
|
|
||||||
*.taler-ops.ch 22
|
|
||||||
*.lefrancpaysan.ch 22
|
|
||||||
*.taler-systems.com 22
|
|
||||||
*.gnunet.org 22
|
|
||||||
*.anastasis.lu 22
|
|
||||||
*.infomaniak.ch 22
|
|
||||||
anastasis.lu 22,25,143,465,587,993
|
|
||||||
taler-systems.com 22
|
|
||||||
gnunet.org 22
|
|
||||||
|
|
||||||
# Shared mail/edge ports often co-hosted on public FQDNs (soft noise otherwise):
|
|
||||||
*.taler.net 25,143,465,587,993,8080
|
|
||||||
*.test.taler.net 25,143,465,587,993,8080
|
|
||||||
firefly.gnunet.org 22,25,143,465,587,993,9418
|
|
||||||
pixel.taler-systems.com 22,25,143,465,587,993
|
|
||||||
mail.anastasis.lu 22,25,143,465,587,993
|
|
||||||
|
|
||||||
# Firefly git daemon (admin-log firefly/14-git-daemon.txt · ListenStream=9418)
|
|
||||||
git.taler.net 22,9418
|
|
||||||
git.gnunet.org 22,9418
|
|
||||||
|
|
||||||
# VeciGate WAN edge (vecigate-admin-log · public A 212.51.151.254)
|
|
||||||
# 80/443 baseline; 8080 Tor OR; 9200 Forgejo git-SSH; 2323x host SSH DNAT;
|
|
||||||
# 1789/1790 Nym; tortuna companion 23230 + 52323 (nat README)
|
|
||||||
212.51.151.254 22,8080,9200,1789,1790,23230,23235,23236,23237,23238,52323
|
|
||||||
|
|
||||||
# Optional — add more as you confirm intentional opens:
|
|
||||||
# lefrancpaysan.ch 21
|
|
||||||
|
|
@ -26,14 +26,6 @@
|
||||||
# MERCHANT_TOKEN=
|
# MERCHANT_TOKEN=
|
||||||
# MERCHANT_INSTANCE_PASSWORD=
|
# MERCHANT_INSTANCE_PASSWORD=
|
||||||
|
|
||||||
# --- regio.taler.net (FLODOS / taler-deployment) ---
|
|
||||||
# Prefer path via taler-monitoring-env (no password here):
|
|
||||||
# TALER_DEPLOYMENT_SECRETS=$HOME/src/taler-deployment-secrets
|
|
||||||
# REGIO_BANK_ADMIN_USER=admin
|
|
||||||
# REGIO_BANK_ADMIN_SECRET=$HOME/src/taler-deployment-secrets/regio/bank-admin-password.txt
|
|
||||||
# One-shot override:
|
|
||||||
# E2E_BANK_ADMIN_PASS=… # same as bank admin password
|
|
||||||
|
|
||||||
# --- FrancPaysan STAGE (TESTPAYSAN) e2e ---
|
# --- FrancPaysan STAGE (TESTPAYSAN) e2e ---
|
||||||
# Default phases for -d stage.lefrancpaysan.ch: urls + inside + versions + e2e
|
# Default phases for -d stage.lefrancpaysan.ch: urls + inside + versions + e2e
|
||||||
# inside uses francpaysan-stage-user (stagepaysan); stats from public HTTPS.
|
# inside uses francpaysan-stage-user (stagepaysan); stats from public HTTPS.
|
||||||
|
|
|
||||||
|
|
@ -1,33 +0,0 @@
|
||||||
# services-catalog.conf — non-Taler company / firefly / pixel hosts (monitord)
|
|
||||||
#
|
|
||||||
# Format (whitespace-separated; # comments):
|
|
||||||
# id host label zone
|
|
||||||
#
|
|
||||||
# zone hints: pixel | firefly | tops | public
|
|
||||||
# Override: TALER_SERVICES_CONF=/path/to/file
|
|
||||||
|
|
||||||
# --- Pixel / TSA / Anastasis ---
|
|
||||||
taler-systems.com taler-systems.com company pixel
|
|
||||||
www.taler-systems.com www.taler-systems.com company pixel
|
|
||||||
erp.stage.taler-systems.com erp.stage.taler-systems.com ERPnext pixel
|
|
||||||
anastasis.lu anastasis.lu anastasis pixel
|
|
||||||
www.anastasis.lu www.anastasis.lu anastasis pixel
|
|
||||||
pixel.taler-systems.com pixel.taler-systems.com pixel pixel
|
|
||||||
|
|
||||||
# TOPS company web (CHF Taler stack is in domains.conf as taler-ops.ch)
|
|
||||||
www.taler-ops.ch www.taler-ops.ch company tops
|
|
||||||
|
|
||||||
# --- Firefly / GNUnet (mail plane hosts are in mail-catalog.conf) ---
|
|
||||||
gnunet.org gnunet.org gnunet firefly
|
|
||||||
www.gnunet.org www.gnunet.org gnunet firefly
|
|
||||||
bugs.gnunet.org bugs.gnunet.org MantisBT firefly
|
|
||||||
taler.net-apex taler.net taler-net firefly
|
|
||||||
git.taler.net git.taler.net Git firefly
|
|
||||||
git.gnunet.org git.gnunet.org Git firefly
|
|
||||||
mattermost.taler.net mattermost.taler.net Mattermost firefly
|
|
||||||
buildbot.gnunet.org buildbot.gnunet.org Buildbot firefly
|
|
||||||
buildbot.taler.net buildbot.taler.net Buildbot firefly
|
|
||||||
|
|
||||||
# --- VeciGate edge (+ tortuna companion via WAN DNAT) ---
|
|
||||||
# Facts: PATH[vecigate-admin-log] docs/port-map-*.md · ip/firewall/nat/README.md
|
|
||||||
vecigate-wan 212.51.151.254 VeciGate vecigate
|
|
||||||
|
|
@ -1,97 +1,14 @@
|
||||||
# site-gen — HTML converter + Caddy notes
|
# site-gen — HTML converter + Caddy notes
|
||||||
|
|
||||||
**Product:** **LLM** = **Lightweight Monitoring** (Taler-stack watcher). CLI/tree still `taler-monitoring` / `taler-monitord`.
|
|
||||||
|
|
||||||
**Required for host-agent:** `console_to_html.py` (log → sticky HTML).
|
**Required for host-agent:** `console_to_html.py` (log → sticky HTML).
|
||||||
|
|
||||||
| File | Role |
|
| File | Role |
|
||||||
|------|------|
|
|------|------|
|
||||||
| `console_to_html.py` | Convert suite console log → public mon HTML |
|
| `console_to_html.py` | Convert suite console log → public mon HTML |
|
||||||
| `stack_status_lib.py` | Domain-stack traffic lights + aggregation |
|
|
||||||
| `taler-monitord` | Daemon: probe stacks → `stack-status.json` (UI detached) |
|
|
||||||
| `stack_status_html.py` | Render status cards from JSON only |
|
|
||||||
| `caddy-monitoring-handles.snippet` | Caddy `handle` / bare redir notes |
|
| `caddy-monitoring-handles.snippet` | Caddy `handle` / bare redir notes |
|
||||||
| `ROOT-ON-KOOPA.md` | Publish staging → `/var/www` (paths from env) |
|
| `ROOT-ON-KOOPA.md` | Publish staging → `/var/www` (paths from env) |
|
||||||
| `settings.conf.example` | Optional settings for **legacy** multi-host generator |
|
| `settings.conf.example` | Optional settings for **legacy** multi-host generator |
|
||||||
|
|
||||||
## Stack status (taler-monitord)
|
|
||||||
|
|
||||||
Like `llm-controld`: the daemon collects; the page only reads JSON.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./site-gen/taler-monitord --once
|
|
||||||
./site-gen/stack_status_html.py --out /tmp/taler-stack-status.html
|
|
||||||
# optional: ./site-gen/taler-monitord --interval 20
|
|
||||||
# local GOA (hacktivism.ch / taler.hacktivism.ch) is included by default;
|
|
||||||
# skip with --no-include-local or TALER_MONITORD_INCLUDE_LOCAL=0
|
|
||||||
```
|
|
||||||
|
|
||||||
Lights: green ok · blue present · yellow soft · orange hard/recent · red acute · black long · white gone.
|
|
||||||
Box colour = weakest member; **muted** if partial; **full** if all members red/black.
|
|
||||||
|
|
||||||
**Total score** (header): `N/M green` across stack+service+mail cards. Ratio thresholds (env override):
|
|
||||||
1.0 green · ≥0.85 blue · ≥0.60 yellow · ≥0.35 orange · ≥0.01 red · 0 white (gone) / all-black → black.
|
|
||||||
|
|
||||||
Stacks with currency `-` in `domains.conf` are **`static_pending`** (e.g. lefrancpaysan.ch: static site, no Taler prod yet). monitord probes the apex as `site` and treats missing bank/exchange/merchant as **blue `no_prod_yet`**, not orange/red. Stage TESTPAYSAN remains a normal Taler stack.
|
|
||||||
|
|
||||||
**Bank backend:** exchange `/keys` `payto://` methods decide whether `bank.*` is expected.
|
|
||||||
`payto://x-taler-bank/…` → probe bank (libeufin).
|
|
||||||
`payto://iban/…` only (EBICS / TOPS CHF) → bank member **blue**, host **`IBAN`**, reason `ebics_iban` (not red).
|
|
||||||
Mixed (e.g. regio) still probes bank. Override in `domains.conf`: bank column `-` / `ebics` / `none`.
|
|
||||||
|
|
||||||
**Network / SSH:** each stack gets a coarse **zone** (GOA/local, FP stage, TOPS, pixel, firefly, public demo).
|
|
||||||
SSH = **TCP:22 banner** (service present), not Magikoopa jump login.
|
|
||||||
Zone-mapped SSH targets come **only** from `~/.config/taler-monitoring/env` (`TALER_MONITORD_SSH_TOPS` / `_FP` / `_GOA` / `_TOPS_PROD` via **taler-monitoring-env**); hostnames are **hidden** in the UI when set. Without env, probes use public stack FQDNs. Disable: `TALER_MONITORD_SSH=0`.
|
|
||||||
|
|
||||||
**Outside CVEs:** `Server` headers (nginx/apache/caddy + version) → OSV.
|
|
||||||
**CRITICAL → red**, **HIGH → orange**. UI: CVE tags **link to osv.dev**; card click opens detail (severity / version / summary). Logos for nginx/apache/caddy. Disable: `TALER_MONITORD_CVE=0`.
|
|
||||||
|
|
||||||
**Why Degraded:** each card shows a one-line aggregation (`4× OK, 3× Soft → Degraded`); click opens a dialog with per-member reasons.
|
|
||||||
|
|
||||||
**Company / services:** `services-catalog.conf` (ERPnext, MantisBT, company apexes, …). Disable: `TALER_MONITORD_SERVICES=0`.
|
|
||||||
|
|
||||||
**Mail:** `mail-catalog.conf` → MX / SPF / DMARC / A+AAAA / rDNS + SMTP/IMAP TCP ports (587 soft).
|
|
||||||
**Deliverability** (regular, not every cycle): SMTP `MAIL FROM`/`RCPT TO` smoke to `postmaster@domain` (no DATA).
|
|
||||||
Cached under monitord state (`mail-deliverability.json`); default interval **6h**, stale after **30h** → yellow.
|
|
||||||
Send/server failures → orange/red (degraded). Env: `TALER_MONITORD_DELIVER=0`, `_INTERVAL_S`, `_STALE_S`, `_FROM`, `_RCPT`.
|
|
||||||
Disable plane: `TALER_MONITORD_MAIL=0`; ports only: `TALER_MONITORD_MAIL_PORTS=0`.
|
|
||||||
|
|
||||||
**TLS certs + orphans:** cert days-left + SANs. Orphans = **DNS-missing** SANs not covered by monitored hosts; wildcards/siblings skipped.
|
|
||||||
HTTP orphans opt-in: `TALER_MONITORD_ORPHAN_HTTP=1`. Disable: `TALER_MONITORD_CERT=0` / `TALER_MONITORD_ORPHAN=0`.
|
|
||||||
|
|
||||||
**Port scan (outside-in only):** TCP connect to notable ports on publicly named hosts. No internal firewall maps required.
|
|
||||||
Baseline allows 80/443 (+ mail ports on mail-catalog hosts). Anything else open → **suspicious** until listed in `ports-whitelist.conf`.
|
|
||||||
Many odd open ports ⇒ worth investigating. Whitelist only silences *known intentional public* opens (e.g. hacktivism **8080** Tor OR, **9200** git-SSH) so they do not turn cards orange.
|
|
||||||
Only **hard** findings (DB/RDP/SMB/…) raise card colour; soft stays informational. Disable: `TALER_MONITORD_PORTSCAN=0`.
|
|
||||||
|
|
||||||
**Software versions:** per card from Taler `/config` (`build_version`), `Server:` (nginx/apache/caddy), SMTP banner (Postfix).
|
|
||||||
Compared to **published** packages on `deb.taler.net` (`TALER_MONITORD_APT_SUITE`, default trixie): current / behind / ahead; chips show `installed → published` when behind.
|
|
||||||
**Apps / SPAs:** page section with published WebUI + wallet-cli (+ GNU FTP wallet tarball when listed). Disable: `TALER_MONITORD_PUBLISHED=0`.
|
|
||||||
|
|
||||||
**Publish:** Cyon **`hacktivism`**. Defaults (both):
|
|
||||||
`…/public_html/stacks/index.html` → **https://hacktivism.ch/stacks/**
|
|
||||||
`…/public_html/monitoring.hacktivism.ch/index.html` (addon when DNS ready)
|
|
||||||
Override: `TALER_MONITORD_PUBLISH_PATH` or `TALER_MONITORD_PUBLISH_PATHS=a,b`. Off: `TALER_MONITORD_PUBLISH=0`.
|
|
||||||
|
|
||||||
**Result index (optional):** bottom of the stacks page lists every probe grouped by traffic-light colour.
|
|
||||||
`TALER_MONITORD_RESULT_INDEX=1` (or `colors`) · with concrete command: `=cmd` / `full`, or `=1` plus `TALER_MONITORD_RESULT_INDEX_CMD=1`. Off by default (`=0`).
|
|
||||||
|
|
||||||
**Git soft probe:** Git-labelled services run infrequent `git ls-remote` over HTTPS + SSH (cached ~6h). Soft yellow on fail. Disable: `TALER_MONITORD_GIT=0`.
|
|
||||||
|
|
||||||
**VeciGate:** `services-catalog.conf` zone `vecigate` (WAN IP) + intentional DNAT ports in `ports-whitelist.conf` (incl. tortuna companion).
|
|
||||||
|
|
||||||
**Payability:** `TALER_MONITORD_PAY=1` shows CHF:0 / zero-pay status on matching stacks. Live wallet run only with `TALER_MONITORD_PAY_RUN=1` (interval `TALER_MONITORD_PAY_INTERVAL_S`, default 4h). Domains: `TALER_MONITORD_PAY_DOMAINS`.
|
|
||||||
|
|
||||||
**Ansible smoke (opt-in phase):** `./taler-monitoring.sh ansible-smoke` → `check_ansible_smoke.sh` against `TALER_DEPLOYMENT` (`regional-currency-ng/site.yml`). Podman tracks: `ANSIBLE_SMOKE_PODMAN=1` on koopa (else reuse `aptdeploy`).
|
|
||||||
|
|
||||||
**Local VCS mirror:** `~/git/www-hacktivism-wiki` rsyncs Cyon → git every **4h**
|
|
||||||
(`ch.hacktivism.www-wiki-sync`). Tracks `stacks/` (+ optional addon). No push by default.
|
|
||||||
|
|
||||||
**Custom /intro:** only **GOA** (hacktivism) and **FrancPaysan / TESTPAYSAN stage**.
|
|
||||||
**Landing stats:** on intro stacks, `…/intro/stats.json` → compact tag.
|
|
||||||
**Currency / coins:** from exchange `/config` `currency_specification`.
|
|
||||||
**Endpoints:** suite-like paths; soft ≤ yellow. **`/terms` + `/privacy`:** medium → orange **only TOPS prod**.
|
|
||||||
|
|
||||||
## Optional / moved
|
## Optional / moved
|
||||||
|
|
||||||
Legacy multi-host generate/deploy and firecuda launchd tooling live under
|
Legacy multi-host generate/deploy and firecuda launchd tooling live under
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -1,224 +0,0 @@
|
||||||
#!/usr/bin/env python3
|
|
||||||
"""taler-monitord — probe domain stacks on an interval; UI reads JSON only."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
HERE = Path(__file__).resolve().parent
|
|
||||||
sys.path.insert(0, str(HERE))
|
|
||||||
import stack_status_html as H # noqa: E402
|
|
||||||
import stack_status_lib as L # noqa: E402
|
|
||||||
|
|
||||||
# Match lib.sh: load ~/.config/taler-monitoring/env (or TALER_MONITORING_ENV), then
|
|
||||||
# refresh module flags that were read at import time.
|
|
||||||
_ENV_LOADED = L.load_monitoring_env()
|
|
||||||
if _ENV_LOADED:
|
|
||||||
L.refresh_runtime_flags_from_env()
|
|
||||||
|
|
||||||
|
|
||||||
def _pid_path() -> Path:
|
|
||||||
return L.monitord_state_dir() / "taler-monitord.pid"
|
|
||||||
|
|
||||||
|
|
||||||
def _write_pid() -> None:
|
|
||||||
_pid_path().write_text(str(os.getpid()) + "\n", encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def _clear_pid() -> None:
|
|
||||||
p = _pid_path()
|
|
||||||
try:
|
|
||||||
p.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _publish_one(local: Path, ssh_host: str, remote: str) -> str | None:
|
|
||||||
remote_dir = str(Path(remote).parent)
|
|
||||||
mk = subprocess.run(
|
|
||||||
[
|
|
||||||
"ssh",
|
|
||||||
"-o",
|
|
||||||
"BatchMode=yes",
|
|
||||||
"-o",
|
|
||||||
"ConnectTimeout=8",
|
|
||||||
ssh_host,
|
|
||||||
f"mkdir -p -- {remote_dir!s}",
|
|
||||||
],
|
|
||||||
capture_output=True,
|
|
||||||
timeout=15,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if mk.returncode != 0:
|
|
||||||
err = (mk.stderr or b"").decode("utf-8", "replace").strip()[:200]
|
|
||||||
print(f"publish mkdir fail ({remote}): {err}", file=sys.stderr, flush=True)
|
|
||||||
return None
|
|
||||||
r = subprocess.run(
|
|
||||||
[
|
|
||||||
"scp",
|
|
||||||
"-o",
|
|
||||||
"BatchMode=yes",
|
|
||||||
"-o",
|
|
||||||
"ConnectTimeout=8",
|
|
||||||
str(local),
|
|
||||||
f"{ssh_host}:{remote}",
|
|
||||||
],
|
|
||||||
capture_output=True,
|
|
||||||
timeout=30,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
err = (r.stderr or b"").decode("utf-8", "replace").strip()[:200]
|
|
||||||
print(f"publish fail ({remote}): {err}", file=sys.stderr, flush=True)
|
|
||||||
return None
|
|
||||||
if "/var/www/monitoring-sites/" in remote:
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
"ssh",
|
|
||||||
"-o",
|
|
||||||
"BatchMode=yes",
|
|
||||||
"-o",
|
|
||||||
"ConnectTimeout=8",
|
|
||||||
ssh_host,
|
|
||||||
(
|
|
||||||
f"chmod 755 -- {remote_dir!s} && chmod 644 -- {remote!s} && "
|
|
||||||
f"setfacl -m u:caddy:rx,m::r-x -- {remote_dir!s} && "
|
|
||||||
f"setfacl -m u:caddy:r,m::r-- -- {remote!s} || true"
|
|
||||||
),
|
|
||||||
],
|
|
||||||
capture_output=True,
|
|
||||||
timeout=15,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
"ssh",
|
|
||||||
"-o",
|
|
||||||
"BatchMode=yes",
|
|
||||||
"-o",
|
|
||||||
"ConnectTimeout=8",
|
|
||||||
ssh_host,
|
|
||||||
f"chmod 755 -- {remote_dir!s} && chmod 644 -- {remote!s} || true",
|
|
||||||
],
|
|
||||||
capture_output=True,
|
|
||||||
timeout=15,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
return f"{ssh_host}:{remote}"
|
|
||||||
|
|
||||||
|
|
||||||
def publish_html(doc: dict) -> str | None:
|
|
||||||
"""Render HTML and scp to Cyon (hacktivism) paths.
|
|
||||||
|
|
||||||
Env:
|
|
||||||
TALER_MONITORD_PUBLISH=0 to disable
|
|
||||||
TALER_MONITORD_PUBLISH_SSH=hacktivism
|
|
||||||
TALER_MONITORD_PUBLISH_PATH=… single path (overrides multi default)
|
|
||||||
TALER_MONITORD_PUBLISH_PATHS=comma-separated paths
|
|
||||||
Default paths: public_html/stacks + public_html/monitoring.hacktivism.ch
|
|
||||||
"""
|
|
||||||
if os.environ.get("TALER_MONITORD_PUBLISH", "1") in ("0", "false", "no", ""):
|
|
||||||
return None
|
|
||||||
ssh_host = os.environ.get("TALER_MONITORD_PUBLISH_SSH", "hacktivism").strip() or "hacktivism"
|
|
||||||
single = (os.environ.get("TALER_MONITORD_PUBLISH_PATH") or "").strip()
|
|
||||||
multi = (os.environ.get("TALER_MONITORD_PUBLISH_PATHS") or "").strip()
|
|
||||||
if single:
|
|
||||||
remotes = [single]
|
|
||||||
elif multi:
|
|
||||||
remotes = [p.strip() for p in multi.split(",") if p.strip()]
|
|
||||||
else:
|
|
||||||
remotes = [
|
|
||||||
"/home/hacktivi/public_html/stacks/index.html",
|
|
||||||
"/home/hacktivi/public_html/monitoring.hacktivism.ch/index.html",
|
|
||||||
]
|
|
||||||
html_text = H.render_html(doc)
|
|
||||||
with tempfile.NamedTemporaryFile("w", encoding="utf-8", suffix=".html", delete=False) as f:
|
|
||||||
f.write(html_text)
|
|
||||||
local = Path(f.name)
|
|
||||||
published: list[str] = []
|
|
||||||
try:
|
|
||||||
for remote in remotes:
|
|
||||||
got = _publish_one(local, ssh_host, remote)
|
|
||||||
if got:
|
|
||||||
published.append(got)
|
|
||||||
return "; ".join(published) if published else None
|
|
||||||
except (OSError, subprocess.TimeoutExpired) as e:
|
|
||||||
print(f"publish error: {e}", file=sys.stderr, flush=True)
|
|
||||||
return None
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
local.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def run_once(*, domains: Path, out_paths: list[Path] | None, public_only: bool) -> Path:
|
|
||||||
stacks = L.parse_domains_conf(domains)
|
|
||||||
prev = L.read_status(out_paths[0] if out_paths else None)
|
|
||||||
if not prev and out_paths:
|
|
||||||
for p in out_paths:
|
|
||||||
prev = L.read_status(p)
|
|
||||||
if prev:
|
|
||||||
break
|
|
||||||
doc = L.build_stack_status(stacks, prev_doc=prev, public_only=public_only)
|
|
||||||
paths = out_paths or L.status_json_paths()
|
|
||||||
path = L.write_stack_status(doc, paths)
|
|
||||||
# always keep a local HTML copy too
|
|
||||||
local_html = L.monitord_state_dir() / "index.html"
|
|
||||||
local_html.write_text(H.render_html(doc), encoding="utf-8")
|
|
||||||
published = publish_html(doc)
|
|
||||||
if published:
|
|
||||||
print(f"published {published}", flush=True)
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
ap = argparse.ArgumentParser(description="Collect Taler stack status for the detached UI")
|
|
||||||
ap.add_argument("--interval", type=float, default=float(os.environ.get("TALER_MONITORD_INTERVAL", "20")))
|
|
||||||
ap.add_argument("--once", action="store_true", help="Single probe cycle then exit")
|
|
||||||
ap.add_argument("--domains-conf", type=Path, default=None)
|
|
||||||
ap.add_argument("--out", type=Path, action="append", default=None, help="Output JSON path (repeatable)")
|
|
||||||
# Default on: include GOA / hacktivism.ch (taler.hacktivism.ch merchant). Opt out: --no-include-local
|
|
||||||
_incl = os.environ.get("TALER_MONITORD_INCLUDE_LOCAL", "1") not in ("0", "false", "no", "")
|
|
||||||
ap.add_argument(
|
|
||||||
"--include-local",
|
|
||||||
action=argparse.BooleanOptionalAction,
|
|
||||||
default=_incl,
|
|
||||||
help="Probe local=1 stacks (hacktivism GOA). Default on; --no-include-local to skip",
|
|
||||||
)
|
|
||||||
args = ap.parse_args()
|
|
||||||
if _ENV_LOADED:
|
|
||||||
print(f"env {_ENV_LOADED}", flush=True)
|
|
||||||
domains = args.domains_conf or L.default_domains_conf()
|
|
||||||
if not domains.is_file():
|
|
||||||
print(f"domains.conf not found: {domains}", file=sys.stderr)
|
|
||||||
return 2
|
|
||||||
public_only = not bool(args.include_local)
|
|
||||||
out_paths = list(args.out) if args.out else None
|
|
||||||
|
|
||||||
if args.once:
|
|
||||||
path = run_once(domains=domains, out_paths=out_paths, public_only=public_only)
|
|
||||||
print(path)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
_write_pid()
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
path = run_once(domains=domains, out_paths=out_paths, public_only=public_only)
|
|
||||||
print(f"ok {path}", flush=True)
|
|
||||||
except Exception as e: # keep loop alive
|
|
||||||
print(f"error {type(e).__name__}: {e}", file=sys.stderr, flush=True)
|
|
||||||
time.sleep(max(1.0, float(args.interval)))
|
|
||||||
finally:
|
|
||||||
_clear_pid()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
|
|
@ -99,7 +99,6 @@ Phases:
|
||||||
max-ladder max-search: highest withdrawable + highest payable (≤ available)
|
max-ladder max-search: highest withdrawable + highest payable (≤ available)
|
||||||
goa-ladder alias of ladder (legacy phase name; same check_amount_ladder.sh)
|
goa-ladder alias of ladder (legacy phase name; same check_amount_ladder.sh)
|
||||||
auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
|
auth401 merchant Basic-auth / case matrix (HTTP 401 paths; may create throwaway instance)
|
||||||
ansible-smoke taler-deployment site.yml syntax + optional podman tracks (TASK112)
|
|
||||||
aptdeploy koopa podman apt-src smoke: taler-merchant in
|
aptdeploy koopa podman apt-src smoke: taler-merchant in
|
||||||
koopa-taler-deploy-test-apt-src-trixie{,-testing}
|
koopa-taler-deploy-test-apt-src-trixie{,-testing}
|
||||||
surface REMOTE-ONLY public inventory (NOT in default/all/full):
|
surface REMOTE-ONLY public inventory (NOT in default/all/full):
|
||||||
|
|
@ -284,7 +283,7 @@ while [ $# -gt 0 ]; do
|
||||||
shift
|
shift
|
||||||
;;
|
;;
|
||||||
--no-probe) NO_PROBE=1; shift ;;
|
--no-probe) NO_PROBE=1; shift ;;
|
||||||
urls|inside|versions|sanity|server|e2e|zero-pay|zeropay|sticker-pay|template-zero|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|ansible-smoke|ansible_smoke|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;;
|
urls|inside|versions|sanity|server|e2e|zero-pay|zeropay|sticker-pay|template-zero|ladder|goa-ladder|max-ladder|maxladder|auth401|aptdeploy|apt-deploy|apt_src|surface|ecosystem|mattermost|mail|monpages|pages|devtesting|franken|fake-franken|fake_franken|all|full) PHASES+=("$1"); shift ;;
|
||||||
*)
|
*)
|
||||||
# bare domain shorthand: ./taler-monitoring.sh taler.net
|
# bare domain shorthand: ./taler-monitoring.sh taler.net
|
||||||
if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then
|
if [[ "$1" == *.* && "$1" != *://* && "$1" != -* ]]; then
|
||||||
|
|
@ -309,11 +308,6 @@ if [ -n "$BANK_OVERRIDE" ]; then BANK_PUBLIC="$BANK_OVERRIDE"; fi
|
||||||
if [ -n "$EXCHANGE_OVERRIDE" ]; then EXCHANGE_PUBLIC="$EXCHANGE_OVERRIDE"; fi
|
if [ -n "$EXCHANGE_OVERRIDE" ]; then EXCHANGE_PUBLIC="$EXCHANGE_OVERRIDE"; fi
|
||||||
if [ -n "$MERCHANT_OVERRIDE" ]; then MERCHANT_PUBLIC="$MERCHANT_OVERRIDE"; fi
|
if [ -n "$MERCHANT_OVERRIDE" ]; then MERCHANT_PUBLIC="$MERCHANT_OVERRIDE"; fi
|
||||||
|
|
||||||
# Wire backend (x-taler-bank vs EBICS) after CLI overrides — sets CHECK_BANK for EBICS
|
|
||||||
if [ "${TALER_SKIP_BANK_BACKEND_DETECT:-0}" != "1" ]; then
|
|
||||||
resolve_taler_bank_backend
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Only koopa may use SSH. Remote domains: public + optional e2e (no SSH).
|
# Only koopa may use SSH. Remote domains: public + optional e2e (no SSH).
|
||||||
if [ "${LOCAL_STACK:-1}" != "1" ]; then
|
if [ "${LOCAL_STACK:-1}" != "1" ]; then
|
||||||
SKIP_SSH=1
|
SKIP_SSH=1
|
||||||
|
|
@ -330,7 +324,6 @@ fi
|
||||||
# Export so check_*.sh (re-source lib) see the same targets via env
|
# Export so check_*.sh (re-source lib) see the same targets via env
|
||||||
export TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC
|
export TALER_DOMAIN BANK_PUBLIC EXCHANGE_PUBLIC MERCHANT_PUBLIC
|
||||||
export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE CHECK_LANDING
|
export EXPECT_CURRENCY SKIP_SSH LOCAL_STACK TALER_DOMAIN_PROBE CHECK_LANDING
|
||||||
export CHECK_BANK CHECK_BANK_SET TALER_BANK_BACKEND TALER_BANK_BACKEND_SET
|
|
||||||
export TALER_DOMAIN_FROM_CLI="${TALER_DOMAIN_FROM_CLI:-0}"
|
export TALER_DOMAIN_FROM_CLI="${TALER_DOMAIN_FROM_CLI:-0}"
|
||||||
export DOMAIN_SET="${DOMAIN_SET:-0}"
|
export DOMAIN_SET="${DOMAIN_SET:-0}"
|
||||||
export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE
|
export WITHDRAW_AMT PAY_AMT CREDIT_AMT MERCHANT_INSTANCE
|
||||||
|
|
@ -585,15 +578,9 @@ fi
|
||||||
printf 'target domain=%s\n' "${TALER_DOMAIN}"
|
printf 'target domain=%s\n' "${TALER_DOMAIN}"
|
||||||
printf ' ui_lang %s\n' "${TALER_MON_LANG:-en}"
|
printf ' ui_lang %s\n' "${TALER_MON_LANG:-en}"
|
||||||
printf ' locale %s\n' "${TALER_MON_LOCALE:-${TALER_DOMAIN_LOCALE:-?}}"
|
printf ' locale %s\n' "${TALER_MON_LOCALE:-${TALER_DOMAIN_LOCALE:-?}}"
|
||||||
if [ "${TALER_BANK_BACKEND:-}" = "ebics" ] || [ -z "${BANK_PUBLIC:-}" ]; then
|
printf ' bank %s\n' "$BANK_PUBLIC"
|
||||||
printf ' bank IBAN (EBICS — no public bank.*)\n'
|
|
||||||
else
|
|
||||||
printf ' bank %s\n' "${BANK_PUBLIC}"
|
|
||||||
fi
|
|
||||||
printf ' exchange %s\n' "$EXCHANGE_PUBLIC"
|
printf ' exchange %s\n' "$EXCHANGE_PUBLIC"
|
||||||
printf ' merchant %s\n' "$MERCHANT_PUBLIC"
|
printf ' merchant %s\n' "$MERCHANT_PUBLIC"
|
||||||
printf ' wire backend=%s CHECK_BANK=%s\n' \
|
|
||||||
"${TALER_BANK_BACKEND:-?}" "${CHECK_BANK:-1}"
|
|
||||||
printf ' currency expect=%s\n' "${EXPECT_CURRENCY:-any}"
|
printf ' currency expect=%s\n' "${EXPECT_CURRENCY:-any}"
|
||||||
printf ' phases %s\n' "${PHASES[*]}"
|
printf ' phases %s\n' "${PHASES[*]}"
|
||||||
printf ' flags LOCAL_STACK=%s SKIP_SSH=%s INSIDE_PODMAN=%s INSIDE_MODE=%s\n' \
|
printf ' flags LOCAL_STACK=%s SKIP_SSH=%s INSIDE_PODMAN=%s INSIDE_MODE=%s\n' \
|
||||||
|
|
@ -665,7 +652,6 @@ while [ "$_phase_idx" -lt "$_n_phases" ]; do
|
||||||
;;
|
;;
|
||||||
auth401) run_phase auth401 "$ROOT/check_auth401.sh" || ec=1 ;;
|
auth401) run_phase auth401 "$ROOT/check_auth401.sh" || ec=1 ;;
|
||||||
aptdeploy|apt-deploy|apt_src) run_phase aptdeploy "$ROOT/check_apt_deploy.sh" || ec=1 ;;
|
aptdeploy|apt-deploy|apt_src) run_phase aptdeploy "$ROOT/check_apt_deploy.sh" || ec=1 ;;
|
||||||
ansible-smoke|ansible_smoke) run_phase ansible-smoke "$ROOT/check_ansible_smoke.sh" || ec=1 ;;
|
|
||||||
surface|ecosystem) run_phase surface "$ROOT/check_surface.sh" || ec=1 ;;
|
surface|ecosystem) run_phase surface "$ROOT/check_surface.sh" || ec=1 ;;
|
||||||
mattermost) run_phase mattermost "$ROOT/check_mattermost.sh" || ec=1 ;;
|
mattermost) run_phase mattermost "$ROOT/check_mattermost.sh" || ec=1 ;;
|
||||||
mail) run_phase mail "$ROOT/check_mail.sh" || ec=1 ;;
|
mail) run_phase mail "$ROOT/check_mail.sh" || ec=1 ;;
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
Reference in a new issue