diff --git a/CLI-AUTOMATION-NOTES.md b/CLI-AUTOMATION-NOTES.md index 3cd311a..36aa315 100644 --- a/CLI-AUTOMATION-NOTES.md +++ b/CLI-AUTOMATION-NOTES.md @@ -5,7 +5,9 @@ stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring e2e/ladder). Ordered roughly by **how often they hurt automation** and how useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be. -Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md). +Companions: +- Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md) +- Git / suite tree → [`GIT-AUTOMATION-NOTES.md`](./GIT-AUTOMATION-NOTES.md) Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`. diff --git a/DEPENDENCIES.md b/DEPENDENCIES.md index 940dea3..192a897 100644 --- a/DEPENDENCIES.md +++ b/DEPENDENCIES.md @@ -7,7 +7,7 @@ | Binary | Package (zypper) | Role | |--------|------------------|------| | `bash` | (base) | runner | -| `curl` | `curl` | HTTPS/API probes | +| `curl` | `curl` | HTTPS/API probes + **monpages** FQDN checks | | `python3` | `python313-base` (provides `/usr/bin/python3`) | JSON, HTML site-gen, metrics | | `podman` | `podman` | **inside**: `podman exec` into taler-hacktivism* | | `git` | `git` | pin commit SHA in monitoring HTML footer | @@ -67,4 +67,4 @@ command -v qrencode zbarimg ## Franc Paysan hosts (francpaysan / stagepaysan) -Document package lists under **`francpaysan-admin-log`** when those host-agents are installed (same idea: curl, python3, podman if inside, qrencode/zbar for full urls). +FP hosts (`francpaysan` / `stagepaysan`): same idea — curl, python3, podman if inside, qrencode/zbar for full urls; suite at `~/src/taler-monitoring`. diff --git a/GIT-AUTOMATION-NOTES.md b/GIT-AUTOMATION-NOTES.md index 4ad80b3..d7f839c 100644 --- a/GIT-AUTOMATION-NOTES.md +++ b/GIT-AUTOMATION-NOTES.md @@ -1,122 +1,85 @@ -# Git / automation notes (taler-monitoring + mobile) +# Git / release notes — **taler-monitoring** -Notes for **automated checks** around the Taler stacks (GOA, stage -TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet -UI automation is separate tooling** and is **not** installed on the -default ops laptop yet. +How we commit, tag, and push this **standalone** suite +(`https://git.hacktivism.ch/hernani/taler-monitoring`). + +**Not here:** Android UI automation → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md) +**Not here:** wallet-cli e2e pain points → [`CLI-AUTOMATION-NOTES.md`](./CLI-AUTOMATION-NOTES.md) --- -## Git commit / push cadence (existing model) +## Canonical tree -Observed in `koopa-admin-log` / `francpaysan-admin-log` histories, plus ops preference: +| | | +|--|--| +| Clone | `~/src/taler-monitoring` | +| Remote | `https://git.hacktivism.ch/hernani/taler-monitoring.git` | +| Branch | `main` | +| Tags | `vX.Y` / `vX.Y.Z` (see [`VERSIONS.md`](./VERSIONS.md)) | +| Symlink (optional) | `~/taler-monitoring` → suite dir (`update-suite.sh`) | + +There is **no** `scripts/` subtree under an ops admin-log for this suite. +Host ops (Caddy, landings, day logs) stay in **koopa-admin-log** (separate repo). + +--- + +## Commit / push cadence | Rule | Practice | |------|----------| -| **Commits** | **Prefer frequent** small commits over rare large ones | -| **Interval** | Soft range **7–13 minutes** while a topic is open — **not** a fixed 9‑minute metronome; denser 0–2 min bursts for tiny follow-ups are fine | -| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …` | -| **Push** | **Only at the end of a work session** (`CADENCE_PUSH=1` or manual `git push`) — not after every commit | - -### Agent / human workflow - -1. Finish a small slice → commit (`area: …`). Repeat often. -2. Do **not** push after each commit. -3. When the topic/session is done → one `git push` (or `CADENCE_PUSH=1`). -4. Soft next-commit window: about **7–13 min** (jitter), sooner if the next slice is ready. - -### Helper script +| **Commits** | Prefer frequent small commits (`area: short description`) | +| **Areas** | e.g. `host-agent: …`, `site-gen: …`, `monpages: …`, `docs: …`, `release: …` | +| **Push** | At end of a work slice / session — not after every micro-commit | +| **Tags** | After a fix/feature lands on `main`, tag + push tag (e.g. `v1.3.3`) | ```bash -# dry-run -CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh - -# commit only if dirty (default: no push; skips secrets) -./scripts/git-cadence-commit-push.sh -./scripts/git-cadence-commit-push.sh ~/taler/src/francpaysan-admin-log - -# end of session — push commits that are already local -CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh - -# soft timer: only commit if last commit age ≥ random target in 7–13 min -CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh +cd ~/src/taler-monitoring +git status -sb +git add -A && git commit -m "area: description" +# session end: +git push origin main +git tag -a v1.3.3 -m "v1.3.3: …" +git push origin v1.3.3 ``` -Optional user cron (check every 7 min; **jitter** keeps effective gaps in **7–13 min**): +### Push from koopa (HTTPS + Forgejo token) + +Laptop SSH keys often cannot push to Forgejo; on **koopa** a token file is used +(ops-only, never commit it): ```bash -*/7 * * * * CADENCE_JITTER=1 $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1 +# example — do not print the token +TOK=$(tr -d '\n\r ' < ~/.config/forgejo-taler-monitoring.token) +git push "https://hernani:${TOK}@git.hacktivism.ch/hernani/taler-monitoring.git" main ``` -**Auth note:** `francpaysan-admin-log` (gnunet SSH) pushes OK. -`koopa-admin-log` (hacktivism HTTPS) needs credentials/SSH key before end-of-session push works from this host. +Prefer `git push origin …` once `credential.helper` / remote URL is set up. -## What runs without Android +### Host-agent auto-update -| Phase | Needs | -|-------|--------| -| `urls` / `sanity` | public HTTPS only | -| `inside` / `versions` | SSH (`koopa` or `francpaysan-stage-user`) | -| `e2e` / `ladder` | `taler-wallet-cli` + bank/merchant secrets | - -```bash -./taler-monitoring.sh -d stage.lefrancpaysan.ch urls -./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env -``` - -Mobile app checks (manual withdraw/pay, then automation) are **out of -band** until the software below is present. +`host-agent/update-suite.sh` does `git fetch` + `reset --hard` to `SUITE_GIT_REF` +(default `main`). Local uncommitted suite edits under `~/src/taler-monitoring` +can be wiped — keep work in a branch or commit first. Stack env lives only in +`~/.config/taler-monitoring/env` (never overwritten by git). --- -## Android automation software — **required** (not installed yet) +## What the suite runs (no Android) -To automate **GNU Taler Wallet** on Android (stable + master builds; -manual then auto withdraw; fixed + variable pay) against stage -TESTPAYSAN, the following must be available on the runner host. +| Phase | Needs | +|-------|--------| +| `urls` / `sanity` | public HTTPS | +| `inside` / `versions` | SSH / podman on local stacks | +| `monpages` | public FQDN of **monitoring HTML** (not merchant JSON code 21) | +| `e2e` / `ladder` | `taler-wallet-cli` + secrets | +| `surface` / `aptdeploy` | explicit only (timers on koopa) | -### Status on this host (2026-07-17) +```bash +./taler-monitoring.sh -d hacktivism.ch urls monpages +./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env +``` -| Component | Status | -|-----------|--------| -| Java (OpenJDK 21) | present (`/usr/bin/java`) | -| `aapt` / `aapt2` | present | -| **`adb`** (platform-tools) | **missing** (package was removed: `android-tools-adb` rc) | -| **Android SDK** (`ANDROID_HOME` / `ANDROID_SDK_ROOT`) | **missing** / empty | -| **Emulator + AVD** | **missing** | -| Appium / Maestro / `uiautomator2` | **missing** | -| scrcpy | missing (optional) | - -### Must install before mobile automation - -1. **Android platform-tools** — at least `adb` - - Debian: `android-tools-adb` (and preferably full SDK platform-tools) -2. **Android SDK command-line tools** — `sdkmanager`, platforms, build-tools - - set `ANDROID_HOME` (or `ANDROID_SDK_ROOT`) permanently -3. **System image + AVD** — for emulator runs (x86_64 or host-matching ABI) - - `emulator`, `avdmanager`, KVM/acceleration if available -4. **UI automation stack** (pick one, prefer the leaner path for wallet flows): - - **Maestro**, or - - **Appium** + **UiAutomator2** (+ Node/Python bindings as needed) -5. **Optional:** `scrcpy` for live view during manual debugging - -### Explicitly out of scope until freigabe - -- Installing production wallet APKs and driving real stage withdraw/pay - without a human go-ahead -- Committing secrets, device serials, or APK binaries into git - -### Operator still needs to supply - -- Stable / master **APK** (or store/build URLs) for the wallet -- USB debugging consent on a physical device, **or** agreement to use AVD only -- Which flows are in-scope first (withdraw communal QR → shop pay, …) - -### Suggested freigabe prompt (for an agent) - -See session notes: full copy-paste block under “Android-Automatisierung -vollständig einrichten” — install tooling, verify `adb devices` + emulator, -document paths; **no** real wallet payments until approved. +Mobile / GUI → **GUI-AUTOMATION-NOTES** and `android-test/`. --- @@ -125,74 +88,9 @@ document paths; **no** real wallet payments until approved. | Doc | Role | |-----|------| | [README.md](./README.md) | how to run `taler-monitoring.sh` | +| [VERSIONS.md](./VERSIONS.md) | release tags | | [TESTS.md](./TESTS.md) | grouped check IDs | | [secrets.env.example](./secrets.env.example) | e2e secrets layout | -| `francpaysan-admin-log/TODO.md` | stage wallet mobile test backlog | - -When Android tooling is installed, append a short “installed on …” -section here (date, `adb version`, AVD name, `ANDROID_HOME`). - ---- - -## Android pay smoke (published APK, no root preferred) - -Script: [`android-test/run-android-pay-smoke.sh`](./android-test/run-android-pay-smoke.sh) - -| Step | Detail | -|------|--------| -| APK | F-Droid **Taler Wallet 1.6.1** (`net.taler.wallet.fdroid_854.apk`) | -| Install | `adb install -r` (no root) | -| Withdraw | `taler://withdraw/…` from `…/intro/demo-withdraw.json` (GOA or stage) | -| Pay | stage: shop template order → `taler://pay/…`; GOA: `taler://pay-template/…/goa-shop/paivana` | -| Evidence | screenshots + logcat under `android-test/out/` | - -```bash -# device or emulator already booted -./android-test/run-android-pay-smoke.sh -STACK=goa ./android-test/run-android-pay-smoke.sh -STACK=stage ./android-test/run-android-pay-smoke.sh -``` - -**User-local emulator (no apt root if SDK already under `$HOME/Android/Sdk`):** - -```bash -# Default = headless (no host window); guest GLES via SwiftShader. -# Prefer the helper — same flags smokes use: -./android-test/start-android-emulator.sh --wait -./android-test/run-android-pay-smoke.sh -# Or let smoke auto-start the AVD (AUTO_START_EMULATOR=1). -# Windowed laptop: WINDOWED=1 ./android-test/start-android-emulator.sh --wait -``` - -**Limits (clementine 2026-07-17):** host RAM ~3.7 GiB → emulator often ANRs System UI; -wallet-core still receives withdraw intents (`prepareBankIntegratedWithdrawal` in logcat). -For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 GiB** free RAM. - -### Status snapshot (2026-07-17) - -| Item | Status | -|------|--------| -| `adb` | present (`android-sdk-platform-tools`) | -| F-Droid APK | downloaded under `android-test/apks/` | -| User SDK + emulator + AVD `TalerWallet34` | installed under `$HOME/Android/Sdk` (~5 GiB) | -| `adb install` wallet | **Success** on emulator-5554 | -| Deep-link withdraw | **delivered**; wallet-core started `prepareBankIntegratedWithdrawal` for GOA | -| Unattended UI confirm | flaky (ANR) on this host’s RAM | -| Source build `assembleFdroidDebug` | **OK** (2026-07-17, `taler-android@f65a976c0d4a`, ~12 min, APK ~59 MiB) | - -### From-source smoke (same tests as F-Droid) - -```bash -# single ref (default: inference fix branch) -./android-test/run-android-build-and-smoke.sh -BRANCH=master ./android-test/run-android-build-and-smoke.sh -# self-build published stable tag (parity with F-Droid 1.6.1 source) -BRANCH=wallet-1.6.1 LABEL=stable-self ./android-test/run-android-build-and-smoke.sh -# matrix: published APK + stable self-build + master (anti–weird-build) -./android-test/run-android-variant-matrix.sh -STACK=goa ./android-test/run-android-build-and-smoke.sh -# already built: -APK_PATH=$HOME/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \ - PKG=net.taler.wallet.fdroid.debug \ - ./android-test/run-android-pay-smoke.sh -``` +| [host-agent/README.md](./host-agent/README.md) | systemd timers / install | +| [android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md) | Android GUI automation | +| [CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md) | wallet-cli automation issues | diff --git a/README.md b/README.md index 4353947..953d1ea 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Standalone suite (formerly under host ops log). Canonical path: `~/src/taler-mon Host packages (koopa / runners): **[DEPENDENCIES.md](./DEPENDENCIES.md)**. Host-agents (GOA + FP; shared report pipeline): **[host-agent/README.md](./host-agent/README.md)**. Public HTML site-gen (same timeout/log/HTML defaults): **[site-gen/README.md](./site-gen/README.md)**. -FP install notes: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`. +FP host-agent: `host-agent/run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` + `host-agent/env/*.env.example` (see `host-agent/README.md`). Report for the **GOA** stack with boxed severity badges and **grouped test IDs** `area.group-NN` (e.g. `www.exchange-01`, `e2e.pay-03`): @@ -25,9 +25,9 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs** IDs reset per **group** inside an area — so “too many www tests” become `www.exchange-*`, `www.bank-*`, `www.landing-*`, etc. Catalog: **[TESTS.md](./TESTS.md)**. -Android / tooling: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**. +Git / tags / suite tree: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**. GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**. -CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**. +CLI / wallet-cli recurring issues: **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**. ```text ┌ OK ┐ www.exchange-01 exchange /config · HTTP 200 @@ -378,7 +378,7 @@ Phase **`monpages`** checks that each host’s monitoring URL is live HTML If pages are missing publicly: `sudo ~/koopa-caddy/apply-monitoring-live.sh` on koopa. -## Version on monitoring pages (v1.3+) +## Version on monitoring pages (v1.3+ / sticky bar) -The sticky bar shows the suite **tag** (e.g. `v1.3.1`) and links to that tag on Forgejo. +The sticky bar shows the suite **tag** (e.g. `v1.3.3`) and links to that tag on Forgejo. Planned **v1.4.0**: fuller L10n (fr-CH for FrancPaysan, de-CH for hacktivism/Taler CH). diff --git a/TESTS.md b/TESTS.md index a762f39..66b54cc 100644 --- a/TESTS.md +++ b/TESTS.md @@ -20,6 +20,7 @@ Every check line has a **global** run number and a **grouped** id: | **versions** | `check_versions.sh` | `outside` `inside` `compare` | | **aptdeploy** | `check_apt_deploy.sh` | `trixie` `trixie-testing` (koopa podman apt-src merchant smoke) | | **surface** | `check_surface.sh` | **remote-only** ecosystem / `-d` domain inventory (not in default/all/full) | +| **monpages** | `check_monitoring_pages.sh` | public monitoring HTML via FQDN (`https://host/monitoring/`, not merchant code 21) | | **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` | | **server** | `check_server.sh` | (flat `server-NN` or host groups) | | **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `atm` `settle` `pay` `shop` `paivana` `dig` `report` | diff --git a/VERSION b/VERSION index d5e98f7..785cda8 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.3.2 \ No newline at end of file +1.3.3 \ No newline at end of file diff --git a/VERSIONS.md b/VERSIONS.md index 8a58906..3478382 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -5,6 +5,7 @@ Release history for the standalone **taler-monitoring** suite | Tag | Date (UTC) | Notes | |-----|------------|--------| +| **v1.3.3** | 2026-07-18 | **Docs/fix:** `GIT-AUTOMATION-NOTES.md` is git/suite only (no Android blob, no dead `koopa-admin-log` cadence paths); Android stays in `android-test/GUI-AUTOMATION-NOTES.md`. monpages in TESTS; README/host-agent/DEPENDENCIES cleanup. | | **v1.3.2** | 2026-07-18 | **Bugfix:** drop remaining `koopa-admin-log` suite paths — host-agent/install, path unit, aptdeploy dedupe use **only** `~/src/taler-monitoring`. Reinstall systemd units. | | **v1.4.0** | *planned* | **L10n** for user-facing outputs that need localization: **fr-CH** (FrancPaysan, Suisse romande) and **de-CH** (hacktivism / Taler CH German). Not implemented yet — after **1.3.x**. | | **v1.3.1** | 2026-07-18 | **Bugfix:** silent skip when `DEPLOY_WWW_ROOT` not writable left public pages missing (merchant JSON code 21). **monpages** phase + host-agent post-check: verify monitoring HTML via **FQDN** (`https:///monitoring/` etc.). | @@ -53,5 +54,5 @@ sudo ~/koopa-caddy/apply-monitoring-live.sh ```bash git clone https://git.hacktivism.ch/hernani/taler-monitoring.git ~/src/taler-monitoring -cd ~/src/taler-monitoring && git checkout v1.3.2 +cd ~/src/taler-monitoring && git checkout v1.3.3 ``` diff --git a/android-test/GUI-AUTOMATION-NOTES.md b/android-test/GUI-AUTOMATION-NOTES.md index 18fe398..87cb77d 100644 --- a/android-test/GUI-AUTOMATION-NOTES.md +++ b/android-test/GUI-AUTOMATION-NOTES.md @@ -1,5 +1,10 @@ # GUI automation notes (Android wallet) + +Companion docs: +- CLI / wallet-cli: [`../CLI-AUTOMATION-NOTES.md`](../CLI-AUTOMATION-NOTES.md) +- Git / tags / suite tree: [`../GIT-AUTOMATION-NOTES.md`](../GIT-AUTOMATION-NOTES.md) + Canonical notes for **graphical** Android wallet tests against GOA / stage. --- diff --git a/host-agent/README.md b/host-agent/README.md index 9382487..fe42dbe 100644 --- a/host-agent/README.md +++ b/host-agent/README.md @@ -64,7 +64,7 @@ systemctl --user start taler-monitoring-aptdeploy.service # Caddy: handles only in taler.hacktivism.ch block (root) ``` -Env examples: **`env/*.env.example`**. FP docs: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`. +Env examples: **`env/*.env.example`**. FP: `run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` (same `run-host-report.sh` pipeline). ## GOA (koopa) — host-podman