hernani/taler-monitoring
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

release 1.3.3: fix GIT notes vs GUI notes; docs cleanup

Browse source
This commit is contained in:
Hernâni Marques 2026-07-18 19:43:49 +02:00
parent f40f593ba1
commit 43494c1c8c
No known key found for this signature in database
GPG key ID: CB5738652768F7E9
9 changed files with 82 additions and 175 deletions
Download patch file Download diff file Expand all files Collapse all files

4
CLI-AUTOMATION-NOTES.md
View file

@ -5,7 +5,9 @@ stage TESTPAYSAN via **CLI** (`taler-wallet-cli`, bank/merchant HTTP, monitoring
e2e/ladder). Ordered roughly by **how often they hurt automation** and how
useful a fix in **wallet-cli / wallet-core** (or clearer APIs) would be.
Companion: Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md).
Companions:
- Android GUI → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md)
- Git / suite tree → [`GIT-AUTOMATION-NOTES.md`](./GIT-AUTOMATION-NOTES.md)
Scripts: `check_e2e.sh`, `check_goa_ladder.sh`, `taler-monitoring.sh`.

4
DEPENDENCIES.md
View file

@ -7,7 +7,7 @@
| Binary | Package (zypper) | Role |
|--------|------------------|------|
| `bash` | (base) | runner |
| `curl` | `curl` | HTTPS/API probes |
| `curl` | `curl` | HTTPS/API probes + **monpages** FQDN checks |
| `python3` | `python313-base` (provides `/usr/bin/python3`) | JSON, HTML site-gen, metrics |
| `podman` | `podman` | **inside**: `podman exec` into taler-hacktivism* |
| `git` | `git` | pin commit SHA in monitoring HTML footer |
@ -67,4 +67,4 @@ command -v qrencode zbarimg
## Franc Paysan hosts (francpaysan / stagepaysan)
Document package lists under **`francpaysan-admin-log`** when those host-agents are installed (same idea: curl, python3, podman if inside, qrencode/zbar for full urls).
FP hosts (`francpaysan` / `stagepaysan`): same idea — curl, python3, podman if inside, qrencode/zbar for full urls; suite at `~/src/taler-monitoring`.

226
GIT-AUTOMATION-NOTES.md
View file

@ -1,122 +1,85 @@
# Git / automation notes (taler-monitoring + mobile)
# Git / release notes — **taler-monitoring**
Notes for **automated checks** around the Taler stacks (GOA, stage
TESTPAYSAN, …). CLI monitoring lives in this directory; **Android wallet
UI automation is separate tooling** and is **not** installed on the
default ops laptop yet.
How we commit, tag, and push this **standalone** suite
(`https://git.hacktivism.ch/hernani/taler-monitoring`).
**Not here:** Android UI automation → [`android-test/GUI-AUTOMATION-NOTES.md`](./android-test/GUI-AUTOMATION-NOTES.md)
**Not here:** wallet-cli e2e pain points → [`CLI-AUTOMATION-NOTES.md`](./CLI-AUTOMATION-NOTES.md)
---
## Git commit / push cadence (existing model)
## Canonical tree
Observed in `koopa-admin-log` / `francpaysan-admin-log` histories, plus ops preference:
| | |
|--|--|
| Clone | `~/src/taler-monitoring` |
| Remote | `https://git.hacktivism.ch/hernani/taler-monitoring.git` |
| Branch | `main` |
| Tags | `vX.Y` / `vX.Y.Z` (see [`VERSIONS.md`](./VERSIONS.md)) |
| Symlink (optional) | `~/taler-monitoring` → suite dir (`update-suite.sh`) |
There is **no** `scripts/` subtree under an ops admin-log for this suite.
Host ops (Caddy, landings, day logs) stay in **koopa-admin-log** (separate repo).
---
## Commit / push cadence
| Rule | Practice |
|------|----------|
| **Commits** | **Prefer frequent** small commits over rare large ones |
| **Interval** | Soft range **7–13 minutes** while a topic is open — **not** a fixed 9‑minute metronome; denser 0–2 min bursts for tiny follow-ups are fine |
| **Message** | `area: short description` — e.g. `monitoring: …`, `landings: …`, `docs: …` |
| **Push** | **Only at the end of a work session** (`CADENCE_PUSH=1` or manual `git push`) — not after every commit |
### Agent / human workflow
1. Finish a small slice → commit (`area: …`). Repeat often.
2. Do **not** push after each commit.
3. When the topic/session is done → one `git push` (or `CADENCE_PUSH=1`).
4. Soft next-commit window: about **7–13 min** (jitter), sooner if the next slice is ready.
### Helper script
| **Commits** | Prefer frequent small commits (`area: short description`) |
| **Areas** | e.g. `host-agent: …`, `site-gen: …`, `monpages: …`, `docs: …`, `release: …` |
| **Push** | At end of a work slice / session — not after every micro-commit |
| **Tags** | After a fix/feature lands on `main`, tag + push tag (e.g. `v1.3.3`) |
```bash
# dry-run
CADENCE_DRY=1 ./scripts/git-cadence-commit-push.sh
# commit only if dirty (default: no push; skips secrets)
./scripts/git-cadence-commit-push.sh
./scripts/git-cadence-commit-push.sh ~/taler/src/francpaysan-admin-log
# end of session — push commits that are already local
CADENCE_PUSH=1 ./scripts/git-cadence-commit-push.sh
# soft timer: only commit if last commit age ≥ random target in 7–13 min
CADENCE_JITTER=1 ./scripts/git-cadence-commit-push.sh
cd ~/src/taler-monitoring
git status -sb
git add -A && git commit -m "area: description"
# session end:
git push origin main
git tag -a v1.3.3 -m "v1.3.3: …"
git push origin v1.3.3
```
Optional user cron (check every 7 min; **jitter** keeps effective gaps in **7–13 min**):
### Push from koopa (HTTPS + Forgejo token)
Laptop SSH keys often cannot push to Forgejo; on **koopa** a token file is used
(ops-only, never commit it):
```bash
*/7 * * * * CADENCE_JITTER=1 $HOME/src/koopa/koopa-admin-log/scripts/git-cadence-commit-push.sh >>$HOME/.cache/git-cadence-koopa.log 2>&1
# example — do not print the token
TOK=$(tr -d '\n\r ' < ~/.config/forgejo-taler-monitoring.token)
git push "https://hernani:${TOK}@git.hacktivism.ch/hernani/taler-monitoring.git" main
```
**Auth note:** `francpaysan-admin-log` (gnunet SSH) pushes OK.
`koopa-admin-log` (hacktivism HTTPS) needs credentials/SSH key before end-of-session push works from this host.
Prefer `git push origin …` once `credential.helper` / remote URL is set up.
## What runs without Android
### Host-agent auto-update
| Phase | Needs |
|-------|--------|
| `urls` / `sanity` | public HTTPS only |
| `inside` / `versions` | SSH (`koopa` or `francpaysan-stage-user`) |
| `e2e` / `ladder` | `taler-wallet-cli` + bank/merchant secrets |
```bash
./taler-monitoring.sh -d stage.lefrancpaysan.ch urls
./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env
```
Mobile app checks (manual withdraw/pay, then automation) are **out of
band** until the software below is present.
`host-agent/update-suite.sh` does `git fetch` + `reset --hard` to `SUITE_GIT_REF`
(default `main`). Local uncommitted suite edits under `~/src/taler-monitoring`
can be wiped — keep work in a branch or commit first. Stack env lives only in
`~/.config/taler-monitoring/env` (never overwritten by git).
---
## Android automation software — **required** (not installed yet)
## What the suite runs (no Android)
To automate **GNU Taler Wallet** on Android (stable + master builds;
manual then auto withdraw; fixed + variable pay) against stage
TESTPAYSAN, the following must be available on the runner host.
| Phase | Needs |
|-------|--------|
| `urls` / `sanity` | public HTTPS |
| `inside` / `versions` | SSH / podman on local stacks |
| `monpages` | public FQDN of **monitoring HTML** (not merchant JSON code 21) |
| `e2e` / `ladder` | `taler-wallet-cli` + secrets |
| `surface` / `aptdeploy` | explicit only (timers on koopa) |
### Status on this host (2026-07-17)
```bash
./taler-monitoring.sh -d hacktivism.ch urls monpages
./taler-monitoring.sh -d stage.lefrancpaysan.ch e2e # after secrets.env
```
| Component | Status |
|-----------|--------|
| Java (OpenJDK 21) | present (`/usr/bin/java`) |
| `aapt` / `aapt2` | present |
| **`adb`** (platform-tools) | **missing** (package was removed: `android-tools-adb` rc) |
| **Android SDK** (`ANDROID_HOME` / `ANDROID_SDK_ROOT`) | **missing** / empty |
| **Emulator + AVD** | **missing** |
| Appium / Maestro / `uiautomator2` | **missing** |
| scrcpy | missing (optional) |
### Must install before mobile automation
1. **Android platform-tools** — at least `adb`
- Debian: `android-tools-adb` (and preferably full SDK platform-tools)
2. **Android SDK command-line tools** — `sdkmanager`, platforms, build-tools
- set `ANDROID_HOME` (or `ANDROID_SDK_ROOT`) permanently
3. **System image + AVD** — for emulator runs (x86_64 or host-matching ABI)
- `emulator`, `avdmanager`, KVM/acceleration if available
4. **UI automation stack** (pick one, prefer the leaner path for wallet flows):
- **Maestro**, or
- **Appium** + **UiAutomator2** (+ Node/Python bindings as needed)
5. **Optional:** `scrcpy` for live view during manual debugging
### Explicitly out of scope until freigabe
- Installing production wallet APKs and driving real stage withdraw/pay
without a human go-ahead
- Committing secrets, device serials, or APK binaries into git
### Operator still needs to supply
- Stable / master **APK** (or store/build URLs) for the wallet
- USB debugging consent on a physical device, **or** agreement to use AVD only
- Which flows are in-scope first (withdraw communal QR → shop pay, …)
### Suggested freigabe prompt (for an agent)
See session notes: full copy-paste block under “Android-Automatisierung
vollständig einrichten” — install tooling, verify `adb devices` + emulator,
document paths; **no** real wallet payments until approved.
Mobile / GUI → **GUI-AUTOMATION-NOTES** and `android-test/`.
---
@ -125,74 +88,9 @@ document paths; **no** real wallet payments until approved.
| Doc | Role |
|-----|------|
| [README.md](./README.md) | how to run `taler-monitoring.sh` |
| [VERSIONS.md](./VERSIONS.md) | release tags |
| [TESTS.md](./TESTS.md) | grouped check IDs |
| [secrets.env.example](./secrets.env.example) | e2e secrets layout |
| `francpaysan-admin-log/TODO.md` | stage wallet mobile test backlog |
When Android tooling is installed, append a short “installed on …”
section here (date, `adb version`, AVD name, `ANDROID_HOME`).
---
## Android pay smoke (published APK, no root preferred)
Script: [`android-test/run-android-pay-smoke.sh`](./android-test/run-android-pay-smoke.sh)
| Step | Detail |
|------|--------|
| APK | F-Droid **Taler Wallet 1.6.1** (`net.taler.wallet.fdroid_854.apk`) |
| Install | `adb install -r` (no root) |
| Withdraw | `taler://withdraw/…` from `…/intro/demo-withdraw.json` (GOA or stage) |
| Pay | stage: shop template order → `taler://pay/…`; GOA: `taler://pay-template/…/goa-shop/paivana` |
| Evidence | screenshots + logcat under `android-test/out/` |
```bash
# device or emulator already booted
./android-test/run-android-pay-smoke.sh
STACK=goa ./android-test/run-android-pay-smoke.sh
STACK=stage ./android-test/run-android-pay-smoke.sh
```
**User-local emulator (no apt root if SDK already under `$HOME/Android/Sdk`):**
```bash
# Default = headless (no host window); guest GLES via SwiftShader.
# Prefer the helper — same flags smokes use:
./android-test/start-android-emulator.sh --wait
./android-test/run-android-pay-smoke.sh
# Or let smoke auto-start the AVD (AUTO_START_EMULATOR=1).
# Windowed laptop: WINDOWED=1 ./android-test/start-android-emulator.sh --wait
```
**Limits (clementine 2026-07-17):** host RAM ~3.7 GiB → emulator often ANRs System UI;
wallet-core still receives withdraw intents (`prepareBankIntegratedWithdrawal` in logcat).
For reliable full UI confirm/pay use a physical phone or a host with **≥6–8 GiB** free RAM.
### Status snapshot (2026-07-17)
| Item | Status |
|------|--------|
| `adb` | present (`android-sdk-platform-tools`) |
| F-Droid APK | downloaded under `android-test/apks/` |
| User SDK + emulator + AVD `TalerWallet34` | installed under `$HOME/Android/Sdk` (~5 GiB) |
| `adb install` wallet | **Success** on emulator-5554 |
| Deep-link withdraw | **delivered**; wallet-core started `prepareBankIntegratedWithdrawal` for GOA |
| Unattended UI confirm | flaky (ANR) on this host’s RAM |
| Source build `assembleFdroidDebug` | **OK** (2026-07-17, `taler-android@f65a976c0d4a`, ~12 min, APK ~59 MiB) |
### From-source smoke (same tests as F-Droid)
```bash
# single ref (default: inference fix branch)
./android-test/run-android-build-and-smoke.sh
BRANCH=master ./android-test/run-android-build-and-smoke.sh
# self-build published stable tag (parity with F-Droid 1.6.1 source)
BRANCH=wallet-1.6.1 LABEL=stable-self ./android-test/run-android-build-and-smoke.sh
# matrix: published APK + stable self-build + master (anti–weird-build)
./android-test/run-android-variant-matrix.sh
STACK=goa ./android-test/run-android-build-and-smoke.sh
# already built:
APK_PATH=$HOME/taler/taler-android/wallet/build/outputs/apk/fdroid/debug/wallet-fdroid-debug.apk \
PKG=net.taler.wallet.fdroid.debug \
./android-test/run-android-pay-smoke.sh
```
| [host-agent/README.md](./host-agent/README.md) | systemd timers / install |
| [android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md) | Android GUI automation |
| [CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md) | wallet-cli automation issues |

10
README.md
View file

@ -9,7 +9,7 @@ Standalone suite (formerly under host ops log). Canonical path: `~/src/taler-mon
Host packages (koopa / runners): **[DEPENDENCIES.md](./DEPENDENCIES.md)**.
Host-agents (GOA + FP; shared report pipeline): **[host-agent/README.md](./host-agent/README.md)**.
Public HTML site-gen (same timeout/log/HTML defaults): **[site-gen/README.md](./site-gen/README.md)**.
FP install notes: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
FP host-agent: `host-agent/run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` + `host-agent/env/*.env.example` (see `host-agent/README.md`).
Report for the **GOA** stack with boxed severity badges and **grouped test IDs**
`area.group-NN` (e.g. `www.exchange-01`, `e2e.pay-03`):
@ -25,9 +25,9 @@ Report for the **GOA** stack with boxed severity badges and **grouped test IDs**
IDs reset per **group** inside an area — so “too many www tests” become
`www.exchange-*`, `www.bank-*`, `www.landing-*`, etc.
Catalog: **[TESTS.md](./TESTS.md)**.
Android / tooling: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**.
Git / tags / suite tree: **[GIT-AUTOMATION-NOTES.md](./GIT-AUTOMATION-NOTES.md)**.
GUI Android notes: **[android-test/GUI-AUTOMATION-NOTES.md](./android-test/GUI-AUTOMATION-NOTES.md)**.
CLI / wallet-cli recurring issues (for upstream improvements): **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**.
CLI / wallet-cli recurring issues: **[CLI-AUTOMATION-NOTES.md](./CLI-AUTOMATION-NOTES.md)**.
```text
┌ OK ┐ www.exchange-01 exchange /config · HTTP 200
@ -378,7 +378,7 @@ Phase **`monpages`** checks that each host’s monitoring URL is live HTML
If pages are missing publicly: `sudo ~/koopa-caddy/apply-monitoring-live.sh` on koopa.
## Version on monitoring pages (v1.3+)
## Version on monitoring pages (v1.3+ / sticky bar)
The sticky bar shows the suite **tag** (e.g. `v1.3.1`) and links to that tag on Forgejo.
The sticky bar shows the suite **tag** (e.g. `v1.3.3`) and links to that tag on Forgejo.
Planned **v1.4.0**: fuller L10n (fr-CH for FrancPaysan, de-CH for hacktivism/Taler CH).

1
TESTS.md
View file

@ -20,6 +20,7 @@ Every check line has a **global** run number and a **grouped** id:
| **versions** | `check_versions.sh` | `outside` `inside` `compare` |
| **aptdeploy** | `check_apt_deploy.sh` | `trixie` `trixie-testing` (koopa podman apt-src merchant smoke) |
| **surface** | `check_surface.sh` | **remote-only** ecosystem / `-d` domain inventory (not in default/all/full) |
| **monpages** | `check_monitoring_pages.sh` | public monitoring HTML via FQDN (`https://host/monitoring/`, not merchant code 21) |
| **sanity** | `check_sanity.sh` | `bank` `exchange` `merchant` |
| **server** | `check_server.sh` | (flat `server-NN` or host groups) |
| **e2e** | `check_e2e.sh` | `prereq` `load` `bank` `wallet` `atm` `settle` `pay` `shop` `paivana` `dig` `report` |

2
VERSION
View file

@ -1 +1 @@
1.3.2
1.3.3

3
VERSIONS.md
View file

@ -5,6 +5,7 @@ Release history for the standalone **taler-monitoring** suite
| Tag | Date (UTC) | Notes |
|-----|------------|--------|
| **v1.3.3** | 2026-07-18 | **Docs/fix:** `GIT-AUTOMATION-NOTES.md` is git/suite only (no Android blob, no dead `koopa-admin-log` cadence paths); Android stays in `android-test/GUI-AUTOMATION-NOTES.md`. monpages in TESTS; README/host-agent/DEPENDENCIES cleanup. |
| **v1.3.2** | 2026-07-18 | **Bugfix:** drop remaining `koopa-admin-log` suite paths — host-agent/install, path unit, aptdeploy dedupe use **only** `~/src/taler-monitoring`. Reinstall systemd units. |
| **v1.4.0** | *planned* | **L10n** for user-facing outputs that need localization: **fr-CH** (FrancPaysan, Suisse romande) and **de-CH** (hacktivism / Taler CH German). Not implemented yet — after **1.3.x**. |
| **v1.3.1** | 2026-07-18 | **Bugfix:** silent skip when `DEPLOY_WWW_ROOT` not writable left public pages missing (merchant JSON code 21). **monpages** phase + host-agent post-check: verify monitoring HTML via **FQDN** (`https://<host>/monitoring/` etc.). |
@ -53,5 +54,5 @@ sudo ~/koopa-caddy/apply-monitoring-live.sh
```bash
git clone https://git.hacktivism.ch/hernani/taler-monitoring.git ~/src/taler-monitoring
cd ~/src/taler-monitoring && git checkout v1.3.2
cd ~/src/taler-monitoring && git checkout v1.3.3
```

5
android-test/GUI-AUTOMATION-NOTES.md
View file

@ -1,5 +1,10 @@
# GUI automation notes (Android wallet)
Companion docs:
- CLI / wallet-cli: [`../CLI-AUTOMATION-NOTES.md`](../CLI-AUTOMATION-NOTES.md)
- Git / tags / suite tree: [`../GIT-AUTOMATION-NOTES.md`](../GIT-AUTOMATION-NOTES.md)
Canonical notes for **graphical** Android wallet tests against GOA / stage.
---

2
host-agent/README.md
View file

@ -64,7 +64,7 @@ systemctl --user start taler-monitoring-aptdeploy.service
# Caddy: handles only in taler.hacktivism.ch block (root)
```
Env examples: **`env/*.env.example`**. FP docs: `francpaysan-admin-log/docs/taler-monitoring-host-agents.md`.
Env examples: **`env/*.env.example`**. FP: `run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` (same `run-host-report.sh` pipeline).
## GOA (koopa) — host-podman