# host-agent — reporting generation (GOA + FP + any stack)

**Global pipeline** lives in **`run-host-report.sh`** (all stacks share it):

| Behaviour | Default |
|-----------|---------|
| Suite refresh | `update-suite.sh` → `origin/main` |
| Wall clock | `RUN_TIMEOUT=600` |
| Log | line-buffered `run-*.log` (continuous write) |
| HTML | always (first run / empty tree too) + commit URL |
| Timeout notice | end of log + top jump on `/monitoring_err/` |

Thin wrappers only set stack defaults:

| Wrapper | Stack | Typical user |
|---------|-------|----------------|
| `run-hacktivism-monitoring.sh` | GOA `hacktivism.ch` | `hernani` @ koopa |
| `run-fp-stage-monitoring.sh` | FP stage | `stagepaysan` |
| `run-fp-prod-monitoring.sh` | FP prod | `francpaysan` |
| `run-aptdeploy.sh` | CLI: ensure + **aptdeploy** (no HTML) | local if hostname=koopa, else **ssh koopa-external** |
| `run-aptdeploy-monitoring.sh` | **aptdeploy** → `/taler-monitoring-aptdeploy*` | `hernani` @ koopa (4h timer, taler only) |
| `run-surface-monitoring.sh` | **surface** → `/taler-monitoring-surface*` | `hernani` @ koopa (hourly, taler only) |

### apt-src deploy tests (hacktivism `/monitoring`)

| Container | Mode |
|-----------|------|
| `koopa-taler-deploy-test-apt-src-trixie` | **fresh**: recreate when repo package fingerprint changes |
| `koopa-taler-deploy-test-apt-src-trixie-testing` | **fresh** for `trixie-testing` |
| `koopa-taler-deploy-test-apt-src-trixie-upgrade` | **upgrade**: initial install, then mytops-style `apt install` + `upgrade` |
| `koopa-taler-deploy-test-apt-src-trixie-testing-upgrade` | **upgrade** for testing suite |

```bash
# from ops workstation → triggers on koopa-external
./host-agent/run-aptdeploy.sh
./host-agent/run-aptdeploy.sh upgrade
./host-agent/run-aptdeploy.sh fresh
./host-agent/run-aptdeploy.sh check-only

# on koopa
./taler-monitoring.sh -d hacktivism.ch aptdeploy
```

Checks: package table, `httpd --version`, libtalerutil/`ldd` on failure, `systemctl` for
`taler-merchant.target`, basic local `/config` probe. No nginx/HTTPS in containers.

### surface + aptdeploy HTML (taler.hacktivism.ch only)

| URL | Role | Timer |
|-----|------|-------|
| `…/taler-monitoring-surface/` | ecosystem surface scan | 1h |
| `…/taler-monitoring-surface_err/` | errors + jump list | ″ |
| `…/taler-monitoring-aptdeploy/` | apt-src container deploy tests | 4h |
| `…/taler-monitoring-aptdeploy_err/` | errors + jump list | ″ |

Not on bank/exchange. Staging:
`~/monitoring-sites-staging/taler.hacktivism.ch/taler-monitoring-{surface,aptdeploy}*`.

```bash
systemctl --user enable --now taler-monitoring-surface.timer
systemctl --user enable --now taler-monitoring-aptdeploy.timer
systemctl --user start taler-monitoring-surface.service
systemctl --user start taler-monitoring-aptdeploy.service
# Caddy: handles only in taler.hacktivism.ch block (root)
```

Env examples: **`env/*.env.example`**. FP: `run-fp-prod-monitoring.sh` / `run-fp-stage-monitoring.sh` (same `run-host-report.sh` pipeline).

## GOA (koopa) — host-podman

Runs **as user `hernani`** (systemd user, linger). **Primary** mode:

- **`INSIDE_PODMAN=1`** / **`INSIDE_MODE=local-podman`**: access mode **`host-podman`** —  
  `podman exec` on this host into  
  `taler-hacktivism-bank` / `taler-hacktivism-exchange-ansible` / `taler-hacktivism`  
  (IDs `inside.host-*`, not `inside.ssh-*`).  
  From an **ops workstation**, leave `INSIDE_PODMAN` off → access mode **`ssh`** via `KOOPA_SSH`  
  (`inside.ssh-*`).
- Public **`urls` + `versions`** for the stack (`-d hacktivism.ch`).
- Writes console HTML for all three sites under  
  `~/monitoring-sites-staging/{bank,exchange,taler}.hacktivism.ch/`.

Caddy (host root) can serve that tree at `/monitoring` — you configure Caddy.

## Latest software + config isolation

**Source of truth is always Forgejo** (`git.hacktivism.ch`), not a laptop rsync copy.

Every host-agent / aptdeploy run:

1. **`update-suite.sh`**: `git fetch` + **`SUITE_UPDATE_MODE=reset`** (default) → `origin/main`  
   into `SUITE_DIR` (prefer `~/src/taler-monitoring`).
2. **Re-exec** host-agent scripts from that tree so the running code matches `COMMIT_URL`.
3. Local overrides only in **`~/.config/taler-monitoring/env`** — never overwritten by git.
4. HTML footer = exact commit:  
   `https://git.hacktivism.ch/hernani/taler-monitoring/src/commit/<full-sha>`
5. **`RUN_TIMEOUT=600`**, line-buffered logs, HTML always written.

If fetch fails → previous tree + WARN.  
**To ship features (aptdeploy, surface, …): push to `origin/main`.** Local-only edits vanish on the next reset.

If pull fails (no network / auth), the previous tree is used and a WARN is logged.

## Install (on koopa as hernani)

```bash
# once: local config (optional)
mkdir -p ~/.config/taler-monitoring
cp ~/src/taler-monitoring/host-agent/env.example \
   ~/.config/taler-monitoring/env

# ops workstation:
./host-agent/install-host-agent.sh --remote koopa-external

# or on koopa:
~/src/taler-monitoring/host-agent/install-host-agent.sh
```

Units:

| Unit | Role |
|------|------|
| `taler-monitoring-hacktivism.path` | fires on software stamp (`touch-software-stamp.sh`) |
| `taler-monitoring-hacktivism.timer` | every **4h** fallback |
| `taler-monitoring-hacktivism.service` | oneshot → `/monitoring*` (urls inside versions) |
| `taler-monitoring-surface.timer` | every **1h** |
| `taler-monitoring-surface.service` | → `/taler-monitoring-surface*` (taler only) |
| `taler-monitoring-mattermost.timer` | every **1h** |
| `taler-monitoring-mattermost.service` | → `/taler-monitoring-mattermost*` (mattermost.taler.net) |
| `taler-monitoring-mail.timer` | every **4h** |
| `taler-monitoring-mail.service` | → `/taler-monitoring-mail*` (firefly + pixel MX/SMTP/IMAP) |
| `taler-monitoring-aptdeploy.timer` | every **4h** |
| `taler-monitoring-aptdeploy.service` | → `/taler-monitoring-aptdeploy*` (taler only) |

## After package / image / landing upgrades

```bash
~/src/taler-monitoring/host-agent/touch-software-stamp.sh
# → path unit starts the service
```

## Manual

```bash
systemctl --user start taler-monitoring-hacktivism.service
systemctl --user status taler-monitoring-hacktivism.service --no-pager
journalctl --user -u taler-monitoring-hacktivism.service -n 50 --no-pager
ls ~/monitoring-sites-staging/bank.hacktivism.ch/monitoring_err/
```

## Relation to optional outside-only timer

| | host-agent (koopa) | outside-runner site-gen |
|--|--------------------|-------------------|
| Access | **podman on host** | public HTTPS only |
| Phases | urls **inside** versions | urls versions |
| Trigger | software path + 4h | *could* launchd 4h |
| Primary for GOA | **yes** | **off** (optional only) |

**outside-runner is not scheduled** right now (launchd disabled). Scripts under
`site-gen/` can still one-shot a public run if needed; see `site-gen/README.md`.

No secrets required for default phases. `secrets.env` optional for e2e later.

## Git commit on HTML pages

Header of each generated page includes a link like:

`https://git.hacktivism.ch/hernani/taler-monitoring/src/commit/<full-sha>`

That SHA is the tree **after** `update-suite.sh` for that run.

## Packages on koopa

See **[../DEPENDENCIES.md](../DEPENDENCIES.md)**.

**Default host-agent needs no new packages** if curl/python3/podman/git are present (they are).

**Root once** only for full QR checks in `urls`:

```bash
zypper in qrencode zbar
```

Without them, monitoring still runs; QR group WARNs and skips encode/decode.

Caddy `/var/www/monitoring-sites` is separate (you configured hernani write access).