monitoring: koopa login-health timer snapshot on SSH login

This commit is contained in:
Hernâni Marques 2026-08-06 17:12:00 +02:00
parent dc94b30fed
commit e4dc9abac5
No known key found for this signature in database
GPG key ID: CB5738652768F7E9
6 changed files with 739 additions and 0 deletions

View file

@ -0,0 +1,102 @@
# login-health — status on SSH login (koopa)
When you **SSH into koopa**, show a short **pre-collected** health snapshot so you see whether **git.hacktivism.ch** (Forgejo), Taler GOA, Caddy edge, and the rest of the LIVE inventory are up.
Inventory source: `host/overview/LIVE.md`, `configs/ports.md`.
## Design: no probes at login
| Path | What runs |
|------|-----------|
| **systemd user timer** (every **5 min** + soon after boot) | `koopa-login-health collect` — podman, Caddy, loopback HTTP, write snapshot |
| **SSH / bashrc** | `koopa-login-health show` only — **cat** last snapshot (milliseconds, no curl/podman) |
Snapshot files (`XDG_STATE_HOME`, default `~/.local/state/koopa-login-health/`):
| File | Content |
|------|---------|
| `status.compact` | login view (WARN/FAIL + overall) |
| `status.verbose` | full line list |
| `status.meta` | `overall=`, `collected_at=` (unix), RFC date |
If the snapshot is older than **`KOOPA_HEALTH_STALE_SEC`** (default **900** = 15 min), `show` prints a **stale** warning (timer likely down).
## Login must never break
| Guard | What it does |
|-------|----------------|
| **show only** | bashrc never runs `collect` |
| **`KOOPA_HEALTH_FROM_LOGIN=1`** | always exit 0 |
| **`timeout 2s`** | wall cap to cat a few KB |
| **`\|\| true` / `set +e`** | hook cannot abort the shell |
| **`bash -n`** on install | refuse a broken bashrc hook |
Skip: `KOOPA_SKIP_HEALTH=1 ssh koopa`
## What you see
```text
koopa health koopa Wed, …
git.hacktivism.ch = Forgejo :9024/:9200 · edge Caddy :9000/:9001
OK all critical checks passed (42 ok)
overall: OK ok=42 warn=0 fail=0
checked: Wed, … (3m ago) [timer snapshot; not live at login]
```
## Checks (on collect only)
| Area | What |
|------|------|
| Edge | `caddy.service`, proxy sockets |
| Listen | critical + optional ports |
| Podman | LIVE container names |
| HTTP | loopback backends (2s curl, parallel) |
| Git SSH | TCP `:9200` |
**Critical** (→ overall FAIL): merchant, exchange, bank, forgejo, edge/critical ports.
**Optional** (→ WARN): castopod, bonfire, prime, bt, paivana, tops, tor, nym.
Read-only. No restarts, no secrets.
## Install (on koopa as `hernani`)
```bash
cd ~/src/koopa/koopa-admin-log
./scripts/login-health/install-login-health.sh
```
Installs binary, user units, bashrc hook, enables timer, runs one collect.
Linger (so the timer runs after reboot without an open SSH session):
```bash
sudo loginctl enable-linger hernani
```
## Manual
```bash
koopa-login-health # show compact snapshot
koopa-login-health show -v # verbose snapshot
koopa-login-health show -q # overall (+ stale note)
koopa-login-health collect # refresh now (same as the timer unit)
systemctl --user start koopa-login-health.service
systemctl --user status koopa-login-health.timer
```
## Uninstall
```bash
systemctl --user disable --now koopa-login-health.timer
rm -f ~/.config/systemd/user/koopa-login-health.{service,timer}
systemctl --user daemon-reload
rm -f ~/.local/bin/koopa-login-health
# remove marked block in ~/.bashrc
rm -rf ~/.local/state/koopa-login-health
```
## Related
- Grok workflow `koopa-services-health` (laptop public HTTPS + generated mon pages)
- `scripts/taler-sanity/check_stack-health.sh` (deeper Taler, often root)
- units: `configs/systemd/user/koopa-login-health.{service,timer}`