monitoring: koopa login-health timer snapshot on SSH login
This commit is contained in:
parent
dc94b30fed
commit
e4dc9abac5
6 changed files with 739 additions and 0 deletions
102
scripts/login-health/README.md
Normal file
102
scripts/login-health/README.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
# login-health — status on SSH login (koopa)
|
||||
|
||||
When you **SSH into koopa**, show a short **pre-collected** health snapshot so you see whether **git.hacktivism.ch** (Forgejo), Taler GOA, Caddy edge, and the rest of the LIVE inventory are up.
|
||||
|
||||
Inventory source: `host/overview/LIVE.md`, `configs/ports.md`.
|
||||
|
||||
## Design: no probes at login
|
||||
|
||||
| Path | What runs |
|
||||
|------|-----------|
|
||||
| **systemd user timer** (every **5 min** + soon after boot) | `koopa-login-health collect` — podman, Caddy, loopback HTTP, write snapshot |
|
||||
| **SSH / bashrc** | `koopa-login-health show` only — **cat** last snapshot (milliseconds, no curl/podman) |
|
||||
|
||||
Snapshot files (`XDG_STATE_HOME`, default `~/.local/state/koopa-login-health/`):
|
||||
|
||||
| File | Content |
|
||||
|------|---------|
|
||||
| `status.compact` | login view (WARN/FAIL + overall) |
|
||||
| `status.verbose` | full line list |
|
||||
| `status.meta` | `overall=`, `collected_at=` (unix), RFC date |
|
||||
|
||||
If the snapshot is older than **`KOOPA_HEALTH_STALE_SEC`** (default **900** = 15 min), `show` prints a **stale** warning (timer likely down).
|
||||
|
||||
## Login must never break
|
||||
|
||||
| Guard | What it does |
|
||||
|-------|----------------|
|
||||
| **show only** | bashrc never runs `collect` |
|
||||
| **`KOOPA_HEALTH_FROM_LOGIN=1`** | always exit 0 |
|
||||
| **`timeout 2s`** | wall cap to cat a few KB |
|
||||
| **`\|\| true` / `set +e`** | hook cannot abort the shell |
|
||||
| **`bash -n`** on install | refuse a broken bashrc hook |
|
||||
|
||||
Skip: `KOOPA_SKIP_HEALTH=1 ssh koopa`
|
||||
|
||||
## What you see
|
||||
|
||||
```text
|
||||
koopa health koopa Wed, …
|
||||
git.hacktivism.ch = Forgejo :9024/:9200 · edge Caddy :9000/:9001
|
||||
OK all critical checks passed (42 ok)
|
||||
overall: OK ok=42 warn=0 fail=0
|
||||
checked: Wed, … (3m ago) [timer snapshot; not live at login]
|
||||
```
|
||||
|
||||
## Checks (on collect only)
|
||||
|
||||
| Area | What |
|
||||
|------|------|
|
||||
| Edge | `caddy.service`, proxy sockets |
|
||||
| Listen | critical + optional ports |
|
||||
| Podman | LIVE container names |
|
||||
| HTTP | loopback backends (2s curl, parallel) |
|
||||
| Git SSH | TCP `:9200` |
|
||||
|
||||
**Critical** (→ overall FAIL): merchant, exchange, bank, forgejo, edge/critical ports.
|
||||
**Optional** (→ WARN): castopod, bonfire, prime, bt, paivana, tops, tor, nym.
|
||||
|
||||
Read-only. No restarts, no secrets.
|
||||
|
||||
## Install (on koopa as `hernani`)
|
||||
|
||||
```bash
|
||||
cd ~/src/koopa/koopa-admin-log
|
||||
./scripts/login-health/install-login-health.sh
|
||||
```
|
||||
|
||||
Installs binary, user units, bashrc hook, enables timer, runs one collect.
|
||||
|
||||
Linger (so the timer runs after reboot without an open SSH session):
|
||||
|
||||
```bash
|
||||
sudo loginctl enable-linger hernani
|
||||
```
|
||||
|
||||
## Manual
|
||||
|
||||
```bash
|
||||
koopa-login-health # show compact snapshot
|
||||
koopa-login-health show -v # verbose snapshot
|
||||
koopa-login-health show -q # overall (+ stale note)
|
||||
koopa-login-health collect # refresh now (same as the timer unit)
|
||||
systemctl --user start koopa-login-health.service
|
||||
systemctl --user status koopa-login-health.timer
|
||||
```
|
||||
|
||||
## Uninstall
|
||||
|
||||
```bash
|
||||
systemctl --user disable --now koopa-login-health.timer
|
||||
rm -f ~/.config/systemd/user/koopa-login-health.{service,timer}
|
||||
systemctl --user daemon-reload
|
||||
rm -f ~/.local/bin/koopa-login-health
|
||||
# remove marked block in ~/.bashrc
|
||||
rm -rf ~/.local/state/koopa-login-health
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- Grok workflow `koopa-services-health` (laptop public HTTPS + generated mon pages)
|
||||
- `scripts/taler-sanity/check_stack-health.sh` (deeper Taler, often root)
|
||||
- units: `configs/systemd/user/koopa-login-health.{service,timer}`
|
||||
Loading…
Add table
Add a link
Reference in a new issue