docs: update goa-regio-ng login-health monitoring

scripts/firecuda-probe/README.md

@@ -0,0 +1,80 @@
+# firecuda-probe — external last-alive from koopa
+
+**Why:** firecuda panics (e.g. 1GbE PCIe completion timeout 2026-09-20) and hard hangs leave the internal HB dead with the box. External ICMP+SSH from **koopa** distinguishes:
+
+| Pattern | Likely meaning |
+|---------|----------------|
+| ICMP fail + SSH fail, long gap | hang, power cut, or NIC/kernel down |
+| ICMP ok + SSH fail | userspace/sshd stuck, or max lock |
+| both ok, HB stamp stale vs wall | rare clock skew / forced-cmd path oddity / HB LaunchDaemon stopped |
+| both ok, HB fresh | alive |
+
+Log is **hernani-readable** on koopa (`~/.local/state/firecuda-probe/`).
+SSH key is **max-only** on firecuda, restricted to koopa LAN IP + forced `cat` of HB mirror.
+
+Requires firecuda crash-debug HB live (`/usr/local/var/lib/firecuda/heartbeat.last`) — see `firecuda-admin-log/configs/crash-debug/`.
+
+## Pieces
+
+| Piece | Path |
+|-------|------|
+| Script | `scripts/firecuda-probe/koopa-firecuda-probe.sh` → `~/.local/bin/koopa-firecuda-probe` |
+| Timer | `configs/systemd/user/koopa-firecuda-probe.{service,timer}` (~15 s) |
+| Key | `~/.ssh/id_ed25519_firecuda_probe` (hernani on koopa) |
+| Log | `~/.local/state/firecuda-probe/firecuda-probe.log` (+ `.last` / `.meta`) |
+| SSH Host | `firecuda-probe` → `max@192.168.100.84` with that IdentityFile |
+
+## Install (koopa as hernani) — Freigabe
+
+Laptop SoT: `$HOME/git/admin-logs/local/koopa-admin-log`. Live mirror: `~/src/koopa/koopa-admin-log` (no git commit on koopa).
+
+```bash
+# on koopa after syncing the tree:
+cd ~/src/koopa/koopa-admin-log
+./scripts/firecuda-probe/install-firecuda-probe.sh --keygen
+# copy the printed authorized_keys line → firecuda max (see below)
+./scripts/firecuda-probe/install-firecuda-probe.sh   # units + first probe
+```
+
+Linger already yes for hernani → timer survives logout.
+
+## firecuda max authorized_keys (one line)
+
+After `--keygen`, install prints the exact line. Template:
+
+```text
+from="192.168.100.95",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,command="/bin/cat /usr/local/var/lib/firecuda/heartbeat.last" ssh-ed25519 <PUBKEY> koopa-firecuda-probe@hernani-koopa
+```
+
+- **User:** `max` only (not root).
+- **from=:** only koopa LAN `192.168.100.95`.
+- **forced command:** read-only HB mirror (world-readable after crash-debug install).
+- Daylog: `firecuda-admin-log/2026/2026-09-20--pcie-lan-panic+crash-debug.md`.
+
+## Verify
+
+```bash
+# koopa
+systemctl --user list-timers koopa-firecuda-probe.timer
+koopa-firecuda-probe status
+koopa-firecuda-probe show
+ssh firecuda-probe   # should print one HB line, no shell
+
+# gaps after a crash: look for icmp=fail/ssh=fail with large gap_s=
+```
+
+## Uninstall
+
+```bash
+systemctl --user disable --now koopa-firecuda-probe.timer
+rm -f ~/.local/bin/koopa-firecuda-probe \
+  ~/.config/systemd/user/koopa-firecuda-probe.{service,timer}
+systemctl --user daemon-reload
+# optional: remove key + Host block + max authorized_keys line
+```
+
+## Do not
+
+- Put this key on **root** authorized_keys.
+- Enable timer / write authorized_keys without Freigabe.
+- Expect this to fix PCIe/Ethernet panics — it only timestamps last-alive.