79 lines
3 KiB
Markdown
79 lines
3 KiB
Markdown
# amanita-probe — external last-alive from koopa
|
||
|
||
**Why:** amanita hard hangs leave no panic; internal HB stops with the box.
|
||
External ICMP+SSH from **koopa** distinguishes:
|
||
|
||
| Pattern | Likely meaning |
|
||
|---------|----------------|
|
||
| ICMP fail + SSH fail, long gap | hang, power cut, or **PSU** (shared wall outlet with rabbithole ≠ whole-circuit outage) |
|
||
| ICMP ok + SSH fail | kernel/userspace hang, sshd stuck, or vx lock |
|
||
| both ok, HB stamp stale vs wall | rare clock skew / forced-cmd path oddity |
|
||
| both ok, HB fresh | alive |
|
||
|
||
Log is **hernani-readable** on koopa (`~/.local/state/amanita-probe/`).
|
||
SSH key is **vx-only** on amanita, restricted to koopa LAN IP + forced `cat` of HB mirror.
|
||
|
||
## Pieces
|
||
|
||
| Piece | Path |
|
||
|-------|------|
|
||
| Script | `scripts/amanita-probe/koopa-amanita-probe.sh` → `~/.local/bin/koopa-amanita-probe` |
|
||
| Timer | `configs/systemd/user/koopa-amanita-probe.{service,timer}` (~15 s) |
|
||
| Key | `~/.ssh/id_ed25519_amanita_probe` (hernani on koopa) |
|
||
| Log | `~/.local/state/amanita-probe/amanita-probe.log` (+ `.last` / `.meta`) |
|
||
| SSH Host | `amanita-probe` → `vx@192.168.100.6` with that IdentityFile |
|
||
|
||
## Install (koopa as hernani)
|
||
|
||
Laptop SoT: `$HOME/git/admin-logs/local/koopa-admin-log`. Live mirror: `~/src/koopa/koopa-admin-log` (no git commit on koopa).
|
||
|
||
```bash
|
||
# on koopa after syncing the tree:
|
||
cd ~/src/koopa/koopa-admin-log
|
||
./scripts/amanita-probe/install-amanita-probe.sh --keygen
|
||
# copy the printed authorized_keys line → amanita vx (see below)
|
||
./scripts/amanita-probe/install-amanita-probe.sh # units + first probe
|
||
```
|
||
|
||
Linger already yes for hernani → timer survives logout.
|
||
|
||
## amanita vx authorized_keys (one line)
|
||
|
||
```text
|
||
from="192.168.100.95",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,command="/bin/cat /var/lib/amanita/heartbeat.last" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBeMenvp0D9EarfAEUtu49Q7bCbGhw+R/E1nJPQSSmZc koopa-amanita-probe@hernani-koopa
|
||
```
|
||
|
||
- **User:** `vx` only (not root).
|
||
- **from=:** only koopa LAN `192.168.100.95`.
|
||
- **forced command:** read-only HB mirror (world-readable on amanita).
|
||
- **Fingerprint:** `SHA256:R2i2nZfI1svR6Ts3DaSw5eZj2fvnGgm1R9PoMFS4Jj0`
|
||
- hernani on koopa owns the private key and the probe log (live since 2026-09-18).
|
||
- Daylog: `2026/2026-09-18--amanita-external-probe.md`.
|
||
|
||
## Verify
|
||
|
||
```bash
|
||
# koopa
|
||
systemctl --user list-timers koopa-amanita-probe.timer
|
||
koopa-amanita-probe status
|
||
koopa-amanita-probe show
|
||
ssh amanita-probe # should print one HB line, no shell
|
||
|
||
# gaps after a freeze: look for icmp=fail/ssh=fail with large gap_s=
|
||
```
|
||
|
||
## Uninstall
|
||
|
||
```bash
|
||
systemctl --user disable --now koopa-amanita-probe.timer
|
||
rm -f ~/.local/bin/koopa-amanita-probe \
|
||
~/.config/systemd/user/koopa-amanita-probe.{service,timer}
|
||
systemctl --user daemon-reload
|
||
# optional: remove key + Host block + vx authorized_keys line
|
||
```
|
||
|
||
## Do not
|
||
|
||
- Put this key on **root** authorized_keys.
|
||
- `modprobe ramoops` without reserved mem (known hard reset 2026-09-16).
|
||
- Expect this to replace internal HB — both are needed.
|