koopa-admin-log/scripts/amanita-probe

amanita-probe — external last-alive from koopa

Why: amanita hard hangs leave no panic; internal HB stops with the box. External ICMP+SSH from koopa distinguishes:

Pattern	Likely meaning
ICMP fail + SSH fail, long gap	hang, power cut, or PSU (shared wall outlet with rabbithole ≠ whole-circuit outage)
ICMP ok + SSH fail	kernel/userspace hang, sshd stuck, or vx lock
both ok, HB stamp stale vs wall	rare clock skew / forced-cmd path oddity
both ok, HB fresh	alive

Log is hernani-readable on koopa (~/.local/state/amanita-probe/). SSH key is vx-only on amanita, restricted to koopa LAN IP + forced cat of HB mirror.

Pieces

Piece	Path
Script	scripts/amanita-probe/koopa-amanita-probe.sh → ~/.local/bin/koopa-amanita-probe
Timer	configs/systemd/user/koopa-amanita-probe.{service,timer} (~15 s)
Key	~/.ssh/id_ed25519_amanita_probe (hernani on koopa)
Log	~/.local/state/amanita-probe/amanita-probe.log (+ .last / .meta)
SSH Host	amanita-probe → vx@192.168.100.6 with that IdentityFile

Install (koopa as hernani)

Laptop SoT: $HOME/git/admin-logs/local/koopa-admin-log. Live mirror: ~/src/koopa/koopa-admin-log (no git commit on koopa).

# on koopa after syncing the tree:
cd ~/src/koopa/koopa-admin-log
./scripts/amanita-probe/install-amanita-probe.sh --keygen
# copy the printed authorized_keys line → amanita vx (see below)
./scripts/amanita-probe/install-amanita-probe.sh   # units + first probe

Linger already yes for hernani → timer survives logout.

amanita vx authorized_keys (one line)

from="192.168.100.95",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,command="/bin/cat /var/lib/amanita/heartbeat.last" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBeMenvp0D9EarfAEUtu49Q7bCbGhw+R/E1nJPQSSmZc koopa-amanita-probe@hernani-koopa

• User: vx only (not root).
• from=: only koopa LAN 192.168.100.95.
• forced command: read-only HB mirror (world-readable on amanita).
• Fingerprint: SHA256:R2i2nZfI1svR6Ts3DaSw5eZj2fvnGgm1R9PoMFS4Jj0
• hernani on koopa owns the private key and the probe log (live since 2026-09-18).
• Daylog: 2026/2026-09-18--amanita-external-probe.md.

Verify

# koopa
systemctl --user list-timers koopa-amanita-probe.timer
koopa-amanita-probe status
koopa-amanita-probe show
ssh amanita-probe   # should print one HB line, no shell

# gaps after a freeze: look for icmp=fail/ssh=fail with large gap_s=

Uninstall

systemctl --user disable --now koopa-amanita-probe.timer
rm -f ~/.local/bin/koopa-amanita-probe \
  ~/.config/systemd/user/koopa-amanita-probe.{service,timer}
systemctl --user daemon-reload
# optional: remove key + Host block + vx authorized_keys line

Do not

• Put this key on root authorized_keys.
• modprobe ramoops without reserved mem (known hard reset 2026-09-16).
• Expect this to replace internal HB — both are needed.