The Router From Hades maxdevnet.cc

Built for the box

Custom software

Off-the-shelf monitoring assumes an off-the-shelf router. Magikarp isn’t one — so it grew its own tooling: a couple of purpose-built C programs, a tuning script, and a hand-written kernel sysctl profile, all named to match the theme.

Drop / error watcher (TUI) C

Thanatos

A single-binary, ncurses-free terminal dashboard purpose-built for Magikarp. Three live pages — Overview, Interfaces, Burst catcher — switchable with 1/2/3. Delta-tracks the full netdev error family (missed / fifo / overrun / carrier / collisions), watches WireGuard peers and handshake age, tails BIRD OSPF neighbours (and flags MISSING ones parsed straight from bird.conf), and reads i40e private flags via the ETHTOOL_GSTATS ioctl.

Signature move: The burst catcher arms a kfree_skb ftrace trigger the instant tx_dropped ticks, snapshots the trace buffer and forks a tcpdump for the capture window — all without blocking the UI. Evidence lands in /tmp/burst-<timestamp>/.

gcc -O2 -Wall -Wextra -o thanatos thanatos.c

XDP drop-plane daemon + CLI C + eBPF/XDP

styxgated

One binary, two personalities: a daemon that loads an embedded XDP program (bpftool skeleton compiled in — no .o on disk, no bpftool at runtime) and pins its maps under /sys/fs/bpf/styxgate so counters and rules survive restarts; and a CLI that talks the same words over a root-only unix socket.

Signature move: It watches RTNETLINK and re-attaches automatically if the NIC reappears (driver reload, boot ordering — or the day the X710 finally gets swapped: styxgate move <newif>). Guard rails are enforced centrally: it refuses to drop dport 22 or the live WireGuard listen-port no matter which client asks. Steady-state footprint ~1 MB RSS, zero mallocs, zero forks.

clang XDP object → embedded skeleton → gcc daemon

10G interface tuner bash + ethtool

lethe.sh

Drives the X710 hardware offloads: rx/tx/gso/gro/tso, checksums, scatter-gather, hw-tc-offload, 4096-deep ring buffers, adaptive coalescing and flow control. Then it pins each queue's IRQ to a distinct CPU for clean RSS spread. The 1GbE emergency port gets a lighter touch. Appropriately named after the river lethe, setting interface parameters exactly to what the firewall needs to run as fast as possible

Signature move: Ends, correctly, with: "The Router from Hades is tuned and ready!" after setting IRQs, Ethtool settings, and runtime sysctl settings

./lethe.sh

Kernel network tuning sysctl

99-Phlegethon.conf

The 10G sysctl profile: 128 MiB socket buffers, BBR + fq qdisc, TCP fast-open, 2M-entry conntrack table with tuned timeouts, big netdev backlog/budget, loose rp_filter for asymmetric routing, ARP tuning for the VLAN sub-interfaces, and a full anti-spoof / redirect-hardening block.

Signature move: conntrack max = 2,097,152. Because residential doesn't mean small tables.

/etc/sysctl.d/99-Phlegethon.conf

Data plane

Styxgate: dropping packets before they exist

The most interesting piece is styxgate — an eBPF/XDP program that drops bogons and abusive sources at the earliest point in the stack, before the kernel even allocates a socket buffer for the packet. That skips the entire netfilter path for traffic that was only going to be dropped anyway.

The XDP object is compiled into the daemon as a skeleton — there’s no .o file on disk and no bpftool at runtime. Maps are pinned under /sys/fs/bpf/styxgate, so block rules and counters survive a daemon restart. It watches RTNETLINK and re-attaches itself automatically if the NIC reappears — including the day the X710 finally gets swapped (styxgate move <newif>).

Styxgate is also controlled and monitored by the custom TUI firewall monitor, Thanatos!

# styxgate — CLI == wire protocol, same words styxgate status styxgate block 1.2.3.0/24 styxgate unblock 1.2.3.0/24 styxgate drop-port 4444 styxgate move enp16s0f1np1 # re-attach to a new NIC # Guard rails enforced centrally in the daemon: # refuses to drop dport 22 or the live WireGuard # listen-port, no matter which client asks. # Steady state: ~1 MB RSS, zero mallocs, zero forks.

Observability

Thanatos: watching the drops in real time

Thanatos is a single-binary terminal dashboard that replaced a pile of shell scripts. With four live pages — Overview, Interfaces, Burst catcher and Sytxgate — switch with the number keys. Counters are delta-tracked and colour-coded: red means it ticked this interval, yellow means since start, dim means quiet.

It reads NIC counters straight from the ETHTOOL_GSTATS ioctl, surfaces i40e private flags, tracks WireGuard peers and handshake age, and tails BIRD OSPF neighbours — flagging any that are MISSING by parsing the expected set out of bird.conf itself.

Thanatos also ties directly into Styxgate acting as the TUI controller and monitor for the styxgate daemon, allowing the operator to block & unblock entire subnets and port ranges without exiting the monitor.

The burst catcher is the clever bit. The instant tx_dropped ticks up on the watched interface, it arms a kfree_skb ftrace stack-trace trigger, snapshots the trace buffer, and forks a tcpdump for the capture window — then summarises the TX-path frames. All of it runs as a non-blocking state machine, so the UI keeps updating mid-capture. Evidence lands in /tmp/burst-YYYYmmdd-HHMMSS/. It catches the kind of transient drop that’s gone before you can SSH in and look.

Tuning

Making 10 gigabits behave

Lethe.sh drives the X710’s hardware offloads — checksums, GSO/GRO/TSO, scatter-gather, hardware TC offload, 4096-deep ring buffers, adaptive interrupt coalescing — then pins each queue’s IRQ to its own CPU so receive-side scaling spreads cleanly across the cores.

The kernel side is a hand-tuned sysctl profile: 128 MiB socket buffers, BBR congestion control on an fq qdisc, TCP fast-open, a conntrack table two million entries deep, and a full anti-spoofing / redirect-hardening block. Loose reverse-path filtering keeps asymmetric overlay routing working.

# Lethe.sh (excerpt) for IF in $WAN_IF $LAN_10G_IF; do ethtool -K $IF gso on gro on tso on ethtool -K $IF hw-tc-offload on ethtool -G $IF rx 4096 tx 4096 ethtool -C $IF adaptive-rx on adaptive-tx on done # ... then distribute IRQs across every CPU ... echo "🔥 The Router from Hades is tuned and ready!" echo " "