The Router From Hell maxdevnet.cc

Built for the box

Custom software

Off-the-shelf monitoring assumes an off-the-shelf router. Magikarp isn’t one — so it grew its own tooling: a couple of purpose-built C programs, a tuning script, and a hand-written kernel sysctl profile, all named to match the theme.

Drop / error watcher (TUI) C

hellmon

A single-binary, ncurses-free terminal dashboard purpose-built for Magikarp. Three live pages — Overview, Interfaces, Burst catcher — switchable with 1/2/3. Delta-tracks the full netdev error family (missed / fifo / overrun / carrier / collisions), watches WireGuard peers and handshake age, tails BIRD OSPF neighbours (and flags MISSING ones parsed straight from bird.conf), and reads i40e private flags via the ETHTOOL_GSTATS ioctl.

Signature move: The burst catcher arms a kfree_skb ftrace trigger the instant tx_dropped ticks, snapshots the trace buffer and forks a tcpdump for the capture window — all without blocking the UI. Evidence lands in /tmp/burst-<timestamp>/.

gcc -O2 -Wall -Wextra -o hellmon hellmon.c

XDP drop-plane daemon + CLI C + eBPF/XDP

hellgated

One binary, two personalities: a daemon that loads an embedded XDP program (bpftool skeleton compiled in — no .o on disk, no bpftool at runtime) and pins its maps under /sys/fs/bpf/hellgate so counters and rules survive restarts; and a CLI that talks the same words over a root-only unix socket.

Signature move: It watches RTNETLINK and re-attaches automatically if the NIC reappears (driver reload, boot ordering — or the day the X710 finally gets swapped: hellgate move <newif>). Guard rails are enforced centrally: it refuses to drop dport 22 or the live WireGuard listen-port no matter which client asks. Steady-state footprint ~1 MB RSS, zero mallocs, zero forks.

clang XDP object → embedded skeleton → gcc daemon

10G interface tuner bash + ethtool

tune-hell.sh

Drives the X710 hardware offloads: rx/tx/gso/gro/tso, checksums, scatter-gather, hw-tc-offload, 4096-deep ring buffers, adaptive coalescing and flow control. Then it pins each queue's IRQ to a distinct CPU for clean RSS spread. The 1GbE emergency port gets a lighter touch.

Signature move: Ends, correctly, with: "God help you, you crazy bastard."

./tune-hell.sh

Kernel network tuning sysctl

99-router-from-hell.conf

The 10G sysctl profile: 128 MiB socket buffers, BBR + fq qdisc, TCP fast-open, 2M-entry conntrack table with tuned timeouts, big netdev backlog/budget, loose rp_filter for asymmetric routing, ARP tuning for the VLAN sub-interfaces, and a full anti-spoof / redirect-hardening block.

Signature move: conntrack max = 2,097,152. Because residential doesn't mean small tables.

/etc/sysctl.d/99-router-from-hell.conf

Data plane

hellgate: dropping packets before they exist

The most interesting piece is hellgate — an eBPF/XDP program that drops bogons and abusive sources at the earliest point in the stack, before the kernel even allocates a socket buffer for the packet. That skips the entire netfilter path for traffic that was only going to be dropped anyway.

The XDP object is compiled into the daemon as a skeleton — there’s no .o file on disk and no bpftool at runtime. Maps are pinned under /sys/fs/bpf/hellgate, so block rules and counters survive a daemon restart. It watches RTNETLINK and re-attaches itself automatically if the NIC reappears — including the day the X710 finally gets swapped (hellgate move <newif>).

# hellgate — CLI == wire protocol, same words hellgate status hellgate block 1.2.3.0/24 hellgate unblock 1.2.3.0/24 hellgate drop-port 4444 hellgate move enp16s0f1np1 # re-attach to a new NIC # Guard rails enforced centrally in the daemon: # refuses to drop dport 22 or the live WireGuard # listen-port, no matter which client asks. # Steady state: ~1 MB RSS, zero mallocs, zero forks.

Observability

hellmon: watching the drops in real time

hellmon is a single-binary terminal dashboard that replaced a pile of shell scripts. Three live pages — Overview, Interfaces and Burst catcher — switch with the number keys. Counters are delta-tracked and colour-coded: red means it ticked this interval, yellow means since start, dim means quiet.

It reads NIC counters straight from the ETHTOOL_GSTATS ioctl, surfaces i40e private flags, tracks WireGuard peers and handshake age, and tails BIRD OSPF neighbours — flagging any that are MISSING by parsing the expected set out of bird.conf itself.

The burst catcher is the clever bit. The instant tx_dropped ticks up on the watched interface, it arms a kfree_skb ftrace stack-trace trigger, snapshots the trace buffer, and forks a tcpdump for the capture window — then summarises the TX-path frames. All of it runs as a non-blocking state machine, so the UI keeps updating mid-capture. Evidence lands in /tmp/burst-YYYYmmdd-HHMMSS/. It catches the kind of transient drop that’s gone before you can SSH in and look.

Tuning

Making 10 gigabits behave

tune-hell.sh drives the X710’s hardware offloads — checksums, GSO/GRO/TSO, scatter-gather, hardware TC offload, 4096-deep ring buffers, adaptive interrupt coalescing — then pins each queue’s IRQ to its own CPU so receive-side scaling spreads cleanly across the cores.

The kernel side is a hand-tuned sysctl profile: 128 MiB socket buffers, BBR congestion control on an fq qdisc, TCP fast-open, a conntrack table two million entries deep, and a full anti-spoofing / redirect-hardening block. Loose reverse-path filtering keeps asymmetric overlay routing working.

# tune-hell.sh (excerpt) for IF in $WAN_IF $LAN_10G_IF; do ethtool -K $IF gso on gro on tso on ethtool -K $IF hw-tc-offload on ethtool -G $IF rx 4096 tx 4096 ethtool -C $IF adaptive-rx on adaptive-tx on done # ... then distribute IRQs across every CPU ... echo "🔥 The Router from Hell is tuned and ready!" echo " God help you, you crazy bastard"