The effort
The work that went into this
A network like this isn’t bought, it’s accreted — one audit, one fix, one 2 a.m. packet capture at a time. This is the log of the decisions and the corrections, credited where it’s due: to Max, and to the copious amounts of alcohol that made staring at OSPF adjacencies bearable.
Router From Hell replaces the NanoPi R5c
Retired the little NanoPi R5c and stood up Magikarp — a hand-built Ryzen 7 5700G box with an Intel X710, terminating 10G WAN and LAN. The NanoPi is now a spare; the next site gets a MikroTik hAP ax3 instead of the NanoPi.
Whole-network config audit
Reviewed every device and produced six corrected config artifacts (BIRD, ICX, both MikroTiks, Magikarp misc, EX3300). Findings triaged High / Medium / Low with specific remediation.
BIRD kernel export filter
Scoped what OSPF is allowed to install in the kernel table — without an explicit export filter, an OSPF-learned default could have clobbered the real WAN default route.
WireGuard AllowedIPs de-dup
The 224.0.0.0/24 multicast carrier must appear exactly once per peer; duplicated across peers it broke routing. Fixed so OSPF hellos traverse the tunnel cleanly.
Switch VLAN + bypass fixes
Resolved ICX VLAN membership conflicts and guest-port VLAN-hopping exposure, and flagged that management SVIs on the switches create implicit inter-VLAN paths that bypass Magikarp's firewall.
LibreNMS SNMP polling
Chased down a stale hosts entry inside the Pleco container plus two snmpd.conf bugs (malformed rocommunity line, a view reference that didn't exist). Polling restored.
DNS resolution rework
Switched dnsmasq from address= to host-record= entries, and recommended a RouterOS static FWD entry for .fishtank so the internal domain resolves deterministically instead of racing ISP DNS.
FusionPBX + Telnyx deployment guide
Serial-console install, registering trunk setup, E911 routing, the nftables changes, ICX port options, and a full acceptance-test matrix.
IPv6 paused
Pulled the Hurricane Electric 6in4 tunnel — it caused severe packet loss on this network. IPv6 stays commented (not deleted) until Socket ships native v6.
The audit, quantified
One pass, six corrected configs
A full audit produced corrected configs for BIRD, the ICX 7450, both MikroTiks, an assortment of Magikarp fixes, and the EX3300. The headline catches: a missing BIRD kernel export filter that risked clobbering the WAN default route; duplicated WireGuard AllowedIPs breaking OSPF; missing MikroTik input-chain drops; ICX VLAN-membership conflicts and guest-port VLAN-hopping exposure; and management SVIs on the switches quietly creating firewall-bypass paths.
What it taught
Lessons, kept
- WireGuard + OSPF is the right call at this scale. IPsec would need GRE/IPIP to get multicast dynamic routing back — more overhead, no throughput win, because residential WAN is the real bottleneck, not the crypto.
- Flowtable offload creates monitoring blind spots. Offloaded flows skip the forward chain, so NFQUEUE-based tooling needs flowtable exemptions (e.g. VLAN 77 SIP to avoid one-way audio).
- SIP ALG stays off. The hosted PBX handles NAT at its SBC edge; the Linux conntrack SIP helper would only interfere. MikroTik SIP service-port is disabled everywhere too.
- Hardware crypto offload is moot at residential speeds. ChaCha20-Poly1305 on Zen 3 AVX2 is not the bottleneck, and GPU WireGuard offload is architecturally impossible anyway.
- XDP is the right tool for early drop. Bogon/DDoS dropping before skb allocation skips the whole netfilter path.
- RouterOS does not do ordered DNS failover — all servers race. Use static FWD entries to steer internal domains deterministically.
Read the technical detail on the Routing & Security page, or see what the effort produced on the Custom Software page.