About me: My name is Solène Rapenne, pronouns she/her. I like learning and sharing knowledge. Hobbies: '(BSD OpenBSD Qubes OS Lisp cmdline gaming security QubesOS internet-stuff). I love percent and lambda characters. OpenBSD developer solene@. No AI is involved in this blog.

Contact me: solene at dataswamp dot org or @solene@bsd.network (mastodon).

You can sponsor my work financially if you want to help me writing this blog and contributing to Free Software as my daily job.

Solving a bad ARP behavior on a Linux router

Written by Solène, on 05 August 2022.
Tags: #linux #networking

Comments on Fediverse/Mastodon

1. Introduction §

So, I recently switched my home router to Linux but had a network issues for devices that would get/renew their IP with DHCP. They were obtaining an IP, but they couldn't reach the router before a while (between 5 seconds to a few minutes), which was very annoying and unreliable.

After spending some time with tcpdump on multiple devices, I found the issue, it was related to ARP (the protocol to discover MAC addresses associate them with IPs).

Wikipedia page about the ARP protocol

The arp flux problem explained

2. My setup §

I have an unusual network setup at home as I use my ISP router for Wi-Fi, switch and as a modem, the issue here is that there are two subnets on its switch.

      +------------------+                                +-----------------+
      | ISP MODEM        | ethernet #1         ethernet #1|                 |
      |                  |<------------------------------>|                 |
      |                  ||                 |
      |                  |                                |  linux router   |
      |                  |                                |                 |
      |                  | ethernet #2         ethernet #2|                 |
      |                  |<------------------------------>|                 |
      |                  |           |                 |
      |                  |                                |                 |
      |                  |                                |                 |
      +------------------+                                +-----------------+
       ^ethernet #4     ^ ethernet #3
       |                |
       |                |
       |                +----> some switch with many devices

Because the modem is reachable over and is used by the router on that switch, but that the LAN network uses the same switch with, ARP packets arrives on two network interfaces of the router, for addresses that are non routables (ARP packets for would arrive at the interface or the opposite).

3. Solution §

There is simple solution, but it was very complicated to find as it's not obvious. We can configure the Linux kernel to discard ARP packets that are related to non routable addresses, so the interface with a address will discard packets for the network and vice-versa.

You need to define the sysctl net.ipv4.conf.all.arp_filter to 1.

sysctl net.ipv4.conf.all.arp_filter=1

This can be set per interface if you have specific need.

Documentation of the sysctl available on Linux

4. Conclusion §

This was a very annoying issue, incredibly hard to troubleshoot. I suppose OpenBSD has this strict behavior by default because I didn't have this problem when the router was running OpenBSD.