BPFTUNE-NEIGH(8)

💅 BPFTUNE-NEIGH 💅

okay bestie, welcome to BPFTUNE-NEIGH — straight out of the System Manager's Manual, section 8, and it's giving networking glow-up. this is the neighbor table bpftune plugin, and its whole job is managing neighbor table sizing. let's get into it 🎀

---

✨ NAME ✨

BPFTUNE-NEIGH — the neighbor table bpftune plugin, here to manage neighbor table sizing. that's the vibe. 🦋

---

🌸 DESCRIPTION 🌸

so the neighbor table is basically the group chat between layer 3 and layer 2 — it holds layer 3 -> layer 2 mappings plus reachability information about remote systems. we look up by layer 3 address (the IP address) and it tells us the layer 2 address (the MAC address) associated with it. iconic. 💖

the table gets populated with both static and garbage-collected values. when we're adding entries we can specify that they should be PERMANENT, in which case they are not (exempt from) garbage-collected. no take-backs, bestie. 💅

Periodic garbage collection happens for non-permanent failed or expired entries — and it runs immediately if we can't alloc a new neighbor table entry. it's got main character energy when it needs to. ✨

there are a few pathologies we wanna dodge here, principally:

• neighbor table full: if we see /var/log/messages saying "Neighbour table overflow." then bestie… we have run out of space 😭. this can happen if garbage collection isn't run quickly enough, or if we're full with entries that are not subject to garbage collection.
  • in the former case, we could auto-tune by reducing gc_thresh2, since this makes GC run more quickly.
  • in the latter case, with a large number (75% or more) of exempt-from-GC entries, garbage collection won't help, so we have to increase gc_thresh3. this is done on a per-table basis via netlink, so the resource costs are limited rather than setting a system-wide tunable. size is increased by 25% of the current value (so 1024 -> 1280, etc).
  • 📌 note that by increasing gc_thresh3 only, garbage collection gets more time to run from table sized gc_thresh2 until we reach gc_thresh3. so it effectively helps with both scenarios. slay.
• neighbor table thrashing: too-aggressive GC eviction might lead to excessive overhead in re-establishing L3->L2 reachability information. TBD. 👀

🧚 Tunables 🧚

• gc_interval: how often garbage collection should happen; default 30 seconds. ⏰
• gc_stale_time: how often to check for stale entries. if neighbor goes stale, it is resolved again before sending data; defaults to 60sec.
• base_reachable_time_ms: how long neighbor entry is considered reachable for; defaults to 30sec.
• gc_thresh1: with a table size below this value, no GC happens; default 128.
• gc_thresh2: soft max of entries in table; wait 5 secs if we exceed this value to do GC; default 512.
• gc_thresh3: hard max for table size, GC will run if more entries than this exist, default 1024.

📢 Note: to set table size we need to use the equivalent of "ip ntable"; i.e. "ip ntable change name arp_cache dev eth0 thresh3 1024" (this is done directly in bpftune via netlink). we do this for gc_thresh1, thresh2 and thresh3 to ensure that GC scales with overall table size.

however, we set an upper limit on the adjustments based on gc_thresh3 size comparison to the prefixlens of the IP addresses configured on the netdevice the table manages. for example 🍓 — for a single IPv4 address with a /24 prefixlen, there are 256 possible addresses, and allowing for multicast etc we multiply this by a scaling factor (currently 4). so in this case we will not increase beyond 1024. but with larger networks / more devices, we will. we also scale down the table size when there are too few entries, upto the minimum of kernel default values, to minimize memory usage. 🫶

contrast this approach with simply choosing a large net.ipv4.neigh.gc_thresh3. if thresh2 and thresh3 are far apart, we may over-garbage collect, whereas if they are close we may end up keeping around too many entries. in either case, we're mistuned because we've had to choose coarse-grained defaults rather than adapting on a per-table basis as the need arises. periodt. 💀

---

👑 SEE ALSO 👑

• bpf(2)
• bpftune(8)

---

and that's the whole page, bestie 💖 shoutout to the System Manager's Manual for hosting us, and to the page itself — BPFTUNE-NEIGH(8), section 8, that's the one. go tune some neighbor tables, you've got this 🌈✨