IFLIB(4)

bestie, you've landed on IFLIB(4), straight out of the Kernel Interfaces Manual 💖 grab a snack, this one's a whole framework

---

🌸 NAME 🌸

iflib — Network Interface Driver Framework

---

✨ SYNOPSIS ✨

device pci
device iflib

---

🦋 DESCRIPTION 🦋

So iflib is the framework that network interface drivers for FreeBSD get to live inside, and honestly? It's a gift. It exists to delete a huge chunk of the boilerplate that basically every modern network interface device needs, which means driver authors get to put their whole entire brain on the code that's actually specific to their hardware instead of re-typing the same scaffolding forever. 🙈

And the glow-up doesn't stop there: because everybody's using the framework, everyone shares one set of sysctl(8) names instead of each driver inventing its own. iconic, no cap 💅

---

🎀 SYSCTL VARIABLES 🎀

Heads up bestie: these ones have to be set before you load the driver — either through loader.conf(5) or by using kenv(1). And they're all prefixed with dev.X.Y.iflib. where X is the driver name and Y is the instance number. Keep that in your notes 📝

• override_nrxds — overrides how many RX descriptors each queue gets. The value is a comma separated list of positive integers — some drivers only use a single value, but others may use more. These numbers must be powers of two, and 0 means "just use the default." Individual drivers may have additional restrictions on what values are allowed. Defaults to all zeros.
• override_ntxds — same deal but for the number of TX descriptors per queue. Comma separated list of positive integers; some drivers use one value, others use more; must be powers of two; 0 means use the default. Individual drivers may have additional restrictions on allowable values. Defaults to all zeros.
• override_qs_enable — when this is set, the number of transmit queues and receive queues are allowed to be different. If it's not set, the lower of the number of TX or RX queues gets used for both. so it's giving ✨compromise✨
• override_nrxqs — sets the number of RX queues. If it's 0, the number of RX queues is derived from the number of cores on the socket connected to the controller. Defaults to 0.
• override_ntxqs — sets the number of TX queues. If it's 0, the number of TX queues is derived from the number of cores on the socket connected to the controller.
• disable_msix — disables MSI-X interrupts for the device. that's it, that's the whole thing 💀
• core_offset — specifies a starting core offset to assign queues to. If the value is unspecified, or if it's 65535, cores get assigned sequentially across controllers.
• separate_txrx — requests that RX and TX queues not be paired on the same core. If it's zero or not set, one RX and TX queue pair gets assigned to each core. When it's set to a non-zero value, TX queues are assigned to cores following the last RX queue.

---

💜 these sysctl(8) variables can be changed at any time

• tx_abdicate — controls how the transmit ring is serviced. If set to zero, then when a frame is submitted to the transmission ring, the same task that's submitting it will service the ring — unless there's already a task servicing the TX ring. This is what makes sure that whenever there's a pending transmission, the transmit ring is being serviced, and it results in higher transmit throughput. If set to a non-zero value, the task returns immediately and the transmit ring gets serviced by a different task. That hands control back to the caller faster, and under high receive load it may result in fewer dropped RX frames.
• rx_budget — sets the max number of frames that get received at a time. Zero (which is the default) means the default (currently 16) should be used.

---

🌈 global sysctls, they change vibes for all drivers

These affect the behaviour for every driver, and they may be changed at any time.

• net.iflib.min_tx_latency — if this is set to a non-zero value, iflib will avoid any attempt to combine multiple transmits, and it notifies the hardware as quickly as possible about new descriptors. This lowers the maximum throughput, but it also lowers transmit latency. tradeoffs, we love them 💅
• net.iflib.no_tx_batch — some NICs allow processing completed transmit descriptors in batches. Doing that usually increases transmit throughput by reducing the number of transmit interrupts. Setting this to a non-zero value disables the use of this feature.

---

⭐ read-only sysctl(8) variables

• driver_version — a string indicating the internal version of the driver. look but don't touch 🫶

---

👑 queue state sysctl(8) variables

There are a whole bunch of these too, bestie:

• txqZ — the following are repeated for each transmit queue, where Z is the transmit queue instance number:
  • r_abdications — number of consumer abdications in the MP ring for this queue. An abdication occurs on every ring submission when tx_abdicate is true.
  • r_restarts — number of consumer restarts in the MP ring for this queue. A restart occurs when an attempt to drain a non-empty ring fails, and the ring is already in the STALLED state.
  • r_stalls — number of consumer stalls in the MP ring for this queue. A stall occurs when an attempt to drain a non-empty ring fails.
  • r_starts — number of normal consumer starts in the MP ring for this queue. A start occurs when the MP ring transitions from IDLE to BUSY.
  • r_drops — number of drops in the MP ring for this queue. A drop occurs when there's an attempt to add an entry to an MP ring that has no available space.
  • r_enqueues — number of entries which have been enqueued to the MP ring for this queue.
  • ring_state — MP (soft) ring state. This provides a snapshot of the current MP ring state, including the producer head and tail indexes, the consumer index, and the state. The state is one of "IDLE", "BUSY", "STALLED", or "ABDICATED".
  • txq_cleaned — the number of transmit descriptors which have been reclaimed. Total cleaned.
  • txq_processed — the number of transmit descriptors which have been processed, but may not yet have been reclaimed.
  • txq_in_use — descriptors which have been added to the transmit queue, but have not yet been cleaned. This value will include both untransmitted descriptors as well as descriptors which have been processed.
  • txq_cidx_processed — the transmit queue consumer index of the next descriptor to process.
  • txq_cidx — the transmit queue consumer index of the oldest descriptor to reclaim.
  • txq_pidx — the transmit queue producer index where the next descriptor to transmit will be inserted.
  • no_tx_dma_setup — number of times DMA mapping a transmit mbuf failed for reasons other than EFBIG.
  • txd_encap_efbig — number of times DMA mapping a transmit mbuf failed due to requiring too many segments.
  • tx_map_failed — number of times DMA mapping a transmit mbuf failed for any reason (sum of no_tx_dma_setup and txd_encap_efbig)
  • no_desc_avail — number of times a descriptor couldn't be added to the transmit ring because the transmit ring was full.
  • mbuf_defrag_failed — number of times both m_collapse(9) and m_defrag(9) failed after an EFBIG error result from DMA mapping a transmit mbuf.
  • m_pullups — number of times m_pullup(9) was called attempting to parse a header.
  • mbuf_defrag — number of times m_defrag(9) was called.
• rxqZ — the following are repeated for each receive queue, where Z is the receive queue instance number:
  • rxq_fl0.credits — credits currently available in the receive ring.
  • rxq_fl0.cidx — current receive ring consumer index.
  • rxq_fl0.pidx — current receive ring producer index.

🧚 psst — additional OIDs that are useful for driver and iflib development get exposed when the INVARIANTS and/or WITNESS options are enabled in the kernel. freebies, but only for the devs ✨

---

🔥 SEE ALSO 🔥

• iflib(9)

---

💖 HISTORY 💖

This framework was introduced in FreeBSD 11.0. baby framework, all grown up now 🥹

---

and that's the whole page, bestie!! this one comes to you from Debian, dated September 27, 2018 🎀 go forth and configure your queues responsibly, ily 💜🦋