RFC 3549
Linux Netlink as an IP Services Protocol
RFC 3549: Linux Netlink as an IP Services Protocol 🎀
Okay bestie, the header info first! ✨ This is RFC 3549, titled Linux Netlink as an IP Services Protocol. It came out of the Network Working Group in July 2003 and is labeled Category: Informational 📚. The authors are J. Salim (Znyx Networks), H. Khosravi (Intel), A. Kleen (Suse), and A. Kuznetsov (INR/Swsoft) 💅. The title block doesn't list anything it obsoletes or updates, so it's standing on its own 🦋.
Status of this Memo 🌸
This memo gives information to the Internet community. It is not a standard of any kind, so don't treat it as a spec you have to follow 🫶. Anyone can distribute it freely, with no limits 💜.
Copyright Notice 🔖
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract ✨
Okay so this doc is all about Linux Netlink 🔥. In Linux, Netlink is used for messaging inside the kernel AND between the kernel and user space, which is kinda iconic 👑. But this doc only cares about one job: Netlink as a protocol between a Forwarding Engine Component (FEC) and a Control Plane Component (CPC). Those two together are what define an IP service 🧚.
Because of that focus, the doc deliberately ignores everything else Netlink does 🙈. That includes its use as an intra-kernel messaging system, as an inter-process communication scheme (IPC), and as a config tool for non-networking or non-IP network services (like decnet, etc.) 💀.
The doc is also meant as informational prior art for the ForCES IETF working group 🍓.
Table of Contents 📋
1. Introduction ............................................... 2
1.1. Definitions ........................................... 3
1.1.1. Control Plane Components (CPCs)................ 3
1.1.2. Forwarding Engine Components (FECs)............ 3
1.1.3. IP Services ................................... 5
2. Netlink Architecture ....................................... 7
2.1. Netlink Logical Model ................................. 8
2.2. Message Format......................................... 9
2.3. Protocol Model......................................... 9
2.3.1. Service Addressing............................. 10
2.3.2. Netlink Message Header......................... 10
2.3.3. FE System Services' Templates.................. 13
3. Currently Defined Netlink IP Services....................... 16
3.1. IP Service NETLINK_ROUTE............................... 16
3.1.1. Network Route Service Module................... 16
3.1.2. Neighbor Setup Service Module.................. 20
3.1.3. Traffic Control Service........................ 21
3.2. IP Service NETLINK_FIREWALL............................ 23
3.3. IP Service NETLINK_ARPD................................ 27
4. References.................................................. 27
4.1. Normative References................................... 27
4.2. Informative References................................. 28
5. Security Considerations..................................... 28
6. Acknowledgements............................................ 28
Appendix 1: Sample Service Hierarchy .......................... 29
Appendix 2: Sample Protocol for the Foo IP Service............. 30
Appendix 2a: Interacting with Other IP services................. 30
Appendix 3: Examples........................................... 31
Authors' Addresses.............................................. 32
Full Copyright Statement........................................ 331. Introduction 🦋
Okay, history time! 📜 The idea of IP Service control-forwarding separation first showed up in the early 1990s with the BSD 4.4 routing sockets [9]. Back then the focus was a simple IP(v4) forwarding service 🍼. The CPC could control the forwarding tables for that IPv4 service, either through a command line config tool or a dynamic route daemon 🛠️.
The IP world has glowed up A LOT since then 💅. Looking at Linux Netlink from a service provisioning and management point of view, it takes routing sockets one step further by breaking the whole IPv4-forwarding-only vibe 🔥. Since the Linux 2.1 kernel, Netlink has been giving the IP service abstraction to a bunch of services beyond the classic RFC 1812 IPv4 forwarding 🌈.
Now, the goal here is NOT to list every single service Netlink is used for 🙈. We skip a lot of them, like multicast routing, tunneling, policy routing, etc. This also isn't meant to be a Netlink tutorial 📖. The idea is to explain the big-picture Netlink view with a special focus on the mandatory building blocks inside the ForCES charter, which are IPv4 and QoS ⭐. The doc also captures prior art for lots of mechanisms that are useful for ForCES ✨.
Lowkey, the text is limited to a subset of what's in kernel 2.4.6, which was the newest kernel when this was first written 🕰️. It's also limited to IPv4 functionality, so no IPv6 realness here 💀.
We're gonna start with some concept definitions and then show how Netlink fits in 🧚.
1.1. Definitions 🌸
A Control Plane (CP) is an execution environment that can have several sub-components, which we call CPCs 🫶. Each CPC controls a different IP service being run by a Forwarding Engine (FE) component 🔥. So there can be several CPCs on one physical CP, if it's controlling several IP services 👑. Basically the cohesion between a CP component and an FE component is the service abstraction 💜.
1.1.1. Control Plane Components (CPCs) 🎀
CPCs cover signalling protocols, and the range is WILD 🌈. On one end there are dynamic routing protocols, like OSPF [5], and on the other there are tag distribution protocols, like CR-LDP [7] 🦋. Classical management protocols and activities count too, including SNMP [6], COPS [4], and proprietary CLI/GUI config mechanisms 🛠️.
The whole point of the control plane is to give an execution environment for all of those activities 🍓. The ultimate goal is to configure and manage the second Network Element (NE) component, which is the FE 🧚. The config result defines how packets going through the FE get treated 💅. Main character energy 👑.
1.1.2. Forwarding Engine Components (FECs) 🌸
The FE is the part of the NE that incoming packets (coming from the network into the NE) hit FIRST ✨.
The FE's service-specific component basically massages the packet so it gets the right treatment to achieve an IP service, as defined by the CPCs for that service 💖. Different services use different FECs 🦋. Service modules can also be chained together to make a more complex service (more on that in the Linux FE model later) 🔗. And when a FE service component is built for a specific service, it sticks to a forwarding model 💅.
1.1.2.1. Linux IP Forwarding Engine Model 🦋
Here's the Linux FE model, per device 👀. The only mandatory part of the datapath is the Forwarding module, which is RFC 1812 conformant 🔥. The Firewall (FW), Ingress Traffic Control, and Egress Traffic Control blocks are NOT mandatory in the datapath, and they can even be used to bypass the RFC 1812 module 🙈. Those blocks are drawn as simple boxes, but they could actually be multiple cascaded, independent submodules inside each box 🌈. More info is in [10] and [11] 📚.
____ +---------------+
+->-| FW |---> | TCP, UDP, ... |
| +----+ +---------------+
| |
^ v
| _|_
+----<----+ | FW |
| +----+
^ |
| Y
To host From host
stack stack
^ |
|_____ |
Ingress ^ Y
device ____ +-------+ +|---|--+ ____ +--------+ Egress
->----->| FW |-->|Ingress|-->---->| Forw- |->| FW |->| Egress | device
+----+ | TC | | ard | +----+ | TC |-->
+-------+ +-------+ +--------+Okay so walk with me through the path 🧚. Packets arriving at the ingress device go through a firewall module FIRST 🔥. The firewall can drop packets, munge them, etc., depending on the set policy 🙈. Then the packet may pass through the Ingress Traffic Control module, where metering and policing happen 📊. The packet can get dropped here based on metering results and policing policies 💀.
Next up is the only non-optional module, the RFC 1812-conformant Forwarding module 👑. The packet can get dropped if it's nonconformant (to the many RFCs that complement 1812 and 1122) 🙅♀️. This is also the juncture point where packets destined for the forwarding NE can get sent up to the host stack 🫶.
Packets that aren't for the NE can traverse a policy routing submodule (inside the forwarding module), if that's been provisioned 🦋. After that, another firewall module gets walked 🔥. This firewall can drop or munge/transform packets, depending on the configured sub-modules it runs into and their policies 💅. If everything goes well, the packet reaches the Egress TC module ✨.
Last stop, the Egress TC can drop packets for policing, scheduling, congestion control, or rate control reasons 💖. Egress queues exist at this point, so drops or delays can happen before OR after the packet gets queued 😭. It all depends on the algorithms and policies configured for each module 🌸.
1.1.3. IP Services ✨
okay so an IP service is basically how an IP packet gets treated inside the NE 💅 and it's a team effort, because the CPC and the FEC both bring something to the table 🫶. The treatment starts the moment the packet arrives at the NE and ends the moment it departs 🦋. In this context an IP service is literally a Per-Hop Behavior 💖. CP components on NEs set up the end-to-end path control for a service by running control/signaling protocols and management applications. These distributed CPCs unify the end-to-end view of the service 👑. Then those CP components define how the FE (and therefore the NE) behaves for a described packet ✨.
A classic example is IPv4 Forwarding 🎀. Control components, like routing protocols (OSPF, RIP, etc.) and proprietary CLI/GUI configurations, tweak the FE's forwarding tables so it can offer the simple service of forwarding packets to the next hop. NEs that only offer this simple service are traditionally called routers 🍓.
Here's a diagram of a simple FE<->CP setup showing the classic IPv4 service with an extra bit of basic QoS egress scheduling, so you can see how it fits this model 💜:
Control Plane (CP)
.------------------------------------
| /^^^^^^\ /^^^^^^\ |
| | | | COPS |-\ |
| | ospfd | | PEP | \ |
| \ / \_____/ | |
/------\_____/ | / |
| | | | / |
| |_________\__________|____|_________|
| | | |
******************************************
Forwarding ************* Netlink layer ************
Engine (FE) *****************************************
.-------------|-----------|----------|---|-------------
| IPv4 forwarding | | |
| FE Service / / |
| Component / / |
| ---------------/---------------/--------- |
| | | / | |
packet | | --------|-- ----|----- | packet
in | | | IPv4 | | Egress | | out
-->--->|------>|---->|Forwarding|----->| QoS |--->| ---->|->
| | | | | Scheduler| | |
| | ----------- ---------- | |
| | | |
| --------------------------------------- |
| |
-------------------------------------------------------The diagram above shows ospfd, an OSPF protocol control daemon, and a COPS Policy Enforcement Point (PEP) as separate CPCs 🌸. The IPv4 FE component includes the IPv4 Forwarding service module as well as the Egress Scheduling service module. Another service could slot a policy forwarder in between the IPv4 forwarder and the QoS egress scheduler 🧚. A simpler classic service would be just the IPv4 forwarder, no extras needed.
Over the years, routers have needed more and more services to meet new requirements 📈. More complex services that extend classic forwarding have been added and standardized, and some of them go beyond the layer 3 contents of the packet header 👀. The name "router" is a bit of a misnomer at this point, but everyone still uses it for these NEs 💀. Services that may look past the classic L3 headers include firewalling, QoS in Diffserv and RSVP, NAT, policy-based routing, and so on. Newer control protocols and management activities arrive alongside these new services 🔥.
One extreme definition of a IP service is something for which a service provider would be able to charge 💸.
2. Netlink Architecture 🌈
Okay bestie, here's the big idea: control of IP service components is defined by using templates 📝.
The FEC and CPC work together to deliver the IP service by talking to each other through these templates 🫶. The FEC might be continuously getting updates from the Control Plane Component about how to operate the service, like for v4 forwarding or for route additions and deletions 🛣️.
Now the interaction between the FEC and the CPC, in the Netlink context, defines a protocol 🔐. Netlink gives the CPC (living in user space) and the FEC (living in kernel space) a way to each have their own protocol definition. Kernel space and user space just mean different protection domains, though, so a wire protocol is needed to actually communicate 🧵. That wire protocol is normally provided by some privileged service that can copy between multiple protection domains. We'll call this service the Netlink service 💌. The Netlink service can also be wrapped in a different transport layer if the CPC runs on a different node than the FEC 🌍. The FEC and CPC, using Netlink mechanisms, may choose to define a reliable protocol between each other. By default, though, Netlink provides an unreliable communication 😬.
Also, the FEC and CPC can both live in the same memory protection domain and use the connect() system call to create a path to the peer and talk to each other 🧚♀️. We won't dig into that mechanism further, other than to say it's available. Throughout this document, we'll use "FEC" to mean kernel space and "CPC" to mean user space interchangeably 💜. This naming isn't meant to restrict the two components to those protection domains or to the same compute node, though.
Note: Netlink allows participation in IP services by both service components 🙌.
2.1. Netlink Logical Model 🦋
Here's a simple FEC<->CPC logical relationship in a diagram 🎀. We'll use the IPv4 forwarding FEC (NETLINK_ROUTE, which gets discussed further below) as our example.
Control Plane (CP)
.------------------------------------
| /^^^^^\ /^^^^^\ |
| | | / CPC-2 \ |
| | CPC-1 | | COPS | |
| | ospfd | | PEP | |
| | / \____ _/ |
| \____/ | |
| | | |
****************************************|
************* BROADCAST WIRE ************
FE---------- *****************************************.
| IPv4 forwarding | | | |
| FEC | | | |
| --------------/ ----|-----------|-------- |
| | / | | | |
| | .-------. .-------. .------. | |
| | |Ingress| | IPv4 | |Egress| | |
| | |police | |Forward| | QoS | | |
| | |_______| |_______| |Sched | | |
| | ------ | |
| --------------------------------------- |
| |
-----------------------------------------------------Netlink logically models FECs and CPCs as nodes that are hooked up to each other through a broadcast wire 🔌. The wire is specific to a service. The example above shows the broadcast wire belonging to the extended IPv4 forwarding service ✨.
Nodes (CPCs or FECs, as shown above) connect to the wire and register to receive specific messages 📡. CPCs can connect to multiple wires if that helps them control the service better 💅. All nodes (CPCs and FECs) dump packets on the broadcast wire. The wire can discard packets if they're malformed or not specifically formatted for the wire 🗑️. Dropped packets aren't seen by any of the nodes 🙈. The Netlink service may signal an error to the sender if it detects a malformatted Netlink packet 🚨.
Packets sent on the wire can be broadcast, multicast, or unicast 📮. FECs or CPCs register for specific messages they're interested in, either for processing or just for monitoring purposes 👀.
Appendices 1 and 2 have a high-level overview of this interaction 🧚.
2.2. Message Format 💖
There are three levels to a Netlink message: the general Netlink message header, the IP service specific template, and the IP service specific data 🍓.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Netlink message header |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IP Service Template |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IP Service specific data in TLVs |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+The Netlink message is used to communicate between the FEC and CPC for parameterizing the FECs 🎛️, for asynchronous event notification of FEC events to the CPCs 🔔, and for statistics querying and gathering (typically done by a CPC) 📊.
The Netlink message header is generic for all services 🌐, whereas the IP Service Template header is specific to a service 🎯. Each IP Service then carries parameterization data (CPC->FEC direction) or a response (FEC->CPC direction) 🔄. These parameterizations are in TLV (Type-Length-Value) format and are unique to the service 💜.
The different parts of the Netlink message are discussed in the following sections 📖.
2.3. Protocol Model 🦋
This section expands on how Netlink provides the mechanism for service-oriented FEC and CPC interaction 🧠.
2.3.1. Service Addressing 🎀
Access is provided by first connecting to the service on the FE 🔗. The connection is made by calling the socket() system call on the PF_NETLINK domain 🧦. Each FEC is identified by a protocol number 🔢. You can open either SOCK_RAW or SOCK_DGRAM type sockets, although Netlink doesn't distinguish between the two 🤷♀️. The socket connection is the basis for the FE<->CP addressing 🌸.
Once you're connected to a service, at any point during the life of the connection you can do one of three things: issue a service-specific command (from the CPC to the FEC, mostly for configuration purposes) 🛠️, issue a statistics-collection command 📈, or subscribe or unsubscribe to service events 🔔. Closing the socket terminates the transaction 🚪. Refer to Appendices 1 and 2 for examples 🧚♀️.
2.3.2. Netlink Message Header ✨
okay so a Netlink message is basically a byte stream 🌊 made of one or more Netlink headers, each with its own payload 💖. If the payload is too big to fit in one message, it can get split across several messages. Together those are called a multipart message 🧩. In a multipart message, the first header and every header after it have the NLM_F_MULTI flag set 🚩, except the last one, which has the type NLMSG_DONE 🏁.
The Netlink message header looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Process ID (PID) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Here's the rundown on each field 👇
Length: 32 bits 📏 The length of the message in bytes, including the header.
Type: 16 bits 🏷️ This tells you what the message is about. The standard message types are:
NLMSG_NOOP Message is ignored.
NLMSG_ERROR The message signals an error and the payload
contains a nlmsgerr structure. This can be looked
at as a NACK and typically it is from FEC to CPC.
NLMSG_DONE Message terminates a multipart message.Individual IP services can add their own message types too 🙌. For example, the NETLINK_ROUTE service has several, like RTM_NEWLINK, RTM_DELLINK, RTM_GETLINK, RTM_NEWADDR, RTM_DELADDR, RTM_NEWROUTE, RTM_DELROUTE, and more.
Flags: 16 bits 🚩 The standard Netlink flag bits are:
NLM_F_REQUEST Must be set on all request messages (typically
from user space to kernel space)
NLM_F_MULTI Indicates the message is part of a multipart
message terminated by NLMSG_DONE
NLM_F_ACK Request for an acknowledgment on success.
Typical direction of request is from user
space (CPC) to kernel space (FEC).
NLM_F_ECHO Echo this request. Typical direction of
request is from user space (CPC) to kernel
space (FEC).There are also some extra flag bits for GET requests on config information in the FEC 🔍:
NLM_F_ROOT Return the complete table instead of a
single entry.
NLM_F_MATCH Return all entries matching criteria passed in
message content.
NLM_F_ATOMIC Return an atomic snapshot of the table being
referenced. This may require special
privileges because it has the potential to
interrupt service in the FE for a longer time.Convenience macro for flag bits 🍬:
NLM_F_DUMP This is NLM_F_ROOT or'ed with NLM_F_MATCHAnd these are the extra flag bits for NEW requests ✨:
NLM_F_REPLACE Replace existing matching config object with
this request.
NLM_F_EXCL Don't replace the config object if it already
exists.
NLM_F_CREATE Create config object if it doesn't already
exist.
NLM_F_APPEND Add to the end of the object list.For my BSD-brained besties who know route sockets 🧠, here's how the operations translate:
- BSD ADD operation equates to
NLM_F_CREATEor-ed withNLM_F_EXCL - BSD CHANGE operation equates to
NLM_F_REPLACE - BSD Check operation equates to
NLM_F_EXCL - BSD APPEND equivalent is actually mapped to
NLM_F_CREATE
Sequence Number: 32 bits 🔢 The sequence number of the message.
Process ID (PID): 32 bits 🆔 The PID of the process sending the message. The kernel uses the PID to multiplex to the correct sockets. A PID of zero is used when sending messages to user space from the kernel 🧚.
2.3.2.1. Mechanisms for Creating Protocols
okay this one is so fun 🤩. You could build a reliable protocol between an FEC and a CPC by combining sequence numbers, ACKs, and retransmit timers. Netlink gives you the sequence numbers and ACKs, and Linux gives you the timers ⏰.
You could also build a heartbeat protocol between the FEC and CPC using the ECHO flags and the NLMSG_NOOP message 💓.
2.3.2.2. The ACK Netlink Message
this message is used for BOTH an ACK and a NACK 🙃. Usually it goes from FEC to CPC, as a reply to an ACK request message. The CPC should also be able to send ACKs back to the FEC when asked to. What that looks like depends on the IP service 💅.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Netlink message header |
| type = NLMSG_ERROR |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Error code |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OLD Netlink message header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Error code: integer (typically 32 bits) 🚦
- An error code of zero means the message is an ACK ✅. An ACK response contains the original Netlink message header, which you can compare against (sent sequence numbers, etc.) 🔍.
- A non-zero error code is a Negative ACK (NACK) ❌. In that case, the Netlink data that was sent down to the kernel comes back appended to the original Netlink message header. An error code you can print with
perror()is also set, but not in the message header. It lives in the executing environment's state variable 🧠.
2.3.3. FE System Services' Templates
these are services the system offers for general use by other services 🎀. They cover configuring things, gathering statistics, and listening for changes in shared resources. IP address management, link events, and so on all live here. This section gets its own spot for logical separation, even though these services are accessed through the NETLINK_ROUTE FEC. They live inside NETLINK_ROUTE because of historical cruft 🦕: the BSD 4.4 Route Sockets implemented them as part of the IPv4 forwarding sockets.
2.3.3.1. Network Interface Service Module
this service lets you create, remove, or get info about a specific network interface 🔌. The interface can be physical or virtual, and it's network protocol independent (an x.25 interface, for example, can be defined with this message) 🌈. Here's the Interface service message template:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Family | Reserved | Device Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Interface Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Device Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Change Mask |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Family: 8 bits 👪 This is always set to AF_UNSPEC.
Device Type: 16 bits 🔗 This defines the type of the link, which could be Ethernet, a tunnel, and so on. We're only interested in IPv4, though the link type is L3 protocol-independent 🌐.
Interface Index: 32 bits 🪪 Uniquely identifies the interface.
Device Flags: 32 bits 🎛️ The device flags are:
IFF_UP Interface is administratively up.
IFF_BROADCAST Valid broadcast address set.
IFF_DEBUG Internal debugging flag.
IFF_LOOPBACK Interface is a loopback interface.
IFF_POINTOPOINT Interface is a point-to-point link.
IFF_RUNNING Interface is operationally up.
IFF_NOARP No ARP protocol needed for this interface.
IFF_PROMISC Interface is in promiscuous mode.
IFF_NOTRAILERS Avoid use of trailers.
IFF_ALLMULTI Receive all multicast packets.
IFF_MASTER Master of a load balancing bundle.
IFF_SLAVE Slave of a load balancing bundle.
IFF_MULTICAST Supports multicast.
IFF_PORTSEL Is able to select media type via ifmap.
IFF_AUTOMEDIA Auto media selection active.
IFF_DYNAMIC Interface was dynamically created.Change Mask: 32 bits 🎭 Reserved for future use. It must be set to 0xFFFFFFFF.
Applicable attributes 🧾:
Attribute Description
..........................................................
IFLA_UNSPEC Unspecified.
IFLA_ADDRESS Hardware address interface L2 address.
IFLA_BROADCAST Hardware address L2 broadcast
address.
IFLA_IFNAME ASCII string device name.
IFLA_MTU MTU of the device.
IFLA_LINK ifindex of link to which this device
is bound.
IFLA_QDISC ASCII string defining egress root
queuing discipline.
IFLA_STATS Interface statistics.And the Netlink message types specific to this service are RTM_NEWLINK, RTM_DELLINK, and RTM_GETLINK 🦋
2.3.3.2. IP Address Service Module
Okay so this service is all about adding, removing, or peeking at info on an IP address sitting on an interface 🌸 The address provisioning service message template is right here, bestie:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Family | Length | Flags | Scope |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Interface Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- Family (8 bits): the address family. It's
AF_INETfor IPv4, andAF_INET6for IPV6. 💅 - Length (8 bits): the length of the address mask. 🎀
- Flags (8 bits): the flag options are:
IFA_F_SECONDARY: for a secondary address (alias interface). ✨IFA_F_PERMANENT: for a permanent address set by the user. When this is not set, it means the address was dynamically created (e.g., by stateless autoconfiguration). 🦋IFA_F_DEPRECATED: defines a deprecated (IPV4) address. 🥀IFA_F_TENTATIVE: defines a tentative (IPV4) address, which means duplicate address detection is still in progress. 🙈
- Scope (8 bits): the address scope where the address stays valid, with these values:
SCOPE_UNIVERSE: global scope. 🌈SCOPE_SITE(IPv6 only): only valid within this site. 🏡SCOPE_LINK: valid only on this device. 🔗SCOPE_HOST: valid only on this host. 🧚
le attributes:
Here's the attribute table, in a code block so nothing gets scrambled 💜
Attribute Description
IFA_UNSPEC Unspecified.
IFA_ADDRESS Raw protocol address of interface.
IFA_LOCAL Raw protocol local address.
IFA_LABEL ASCII string name of the interface.
IFA_BROADCAST Raw protocol broadcast address.
IFA_ANYCAST Raw protocol anycast address.
IFA_CACHEINFO Cache address information.The Netlink messages specific to this service are RTM_NEWADDR, RTM_DELADDR, and RTM_GETADDR. ⭐
3. Currently Defined Netlink IP Services
Okay, so there are a bunch of other IP services that use Netlink too, as we mentioned earlier. But we're only going to talk about a handful of them that got integrated into kernel version 2.4.6. Those are:
NETLINK_ROUTE, NETLINK_FIREWALL, and NETLINK_ARPD3.1. IP Service NETLINK_ROUTE
This one lets CPCs modify the IPv4 routing table in the Forwarding Engine 🛣️ CPCs can also use it to receive routing updates and to collect statistics. It's giving main character energy 👑
3.1.1. Network Route Service Module
This service gives you the power to create, remove, or receive info about a network route. The service message template looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Family | Src length | Dest length | TOS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Table ID | Protocol | Scope | Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- Family (8 bits): the address family. It's
AF_INETfor IPv4, andAF_INET6for IPV6. 💖 - Src length (8 bits): the prefix length of the source IP address. 🌷
- Dest length (8 bits): the prefix length of the destination IP address. 🎯
- TOS (8 bits): the 8-bit TOS (should be deprecated to make room for DSCP). 🫠
- Table ID (8 bits): the table identifier. Up to 255 route tables are supported. The special ones are:
RT_TABLE_UNSPEC An unspecified routing table.
RT_TABLE_DEFAULT The default table.
RT_TABLE_MAIN The main table.
RT_TABLE_LOCAL The local table.The user may assign arbitrary values between RT_TABLE_UNSPEC(0) and RT_TABLE_DEFAULT(253). 🧩
- Protocol (8 bits): this identifies what or who added the route. The options are:
Protocol Route origin.
..............................................
RTPROT_UNSPEC Unknown.
RTPROT_REDIRECT By an ICMP redirect.
RTPROT_KERNEL By the kernel.
RTPROT_BOOT During bootup.
RTPROT_STATIC By the administrator.Values larger than RTPROT_STATIC(4) are not interpreted by the kernel. They're just there for user information, and they may be used to tag the source of routing information or to tell apart multiple routing daemons. See <linux/rtnetlink.h> for the routing daemon identifiers that are already assigned. 🏷️
- Scope (8 bits): the route scope, which means the valid distance to the destination. The values are:
RT_SCOPE_UNIVERSE Global route.
RT_SCOPE_SITE Interior route in the
local autonomous system.
RT_SCOPE_LINK Route on this link.
RT_SCOPE_HOST Route on the local host.
RT_SCOPE_NOWHERE Destination does not exist.The values between RT_SCOPE_UNIVERSE(0) and RT_SCOPE_SITE(200) are available to the user. 🌍
- Type (8 bits): the type of route. Here's the full list:
Route type Description
----------------------------------------------------
RTN_UNSPEC Unknown route.
RTN_UNICAST A gateway or direct route.
RTN_LOCAL A local interface route.
RTN_BROADCAST A local broadcast route
(sent as a broadcast).
RTN_ANYCAST An anycast route.
RTN_MULTICAST A multicast route.
RTN_BLACKHOLE A silent packet dropping route.
RTN_UNREACHABLE An unreachable destination.
Packets dropped and host
unreachable ICMPs are sent to the
originator.
RTN_PROHIBIT A packet rejection route. Packets
are dropped and communication
prohibited ICMPs are sent to the
originator.
RTN_THROW When used with policy routing,
continue routing lookup in another
table. Under normal routing,
packets are dropped and net
unreachable ICMPs are sent to the
originator.
RTN_NAT A network address translation
rule.
RTN_XRESOLVE Refer to an external resolver (not
implemented).- Flags (32 bits): these further qualify the route. The options are:
RTM_F_NOTIFY: if the route changes, notify the user. 🔔RTM_F_CLONED: the route is cloned from another route. 🐑RTM_F_EQUALIZE: allows randomization of the next hop path in multi-path routing (currently not implemented). 🚧
The attributes that apply to this service are:
Attribute Description
---------------------------------------------------
RTA_UNSPEC Ignored.
RTA_DST Protocol address for route
destination address.
RTA_SRC Protocol address for route source
address.
RTA_IIF Input interface index.
RTA_OIF Output interface index.
RTA_GATEWAY Protocol address for the gateway of
the route
RTA_PRIORITY Priority of route.
RTA_PREFSRC Preferred source address in cases
where more than one source address
could be used.
RTA_METRICS Route metrics attributed to route
and associated protocols (e.g.,
RTT, initial TCP window, etc.).
RTA_MULTIPATH Multipath route next hop's
attributes.
RTA_PROTOINFO Firewall based policy routing
attribute.
RTA_FLOW Route realm.
RTA_CACHEINFO Cached route information.Additional Netlink message types that apply to this service are RTM_NEWROUTE, RTM_DELROUTE, and RTM_GETROUTE. 🦋
3.1.2. Neighbor Setup Service Module
This service lets you add, remove, or receive info about a neighbor table entry, like an ARP entry or an IPv4 neighbor solicitation, etc. 🤝 The service message template is shown below:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Family | Reserved1 | Reserved2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Interface Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| State | Flags | Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- Family (8 bits): the address family. It's
AF_INETfor IPv4, andAF_INET6for IPV6. 💜 - Interface Index (32 bits): the unique interface index. 🔢
- State (16 bits): a bitmask of these states:
NUD_INCOMPLETE Still attempting to resolve.
NUD_REACHABLE A confirmed working cache entry
NUD_STALE an expired cache entry.
NUD_DELAY Neighbor no longer reachable.
Traffic sent, waiting for
confirmation.
NUD_PROBE A cache entry that is currently
being re-solicited.
NUD_FAILED An invalid cache entry.
NUD_NOARP A device which does not do neighbor
discovery (ARP).
NUD_PERMANENT A static entry.- Flags (8 bits): the options are:
NTF_PROXY: a proxy ARP entry. 🪞NTF_ROUTER: an IPv6 router. 🛰️
The attributes that apply to this service are:
Attributes Description
------------------------------------
NDA_UNSPEC Unknown type.
NDA_DST A neighbour cache network.
layer destination address
NDA_LLADDR A neighbor cache link layer
address.
NDA_CACHEINFO Cache statistics.Additional Netlink message types that apply to this service are RTM_NEWNEIGH, RTM_DELNEIGH, and RTM_GETNEIGH. 🌸
3.1.3. Traffic Control Service
Okay so the Traffic Control Service 🚦 is where you can set up, look up, or listen for events for all your traffic control stuff 💅. That includes queuing disciplines (the schedulers and queue treatment algorithms, like a priority-based scheduler or RED) and classifiers. Linux Traffic Control is lowkey so flexible because you can stack the blocks in a hierarchy to share traffic resources. It's giving LEGO for packets 🧱✨
Here's the Egress TC block diagram 👀 We're keeping this brief, so for more deets check out [11].
++ ++ +-----+ +-------+ ++ ++ .++
|| . || +------+ | |-->| Qdisc |-->|| || ||
|| ||---->|Filter|--->|Class| +-------+ ||-+ || ||
|| || | +------+ | +---------------+| | || ||
|| . || | +----------------------+ | || .||
|| . || | +------+ | || ||
|| || +->|Filter|-_ +-----+ +-------+ ++ | || .||
|| -->|| | +------+ ->| |-->| Qdisc |-->|| | ||->||
|| . || | |Class| +-------+ ||-+-->|| .||
->dev->|| || | +------+ _->| +---------------+| || ||
|| || +->|Filter|- +----------------------+ || .||
|| || +------+ || .||
|| . |+----------------------------------------------+| ||
|| | Parent Queuing discipline | .||
|| . +------------------------------------------------+ .||
|| . . .. . . .. . . . .. .. .. . .. ||
|+--------------------------------------------------------+|
| Parent Queuing discipline |
| (attached to egress device) |
+----------------------------------------------------------+So a packet first hits a filter that figures out which class it belongs to 🔍. A class is basically a terminal queuing discipline with its own queue. That queue can be super simple, like FIFO, or fancier, like RED or a token bucket 🪣. The outermost queuing discipline is called the parent, and it's usually tied to a scheduler. But inside that scheduler hierarchy there can be other scheduling algorithms too, which is why Linux Egress TC is so flexible 💜
The service message template that makes all this happen is below 💖 It's used for both the ingress and the egress queuing disciplines (check out the egress traffic control model in the FE model section). Every component of the model has its own unique attributes that describe it best, but the common ones are explained right after the diagram.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Family | Reserved1 | Reserved2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Interface Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Qdisc handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Parent Qdisc |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TCM Info |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Family: 8 bits 🏠 The address family. It's AF_INET for IPv4, and AF_INET6 for IPV6.
Interface Index: 32 bits 🔌 The unique interface index.
Qdisc handle: 32 bits 🪪 A unique identifier for an instance of a queuing discipline. It's usually split into major:minor, with 16 bits each. The major number is also the major number of this instance's parent.
Parent Qdisc: 32 bits 👑 This is how queuing disciplines get layered. If this value and the Qdisc handle are the same and equal to TC_H_ROOT, then the defined qdisc is the topmost layer, known as the root qdisc.
TCM Info: 32 bits 🧮 The FE usually sets this to 1, except when the Qdisc instance is in use, and then it's set to imply a reference count. Going from the CPC toward the FEC, it's usually set to 0, unless it's used in the context of filters. In that case, this 32-bit field splits into a 16-bit priority field and a 16-bit protocol field. The protocol is defined in the kernel source <include/linux/if_ether.h>, but the most commonly used one is ETH_P_IP (the IP protocol). The priority is used to break ties when filters overlap in their expressions 🥊
These are the generic attributes that apply to this service 💅
Attribute Description
------------------------------------
TCA_KIND Canonical name of FE component.
TCA_STATS Generic usage statistics of FEC
TCA_RATE rate estimator being attached to
FEC. Takes snapshots of stats to
compute rate.
TCA_XSTATS Specific statistics of FEC.
TCA_OPTIONS Nested FEC-specific attributes.Appendix 3 has an example of configuring an FE component for a FIFO Qdisc 🎀
Lastly, these are the extra Netlink message types that apply to this service 📨 RTM_NEWQDISC, RTM_DELQDISC, RTM_GETQDISC, RTM_NEWTCLASS, RTM_DELTCLASS, RTM_GETTCLASS, RTM_NEWTFILTER, RTM_DELTFILTER, and RTM_GETTFILTER.
3.2. IP Service NETLINK_FIREWALL
Bestie, this one is all about firewalls 🔥 This service lets CPCs receive, manipulate, and re-inject packets through the IPv4 firewall service modules in the FE. First, a firewall rule gets inserted to turn on packet redirection. The CPC tells the FEC whether it wants just the metadata for each packet or the actual data too. If it only wants metadata, it also says the maximum data length to redirect 🧐 The redirected packets stay stored in the FEC while they wait for a verdict from the CPC. The verdict can be a simple accept or drop, and that verdict gets applied to the packet still sitting in the FEC. The verdict can also come with a modified packet to send on instead 🪄
There are two kinds of messages the CPC can send to the FEC: Mode messages and Verdict messages 💌 Mode messages go out right away to tell the FEC what the CPC wants to receive. Verdict messages go out after the CPC has decided what happens to a received packet. The formats are below.
The Mode message comes first 👇
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Mode | Reserved1 | Reserved2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Range |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Mode: 8 bits 🎛️ This is the control information for the packet being sent to the CPC. The types are:
IPQ_COPY_META Copy only packet metadata to CPC.
IPQ_COPY_PACKET Copy packet metadata and packet payloads
to CPC.Range: 32 bits 📏 If the mode is IPQ_COPY_PACKET, this sets the maximum length to copy.
A packet and its metadata, as received from user space, look like this 📦
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Mark |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp_m |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp_u |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| hook |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| indev_name |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| outdev_name |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| hw_protocol | hw_type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| hw_addrlen | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| hw_addr |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| data_len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload . . . |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Packet ID: 32 bits 🆔 The unique packet identifier that the FEC passes to the CPC.
Mark: 32 bits 🏷️ The internal metadata value set to describe the rule the packet was picked up by.
timestamp_m: 32 bits ⏱️ The packet's arrival time, in seconds.
timestamp_u: 32 bits ⏱️ The packet's arrival time in microseconds, added on top of the seconds in timestamp_m.
hook: 32 bits 🪝 The firewall module the packet was picked up from.
indev_name: 128 bits 📥 The ASCII name of the incoming interface.
outdev_name: 128 bits 📤 The ASCII name of the outgoing interface.
hw_protocol: 16 bits 🧬 The hardware protocol, in network order.
hw_type: 16 bits 🔧 The hardware type.
hw_addrlen: 8 bits 📐 The hardware address length.
hw_addr: 64 bits 🧾 The hardware address.
data_len: 32 bits 📦 The length of the packet data.
Payload: size defined by data_len 💎 The payload of the packet that was received.
Now the Verdict message format 💅
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload . . . |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Value: 32 bits ⚖️ This is the verdict to apply to the packet still sitting in the FEC. The verdicts are:
NF_ACCEPT Accept the packet and let it continue its
traversal.
NF_DROP Drop the packet.Packet ID: 32 bits 🆔 The packet identifier that the FEC passed to the CPC.
Data Length: 32 bits 📏 The data length of the modified packet, in bytes. If you don't modify the packet, just set it to 0 🙅♀️
Payload 🎁 Its size is defined by the Data Length field.
3.3. IP Service NETLINK_ARPD 🧚
okay so this service is for CPCs that manage the neighbor table in the FE 💅 The message format between the FEC and CPC lives in the Neighbor Setup Service Module section, so go check that one bestie 🫶
the CPC service is expected to join in on the neighbor solicitation protocol(s) 🔥
the FE sends a neighbor message of type RTM_NEWNEIGH towards the CPC to say "hey, something might have changed on that neighbor's entry" (like the neighbor suddenly looking unreachable 💀). it's giving early warning system 🚨
and RTM_GETNEIGH is how the CPC asks the FE for info on one specific neighbor. simple, iconic, no notes ✨
4. References 📚
4.1. Normative References
[1] Braden, R., Clark, D. and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, June 1994.
[2] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, June 1995.
[3] Blake, S., Black, D., Carlson, M., Davies, E, Wang, Z. and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, December 1998.
[4] Durham, D., Boyle, J., Cohen, R., Herzog, S., Rajan, R. and A. Sastry, "The COPS (Common Open Policy Service) Protocol", RFC 2748, January 2000.
[5] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
[6] Case, J., Fedor, M., Schoffstall, M. and C. Davin, "Simple Network Management Protocol (SNMP)", STD 15, RFC 1157, May 1990.
[7] Andersson, L., Doolan, P., Feldman, N., Fredette, A. and B. Thomas, "LDP Specification", RFC 3036, January 2001.
[8] Bernet, Y., Blake, S., Grossman, D. and A. Smith, "An Informal Management Model for DiffServ Routers", RFC 3290, May 2002.
4.2. Informative References
[9] G. R. Wright, W. Richard Stevens. "TCP/IP Illustrated Volume 2, Chapter 20", June 1995.
[11] http://diffserv.sourceforge.net
5. Security Considerations 🔐
this one is serious bestie, so listen up 👑 Netlink lives in a trusted environment on a single host, split between kernel space and user space. Linux capabilities make sure that only someone with CAP_NET_ADMIN capability (typically, the root user) is allowed to open sockets. no random gremlins getting in 🙅♀️
6. Acknowledgements 💜
shoutouts to everyone who made this possible 🫶
- Andi Kleen, for man pages on netlink and rtnetlink.
- Alexey Kuznetsov is credited for extending Netlink to the IP service delivery model. The original Netlink character device was written by Alan Cox.
- Jeremy Ethridge for taking the role of someone who did not understand Netlink and reviewing the document to make sure that it made sense. iconic behaviour 😭🌸
Appendix 1: Sample Service Hierarchy 🦋
in the diagram below we show a simple IP service, foo, and the interaction it has between CP and FE components for the service (labels 1-3) ✨
the diagram is also used to demonstrate CP<->FE addressing. in this section, we only illustrate the addressing semantics. in Appendix 2, the diagram is referenced again to define the protocol interaction between service foo's CPC and FEC (labels 4-10) 🎀
CP
[--------------------------------------------------------.
| .-----. |
| | . -------. |
| | CLI | / \ |
| | | | CP protocol | |
| /->> -. | component | <-. |
| __ _/ | | For | | |
| | | IP service | ^ |
| Y | foo | | |
| | ___________/ ^ |
| Y 1,4,6,8,9 / ^ 2,5,10 | 3,7 |
--------------- Y------------/---|----------|-----------
| ^ | ^
**|***********|****|**********|**********
************* Netlink layer ************
**|***********|****|**********|**********
FE | | ^ ^
.-------- Y-----------Y----|--------- |----.
| | / |
| Y / |
| . --------^-------. / |
| |FE component/module|/ |
| | for IP Service | |
--->---|------>---| foo |----->-----|------>--
| ------------------- |
| |
| |
------------------------------------------the control plane protocol for IP service foo does the following to connect to its FE counterpart. the steps are also numbered in the diagram above 🧠
- Connect to the IP service foo through a socket connect. A typical connection would be via a call to: socket(AF_NETLINK, SOCK_RAW, NETLINK_FOO).
- Bind to listen to specific asynchronous events for service foo.
- Bind to listen to specific asynchronous FE events.
Appendix 2: Sample Protocol for the Foo IP Service 🍓
our example IP service foo is back to show how you can deploy a simple IP service control using Netlink 💅
these steps continue from Appendix 1 (hence the numbering) 🔢
- Query for current config of FE component.
- Receive response to (4) via channel on (3).
- Query for current state of IP service foo.
- Receive response to (6) via channel on (2).
- Register the protocol-specific packets you would like the FE to forward to you.
- Send service-specific foo commands and receive responses for them, if needed.
Appendix 2a: Interacting with Other IP services 👯♀️
the diagram in Appendix 1 shows another control component configuring the same service. in this case, it's a proprietary Command Line Interface. the CLI may or may not be using the Netlink protocol to talk to the foo component. if the CLI issues commands that will affect the policy of the FEC for service foo, then the foo CPC gets notified. it could then make algorithmic decisions based on this input. for example, if an FE allowed another service to delete policies installed by a different service, and a policy that foo installed was deleted by service bar, there might be a need to propagate this to all the peers of service foo. drama 😭💀
Appendix 3: Examples 🌈
in this example, we show a simple configuration Netlink message sent from a TC CPC to an egress TC FIFO queue. this queue algorithm is based on packet counting and drops packets when the limit goes over 100 packets. we assume the queue sits in a hierarchical setup with a parent 100:0 and a classid of 100:1, and that it gets installed on a device with an ifindex of 4 ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Length (52) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type (RTM_NEWQDISC) | Flags (NLM_F_EXCL | |
| |NLM_F_CREATE | NLM_F_REQUEST)|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number(arbitrary number) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Process ID (0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Family(AF_INET)| Reserved1 | Reserved1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Interface Index (4) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Qdisc handle (0x1000001) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Parent Qdisc (0x1000000) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TCM Info (0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type (TCA_KIND) | Length(4) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Value ("pfifo") |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type (TCA_OPTIONS) | Length(4) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Value (limit=100) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Authors' Addresses 🖊️
Jamal Hadi Salim Znyx Networks Ottawa, Ontario Canada
EMail: hadi@znyx.com
Hormuzd M Khosravi Intel 2111 N.E. 25th Avenue JF3-206 Hillsboro OR 97124-5961 USA
Phone: +1 503 264 0334 EMail: hormuzd.m.khosravi@intel.com
Andi Kleen SuSE Stahlgruberring 28 81829 Muenchen Germany
EMail: ak@suse.de
Alexey Kuznetsov INR/Swsoft Moscow Russia
EMail: kuznet@ms2.inr.ac.ru
Full Copyright Statement ⚖️
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement 🌷
Funding for the RFC Editor function is currently provided by the Internet Society. 💖
📦 where this page comes from
- ietf: rfc3549 2003-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms