CBQ(8)

📦 version 6.1.0, in debian bookworm, ubuntu jammy, ubuntu noble

💖 CBQ(8) — Class Based Queueing 💖

okay bestie, sit down, hydrate, because we're about to talk about the Linux traffic control page for CBQ — Class Based Queueing — and it's giving main character energy the whole way through. 🎀 this is a big one, so buckle up. it comes in multiple parts and this is part one. let's go. ✨

🌸 NAME 🌸

CBQ — Class Based Queueing. that's it, that's the name, periodt. 💅

🦋 SYNOPSIS 🦋

tc qdisc ... dev dev ( parent classid | root) [ handle major: ] cbq
avpkt bytes bandwidth rate [ cell bytes ] [ ewma log ] [ mpu bytes ]
tc class ... dev dev parent major:[minor] [ classid major:minor ] cbq
allot bytes [ bandwidth rate ] [ rate rate ] prio priority [ weight
weight ] [ minburst packets ] [ maxburst packets ] [ ewma log ] [ cell
bytes ] avpkt bytes [ mpu bytes ] [ bounded isolated ] [ split handle &
defmap defmap ] [ estimator interval timeconstant ]

like, copy those exactly, bestie, do not freestyle the flags. 🙈

💜 DESCRIPTION 💜

so. Class Based Queueing is a classful qdisc, which is just a fancy way of saying it implements a rich linksharing hierarchy of classes. it's giving organizational chart but for packets. 📊

it's got shaping elements and prioritizing capabilities, so it can do both at once, which, iconic honestly.

the shaping part? it's done using link idle time calculations, based on the timing of dequeue events and the underlying link bandwidth. that's the whole vibe. 🕰️

🔥 SHAPING ALGORITHM 🔥

okay so shaping is done using link idle time calculations, and then actions get taken if those calculations deviate from the set limits. simple enough, right? bestie, it is not simple enough.

picture this: when you're shaping a 10mbit/s connection down to 1mbit/s, the link will be idle 90% of the time. and if it isn't idle 90% of the time? it needs to be throttled so that it IS idle 90% of the time. that's the assignment. 💀

from the kernel's perspective this is genuinely hard to measure, so CBQ instead derives the idle time from the number of microseconds (in fact, jiffies) that elapse between requests from the device driver for more data. combine that with knowledge of packet sizes, and you get an approximation of how full or empty the link is. no cap.

ngl, this is rather circumspect and it doesn't always arrive at proper results. for example: what even is the actual link speed of an interface that isn't really able to transmit the full 100mbit/s of data, maybe because of a badly implemented driver? 🤔 a PCMCIA network card will also never achieve 100mbit/s because of the way the bus is designed — so again, how do we calculate the idle time? nobody knows, babe.

and the physical link bandwidth may be ill defined in the case of not-quite-real network devices like PPP over Ethernet or PPTP over TCP/IP. in that case the effective bandwidth is probably determined by the efficiency of pipes to userspace — which, and I cannot stress this enough, is not defined. 🫠

during operations, the effective idletime is measured using an exponential weighted moving average (EWMA), which treats recent packets as exponentially more important than past ones. the Unix loadaverage is calculated in the same way. so it's the same technique your uptime is doing. slay, I guess. ⭐

the calculated idle time gets subtracted from the EWMA-measured one, and the resulting number is called avgidle. a perfectly loaded link has an avgidle of zero — packets arrive exactly at the calculated interval. chef's kiss. 👑

an overloaded link has a negative avgidle, and if it gets too negative, CBQ throttles and is then 'overlimit'.

conversely, an idle link might amass a huge avgidle, which would then allow infinite bandwidths after a few hours of silence. 💀 to prevent this, avgidle is capped at maxidle.

if overlimit, in theory, the CBQ could throttle itself for exactly the amount of time that was calculated to pass between packets, and then pass one packet, and then throttle again. in theory. but due to timer resolution constraints, this may not be feasible — see the minburst parameter below. 🎀

🌈 CLASSIFICATION 🌈

within the one CBQ instance, many classes may exist. and each of these classes contains another qdisc — by default, tc-pfifo(8). it's classes all the way down. 🐢

when enqueueing a packet, CBQ starts at the root and uses various methods to figure out which class should receive the data. if a verdict is reached, this process is repeated for the recipient class, which might have further means of classifying traffic to its children, if any. recursion, but make it networking. ✨

CBQ has the following methods available to classify a packet to any child classes:

  1. skb->priority class encoding. can be set from userspace by an application with the SO_PRIORITY setsockopt. the skb->priority class encoding only applies if the skb->priority holds a major:minor handle of an existing class within this qdisc.
  2. tc filters attached to the class.
  3. the defmap of a class, as set with the split & defmap parameters. the defmap may contain instructions for each possible Linux packet priority.

each class also has a level. leaf nodes, attached to the bottom of the class hierarchy, have a level of 0. 🌱

🧚 CLASSIFICATION ALGORITHM 🧚

classification is a loop, which terminates when a leaf class is found. at any point the loop may jump to the fallback algorithm. okay so here's the loop, step by step:

  1. if the packet is generated locally and has a valid classid encoded within its skb->priority, choose it and terminate. 🏁
  2. consult the tc filters, if any, attached to this child. if these return a class which is not a leaf class, restart the loop from the class returned. if it is a leaf, choose it and terminate.
  3. if the tc filters did not return a class, but did return a classid, try to find a class with that id within this qdisc. check if the found class is of a lower level than the current class. if so, and the returned class is not a leaf node, restart the loop at the found class. if it is a leaf node, terminate. if we found an upward reference to a higher level, enter the fallback algorithm. ⬆️
  4. if the tc filters did not return a class, nor a valid reference to one, consider the minor number of the reference to be the priority. retrieve a class from the defmap of this class for the priority. if this did not contain a class, consult the defmap of this class for the BEST_EFFORT class. if this is an upward reference, or no BEST_EFFORT class was defined, enter the fallback algorithm. if a valid class was found, and it is not a leaf node, restart the loop at this class. if it is a leaf, choose it and terminate. if neither the priority distilled from the classid, nor the BEST_EFFORT priority yielded a class, enter the fallback algorithm.

and the fallback algorithm lives outside of the loop, and it goes like this:

  1. consult the defmap of the class at which the jump to fallback occurred. if the defmap contains a class for the priority of the class (which is related to the TOS field), choose this class and terminate.
  2. consult the map for a class for the BEST_EFFORT priority. if found, choose it, and terminate.
  3. choose the class at which break out to the fallback algorithm occurred. terminate. 🫶

the packet is enqueued to the class which was chosen when either algorithm terminated. it is therefore possible for a packet to be enqueued not at a leaf node, but in the middle of the hierarchy. plot twist! 😭

when dequeuing for sending to the network device, CBQ decides which of its classes will be allowed to send. it does so with a Weighted Round Robin process, in which each class with packets gets a chance to send in turn. the WRR process starts by asking the highest priority classes (lowest numerically — highest semantically) for packets, and will continue to do so until they have no more data to offer, in which case the process repeats for lower priorities. 🎠

CERTAINTY ENDS HERE, ANK PLEASE HELP

each class is not allowed to send at length though — they can only dequeue a configurable amount of data during each round.

if a class is about to go overlimit, and it is not bounded, it will try to borrow avgidle from siblings that are not isolated. this process is repeated from the bottom upwards. if a class is unable to borrow enough avgidle to send a packet, it is throttled and not asked for a packet for enough time for the avgidle to increase above zero. sharing is caring but only if you're not isolated, bestie. 💅

I REALLY NEED HELP FIGURING THIS OUT. REST OF DOCUMENT IS PRETTY CERTAIN AGAIN.

⭐ QDISC ⭐

the root qdisc of a CBQ class tree has the following parameters:

  • parent major:minor | root — this mandatory parameter determines the place of the CBQ instance, either at the root of an interface or within an existing class.
  • handle major: — like all other qdiscs, the CBQ can be assigned a handle. should consist only of a major number, followed by a colon. optional.
  • avpkt bytes — for calculations, the average packet size must be known. it is silently capped at a minimum of 2/3 of the interface MTU. mandatory. 😤
  • bandwidth rate — to determine the idle time, CBQ must know the bandwidth of your underlying physical interface, or parent qdisc. this is a vital parameter, more about it later. mandatory.
  • cell — the cell size determines the granularity of packet transmission time calculations. has a sensible default.
  • mpu — a zero sized packet may still take time to transmit. this value is the lower cap for packet transmission time calculations — packets smaller than this value are still deemed to have this size. defaults to zero.
  • ewma log — when CBQ needs to measure the average idle time, it does so using an Exponentially Weighted Moving Average which smooths out measurements into a moving average. the EWMA LOG determines how much smoothing occurs. defaults to 5. lower values imply greater sensitivity. must be between 0 and 31. 🎚️

a CBQ qdisc does not shape out of its own accord. it only needs to know certain parameters about the underlying link. actual shaping is done in classes. so the qdisc is like the group chat admin, and the classes are the ones actually doing the work. periodt. 💖

✨ CLASSES ✨

ok bestie, this is the part where it gets real. Classes have a whole lil host of parameters you gotta configure so they actually do their job. Let's go one by one, no skipping, we don't do that here 💅

  • parent major:minor — this is where the class sits in the hierarchy, its lil spot in the family tree 🌳. If it's attached directly to a qdisc and not to another class, you can just leave minor out, easy. This one is mandatory, no negotiating.
  • classid major:minor — same energy as qdiscs, classes can have names too 🎀. The major number must match the major number of the qdisc it belongs to, that's the rule. It's optional… but bestie, if this class is gonna have children, you need it. Periodt.
  • weight weight — okay so when we're dequeuing to the interface, classes get tried for traffic in a round-robin kinda way. Classes with a higher configured qdisc will generally have more traffic to offer each round, so yeah, it makes sense to let them dequeue more traffic. Here's the cute part: all weights under a class get normalized, so only the ratios matter, the actual numbers are just vibes. Defaults to the configured rate, unless the priority of this class is maximal — in which case it gets set to 1. 🙈
  • allot bytes — allot says how many bytes a qdisc can dequeue during each round of the process. This parameter gets weighted using that renormalized class weight we just talked about, so it's all connected, iykyk.
  • priority priority — in the round-robin process, the classes with the lowest priority field get tried for packets first. Also mandatory 🫶
  • rate rate — the maximum rate this class and all its children combined can send at. Mandatory, obviously, it's the whole point.
  • bandwidth rate — careful bestie, this is different from the bandwidth you specify when creating a CBQ disc ⚠️. This one is only used to determine maxidle and offtime, and those are only calculated when you specify maxburst or minburst. So it's mandatory if you're specifying maxburst or minburst. Otherwise, leave it alone.
  • maxburst — this number of packets is used to calculate maxidle, so that when avgidle is at maxidle, this number of average packets can burst before avgidle drops to 0. Set it higher if you wanna be more tolerant of bursts 🌸. And heads up: you can't set maxidle directly, only through this parameter. That's just how it is.
  • minburst — remember how CBQ needs to throttle when things go overlimit? The ideal solution would be to throttle for exactly the calculated idle time and pass 1 packet. But ngl, Unix kernels generally have a hard time scheduling events shorter than 10ms 😭, so it's better to throttle for a longer period, then pass minburst packets in one go, and then sleep minburst times longer.

The time you wait is called the offtime. Higher values of minburst lead to more accurate shaping in the long term, but to bigger bursts at millisecond timescales. Tradeoffs, bestie, always tradeoffs.

  • minidle — if avgidle is below 0, we're overlimits and we gotta wait until avgidle is big enough to send one packet. But to prevent a sudden burst from shutting down the link for a prolonged period of time, avgidle gets reset to minidle if it gets too low. 🛑

minidle is specified in negative microseconds, so 10 means avgidle is capped at -10us.

  • bounded — this one signifies that this class will not borrow bandwidth from its siblings. Very "I don't need your help, I'm fine" energy 💅
  • isolated — this means the class will not borrow bandwidth to its siblings. (That's exactly what the page says, so that's exactly what I'm telling you 🫶)
  • split major:minor and defmap bitmap[/bitmap] — okay so if consulting the filters attached to a class didn't give a verdict, CBQ can also classify based on the packet's priority. There are 16 priorities available, numbered from 0 to 15 ⭐.

The defmap specifies which priorities this class wants to receive, given as a bitmap. The Least Significant Bit corresponds to priority zero. The split parameter tells CBQ at which class the decision must be made, and that should be a (grand)parent of the class you're adding.

As an example:

'tc class add ... classid 10:1 cbq .. split 10:0 defmap c0'

this configures class 10:0 to send packets with priorities 6 and 7 to 10:1.

And the complimentary configuration would then be:

'tc class add ... classid 10:2 cbq ... split 10:0 defmap 3f'

which would send all packets 0, 1, 2, 3, 4 and 5 to 10:1. See how it all fits together? Obsessed with a well-formed config honestly ✨

  • estimator interval timeconstant — CBQ can measure how much bandwidth each class is using, and tc filters can use that to classify packets with. To figure out the bandwidth it uses a very simple estimator that measures once every interval microseconds how much traffic has passed. And this again is a EWMA, for which the timeconstant can be specified, also in microseconds. The time constant corresponds to the sluggishness of the measurement, or, flip it around, to the sensitivity of the average to short bursts. Higher values mean less sensitivity. 🦋

🌈 SOURCES 🌈

the receipts, bestie, we cite our sources here:

  • Sally Floyd and Van Jacobson, "Link-sharing and Resource Management Models for Packet Networks", IEEE/ACM Transactions on Networking, Vol.3, No.4, 1995
  • Sally Floyd, "Notes on CBQ and Guarantee Service", 1995
  • Sally Floyd, "Notes on Class-Based Queueing: Setting Parameters", 1996
  • Sally Floyd and Michael Speer, "Experimental Results for Class-Based Queueing", 1998, not published.

👀 SEE ALSO 👀

💖 AUTHOR 💖

Alexey N. Kuznetsov, <kuznet@ms2.inr.ac.ru>. This manpage maintained by bert hubert <ahu@ds9a.nl>

and that's a wrap on CBQ(8), bestie 🎀 you made it all the way through, main character energy fr fr. this page came from iproute2 and it's dated 8 December 2001 — yes it's been slaying since 2001, iconic behavior honestly 👑 go shape some traffic, you got this 💅✨

📦 where this page comes from