RFC 2212

Specification of Guaranteed Quality of Service

๐Ÿ“… published September 1997, from ietf

RFC 2212: Specification of Guaranteed Quality of Service ๐Ÿ’–

Okay bestie, here's the title block tea ๐Ÿซถ

  • Network Working Group ๐Ÿงš
  • RFC number: 2212
  • Category: Standards Track ๐Ÿ”ฅ (it's an Internet standards track protocol doc)
  • Date: September 1997 ๐ŸŒธ
  • Authors: S. Shenker (Xerox), C. Partridge (BBN), and R. Guerin (IBM) ๐Ÿ‘‘
  • Obsoletes / Updates: the header doesn't list anything it obsoletes or updates โœจ

Status of This Memo ๐Ÿ“œ

So this doc is an Internet standards track protocol spec, and it's literally begging the community for discussion and suggestions ๐Ÿ™ˆ For the current standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution is unlimited, so share away ๐Ÿ’…

Abstract ๐Ÿฆ‹

This memo spells out the network element behavior you need to deliver a guaranteed service, meaning guaranteed delay and bandwidth, across the Internet. Guaranteed service gives firm, mathematically provable bounds on end-to-end datagram queueing delays, so you can promise both delay and bandwidth at the same time ๐Ÿ”ฅ This spec follows the service specification template described in [1].

Introduction ๐ŸŒˆ

This doc defines the requirements for network elements that support guaranteed service. It's one of a series of docs that describe the network element behavior needed to support different qualities of service in IP internetworks. These services are useful on the global Internet and in private IP networks too โœจ

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Those are the official words, so read them carefully ๐Ÿง

This doc is based on the service specification template in [1]. Go check that one for definitions and extra info about specifying qualities of service in the IP protocol family ๐Ÿ’–

Here's the gist. A flow gets described using a token bucket. Given that description, a service element (a router, a subnet, etc.) computes parameters that describe how it will handle the flow's data. If you combine the parameters from all the service elements on a path, you can compute the maximum delay a piece of data will experience on that path. Main character energy ๐ŸŽ€

There are three things to note about this memo and the service it specifies:

  1. Okay, so the requirements a setup mechanism has to follow to get a guaranteed reservation are carefully specified. But neither the setup mechanism itself nor the way flows get identified is specified. You can create a guaranteed reservation with a protocol like RSVP, with manual configuration of the relevant routers, or with a network management protocol like SNMP. This spec is intentionally independent of the setup mechanism ๐Ÿงš
  2. Getting a bounded delay requires that every service element on the path supports guaranteed service or adequately mimics it. That doesn't mean guaranteed service has to be deployed across the whole Internet to be useful, though! Guaranteed service can have clear benefits even when it's only partly deployed. If it's fully deployed in an intranet, that intranet can support guaranteed service internally. An ISP can also put guaranteed service in its backbone and provide it between customers (or between POPs) ๐ŸŒธ
  3. Service elements produce a delay bound as a result, rather than taking a delay bound as an input to be achieved. So people sometimes assume applications can't control the delay. Lowkey not true: guaranteed service gives applications considerable control over their delay ๐Ÿ’…

Here's the brief version. Delay has two parts: a fixed delay (transmission delays, etc.) and a queueing delay. The fixed delay is a property of the chosen path, and that's determined by the setup mechanism, not by guaranteed service. Only the queueing delay is determined by guaranteed service. As the equations later in this memo show, the queueing delay is mainly a function of two parameters: the token bucket (in particular, the bucket size b) and the data rate (R) the application requests. Both of those values are completely under the application's control. So an application can usually estimate, a priori, what queueing delay guaranteed service will likely promise. And if the delay is bigger than expected, the application can change its token bucket and data rate in predictable ways to get a lower delay ๐Ÿฆ‹

End-to-End Behavior ๐ŸŽ€

The end-to-end behavior from a series of network elements that conform to this document is an assured level of bandwidth. When a policed flow uses that bandwidth, it gets a delay-bounded service with no queueing loss for all conforming datagrams. This assumes no failure of network components and no changes in routing during the life of the flow ๐Ÿ’–

The end-to-end behavior conforms to the fluid model (described under Network Element Data Handling below), in that the delivered queueing delays don't exceed the fluid delays by more than the specified error bounds. More precisely, the end-to-end delay bound is [(b-M)/R*(p-R)/(p-r)]+(M+Ctot)/R+Dtot for p>R>=r, and (M+Ctot)/R+Dtot for r<=p<=R (where b, r, p, M, R, Ctot, and Dtot are defined later in this document) ๐Ÿงฎ

NOTE: The per-hop error terms needed to compute the end-to-end delays are exported by the service module (see Exported Information below). However, the mechanisms needed to collect per-hop bounds and make the end-to-end quantities Ctot and Dtot known to the applications are not described in this specification. Those functions are provided by reservation setup protocols, routing protocols, or other network management functions, and they're outside the scope of this document ๐Ÿ™ˆ

The maximum end-to-end queueing delay (as characterized by Ctot and Dtot) and bandwidth (characterized by R) provided along a path will be stable. In other words, they won't change as long as the end-to-end path doesn't change โœจ

Guaranteed service doesn't control the minimal or average delay of datagrams. It only controls the maximal queueing delay. Also, to compute the maximum delay a datagram will experience, the latency of the path MUST be determined and added to the guaranteed queueing delay. (But as noted below, you can compute a conservative bound of the latency by observing the delay experienced by any one packet ๐Ÿง )

This service is subject to admission control ๐Ÿ’…

Motivation ๐ŸŒŸ

Guaranteed service guarantees that datagrams will arrive within the guaranteed delivery time, and that they won't be discarded due to queue overflows. This holds as long as the flow's traffic stays within its specified traffic parameters. The service is meant for applications that need a firm guarantee that a datagram will arrive no later than a certain time after its source transmitted it. For example, some audio and video "play-back" applications can't tolerate any datagram arriving after their play-back time. Applications with hard real-time requirements will also need guaranteed service ๐ŸŽฌ

This service doesn't try to minimize jitter (the difference between the minimal and maximal datagram delays). It just controls the maximal queueing delay. Because the guaranteed delay bound is a firm one, the delay has to be set large enough to cover extremely rare cases of long queueing delays. Several studies have shown that the actual delay for the vast majority of datagrams can be far lower than the guaranteed delay. So, authors of playback applications should know that datagrams will often arrive way earlier than the delivery deadline, and they'll have to be buffered at the receiving system until the application is ready to process them ๐Ÿ“

This service is one extreme end of delay control for networks. Most other delay control services give much weaker assurances about the resulting delays. To provide this high level of assurance, guaranteed service is typically only useful if every network element along the path provides it (i.e., both the routers and the links that interconnect them). Also, as described in the Exported Information section, providing and using the service effectively requires that the set-up protocol, or another mechanism used to request service, gives service characterizations to intermediate routers and to the endpoints ๐Ÿซถ

Network Element Data Handling Requirements ๐Ÿ› ๏ธ

The network element MUST ensure that the service approximates the "fluid model" of service. The fluid model at service rate R is basically the service a dedicated wire of bandwidth R would provide between the source and receiver. So in the fluid model at a fixed rate R, a flow's service is completely independent of any other flow's service ๐Ÿ’œ

A flow's level of service is characterized at each network element by a bandwidth (or service rate) R and a buffer size B. R represents the share of the link's bandwidth the flow is entitled to, and B represents the buffer space in the network element that the flow may consume. The network element MUST ensure that its service matches the fluid model at that same rate, to within a sharp error bound ๐ŸŽฏ

The definition of guaranteed service relies on this result: the fluid delay of a flow obeying a token bucket (r,b) and served by a line with bandwidth R is bounded by b/R, as long as R is no less than r. Guaranteed service with a service rate R (where R is now a share of bandwidth rather than the bandwidth of a dedicated line) approximates this behavior ๐ŸŒˆ

So, the network element MUST ensure that the queueing delay of any datagram is less than b/R+C/R+D, where C and D describe the maximal local deviation away from the fluid model. It's really important to know that C and D are maximums. For instance, if an implementation has occasional gaps in service (maybe because it's processing routing updates), D needs to be large enough to account for the time a datagram may lose during the gap in service. (C and D are described in more detail in the section on Exported Information ๐Ÿงš)

NOTE: Strictly speaking, this memo only requires that the service a flow receives is never worse than it would receive under this approximation of the fluid model. Giving better service is perfectly acceptable. For instance, if a flow isn't currently using its share, R, algorithms such as Weighted Fair Queueing that temporarily give other flows the unused bandwidth are perfectly acceptable (and they're actually encouraged) ๐Ÿ‘‘

Links are not permitted to fragment datagrams as part of guaranteed service. Datagrams larger than the MTU of the link MUST be policed as nonconformant, which means they'll be policed according to the rules described in the Policing section below ๐Ÿ”ฅ

Invocation Information ๐Ÿ’–

Okay bestie, so guaranteed service gets invoked by handing the network element two things: the traffic (TSpec) and the service you want (RSpec) โœจ. If a flow that already exists shows up with a new TSpec and/or RSpec, it SHOULD be treated as a new invocation, which means admission control SHOULD be reapplied to that flow ๐Ÿ’…. It's basically asking for a fresh vibe check. And flows that shrink their TSpec and/or RSpec (so the new one is strictly smaller than the old one, using the ordering rules in the Ordering section below) SHOULD never be denied service ๐Ÿซถ. Iconic behavior, no notes.

The TSpec is giving token bucket ๐Ÿชฃ, plus a peak rate (p), a minimum policed unit (m), and a maximum datagram size (M) ๐ŸŽ€.

The token bucket has a bucket depth, b, and a bucket rate, r. Both b and r MUST be positive โœจ. The rate, r, is measured in bytes of IP datagrams per second, and it can range from 1 byte per second all the way up to 40 terabytes per second (or close to what is believed to be the maximum theoretical bandwidth of a single strand of fiber) ๐ŸŒˆ. Clearly, especially for big bandwidths, only the first few digits really matter, so floating point representations accurate to at least 0.1% are encouraged ๐Ÿค“.

The bucket depth, b, is also measured in bytes and can range from 1 byte to 250 gigabytes ๐Ÿ“. Again, floating point representations accurate to at least 0.1% are encouraged ๐Ÿ’œ.

The range is intentionally huge so it's ready for future bandwidths ๐Ÿš€. But the range is not meant to imply that a network element has to support the entire thing, so don't panic ๐Ÿ˜Œ.

The peak rate, p, is measured in bytes of IP datagrams per second and has the same range and suggested representation as the bucket rate ๐ŸŒธ. It's the maximum rate at which the source and any reshaping points (reshaping points are defined below) may inject bursts of traffic into the network ๐Ÿ’ฅ. More precisely, for all time periods, the amount of data sent cannot exceed M+pT, where M is the maximum datagram size and T is the length of the time period ๐Ÿ“. Furthermore, p MUST be greater than or equal to the token bucket rate, r ๐Ÿซก. If the peak rate is unknown or unspecified, then p MUST be set to infinity โ™พ๏ธ.

The minimum policed unit, m, is an integer measured in bytes ๐Ÿ”ข. All IP datagrams smaller than size m will be counted, when policed and tested for conformance to the TSpec, as being of size m ๐Ÿ™ˆ. The maximum datagram size, M, is the biggest datagram that will still conform to the traffic specification, and it's also measured in bytes ๐Ÿ“ฆ. The flow MUST be rejected if the requested maximum datagram size is larger than the MTU of the link ๐Ÿšซ. Both m and M MUST be positive, and m MUST be less than or equal to M ๐Ÿงš.

The guaranteed service uses the general TOKEN_BUCKET_TSPEC parameter defined in Reference [8] to describe a data flow's traffic characteristics ๐Ÿ“š. The description above is of that parameter. The TOKEN_BUCKET_TSPEC is general parameter number 127 ๐Ÿ‘‘. Using this parameter for the guaranteed service TSpec keeps things simple in a multi-service environment, which is so smart of them ๐Ÿง .

The RSpec is a rate R and a slack term S, where R MUST be greater than or equal to r, and S MUST be nonnegative ๐ŸŽฏ. The rate R is again measured in bytes of IP datagrams per second and has the same range and suggested representation as the bucket and the peak rates ๐ŸŒŠ. The slack term S is in microseconds โฑ๏ธ. The RSpec rate can be bigger than the TSpec rate because higher rates reduce queueing delay ๐ŸŽ๏ธ. The slack term shows the difference between the desired delay and the delay you'd get by using a reservation level R ๐Ÿซ . The network element can use this slack to reduce its resource reservation for the flow ๐Ÿ’ธ. When a network element chooses to use some of the slack in the RSpec, it MUST follow specific rules for updating the R and S fields of the RSpec, and those rules are specified in the Ordering and Merging section ๐Ÿ“‹. If no slack is specified at the time of service invocation, the slack term, S, is set to zero 0๏ธโƒฃ.

There's no buffer specification in the RSpec, because the network element is expected to work out the buffer space it needs to avoid queueing loss by itself ๐Ÿงฉ. It uses the token bucket and peak rate in the TSpec, the reserved rate and slack in the RSpec, the exported information it receives (Ctot and Dtot, or Csum and Dsum), and its own internal knowledge of how it manages traffic ๐Ÿง โœจ. Main character element behavior, honestly.

The TSpec can be represented by three floating point numbers in single-precision IEEE floating point format, followed by two 32-bit integers in network byte order ๐Ÿ’ป. The first floating point value is the rate (r), the second is the bucket size (b), and the third is the peak rate (p). The first integer is the minimum policed unit (m), and the second integer is the maximum datagram size (M) ๐Ÿฐ.

The RSpec rate term, R, can also be represented using single-precision IEEE floating point ๐Ÿซง.

The Slack term, S, can be represented as a 32-bit integer. Its value can range from 0 to (2**32)-1 microseconds ๐ŸŒ™.

When the r, b, p, and R terms are represented as IEEE floating point values, the sign bit MUST be zero, because all values MUST be non-negative ๐Ÿšจ. Exponents less than 127 (i.e., 0) are prohibited ๐Ÿ™…โ€โ™€๏ธ. Exponents greater than 162 (i.e., positive 35) are discouraged, except for specifying a peak rate of infinity โ™พ๏ธ. Infinity is represented with an exponent of all ones (255) and a sign bit and mantissa of all zeroes ๐Ÿ’€.


Exported Information ๐Ÿ“ค

Each guaranteed service module MUST export at least the following information ๐ŸŽ. All of the parameters described below are characterization parameters ๐Ÿ’….

A network element's implementation of guaranteed service is characterized by two error terms, C and D. They represent how the element's implementation deviates from the fluid model ๐ŸŒ€. These two parameters have an additive composition rule โž•.

The error term C is the rate-dependent error term ๐Ÿƒโ€โ™€๏ธ. It represents the delay a datagram in the flow might experience because of the rate parameters of the flow โณ. An example is the time it takes to serialize a datagram that's been broken up into ATM cells, with the cells sent at a frequency of 1/r ๐Ÿง.

NOTE: It is important to observe that when computing the delay bound, parameter C is divided by the reservation rate R. This division is done because, as with the example of serializing the datagram, the effect of the C term is a function of the transmission rate. Implementors should take care to confirm that their C values, when divided by various rates, give appropriate results. Delay values that are not dependent on the rate SHOULD be incorporated into the value for the D parameter ๐Ÿ“.

The error term D is the rate-independent, per-element error term ๐ŸŒŸ. It represents the worst case non-rate-based transit time variation through the service element ๐Ÿ•ฐ๏ธ. It's generally determined or set at boot or configuration time โš™๏ธ. An example of D is a slotted network, where guaranteed flows are assigned particular slots in a cycle of slots ๐ŸŽฐ. Some part of the per-flow delay may depend on which slots in the cycle are allocated to the flow. In that case, D would measure the maximum amount of time a flow's data, once ready to be sent, might have to wait for a slot ๐Ÿซฅ. (Observe that this value can be computed before slots are assigned, so it can be advertised. For instance, imagine there are 100 slots. In the worst case, a flow might get all of its N slots clustered together, so if a packet was made ready to send just after the cluster ended, the packet might have to wait 100-N slot times before transmitting. In this case, one can easily approximate this delay by setting D to 100 slot times) ๐Ÿงฎ.

If the composition function is applied along the entire path to compute the end-to-end sums of C and D (Ctot and Dtot), and the resulting values are then provided to the end nodes (presumably by the setup protocol), the end nodes can compute the maximal datagram queueing delays ๐Ÿ›ค๏ธ. Moreover, if the partial sums (Csum and Dsum) from the most recent reshaping point (reshaping points are defined below) downstream towards receivers are handed to each network element, then those elements can compute the buffer allocations needed to achieve no datagram loss, as detailed in the section Guidelines for Implementors ๐Ÿช„. The proper use and provision of this service requires that the quantities Ctot, Dtot, Csum, and Dsum be computed. So we assume guaranteed service will mostly be used in contexts where these quantities are made available to end nodes and network elements ๐Ÿซถ.

The error term C is measured in units of bytes ๐Ÿ“. An individual element can advertise a C value between 1 and 2**28 (a little over 250 megabytes), and the total added over all elements can range as high as (2**32)-1. Should the sum of the different elements' delays exceed (2**32)-1, the end-to-end error term MUST be set to (2**32)-1 ๐Ÿšง.

The error term D is measured in units of one microsecond โšก. An individual element can advertise a delay value between 1 and 2**28 (somewhat over two minutes), and the total delay added over all elements can range as high as (2**32)-1. Should the sum of the different elements' delays exceed (2**32)-1, the end-to-end delay MUST be set to (2**32)-1 ๐Ÿฅน.

The guaranteed service is service_name 2 ๐ŸŽ€.

The RSpec parameter is numbered 130 ๐Ÿ”ข.

Error characterization parameters C and D are numbered 131 and 132 ๐Ÿ’œ. The end-to-end composed values for C and D (Ctot and Dtot) are numbered 133 and 134 ๐Ÿฆ‹. The since-last-reshaping-point composed values for C and D (Csum and Dsum) are numbered 135 and 136 โœจ.

Policing โœจ

okay so there are two kinds of policing in guaranteed service ๐Ÿ’… the first one is simple policing (we're just calling it "policing" from here on so it matches the other docs ๐ŸŽ€), where arriving traffic gets compared against a TSpec. the second one is reshaping ๐Ÿฆ‹, where the network tries to nudge traffic that got a little distorted back into the TSpec's shape. and you only find out traffic broke the TSpec during reshaping because the reshaping fails, aka the reshaping buffer overflows ๐Ÿ˜ญ

policing happens at the edge of the network. reshaping happens at all heterogeneous source branch points and all source merge points ๐ŸŒธ a heterogeneous source branch point is where a source's multicast tree splits into several different paths, and the TSpecs of the reservations on those outgoing links aren't all the same. reshaping only needs to happen there if the TSpec on the outgoing link is "less than" the TSpec reserved on the link right upstream (the "less than" is explained in the Ordering section ๐Ÿงš). a source merge point is where paths or trees from two different sources (sharing the same reservation) come together ๐Ÿซถ

it's the invoker's job (a setup protocol, a local config tool, or something similar) to figure out where policing is needed ๐Ÿ‘‘ reshaping can also happen at other spots besides these ones. and policing MUST not be done except at the edge of the network ๐Ÿ™…โ€โ™€๏ธ

now the token bucket and peak rate stuff. traffic MUST obey this rule: over any time period, the amount of data sent can't go over M+min[pT, rT+b-M]. here r and b are the token bucket parameters, M is the maximum datagram size, and T is the length of the time period. (ngl, if p is infinite this just collapses into the regular token bucket requirement ๐Ÿ™ˆ) for this counting, links MUST count any datagram smaller than the minimum policing unit as being of size m ๐Ÿ’– and any datagram that shows up and breaks the M+min[pT, rT+b-M] bound is non-conformant ๐Ÿ’€

at the edge, traffic gets policed so it conforms to the token bucket. non-conforming datagrams SHOULD be treated as best-effort datagrams ๐ŸŒˆ [if and when a marking ability shows up, these non-conformant datagrams SHOULD be "marked" as non-compliant and then treated as best effort at all later routers โœจ]

quick best-effort refresher: it's the default service a network element gives a datagram that isn't part of a flow but was sent between the flow's source and destination ๐Ÿ“ which means if a flow's datagram gets turned into a best-effort datagram, all the flow control that normally applies to best-effort traffic (like RED [2]) applies to it too ๐Ÿ”ฅ

NOTE: There might be situations outside this document's scope, like when a service module's guaranteed service implementation is being used for traffic sharing rather than quality of service. in those cases the goal is to discard non-conforming datagrams ๐Ÿ™ˆ so to allow for that, implementors SHOULD ensure that the action taken for non-conforming datagrams is configurable ๐ŸŽ€

inside the network, policing doesn't give the results we want ๐Ÿ˜ค queueing effects will sometimes make traffic that entered as conformant become non-conformant at some downstream element. so inside the network, any element that wants to police traffic MUST do it by reshaping the traffic to the token bucket ๐Ÿฆ‹ reshaping means delaying datagrams until they're back within TSpec conformance โณ

reshaping works by pairing a buffer with a token bucket and peak rate regulator, and buffering data until it can be sent in line with the token bucket and peak rate parameters ๐Ÿ’… (the token bucket regulator MUST start with its token bucket full of tokens ๐Ÿช™). under guaranteed service, the buffering needed to reshape any conforming traffic back to its original token bucket shape is b+Csum+(Dsum*r), where Csum and Dsum are the sums of the parameters C and D between the last reshaping point and the current one ๐Ÿงฎ you can use the peak rate knowledge at the reshapers to lower these buffer needs (see the section on "Guidelines for Implementors" below ๐Ÿ‘€). and a network element MUST provide the necessary buffers so conforming traffic doesn't get lost at the reshaper ๐Ÿซถ

NOTE: a router that isn't reshaping can still spot non-conforming datagrams (and discard them or schedule them at lower priority) by noticing when queued traffic for the flow goes over b+Csum+(Dsum*r) ๐Ÿ‘€

if a datagram shows up and finds the reshaping buffer full, that datagram is non-conforming ๐Ÿ’€ which means a reshaper is basically policing too ๐Ÿ˜Œ just like a policer, the reshaper SHOULD relegate non-conforming datagrams to best effort ๐ŸŒธ [if marking is available, the non-conforming datagrams SHOULD be marked ๐Ÿท๏ธ]

NOTE: just like with policers, it SHOULD be possible to configure how reshapers handle non-conforming datagrams โš™๏ธ

okay so the big buffer makes it look like reshapers add a ton of delay, but that's not actually the case ๐Ÿ™…โ€โ™€๏ธ if you have a valid TSpec that accurately describes the traffic, reshaping adds very little real delay at the reshaping point, and it won't affect the delay bound at all โœจ and in the normal case, reshaping won't cause any data loss ๐Ÿ’–

BUT (usually at merge or branch points) the TSpec can end up smaller than the actual traffic ๐Ÿ˜ฌ if that happens, reshaping builds a big queue at the reshaping point, which causes lots of extra delay and forces some datagrams to be treated as non-conforming ๐Ÿฅฒ this opens the door to a nasty denial of service attack ๐Ÿšจ a receiver who's happily getting a flow's traffic through best effort gets pushed out by a new receiver who requests a reservation for the flow, but with an inadequate TSpec and RSpec. now the flow's traffic gets policed and maybe reshaped. if the policing was set up to discard datagrams, the best-effort receiver would stop getting traffic entirely ๐Ÿ˜ญ that's why, in the normal case, policers are simply to treat non-conforming datagrams as best effort (and mark them if marking is implemented). this protects against denial of service, but the bad TSpec can still make queueing delays go up ๐Ÿซ 

NOTE: to cut down on datagram reordering problems, reshaping points may want to forward a best-effort datagram from the front of the reshaping queue whenever a new datagram arrives and the reshaping buffer is full ๐Ÿ”„

and readers should also notice that reclassifying datagrams as best effort (instead of dropping them) makes elastic flows easier to support too ๐Ÿฆ‹ they can reserve a modest token bucket, and when their traffic goes over that bucket, the excess just gets sent best effort ๐Ÿ’œ

one more related thing: at all network elements, datagrams bigger than the element's MTU MUST be considered non-conformant ๐Ÿ“ and they SHOULD be classified as best effort (and then they'll either get fragmented or dropped, depending on how the element handles best-effort traffic ๐Ÿงฉ) [again, if marking is available, these reclassified datagrams SHOULD be marked ๐Ÿท๏ธ]

Ordering and Merging โœจ

okay bestie, let's talk about how TSpecs get compared and combined ๐Ÿ’… It's giving math, but it's cute math, I promise ๐ŸŽ€

A TSpec is a substitute ("as good or better than") for another TSpec if all of these are true ๐Ÿ’–

  • both the token rate r and bucket depth b for the first TSpec are greater than or equal to the other's
  • the peak rate p is at least as large in the first one
  • the minimum policed unit m is at least as small in the first one
  • the maximum datagram size M is at least as large in the first one

A TSpec is "less than or equal" to another TSpec if all of these are true ๐ŸŒธ

  • both the token rate r and bucket depth b are less than or equal to the other's
  • the peak rate p is at least as small in the first one
  • the minimum policed unit m is at least as large in the first one
  • the maximum datagram size M is at least as small in the first one

Now, merging! ๐Ÿฆ‹ A merged TSpec is calculated over a set of TSpecs by taking the largest token bucket rate, the largest bucket size, the largest peak rate, the smallest minimum policed unit, and the smallest maximum datagram size across every member of the set. This use of "merging" is similar to the way RSVP [10] uses it. The vibe is that a merged TSpec is one that's adequate to describe the traffic from any one of the constituent TSpecs. Iconic, honestly ๐Ÿ‘‘

A summed TSpec is calculated over a set of TSpecs by adding up the token bucket rates, adding up the bucket sizes, and adding up the peak rates. For the minimum policed unit you take the smallest one, and for the maximum datagram size you take the maximum datagram size parameter. โœจ

A least common TSpec is one that's sufficient to describe the traffic of any one flow in a set of traffic flows. You calculate it over a set of TSpecs by taking the largest token bucket rate, the largest bucket size, the largest peak rate, the smallest minimum policed unit, and the largest maximum datagram size across all members of the set. ๐Ÿงš

Taking the minimum of two TSpecs depends on whether they can be ordered ๐Ÿ™ˆ If one TSpec is less than the other, the smaller TSpec is the minimum. If they can't be ordered that way, you compare the respective values in the two TSpecs and choose the smaller token bucket rate, the larger token bucket size, the smaller peak rate, the smaller minimum policed unit, and the smaller maximum datagram size.

RSpecs get merged in a similar way to TSpecs, so a set of RSpecs gets merged onto a single RSpec by taking the largest rate R and the smallest slack S. More precisely, RSpec A is a substitute for RSpec B if the reserved service rate R in RSpec A is greater than or equal to the value in RSpec B, and the slack S in RSpec A is smaller than or equal to the value in RSpec B. ๐Ÿ’œ

Now the fun part, the actual decision-making. ๐Ÿ”ฅ Each network element gets a service request of the form (TSpec, RSpec), where the RSpec looks like (Rin, Sin). The element processes that request and then does one of two things:

  1. it accepts the request and returns a new RSpec of the form (Rout, Sout);
  2. it rejects the request.

The rules for generating the new RSpec are controlled by the delay constraint, which is:

Sout + b/Rout + Ctoti/Rout <= Sin + b/Rin + Ctoti/Rin

Here, Ctoti is the cumulative sum of the error terms, C, for all the network elements that are upstream of the current element i, including element i itself. So basically, this element uses up (Sin - Sout) of its slack, and it can use that slack to lower its reservation level, as long as the inequality above still holds. ๐ŸŒˆ Also, Rin and Rout MUST also satisfy this constraint:

r <= Rout <= Rin

When several RSpecs, each with a rate Rj (j = 1, 2, ...), are merged at a split point, Rout is the maximum over all the rates Rj, and Sout is the minimum over all the slack terms Sj. ๐Ÿ“

NOTE: The TSpec functions above are used by applications that want to combine TSpecs. It's important to notice, though, that the properties of the actual reservation are determined by combining the TSpec with the RSpec rate (R). ๐Ÿ’€
Because a guaranteed reservation needs both the TSpec and the RSpec rate, shared reservations in RSVP run into some tricky problems, especially where two or more source streams meet. Upstream of the meeting point, it would be ideal to shrink the TSpec and RSpec so they only use as much bandwidth and buffering as each individual source's traffic actually needs. (This can even be necessary if the sender is transmitting over a low bandwidth link.)
The catch is that the RSpec's rate is set to hit a particular delay bound, and it isn't just a function of the TSpec. So changing the RSpec might make the reservation fail to meet the receiver's delay requirements. At the same time, if you don't adjust the RSpec rate, "shared" RSVP reservations using guaranteed service will fail whenever the bandwidth available at a particular link is less than the receiver's requested rate R, even when that bandwidth is enough for the number of senders actually using the link. At this point, this limitation is an open problem for guaranteed service with RSVP. ๐Ÿ˜ญ

Guidelines for Implementors โœจ

okay bestie, this is the implementer's survival guide ๐Ÿงš these bits aren't in any particular order, so just vibe through them ๐Ÿ’…

okay so subnetworks are network elements too ๐Ÿซถ and both routers AND subnetworks MUST support the guaranteed service model if you want guaranteed service to actually work. no cap. subnetworks usually can't negotiate service with IP-based protocols, so as part of providing guaranteed service, routers have to be proxies for the subnets they're attached to ๐ŸŽ€

in some cases the proxy job is super easy ๐Ÿ“ like a leased line run by a WFQ scheduler on the upstream node. the proxy just needs to make sure the sum of all the flows' RSpec rates doesn't go over the line's bandwidth, and it needs to advertise the rate-based and non-rate-based delays of the link as the values of C and D.

other times it's a whole mood ๐Ÿ˜ญ in an ATM network, for example, you might have to set up an ATM VC for the flow and compute the C and D terms for that VC. readers may notice that the token bucket and peak rate used by guaranteed service map directly onto the Sustained Cell Rate, Burst Size, and Peak Cell Rate of ATM's Q.2931 QoS parameters for Variable Bit Rate traffic. iykyk ๐Ÿฆ‹

the promise that datagrams won't get lost comes from setting the router buffer space B equal to the token bucket b plus some error term (described below) โœจ

another subnet thing: the TSpec's token bucket rates measure IP traffic and do not (and cannot) account for link level headers. so the subnetwork network elements MUST adjust the rate and possibly the bucket size to account for adding link level headers. tunnels MUST also account for the extra IP headers they add ๐ŸŒˆ

for datagram networks, you can usually compute a maximum header rate by dividing the rate and bucket sizes by the minimum policed unit. for networks that do internal fragmentation, like ATM, it gets more complicated, since one MUST account for both per-fragment overhead and any wastage (padding bytes transmitted) from mismatches between datagram sizes and fragment sizes. for instance, a conservative estimate of the additional data rate imposed by ATM AAL5 plus ATM segmentation and reassembly is:

((r/48)*5)+((r/m)*(8+52))

that represents the rate divided into 48-byte cells, multiplied by the 5-byte ATM header, plus the maximum datagram rate (r/m) multiplied by the cost of the 8-byte AAL5 header plus the maximum space that can be wasted by ATM segmentation of a datagram (which is the 52 bytes wasted in a cell that contains one byte). but this estimate is likely to be wildly high ๐Ÿ’€ especially if m is small, since ATM wastage is usually much less than 52 bytes. (ATM implementors should be warned that the token bucket may also have to be scaled when setting the VC parameters for call setup, and that this example does not account for overhead incurred by encapsulations such as those specified in RFC 1483.)

to make sure nothing gets lost, network elements will have to allocate some buffering for bursts ๐Ÿซง if every hop implemented the fluid model perfectly, this buffering would simply be b (the token bucket size). however, as noted in the discussion of reshaping earlier, implementations are approximations, and we expect traffic to get more bursty as it goes through the network. still, just like with shaping, the amount of buffering needed to handle the burstiness is bounded by:

b+Csum+Dsum*R

if you account for the peak rate, it can be reduced even further to:

M + (b-M)(p-X)/(p-r) + (Csum/R + Dsum)X

where X is set to r if (b-M)/(p-r) is less than Csum/R+Dsum, and X is R if (b-M)/(p-r) is greater than or equal to Csum/R+Dsum and p>R. otherwise, X is set to p. this reduction comes from the fact that the peak rate limits how fast the burst, b, can be placed into the network. conversely, if a non-zero slack term, Sout, is returned by the network element, the buffer requirements go up by adding Sout to Dsum ๐Ÿ“ˆ

sending applications are encouraged to set the peak rate parameter, and reshaping points are required to conform to it, but it's always acceptable to ignore the peak rate when computing buffer requirements and end-to-end delays. the result is just an overestimate of the buffering and delay ๐Ÿ™ˆ as noted above, if the peak rate is unknown (and thus potentially infinite), the buffering required is b+Csum+Dsum*R. the end-to-end delay without the peak rate is b/R+Ctot/R+Dtot.

the parameter D for each network element SHOULD be set to the maximum datagram transfer delay variation (independent of rate and bucket size) through that network element. for instance, in a simple router, you might compute the difference between the worst case and best case times it takes a datagram to get through the input interface to the processor, and add it to any variation that might happen in how long it takes to get from the processor to the outbound link scheduler (assuming the queueing schemes work correctly) ๐ŸŽ€

for weighted fair queueing in a datagram environment, D is set to the link MTU divided by the link bandwidth. this covers the possibility that a packet arrives just as a maximum-sized packet begins to be transmitted, and that the arriving packet should have departed before the maximum-sized packet. for a frame-based, slotted system like Stop and Go queueing, D is the maximum number of slots a datagram may have to wait before getting a chance to be transmitted โณ

multicasting can make figuring out D trickier ๐Ÿ‘€ in many subnets (ATM being one example), the properties of the subnet may depend on the path taken from the multicast sender to the receiver. there are a few possible approaches. one is to choose a representative latency for the overall subnet and set D to the (non-negative) difference from that latency. another is to estimate subnet properties at exit points from the subnet, since the exit point presumably is best placed to compute the properties of its path from the source ๐Ÿงš

๐Ÿ“ NOTE: There is no fixed set of rules about how a subnet determines its properties, and each subnet technology will have to develop its own set of procedures to accurately compute C and D and slack values.

D is meant to be distinct from the latency through the network element. latency is the minimum time through the device (the speed-of-light delay in a fiber, or the absolute minimum time it would take to move a packet through a router), while parameter D is meant to bound the variability in non-rate-based delay. in practice this distinction is sometimes arbitrary (the latency may be minimal). in cases like that, it's perfectly reasonable to combine the latency with D and advertise any latency as zero ๐ŸŒธ

๐Ÿ“ NOTE: It is implicit in this scheme that, to get a complete guarantee of the maximum delay a packet might experience, a user of this service will need to know both the queueing delay (provided by C and D) and the latency. the latency is not advertised by this service, but it is a general characterization parameter (advertised as specified in [8]).

however, even if latency isn't advertised, this service can still be used ๐Ÿ’– the simplest approach is to measure the delay experienced by the first packet (or the minimum delay of the first few packets) received, and treat this delay value as an upper bound on the latency.

the parameter C is the data backlog resulting from the vagaries of how a specific implementation deviates from a strict bit-by-bit service. so, for instance, for datagramized weighted fair queueing, C is set to M to account for packetization effects ๐Ÿ“

if a network element uses a certain amount of slack, Si, to reduce the amount of resources it has reserved for a particular flow, i, the value Si SHOULD be stored at the network element. later on, if reservation refreshes come in for flow i, the network element MUST use the same slack Si without any further computation. this keeps the reservation process consistent ๐Ÿ”ฅ

here's an example of the slack term in action ๐Ÿง say the required end-to-end delay, Dreq, is larger than the maximum delay of the fluid flow system. the latter is found by setting R=r in the fluid delay formula (for stability, R>=r must be true), and it's given by:

b/r + Ctot/r + Dtot

in this case the slack term is:

S = Dreq - (b/r + Ctot/r + Dtot)

the slack term may be used by network elements to adjust their local reservations, so they can admit flows that would otherwise have been rejected ๐ŸŽ‰ a network element at an intermediate position that can internally differentiate between delay and rate guarantees can use this information to lower the amount of resources allocated to the flow. for example, by taking an amount of slack s <= S, an RCSD scheduler [5] can increase the local delay bound, d, assigned to the flow, to d+s. given an RSpec, (Rin, Sin), it would do that by setting Rout = Rin and Sout = Sin - s.

similarly, a network element using a WFQ scheduler can decrease its local reservation from Rin to Rout by using some of the slack in the RSpec. this can be done with the transformation rules given in the previous section, which make sure that the reduced reservation level won't increase the overall end-to-end delay ๐Ÿ’…


Evaluation Criteria ๐Ÿ”

okay so the scheduling algorithm and admission control algorithm of the element MUST make sure the delay bounds are never violated and datagrams are not lost, as long as a source's traffic conforms to the TSpec. furthermore, the element MUST make sure that misbehaving flows don't affect the service given to other flows ๐Ÿ›ก๏ธ vendors are encouraged to formally prove that their implementation is an approximation of the fluid model ๐Ÿค“


Examples of Implementation ๐Ÿ› ๏ธ

several algorithms and implementations approximate the fluid model. they include Weighted Fair Queueing (WFQ) [2], Jitter-EDD [3], Virtual Clock [4], and a scheme proposed by IBM [5]. a nice theoretical presentation showing these schemes are part of a large class of algorithms can be found in [6] ๐Ÿ“š


Examples of Use ๐ŸŽ€

picture an application that is intolerant of any lost or late datagrams ๐Ÿ˜ฑ it uses the advertised values Ctot and Dtot and the TSpec of the flow to compute the resulting delay bound from a service request with rate R. assuming R < p, it then sets its playback point to:

[(b-M)/R*(p-R)/(p-r)]+(M+Ctot)/R+Dtot

Security Considerations ๐Ÿšจ

๐Ÿšจ bestie, this part is serious, so read it carefully. This memo discusses how this service could be abused to permit denial of service attacks. The service, as defined, does not allow denial of service (although service may degrade under certain circumstances).

Appendix 1: Use of the Guaranteed service with RSVP ๐Ÿงšโ€โ™€๏ธ

the use of guaranteed service together with the RSVP resource reservation setup protocol is specified in reference [9]. that document gives the format of the RSVP FLOWSPEC, SENDER_TSPEC, and ADSPEC objects needed to support applications that want guaranteed service, and it gives information about how RSVP processes those objects. the RSVP protocol itself is specified in Reference [10] ๐Ÿ’–

References โœจ

okay bestie, this is the reading list ๐Ÿ“š every single source the spec leans on is right here, exactly as written so you can go chase them down ๐Ÿ’…

[1] Shenker, S., and J. Wroclawski, "Network Element Service
   Specification Template", RFC 2216, September 1997.

[2] A. Demers, S. Keshav and S. Shenker, "Analysis and Simulation of
   a Fair Queueing Algorithm," in Internetworking: Research and
   Experience, Vol 1, No. 1., pp. 3-26.

[3] L. Zhang, "Virtual Clock: A New Traffic Control Algorithm for
   Packet Switching Networks," in Proc. ACM SIGCOMM '90, pp. 19-29.

[4] D. Verma, H. Zhang, and D. Ferrari, "Guaranteeing Delay Jitter
   Bounds in Packet Switching Networks," in Proc. Tricomm '91.

[5] L. Georgiadis, R. Guerin, V. Peris, and K. N. Sivarajan,
   "Efficient Network QoS Provisioning Based on per Node Traffic
   Shaping," IBM Research Report No. RC-20064.

[6] P. Goyal, S.S. Lam and H.M. Vin, "Determining End-to-End Delay
   Bounds in Heterogeneous Networks," in Proc. 5th Intl. Workshop on
   Network and Operating System Support for Digital Audio and Video,
   April 1995.

[7] A.K.J. Parekh, A Generalized Processor Sharing Approach to Flow
   Control in Integrated Services Networks, MIT Laboratory for
   Information and Decision Systems, Report LIDS-TH-2089, February 1992.

[8] Shenker, S., and J. Wroclawski, "General Characterization
   Parameters for Integrated Service Network Elements", RFC 2215,
   September 1997.

[9] Wroclawski, J., "Use of RSVP with IETF Integrated Services", RFC
   2210, September 1997.

[10] Braden, R., Ed., et. al., "Resource Reservation Protocol (RSVP)
   - Version 1 Functional Specification", RFC 2205, September 1997.

Authors' Addresses ๐Ÿซถ

and finally, the squad behind this spec, with their contact info so you can slide into their inboxes ๐Ÿ’Œ

Scott Shenker
Xerox PARC
3333 Coyote Hill Road
Palo Alto, CA  94304-1314

Phone: 415-812-4840
Fax:   415-812-4471
EMail: shenker@parc.xerox.com


Craig Partridge
BBN
2370 Amherst St
Palo Alto CA 94306

EMail: craig@bbn.com


Roch Guerin
IBM T.J. Watson Research Center
Yorktown Heights, NY 10598

Phone: 914-784-7038
Fax:   914-784-6318
EMail: guerin@watson.ibm.com

that's the whole spec, bestie!! ๐ŸŒˆโœจ the legends who wrote it are Scott, Craig and Roch ๐Ÿ‘‘๐Ÿ’–

๐Ÿ“ฆ where this page comes from