RFC 2990

Next Steps for the IP QoS Architecture

📅 published November 2000, from ietf

RFC 2990: Next Steps for the IP QoS Architecture 🎀

Okay bestie, let's get into it! 💖 This one is Next Steps for the IP QoS Architecture, written by G. Huston of Telstra. Here's the title block tea:

  • Network Working Group 🌸
  • Category: Informational 🦋
  • Date: November 2000 📅
  • Obsoletes / Updates: the header doesn't list any, so it doesn't obsolete or update anything 🙅‍♀️

Status of This Memo 📜

So this memo is giving info to the Internet community 🫶. It does not specify an Internet standard of any kind, and distribution of it is unlimited, so share it with everyone 💅.

Copyright (C) The Internet Society (2000). All Rights Reserved. 🔒

Abstract ✨

There's been loads of progress defining QoS architectures for internet networks 🔥, but some parts still need more elaboration when you try to turn a set of tools into one coherent platform for end-to-end service delivery 🧚. This doc highlights the leftover architectural issues around deploying and using QoS mechanisms inside internet networks 🌈. It also notes where further standards work could help QoS internets get deployed 👑.

This document is the outcome of a collaborative exercise by the Internet Architecture Board 💜.

Table of Contents 🍓

 1. Introduction ...........................................   2
 2. State and Stateless QoS ................................   4
 3. Next Steps for QoS Architectures .......................   6
    3.1 QoS-Enabled Applications ...........................   7
    3.2 The Service Environment ............................   9
    3.3 QoS Discovery ......................................  10
    3.4 QoS Routing and Resource Management ................  10
    3.5 TCP and QoS ........................................  11
    3.6 Per-Flow States and Per-Packet classifiers .........  13
    3.7 The Service Set ....................................  14
    3.8 Measuring Service Delivery .........................  14
    3.9 QoS Accounting .....................................  15
    3.10 QoS Deployment Diversity ..........................  16
    3.11 QoS Inter-Domain signaling ........................  17
    3.12 QoS Deployment Logistics ..........................  17
 4. The objective of the QoS architecture ..................  18
 5. Towards an end-to-end QoS architecture .................  19
 6. Conclusions ............................................  21
 7. Security Considerations ................................  21
 8. References .............................................  22
 9. Acknowledgments ........................................  23
10. Author's Address .......................................  23
11. Full Copyright Statement ...............................  24

1. Introduction 🌟

The default service on the Internet is a best-effort, variable service response 💅. Basically, the network doesn't try to actively tell apart the traffic from different users who are all running at once. When more traffic floods through, the best-effort response changes too, so it can wobble around a lot 🌊.

The whole point of Internet QoS efforts is to add a bunch of selectable service responses on top of that base 🦋. Those responses can be better than best effort through some kind of superior service level 💖, or they can be predictable and unaffected by outside stuff like how many flows are running or how much load they generate ✨.

Any service response depends on two things: the resources available to handle a load, and how much load there actually is 🧠. To offer those distinguished services, the network has to give a differentiated response inside itself AND control how much service-qualified load it lets in. That way the resources it sets aside can actually deliver for the load it's carrying 🎀. This combo of admission control agents and service management elements can be summed up as "rules plus behaviors" 🫶. Using the Differentiated Service architecture terms [4], this admission control is done by a traffic conditioner (an entity that does traffic conditioning and may contain meters, markers, droppers, and shapers). What the conditioner does is governed by explicit or implicit admission control agents 👑.

Honestly, controlling the load is the most troubling bit of QoS architecture 😭. There are lots of well-understood service response mechanisms for IP networks. But getting a set of them, inside a controlled environment, to respond to a set of service loads with a completely consistent response is still a weak spot in existing IP QoS architectures 💀. The control elements cover a bunch of generic requirements: end-to-end application signaling, end-to-network service signaling, and resource management signaling, so policy-based control of network resources is possible 🌈. This control can also have a particular scope, using "edge to edge" signaling meant to support particular service responses within a defined network scope 🧚.

One way to match imposed load to available resources is an application-driven process of service level negotiation, also known as application signaled QoS 🔥. The app first signals its service requirements to the network, and the network responds. The app goes ahead if the network says it can carry the extra load at the requested service level. If the network says it can't accommodate the requirements, the app may still proceed anyway, because the network will handle its data on a best effort basis 🫠. The negotiation can be explicit, with a single negotiation phase followed by a commitment to the service level from the network 🤝.

This application-signaled approach can be used within the Integrated Services architecture, where the app frames its service request inside the resource reservation protocol (RSVP) and passes it into the network 💅. The network can either positively commit to that service profile or reject the request. If the network commits with a resource reservation, the app can pass traffic in, expecting that as long as the traffic stays within the traffic load profile originally tied to the request, the network will meet the requested service levels ✨. The app doesn't need to periodically reconfirm the reservation, because the interaction between RSVP and the network constantly refreshes it while it's active 🔄. The reservation stays in force until the app explicitly requests termination, or the network signals that it's unable to keep the service commitment [3] 🛑.

There are variations on this model. One is an aggregation model where a proxy agent folds a number of app-signaled reservations into one common aggregate reservation along a common sub-path. A matching deaggregator can then rebuild the collection of individual resource reservations when they leave the aggregate region [5] 🧩. The essential feature of this Integrated Services model is its "all or nothing" nature 💅. Either the network commits to the reservation, in which case the requestor doesn't have to keep monitoring the network's level of response, or the network says it cannot meet the resource reservation 😬.

The other approach is to decouple network load control from the application. This is the basis of the Differentiated Services architecture 🦋. Here, the network implements load control as part of admitting traffic into it. It lets in no more traffic within each service category than there are assumed resources to deliver the intended service response 🌸. Some imprecision is unavoidable, since traffic can take an arbitrary path through the network 🙈. From the application's side, this is a service request without prior negotiation: the app asks for a particular service response by simply marking each packet with a code for the desired service 💜.

Architecturally, this decouples the end systems from the network. That lets a network run an active admission function to moderate the workload placed on its resources, without referring to individual resource requests from end systems 🧚. Operators get more ability to manage their resources and keep their services' integrity 👑. The trade-off is a greater potential for imprecision when matching applications' service requirements to the network's service capabilities 😭🫶.

2. State and Stateless QoS 🎀

okay so these two approaches to load control are basically the state-based one and the stateless one 🦋

The Integrated Services model treats the running total of honored service requests as equal to the reserved resource levels in the network 💅 For a reservation to be honored, the network must maintain some form of remembered state that describes what got reserved and which network path the reserved service will run over 🔒 That's how the network keeps the reservation's integrity locked in. On top of that, each active network element along the path must maintain a local state so incoming IP packets can be sorted into a reservation class ✨ That classification puts each packet into a packet flow context tied to a service response that matches the original end-to-end reservation. This local state also covers metering packets for conformance on a flow-by-flow basis, plus the extra overhead of keeping the state of every one of those meters 😭

In the Differentiated Services model, the packet just gets marked with a code that triggers the right service response from whichever network elements handle it 🧚 So there's no strict requirement to install per-reservation state on those elements. The end application or service requestor is not required to tell the network ahead of time where the traffic is going, or what the intended traffic profile or associated service profile is 🙈 Without that info, any per-application or per-path resource reservation is not feasible 💀 In this model there's no maintained per-flow state inside the network 🌸

The state-based IntServ model has the potential to support greater accuracy and a finer level of granularity in how the network responds to service requests 💖 Each individual application's request can generate a reservation state in the network, meant to keep the reserved resources from being reassigned or preempted to serve other reservations or best effort traffic loads 👑 This model is meant to be exclusionary, so other traffic gets displaced to hit the reservation's service targets 😤

As noted in RFC2208 [2], there are several areas of concern about deploying this kind of service architecture 🙃 For per-flow scalability, the resource requirements (computational processing and memory consumption) for per-flow reservations on routers go up in direct proportion to the number of separate reservations that need to be accommodated 📈 Router forwarding performance may be impacted adversely by the packet-classification and scheduling mechanisms that give differentiated services to those resource-reserved flows 🐢 IntServ also challenges the queuing mechanisms, since you have to allocate absolute levels of egress bandwidth to individual flows while still supporting an unmanaged low priority best effort traffic class 🌈

The stateless approach is more approximate in its outcomes 🫠 There's no explicit negotiation between the application's signaling of the service request and the network's ability to deliver a particular service response. If the network can't meet the request, the request simply will not be honored. There's no requirement for the network to tell the application that the request can't be honored, so it's up to the application to figure out whether the service wasn't delivered. The major perk is excellent scaling from the network's point of view 🚀 If the network can support a limited number of discrete service responses and the routers use per-packet marking to trigger them, then the processor and memory requirements in each router do not grow in proportion to the traffic passing through. Of course, there's a compromise: the service response is more approximate as seen by the end client, and scaling up the number of clients and applications may not necessarily result in a highly accurate service response for every client's application 🤷‍♀️

This document isn't going to dig deeper into these service architectures. For an overview of the Integrated Services Architecture (IntServ), check out RFC1633 [3], and for the Differentiated Services architecture (DiffServ), check out RFC2475 [4] 📚

These two approaches are the endpoints of a continuum of control models 🧩 Fine-grained, per-application-invocation reservations can be aggregated into larger, more general, and potentially more approximate aggregate reservation states. End-to-end, element-by-element reservation control can be progressively approximated by treating a collection of subnetworks, or an entire transit network, as a single aggregate service element. There's a bunch of work in progress aiming at these aggregated control models 🍓 including aggregation of RSVP [5], the RSVP DCLASS Object [6] to let Differentiated Services Code Points (DSCPs) ride inside RSVP message objects, and running Integrated Services over Differentiated Services networks [7] 💜

3. Next Steps for QoS Architectures 🌟

Both the Integrated Services architecture and the Differentiated Services architecture have some critical elements, as currently defined, that seem to be acting as deterrents to widespread deployment 😬 Some of these issues will probably be addressed through the efforts to bring aggregated control and response models into these QoS architectures, while others may require further refinement through standards-related activities 🫶

3.1. QoS-Enabled Applications ✨

okay so the big open question for QoS architectures is whether QoS is a per-application service, a transport-layer option, or both 🤔 Per-application services have obvious implications of extending the QoS architecture into some form of Application Protocol Interface (API), so applications could negotiate a QoS response from the network and alter their behavior based on the outcome 💅 Examples of this approach include GQOS [8] and RAPI [9]. With a transport layer option, any application could have its traffic carried by some form of QoS-enabled network services just by changing the host configuration, or by changing the configuration at some other network control point, without making any explicit changes to the application itself 🦋 The strength here is that there's no need to substantially alter application behavior, because the application is totally unaware of the administratively assigned QoS 🫶 The weakness is that the application can't communicate what might be useful information to the network or to the policy systems managing the network's service responses 😭 Without that info, the network might give a service response that's way better than the application's true requirements, or way worse than what the application needs to function correctly 💀 There's also an extra weakness for the class of applications that can adapt their traffic profile to whatever resources are available in the network. As a transport level mechanism, any network availability information the transport level can see never gets passed back up to the application 🙈

In the case of the Integrated Services architecture, the transport layer approach doesn't seem to be an available option, because the application does require some alteration to function correctly here 🎀 The application must be able to give the service reservation module a profile of its anticipated traffic, or in other words, it must be able to predict its own traffic load 🔮 It also must be able to share the reservation state with the network, so that if the network state fails, the application can be told about the failure 🚨 The bigger picture is that a network can only come up with an accurate response to an application's requirements if the application is willing to offer a precise statement of its traffic profile, and is willing to be policed so its traffic fits within that profile 🧚

For Differentiated Services, there's no explicit provision for the application to communicate with the network about service levels 🌸 This does allow a transport level option inside the end system that doesn't require explicitly altering the application to mark its generated traffic with one of the available Differentiated Services service profiles 💜 But whether or not the application knows about those service profiles, there's no level of service assurance for the application in this model 😤 If the Differentiated Services boundary traffic conditioners enter a load shedding state, the application isn't signaled about it, and isn't explicitly aware that the requested service response isn't being provided by the network 👀 If the network itself changes state and can't meet the cumulative traffic loads admitted by the ingress traffic conditioners, neither the ingress traffic conditioners nor the client applications are informed that the associated service quality isn't being maintained 💔 There's no explicit need to alter application behavior in this architecture, since the basic DiffServ mechanism is managed within the network itself. The catch is that an application may not know whether a particular service state is actually being delivered to it 🫠

There's potential in using an explicit signaling model, like the one IntServ uses, but carrying a signal that lets the network manage the application's traffic within an aggregated service class [6] 🌈 Here the application doesn't hand over a complete picture of its intended service profile. Instead, it provides some additional information to help the network manage its resources, both in terms of the generic service class the network can associate with the application's traffic and the intended path of that traffic through the network 🛤️

Receiver capability negotiation is another big factor for QoS-enabled applications ✨ There's no point in the sender setting up a QoS-enabled path across a network to a receiver that can't absorb the resulting data flow 🙅‍♀️ So QoS-enabled applications also need some form of end-to-end capability negotiation, possibly through a generic protocol, letting the sender match its QoS requirements to the minimum of two things: the flow resources the network can provide and the flow resources the receiver can process 🧠 With Integrated Services, the application's end-to-end interaction can be built right into the RSVP negotiation 🎯 With Differentiated Services, there's no clear path to fold that receiver control into the signaling model as it stands 🥲

If high quality services are going to be provided, where "high quality" is implied as being "high precision with a fine level of granularity," then every part of the network that might be involved in servicing the request either has to be over-provisioned so no load state can compromise the service quality, or the network element has to explicitly allocate resources to each flow associated with each service request 💪

For end-to-end service delivery, it does seem like QoS architectures will need to extend all the way to the level of the application requesting the service profile 👑 It looks like further refinement of the QoS architecture is needed to fit DiffServ network services into an end-to-end service delivery model, as noted in [7] 🫶

3.2. The Service Environment ✨

so the outcome of looking at these two approaches to QoS architecture inside the network is that there doesn't seem to be a single comprehensive service environment that has both service accuracy and scaling properties 😩

The maintained reservation state of the Integrated Services architecture and the end-to-end signaling function of RSVP are part of a service management architecture, but it's not cost effective, or even feasible, to run per-application reservation and classification state across the high speed core of a network [2] 🚫

Now, while the aggregated behavior state of Differentiated Services has excellent scaling properties, the lack of end-to-end signaling facilities means it can't operate in isolation in any environment 🥺 DiffServ can be characterized as a boundary-centric operational model 🧱 In this boundary-centric architecture, the signaling of resource availability from the interior of the network to the boundary traffic conditioners isn't defined, and neither is the signaling from the traffic conditioners to the application living on the end system 🕳️ This has been noted as an additional work item in the IntServ operations over DiffServ work, which is concerned with "definition of mechanisms to efficiently and dynamically provision resources in a DiffServ network region". This might include protocols by which an "oracle" (...) conveys information about resource availability within a DiffServ region to border routers." [7] 🔍

What seems to be needed in the Differentiated Services service model is both resource availability signaling from the core of the network out to the DiffServ boundary, and some form of signaling from the boundary to the client application 🎀

3.3. QoS Discovery ✨

there's no robust mechanism for network path discovery with specific service performance attributes 😬 Both IntServ and DiffServ just assume the best effort routing path is used, where the path is either able to sustain the service load or it isn't 🤷‍♀️

Now, if the deployment of service differentiating infrastructure is going to be piecemeal, even in the early stages of a QoS rollout, that assumption might not be warranted 🧐 If so, how can a host application figure out whether there's a distinguished service path to the destination? 🗺️ Neither architecture has existing mechanisms to query the network about its potential to support a specific service profile 🙃 A query like that would need to examine a bunch of candidate paths, rather than just looking at the lowest metric routing path, so this discovery function is likely to be tied to some form of QoS routing functionality 🚦

So from this perspective, the model of service discovery and the related task of resource reservation may still need further refinement 💖

3.4. QoS Routing and Resource Management 🗺️

okay so QoS routing has kinda been developed at a distance from QoS architectures 🙈 like they're in different friend groups lol. the big assumption in the current QoS models is that the routing best effort path gets used for both best effort traffic AND distinguished service traffic 💅

there's no explicit architectural option to let the network service path line up along anything other than the single best routing metric path. that's a problem, because available network resources can't be efficiently applied to meet service requests 😭. maximizing network efficiency would mean some form of path selection is needed inside a QoS architecture, so the whole set of service requirements can be optimally supported within the network's aggregate resource capability ✨

on top of path selection, SPF-based interior routing protocols can flood link metric information across all network elements 🌊. that seems like a promising direction for control-level signaling between the network interior and the network admission elements 🫶. then admission systems can admit traffic based on current resource availability, instead of necessarily conservative, statically defined admission criteria. no more being stuck with overly cautious rules, periodt 💜

there's also a more fundamental issue with resource management and traffic engineering 👀. single path selection with static load characteristics doesn't match a networked environment with a richer mesh of connectivity and dynamic load characteristics. to use a rich connectivity mesh efficiently, you need to direct traffic with a common ingress and egress point across a set of available network paths, spreading the load over a broader collection of network links. at its most basic this is essentially a traffic engineering problem 🎀. to support it, you need to calculate per-path dynamic load metrics and let the network's ingress system distribute incoming traffic across those paths according to some model of desired traffic balance.

applying this to a QoS architecture would mean each path has some form of vector of quality attributes, and incoming traffic gets balanced across a subset of available paths, where the traffic's quality attribute is matched with each path's quality vector. this changes the semantics of traffic engineering, and there's a matching shift in how the path's quality vector is calculated and interpreted 🦋. what needs measuring isn't the path's resource availability level (or idle proportion), but the path's potential to carry additional traffic at a certain level of quality. this potential metric lets existing lower priority traffic be displaced to alternative paths. so the path's quality metric is better read as a metric describing the path's displacement capability, rather than a resource availability metric 🧠

this whole area of active network resource management, plus dynamic network resource discovery and the related control level signaling to network admission systems, is still a topic for further research right now 🔬

3.5. TCP and QoS 🌀

a congestion-managed, rate-adaptive traffic flow (like TCP) uses feedback from the ACK packet stream to time its subsequent data transmissions 📦. the resulting flow rate is an outcome of the service quality given to both the forward data packets and the reverse ACK packets. if the network treats the ACK stream with a different service profile than the outgoing data packets, it's an open question how much the data forwarding service might be compromised in terms of achievable throughput 🤔. high jitter on the ACK stream can cause ACK compression, which in turn causes high burst rates on the subsequent data send 💥. those bursts stress the network's service capacity and compromise TCP throughput rates 😩.

one way to address this is some form of symmetric service, where ACK packets get the same service class as the forward data packets 🪞. but if symmetric service profiles matter for TCP sessions, how do you structure that without incorrectly accounting for service usage? in other words, how can both directions of a TCP flow be accurately accounted to one party? 🧾

then there's the interaction between the routing system and the two TCP data flows 🛣️. the Internet routing architecture doesn't intrinsically preserve TCP flow symmetry, so the network path taken by the forward packets may not exactly match the path used by the reverse packet flow.

TCP also exposes an extra performance constraint related to how traffic conditioning elements work in a QoS-enabled network 🙃. traffic conditioners in QoS architectures are typically specified using a rate enforcement mechanism of token buckets. token bucket conditioners behave analogously to a First In First Out queue, and they impose tail drop behavior on TCP streams. that tail drop can produce TCP timeout retransmission, unduly penalizing average TCP goodput to a level that may be well below the level the token bucket conditioner specifies. honestly, token buckets can be considered TCP-hostile network elements 😤💀

the bigger issue here: providing some form of assured service to congestion-managed flows needs traffic conditioning elements that operate using weighted RED-like control behaviors within the network, with less deterministic traffic patterns as the outcome 🌈. a requirement to manage TCP burst behavior through token bucket control mechanisms is most appropriately handled in the sender's TCP stack 🧁.

there are a bunch of open areas in this topic that would benefit from more research 📚. the interaction between the end-to-end TCP control system and a collection of service differentiation mechanisms within a network has a large number of variables. the issues involve the time constants of the control systems, the amplitude of feedback loops, the extent to which each control system assumes an operating model of other active control systems applied to the same traffic flow, and the mode of convergence to a stable operational state for each control system 🌀.

3.6. Per-Flow States and Per-Packet classifiers 🔍

both IntServ and DiffServ use packet classifiers as an intrinsic part of their architecture 🧩. these classifiers can be thought of as coarse or fine level classifiers. fine-grained classifiers isolate elements of traffic from an invocation of an application (a `micro-flow') and use a number of fields in the IP packet header to do it, typically including source and destination IP addresses and source and destination port addresses. coarse-grained classifiers isolate traffic belonging to an aggregated service state, and typically use the DiffServ code field as the classifying field. with DiffServ, it's possible to use fine-grained classifiers at the network ingress element and coarse-grained classifiers within the network interior 🏠.

in flow-sensitive IntServ deployments, every active network element that does active service discrimination is required to operate fine-grained packet classifiers 🚨. the granularity can be relaxed with aggregate classifiers [5], but at the expense of the precision and accuracy of the service response 😬.

within IntServ, fine-grained classifiers are defined down to the granularity of an individual traffic flow, using the packet's 5-tuple of (source address, destination address, source port, destination port, protocol) to identify it. DiffServ Multi-Field (MF) classifiers can also use this 5-tuple to map individual traffic flows into supported behavior aggregates 🎯.

IPSEC, NAT and various forms of IP tunnels cause an occlusion of flow identification in the IP packet header, combining individual flows into a larger aggregate state that may be too coarse for the network's service policies 🫠. the problem is they may happen in the network path invisibly to the end application, which compromises the application's ability to tell whether the requested service profile is actually being delivered 👁️. for IPSEC, there's a proposal to carry the IPSEC Security Parameter Index (SPI) in the RSVP object [10], as a surrogate for the port addresses 🔐. for NAT and various tunnel forms, there appears to be no coherent way to preserve fine-grained classification characteristics across NAT devices or across tunnel encapsulation 💔.

IP packet fragmentation also affects the network's ability to identify individual flows 🧩, because trailing fragments won't include the TCP or UDP port address information. that means trailing fragments of a packet within a distinguished service class could get classified into the base best effort service category, delaying ultimate delivery of the IP packet to the destination until those trailing best effort fragments arrive 🐌.

the takeaway: QoS services come with caveats that should be placed on both the application and the network 📝. applications should perform path MTU discovery in order to avoid packet fragmentation 🛤️. deploying payload encryption, header address translation and header encapsulation should be done with due attention to their potential impacts on service delivery packet classifiers 💅.

3.7. The Service Set 🍓

the underlying question here: how many distinguished service responses are enough to provide a functionally adequate range of service responses? 🤔

the Differentiated Services architecture doesn't put any limiting restrictions on how many potential services a network operator can offer 🙌. the operator may be limited to up to 64 discrete services in terms of the 6 bit service code point in the IP header, but since each operator can define the mapping from service to code point, there can be any number of potential services 🌸.

as always, there's such a thing as too much of a good thing 😅. a large number of potential services leads to issues around end-to-end service coherency when spanning multiple network domains. a small set of distinguished services can be supported across a large set of service providers by equipment vendors and application designers alike ✨. an ill-defined large set of potential services often serves little productive purpose 🙄. this points to a potential refinement of the QoS architecture: define a small core set of service profiles as "well-known" service profiles, and put all other profiles in a "private use" category 💖

3.8 Measuring Service Delivery ✨

okay so bestie, any QoS architecture needs a network management approach that gives a coherent view of how the network is operating 🫶 This is NOT the usual element-by-element way of looking at things. The goal here is to see the available resources along one particular path through the network, and to map that view onto an admission control function that decides whether a service-differentiated flow can be admitted onto that chosen path 🛣️

Managing admission through resource availability measurement is a big deal, but there's also a need to measure the operating parameters of the service that's actually delivered 📊 The network operator needs measurement methodologies so they can give objective measurements that back up the claim that the delivered service quality matched the service specifications. The client needs a methodology too, so they can measure the quality they're getting and show that any extra cost of premium services is worth it because the application performs better 💸✨

Honestly, these measurement methodologies seem to fall under further refinement of the QoS architecture 🧠

3.9 QoS Accounting 💰

It's reasonable to expect that premium and customized services will come with a higher tariff 💅 Delivering a distinguished service means using some extra network resources, and the price bump should reflect that changed resource allocation. An incremental tariff does two things: it puts the added cost on the clients asking for a disproportionate amount of resources, and it gives a way to control how much demand there is for premium service levels 🎀

If premium services are going to have incremental tariffs, then the network needs some accounting of how premium services get used, tied to a particular client 🧾 So far there's no definition of an accounting model for this, and no definition of how to gather the data needed for resource accounting. Not yet, anyway 🙈

This QoS service model could seriously change how Internet service is provided 😭 The usual model, in both the public Internet and enterprise networks, is an access model: the client's tariff depends on the characteristics of their access to services, not on how much they actually use the service. QoS creates a strong push toward usage-based tariffs, where the price depends on how much of the network's resources are used. That in turn creates a requirement to meter resource use, which is a form of usage accounting 📏 This topic has already been studied in the IETF under the name "Internet Accounting" [11]. Refining those concepts for QoS accounting could be a productive first step toward a standards-based model for QoS accounting 🌈

3.10 QoS Deployment Diversity 🌸

It's extremely unlikely that any single form of service differentiation technology will get rolled out across the entire Internet and all enterprise networks 🙃

Some networks will deploy some form of service differentiation while others won't. Some of these service platforms will interoperate seamlessly, and others less so 🫠 Expecting every application, host system, network router, network policy, and inter-provider arrangement to merge into one homogeneous service environment that supports a broad range of service responses is a bit of a stretch, given how different the available technologies and industry business models are 💀 It's more likely we'll see a bunch of small-scale deployments of service differentiation mechanisms, plus some efforts to bridge those environments together somehow 🌉

In this heterogeneous service environment, service capability discovery is just as critical as being able to invoke service responses and measure the outcomes 🔍 QoS architectures will need protocol capabilities that support service discovery mechanisms 🧚

A deployment this mixed also adds scaling pressure to the operational network, because the network now has an extra dimension of size 📈 Every potential path to every host may be qualified by the service capabilities of that path. One path might be a candidate best effort path, while another offers a closer match between the desired service attributes and what the path can sustain 🎯 Inter-domain policy affects this path choice too, since inter-domain transit agreements may specifically limit the types and total level of quality requests that can be supported between domains 🚧 Most of the scaling brunt will land in the inter-domain and intra-domain routing space, where there's pressure to increase the number of attributes on a routing entry, and to use the routing protocol in some form of service signaling role 😤

3.11 QoS Inter-Domain Signaling 🛰️

Path selection for QoS is both an intra-domain (interior) and an inter-domain (exterior) issue 🧩 In the inter-domain space, current routing technologies let each domain connect to a number of other domains, and express its policies on received traffic through inter-domain route object attributes. Each domain can also express its policies on sent traffic through boundary route object filters, which let a domain state its preference for one domain's advertised routes over another's 🤝 The inter-domain routing space is a state of dynamic equilibrium between all these route policies ⚖️

Differentiated services add another dimension to this policy space ✨ For example, a provider might have an interconnection agreement with one party to exchange best effort traffic, and a separate agreement with a second party to exchange service-qualified traffic. The result is that the service provider needs external route advertisements qualified by the accepted service profiles 📬 Generalizing from that, it's reasonable to suggest we'll need routing advertisements qualified with some form of service quality attributes 💜 That implies we'll need some form of quality vector-based forwarding function, at least in the inter-domain space, plus an associated routing protocol that can pass a quality of service vector in an operationally stable way 🫶

This requirement means the number of objects routing systems have to manage must grow dramatically 📚 As the size and number of objects managed within a routing domain increase, the calculation of a dynamic equilibrium of import and export policies between interconnected providers faces the same scaling pressure 😵‍💫

This also affects inter-domain forwarding, because in a services-differentiated environment the forwarding decision is qualified by some form of service quality vector 🧭 That's needed to pass exterior traffic to the right exterior interconnection gateway 🚪

3.12 QoS Deployment Logistics 🚀

So how does a widespread service-aware network even get started? 🤔 Which gets built first: the host applications, or the network infrastructure?

No network operator will sink significant investment into deploying and supporting distinguished service infrastructure unless there's a set of clients and applications ready to use those facilities right away 💸 Clients won't invest in enhanced services unless they see performance gains in applications designed to take advantage of them 📉 No application designer will bother building service quality features into an application unless there's a widely deployed operating model that makes the extra application complexity worth it, and clients willing to buy those applications 🙄 Everyone's waiting for everyone else to move first, so widespread deployment of distinguished services may need some other external push 🌊

Other parts of this deployment picture involve network provisioning and the related task of traffic engineering 🛠️ Engineering a network for best effort flows follows a well-understood pattern: match the network points where users are concentrated with content delivery network points that have best effort paths 🗺️ Adding QoS-mediated traffic engineering to the provisioning model means provisioning also needs input from a QoS demand model 📊

4. The objective of the QoS architecture 🎯

What exactly is QoS trying to solve? 🤨 This is probably one of the most fundamental questions behind the whole QoS effort, and how many different possible answers there are shows how broad the scope of the QoS effort is 🌈

All of the following responses are part of the QoS intention 💖:

  • To control the network service response such that the response to a specific service element is consistent and predictable. 🔁
  • To control the network service response such that a service element is provided with a level of response equal to or above a guaranteed minimum. 🛡️
  • To allow a service element to establish in advance the service response that can or will be obtained from the network. 🔮
  • To control the contention for network resources such that a service element is provided with a superior level of network resource. 👑
  • To control the contention for network resources such that a service element does not obtain an unfair allocation of resources (to some definition of 'fairness'). ⚖️
  • To allow for efficient total utilization of network resources while servicing a spectrum of directed network service outcomes. ⚡

Broadly speaking, the first three responses can be seen as 'application-centric', and the last as 'network-centric' 🧁 It's critical to keep in mind that none of these responses can be addressed in isolation in any effective QoS architecture 🫵 Within the end-to-end architectural model of the Internet, applications make minimal demands on the underlying IP network 🎀 In TCP's case, the protocol uses an end-to-end control signal approach to dynamically adjust to the prevailing network state 🧚 QoS architectures add a rather different constraint: the network takes an active role in resource allocation and service delivery, instead of being a passive object that requires end systems to adapt 💅✨

5. Towards an end-to-end QoS architecture 🎀

Okay bestie, the big challenge for QoS is fixing the weak spots we just talked about and smooshing all the pieces into one cohesive thing that can keep end-to-end service models working across a huge range of internet platforms 💅. It should be noted that this may not end up as ONE architecture. It's totally possible to see several end-to-end approaches built from different mixes of the existing parts 🦋.

One approach is to combine both architectures into an end-to-end model. IntServ handles the part where applications interact with the network, and DiffServ manages admission to the network's resources [7]. The basic tension here is the per-application view of IntServ clashing with the network boundary-centric view of DiffServ 🤔.

One building block for this kind of end-to-end service architecture is a service signaling protocol. RSVP can cover the needs of applications that want a per-service end-to-end signaling environment 📡. The abstracted RSVP model is a discovery signaling protocol. An application uses a single transaction to tell both the network and the remote party what service it needs, and the response lets those network elements commit to those requirements ✨. The catch is the element-by-element approach to service commitment, which means each network element has to do some signaling and processing for the state being imposed on it. For high precision services, that means per-flow signaling and per-flow processing. This fine-grained approach is seen as an unacceptable amount of overhead on the central core elements of large carrier networks 😩.

The DiffServ approach uses abstraction to create an external view of a compound network as a single subnetwork. From the outside, the network looks like just two boundary service points, ingress and egress 🌊. The big advantage is the potential to get rid of the need for per-flow state and per-flow processing on the interior elements, and to provide aggregate service responses instead 💖.

One approach is for applications to use RSVP to request that their flows be admitted into the network. If a request is accepted, that implies a committed resource reservation within the IntServ-capable components of the network, and that the service requirements have been mapped into a compatible aggregate service class within the DiffServ-capable network [7]. The DiffServ core must be capable of carrying RSVP messages across the DiffServ network, so that further resource reservation is possible within the IntServ network on egress from the DiffServ environment 📨. This approach calls for the DiffServ network to use a per-flow multi-field (MF) classifier, where MF classification is based on the RSVP-signaled flow specification. The service specification of the RSVP-signaled resource reservation is mapped into a compatible aggregate DiffServ behavior aggregate, and the MF classifier marks packets according to the selected behavior 🎯. Alternatively, the boundary between the IntServ and DiffServ networks can use the IntServ egress to mark the flow packets with the appropriate DSCP. That lets the DiffServ ingress element use the BA classifier and skip the per-flow MF classifier entirely 🙌.

A high precision end-to-end QoS model requires that any admission failure within the DiffServ network be communicated to the end application, presumably via RSVP. This lets the application take some corrective action, either by modifying its service requirements or by terminating the application 🛠️. If the service agreement with the DiffServ network is statically provisioned, that static information can be loaded into the IntServ boundary systems, and IntServ can manage the allocation of the available DiffServ behavior aggregate resources 📋. If the service agreement is dynamically variable, some form of signaling is required between the two networks to pass this resource availability information back into the RSVP signaling environment 🔁.

6. Conclusions 👑

None of these observations are meant to say QoS architectures are completely impractical, and they don't give any reason to think the efforts to deploy QoS architectures won't come to fruition 🌸.

What this document is meant to show is that there are still a bunch of activities that are essential precursors to widespread deployment and use of QoS networks. There's also a need to fill in the missing sections with something substantial, in terms of adopting additional refinements to the existing QoS model 📝.

The architectural direction that looks most promising for QoS is not universal adoption of a single architecture. It's a tailored approach: aggregated service elements in the core of a network, where scalability is a major design objective, and per-flow service elements at the edge, where accuracy of the service response is a sustainable outcome 💜.

So architecturally, this points to no single QoS architecture. It points to a set of QoS mechanisms, and a number of ways those mechanisms can be configured to interoperate in a stable and consistent fashion 🧚.

7. Security Considerations 🔐

Okay, this part is serious, so listen up 🚨. The Internet is not an architecture that includes a strict implementation of fairness of access to the common transmission and switching resources. Introducing any form of fairness, and in the case of QoS, weighted fairness, implies a requirement for transparency in how the fairness contract between the network provider and the network's users is implemented. That requires some form of resource accounting and auditing, which in turn requires authentication and access control 🕵️. The balancing factor is that a shared resource should not overtly expose one user's level of resource usage to any other user, so some level of secrecy is required in this environment 🤫.

The QoS environment also exposes the potential for theft of resources through the unauthorized admission of traffic with an associated service profile 😤. QoS signaling protocols that are intended to undertake resource management and admission control require identity authentication and integrity protection to mitigate this potential theft.

Both forms of QoS architecture require the internal elements of the network to be able to classify traffic based on some form of identification carried in the packet header in the clear 👀. Classification systems that use multi-field specifiers, or per-flow specifiers, rely on end-to-end packet header fields being carried in the clear. That creates conflicting requirements for security architectures that try to mask such end-to-end identifiers within an encrypted payload 🙈.

QoS architectures can be considered a means of exerting control over network resource allocation. If resource availability changes rapidly (e.g. disaster), it is an undesirable outcome if the remaining resources are completely allocated to a single class of service to the exclusion of all other classes 💀.

Such an outcome constitutes a denial of service, where the traffic control system (routing) selects paths that are incapable of carrying any traffic of a particular service class. Yikes, that's a whole vibe 😭.

8. References 📚

[1]  Bradner, S., "The Internet Standards Process- Revision 3", BCP
     9, RFC 2026, October 1996.

[2]  Mankin, A., Baker, F., Braden, R., O'Dell, M., Romanow, A.,
     Weinrib, A. and L. Zhang, "Resource ReSerVation Protocol (RSVP)
     Version 1 Applicability Statement", RFC 2208, September 1997.

[3]  Braden. R., Clark, D. and S. Shenker, "Integrated Services in
     the Internet Architecture: an Overview", RFC 1633, June 1994.

[4]  Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z. and W.
     Weiss, "An Architecture for Differentiated Services", RFC 2475,
     December 1998.

[5]  Baker, F., Iturralde, C., Le Faucher, F., Davie, B.,
     "Aggregation of RSVP for IPv4 and IPv6 Reservations", Work in
     Progress.

[6]  Bernet, Y., "Format of the RSVP DCLASS Object", RFC 2996,
     November 2000.

[7]  Bernet, Y., Yavatkar, R., Ford, P., Baker, F., Zhang, L., Speer,
     M., Braden, R., Davie, B., Wroclawski, J. and E. Felstaine, "A
     Framework for Integrated Services Operation Over DiffServ
     Networks", RFC 2998, November 2000.

[8]  "Quality of Service Technical Overview", Microsoft Technical
     Library, Microsoft Corporation, September 1999.

[9]  "Resource Reservation Protocol API (RAPI)", Open Group Technical
     Standard, C809 ISBN 1-85912-226-4, The Open Group, December
     1998.

[10] Berger, L. and T. O'Malley, "RSVP Extensions for IPSEC Data
     Flows", RFC 2007, September 1997.

[11] Mills, C., Hirsh, D. and G. Ruth, "Internet Accounting:
     Background", RFC 1272, November 1991.

9. Acknowledgments 🫶

Huge thanks to Yoram Bernet, Brian Carpenter, Jon Crowcroft, Tony Hain and Henning Schulzrinne, whose valuable contributions made this document possible 💖.

10. Author's Address 📮

Geoff Huston
Telstra
5/490 Northbourne Ave
Dickson ACT 2602
AUSTRALIA

EMail: gih@telstra.net

The legal stuff is kept word-for-word so nothing gets lost in translation 📜.

Copyright (C) The Internet Society (2000).  All Rights Reserved.

This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works.  However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.

The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 🌈

The RFC Editor function is currently funded by the Internet Society 💰.

📦 where this page comes from