RFC 2386

RFC 2386 🎀 A Framework for QoS-based Routing in the Internet

Okay bestie, here's the paperwork vibe check 💅

• RFC number: 2386
• Title: A Framework for QoS-based Routing in the Internet
• Authors: E. Crawley (Argon Networks), R. Nair (Arrowpoint), B. Rajagopalan (NEC USA), H. Sandick (Bay Networks)
• Category: Informational
• Date: August 1998
• Obsoletes / Updates: the header doesn't list anything it obsoletes or updates 🙈

---

Status of This Memo 📜

This memo is giving information for the Internet community 🌸 It does not specify an Internet standard of any kind, so no pressure. Distribution of this memo is unlimited, so share it with everyone you know 🫶

Copyright Notice ©

Copyright (C) The Internet Society (1998). All Rights Reserved.

ABSTRACT ✨

Ngl, QoS-based routing has been called the missing piece in how QoS-based services are evolving on the Internet 🧩 This document goes through some of the QoS-based routing issues and requirements, and proposes a framework for QoS-based routing in the Internet 🦋 The framework is built by extending the current Internet routing model, meaning both intradomain and interdomain routing, so it can support QoS. Main character energy 👑

1. SCOPE OF DOCUMENT & PHILOSOPHY 💖

Okay so this doc is pitching a framework for QoS-based routing 🌈 The goal is to grow an Internet-wide solution while still letting people get creative with the many problems that pop up along the way. QoS-based routing has SO many complex facets, so it is recommended that the following two-pronged approach be used for its development:

1. Encourage the growth and evolution of novel intradomain QoS-based routing architectures. This lets people build independent, innovative solutions for the many QoS-based routing issues. Such solutions may be deployed in autonomous systems (ASs), large and small, based on their specific needs 🍓
2. Encourage simple, consistent and stable interactions between ASs implementing routing solutions developed as above. 🫶

This approach follows the traditional separation between intra and interdomain routing, and honestly it's giving classic 🎀 It allows solutions like QOSPF [GKOP98, ZSSC97], Integrated PNNI [IPNNI] or other schemes to be deployed for intradomain routing without any restriction, other than their ability to interact with a common, and perhaps simple, interdomain routing protocol. So the need to build one single, all-encompassing solution to the complex problem of QoS-based routing is obviated 🙌 Real talk, there are many different views on how QoS-based routing should be done. Much overall progress can be made if various ideas get to be developed and deployed concurrently, while some consensus on the interdomain routing architecture is still being worked out ✨ Finally, this routing model is perhaps the most practical from an evolution point of view. It is superfluous to say that the eventual success of a QoS-based Internet routing architecture would depend on the ease of evolution. Obviously 💅

The aim of this document is to describe the QoS-based routing issues, identify basic requirements on intra and interdomain routing, and describe an extension of the current interdomain routing model to support QoS. It is not an objective of this document to specify the details of intradomain QoS-based routing architectures. That's left up to the various intradomain routing efforts that might follow 🦋 Nor is it an objective to specify the details of the interface between reservation protocols such as RSVP and QoS-based routing. The specific interface functionality needed, however, would be clear from the intra and interdomain routing solutions devised. In the intradomain area, the goal is to develop the basic routing requirements while allowing maximum freedom for the development of solutions. In the interdomain area, the objectives are to identify the QoS-based routing functions, and facilitate the development or enhancement of a routing protocol that allows relatively simple interaction between domains 🧚

In the next section, a glossary of relevant terminology is given. In Section 3, the objectives of QoS-based routing are described and the issues that must be dealt with by QoS-based Internet routing efforts are outlined. In Section 4, some requirements on intradomain routing are defined. These requirements are purposely broad, putting few constraints on solution approaches. The interdomain routing model and issues are described in Section 5 and QoS-based multicast routing is discussed in Section 6. The interaction between QoS-based routing and resource reservation protocols is briefly considered in Section 7. Security considerations are listed in Section 8 and related work is described in Section 9. Finally, summary and conclusions are presented in Section 10 📚

2. GLOSSARY 🧚

The following glossary lists the terminology used in this document and an explanation of what is meant. Some of these terms may have different connotations, but when used in this document, their meaning is as given. Read it like a cheat sheet 🌸

Alternate Path Routing : A routing technique where multiple paths, rather than just the shortest path, between a source and a destination are utilized to route traffic. One of the objectives of alternate path routing is to distribute load among multiple paths in the network. Spreading the load, iconic 🫶

Autonomous System (AS): A routing domain which has a common administrative authority and consistent internal routing policy. An AS may employ multiple intradomain routing protocols internally and interfaces to other ASs via a common interdomain routing protocol.

Source: A host or router that can be identified by a unique unicast IP address.

Unicast destination: A host or router that can be identified by a unique unicast IP address.

Multicast destination: A multicast IP address indicating all hosts and routers that are members of the corresponding group.

IP flow (or simply "flow"): An IP packet stream from a source to a destination (unicast or multicast) with an associated Quality of Service (QoS) (see below) and higher level demultiplexing information. The associated QoS could be "best-effort" 🌈

Quality-of-Service (QoS): A set of service requirements to be met by the network while transporting a flow. 💅

Service class: The definitions of the semantics and parameters of a specific type of QoS.

Integrated services: The Integrated Services model for the Internet defined in RFC 1633 allows for integration of QoS services with the best effort services of the Internet. The Integrated Services (IntServ) working group in the IETF has defined two service classes, Controlled Load Service [W97] and Guaranteed Service [SPG97]. 🦋

RSVP: The ReSerVation Protocol [BZBH97]. A QoS signaling protocol for the Internet. 📣

Path: A unicast or multicast path.

Unicast path: A sequence of links from an IP source to a unicast IP destination, determined by the routing scheme for forwarding packets.

Multicast path (or Multicast Tree): A subtree of the network topology in which all the leaves and zero or more interior nodes are members of the same multicast group. A multicast path may be per-source, in which case the subtree is rooted at the source. 🌳

Flow set-up: The act of establishing state in routers along a path to satisfy the QoS requirement of a flow.

Crankback: A technique where a flow setup is recursively backtracked along the partial flow path up to the first node that can determine an alternative path to the destination. Backtracking era 🔙

QoS-based routing: A routing mechanism under which paths for flows are determined based on some knowledge of resource availability in the network as well as the QoS requirement of flows. 🧠

Route pinning: A mechanism to keep a flow path fixed for a duration of time. Stay put, bestie 📌

Flow Admission Control (FAC): A process by which it is determined whether a link or a node has sufficient resources to satisfy the QoS required for a flow. FAC is typically applied by each node in the path of a flow during flow set-up to check local resource availability. 🚦

Higher-level admission control: A process by which it is determined whether or not a flow set-up should proceed, based on estimates and policy requirements of the overall resource usage by the flow. Higher-level admission control may result in the failure of a flow set-up even when FAC at each node along the flow path indicates resource availability. Not me getting rejected even though every single node said yes 💀

3. QOS-BASED ROUTING: BACKGROUND AND ISSUES 🌸

3.1 Best-Effort and QoS-Based Routing

Routing deployed in today's Internet is focused on connectivity and typically supports only one type of datagram service called "best effort" [WC96]. Current Internet routing protocols, e.g. OSPF, RIP, use "shortest path routing", i.e. routing that is optimized for a single arbitrary metric, administrative weight or hop count. These routing protocols are also "opportunistic," using the current shortest path or route to a destination. Alternate paths with acceptable but non-optimal cost can not be used to route traffic (shortest path routing protocols do allow a router to alternate among several equal cost paths to a destination). So today's routing is kinda one-track-mind 🙈

QoS-based routing must extend the current routing paradigm in three basic ways. First, to support traffic using integrated-services class of services, multiple paths between node pairs will have to be calculated. Some of these new classes of service will require the distribution of additional routing metrics, e.g. delay, and available bandwidth. If any of these metrics change frequently, routing updates can become more frequent, thereby consuming network bandwidth and router CPU cycles 😭

Second, today's opportunistic routing will shift traffic from one path to another as soon as a "better" path is found. The traffic will be shifted even if the existing path can meet the service requirements of the existing traffic. If routing calculation is tied to frequently changing consumable resources (e.g. available bandwidth), this change will happen more often and can introduce routing oscillations as traffic shifts back and forth between alternate paths. Furthermore, frequently changing routes can increase the variation in the delay and jitter experienced by the end users. Shifting lanes for no reason is so messy 🚗💨

Third, as mentioned earlier, today's optimal path routing algorithms do not support alternate routing. If the best existing path cannot admit a new flow, the associated traffic cannot be forwarded even if an adequate alternate path exists. Like, the path is RIGHT THERE and we still can't use it?? 😤

3.2 QoS-Based Routing and Resource Reservation ✨

okay bestie, this part is SO important bc people always mix up these two things 😭. resource reservation protocols like RSVP [BZBH97] are the ones that request and reserve network resources 🎀, but they don't have a way to figure out which network path actually has enough resources for the requested QoS. flip side: QoS-based routing lets you find a path that has a good chance of accommodating the requested QoS, but it doesn't include a mechanism to reserve the required resources 💅. two different jobs, periodt.

so QoS-based routing is usually used together with some form of resource reservation or resource allocation mechanism 🫶. simple forms of QoS-based routing have been used in the past for Type of Service (TOS) routing [M98]. with OSPF, a different shortest-path tree can be computed for each of the 8 TOS values in the IP header [ISI81] 🌸. these mechanisms can be used to select specially provisioned paths, but they don't completely assure that resources aren't overbooked along the path 💀. as long as strict resource management and control aren't needed, TOS-based routing is useful for separating whole classes of traffic over multiple routes 🦋. these mechanisms might work well with the emerging Differential Services efforts [BBCD98] 🌈.

combining a resource reservation protocol with QoS-based routing gives you fine control over the route and resources ⭐, but it costs extra state and setup time. for example, a protocol such as RSVP may be used to trigger QoS-based routing calculations to meet the needs of a specific flow 🧚.

---

3.3 QoS-Based Routing: Objectives 🍓

okay so under QoS-based routing, paths for flows get determined based on some knowledge of resource availability in the network, plus the QoS requirement of each flow 💖. the main objectives are:

1. Dynamic determination of feasible paths: QoS-based routing can determine a path, from among possibly many choices, that has a good chance of accommodating the QoS of the given flow 👑. feasible path selection may be subject to policy constraints, such as path cost, provider selection, etc.
2. Optimization of resource usage: a network state-dependent QoS-based routing scheme can aid in the efficient utilization of network resources by improving the total network throughput 🔥. a routing scheme like this can be the basis for efficient network engineering.
3. Graceful performance degradation: state-dependent routing can compensate for transient inadequacies in network engineering (e.g., during focused overload conditions) 😭, giving better throughput and a more graceful performance degradation compared to a state-insensitive routing scheme [A84].

QoS-based routing in the Internet, though, raises a lot of issues 🙈:

• How do routers determine the QoS capability of each outgoing link and reserve link resources? Note that some of these links may be virtual, over ATM networks, and others may be broadcast multi-access links.
• What is the granularity of routing decision (i.e., destination-based, source and destination-based, or flow-based)?
• What routing metrics are used, and how are QoS-accommodating paths computed for unicast flows?
• How are QoS-accommodating paths computed for multicast flows with different reservation styles and receiver heterogeneity?
• What are the performance objectives while computing QoS-based paths?
• What are the administrative control issues?
• What factors affect the routing overheads?, and
• How is scalability achieved?

some of these issues get discussed briefly next. interdomain routing is discussed in Section 5 🦋.

---

3.4 QoS Determination and Resource Reservation 🌈

okay bestie, to figure out whether the QoS requirements of a flow can be accommodated on a link, a router has to be able to determine the QoS available on that link 🧐. it's still an open issue how QoS availability gets determined for broadcast multiple access links (e.g., Ethernet) 💀. a related problem is reserving resources over those links too. solutions to these problems are just emerging [GPSS98] ✨.

similar problems show up when a router is connected to a large non-broadcast multiple access network, such as ATM 🫠. in this case, if the destination of a flow is outside the ATM network, the router may have multiple egress choices. and the QoS availability on the ATM paths to each egress point may be different 😩. so the issues are:

• how does a router determine all the egress choices across the ATM network?
• how does it determine what QoS is available over the path to each egress point?, and
• what QoS value does the router advertise for the ATM link.

typically, IP routing over ATM (e.g., NHRP) allows the selection of a single egress point in the ATM network, and the procedure doesn't incorporate any knowledge of the QoS required over the path 🙃. an approach like I-PNNI [IPNNI] would be helpful here, although it introduces some complexity 🧩.

another problem with resource reservation is figuring out what resources have already been allocated to a multicast flow 🔍. having this information during path computation improves the chances of finding a path to add a new receiver to a multicast flow 💜. QOSPF [ZSSC97] handles this by letting routers broadcast reserved resource information to other routers in their area 📣.

alternate path routing [ZES97] deals with this issue by using probe messages to find a path with sufficient resources 🕵️‍♀️. the Path QoS Computation (PQC) method, proposed in [GOA97], propagates bandwidth allocation information in RSVP PATH messages. a router receiving the PATH message gets an indication of the resource allocation only on those links in the path from the source to itself 🛤️. allocation for the same flow on other remote branches of the multicast tree isn't available. so the PQC method may not be sufficient to find feasible QoS-accommodating paths to all receivers 😬.

---

3.5 Granularity of Routing Decision 🎀

so routing in the Internet is currently based only on the destination address of a packet 📦. but a lot of multicast routing protocols need routing based on the source AND destination of a packet 👀. the Integrated Services architecture and RSVP allow QoS determination for an individual flow between a source and a destination 💫. this set of routing granularities presents a problem for QoS routing solutions 😤.

if routing is based only on destination address, an intermediate router will route all flows between different sources and a given destination along the same path. that's fine if the path has adequate capacity, but it becomes a problem if there are multiple flows to a destination that exceed the capacity of the link 🚧.

one version of QOSPF [ZSSC97] determines QoS routes based on source and destination address. this means all traffic between a given source and destination, regardless of the flow, will travel down the same route 🛣️. again, the route must have capacity for all the QoS traffic for that source/destination pair. the amount of routing state also goes up, since the routing tables must include source/destination pairs instead of just the destination 📈.

the best granularity is found when routing is based on individual flows, but this costs a tremendous amount in terms of routing state 🥵. each QoS flow can be routed separately between any source and destination. PQC [GOA97] and alternate path routing [ZES97] are examples of solutions that operate at the flow level 🦋.

both source/destination and flow-based routing may be susceptible to packet looping under hop-by-hop forwarding ⚠️. suppose a node along a flow or source/destination-based path loses the state information for the flow. also suppose that the flow-based route is different from the regular destination-based route. the potential then exists for a routing loop to form, when the node forwards a packet belonging to the flow using its destination-based routing table to a node that occurs earlier on the flow-based path. this is because the latter node may use its flow-based routing table to forward the packet again to the former, and this can go on indefinitely 🔁💀.

---

3.6 Metrics and Path Computation 🧮

3.6.1 Metric Selection and Representation 💅

there are some considerations in defining suitable link and node metrics [WC96] 💭. first, the metrics must represent the basic network properties of interest. such metrics include residual bandwidth, delay and jitter 🎯. since the flow QoS requirements have to be mapped onto path metrics, the metrics define the types of QoS guarantees the network can support. alternatively, QoS-based routing cannot support QoS requirements that cannot be meaningfully mapped onto a reasonable combination of path metrics 🙅‍♀️. second, path computation based on a metric or a combination of metrics must not be too complex as to render them impractical. in this regard, it's worth noting that path computation based on certain combinations of metrics (e.g., delay and jitter) is theoretically hard 🤯. so the allowable combinations of metrics must be determined while taking into account the complexity of computing paths based on these metrics and the QoS needs of flows.

a common strategy to allow flexible combinations of metrics while also reducing path computation complexity is to use "sequential filtering" ✨. under this approach, a combination of metrics is ordered in some fashion, reflecting the importance of different metrics (e.g., cost followed by delay, etc.). paths based on the primary metric are computed first (using a simple algorithm, e.g., shortest path), and a subset of them is eliminated based on the secondary metric, and so forth, until a single path is found 🪄. this is an approximation technique, and it trades off global optimality for path computation simplicity (the filtering technique may be simpler, depending on the set of metrics used. For example, with bandwidth and cost as metrics, it is possible to first eliminate the set of links that do not have the requested bandwidth and then compute the least cost path using the remaining links.) 🧠

now, once suitable link and node metrics are defined, a uniform representation of them is required across independent domains (employing possibly different routing schemes) in order to derive path metrics consistently (path metrics are obtained by the composition of link and node metrics) 🤝. encoding of the maximum, minimum, range, and granularity of the metrics is needed. also, the definitions of comparison and accumulation operators are required 📏. in addition, suitable triggers must be defined for indicating a significant change from a minor change. the former will cause a routing update to be generated 🔔. the stability of the QoS routes would depend on the ability to control the generation of updates. with interdomain routing, it is essential to obtain a fairly stable view of the interconnection among the ASs 🫶🌈.

3.6.2 Metric Hierarchy 🌈

Okay so there's a hierarchy among service classes, and it's ranked by how much one class of traffic can mess with the service of lower classes on the same link 💅. Guaranteed constant bit rate traffic sits at the top 👑, and "best-effort" datagram traffic sits at the bottom 🌸. Higher classes can impact the lower ones, but it doesn't work the other way around. A datagram flow literally cannot affect a real-time service 🙈. So in the worst case you might need to distribute and update a different metric for each type of service. BUT ngl, there are several perks to identifying a single default metric 💖. For example, you could derive one metric that combines the availability of datagram and real-time service over a common substrate 🫶.

3.6.3 Datagram Flows 🦋

A delay-sensitive metric is probably the most obvious pick for datagram flows 💜. It does need careful analysis though, to avoid instabilities and to reduce storage and bandwidth requirements 😭. For example, a recursive filtering technique based on a simple and efficient weighted averaging algorithm [NC94] could be used. This filter is used to stabilize the metric. It's adequate for smoothing most loading patterns, but it will not distinguish between regular bursts of traffic and random loading 🤔. Among other stabilizing tools, there's a minimum time between updates that can help filter out high-frequency oscillations ✨.

3.6.4 Real-time Flows 🎀

In real-time QoS, delay variation is generally more critical than delay, as long as the delay isn't too high 💅. Voice-based apps clearly can't tolerate more than a certain level of delay 🙈. Varying delays may be expected to a greater degree in a shared medium environment with datagrams than in a network built over a switched substrate. So routing a real-time flow reduces to an exercise in allocating the required network resources while minimizing fragmentation of bandwidth 🧚. The result is a bandwidth-limited minimum hop path from a source to the destination. In other words, the router does an ordered search through paths of increasing hop count until it finds one that meets all the bandwidth needs of the flow 🔥.

To reduce contention and the probability of false probes (due to inaccuracy in route tables), the router could pick a path randomly from a "window" of paths that meet the needs of the flow and satisfy one of three additional criteria: best-fit, first-fit or worst-fit 💖. Note that there's a similarity between allocating bandwidth and allocating memory in a multiprocessing system 🤯. First-fit seems to be appropriate for a system with high real-time flow arrival rates, and worst-fit is ideal for real-time flows with high holding times. This rather nonintuitive result was shown in [NC94] 👑.

3.6.5 Path Properties 🗺️

Path computation by itself is merely a search technique. For example, Shortest Path First (SPF) is a search technique based on dynamic programming. How useful the computed paths are depends to a large extent on the metrics used to evaluate the cost of a path with respect to a flow 💅.

Each link the path computation engine considers must be evaluated against the requirements of the flow. That is, the cost of providing the services the flow needs must be estimated with respect to the capabilities of the link. This requires a uniform method of combining features such as delay, bandwidth, priority and other service features 🫶. Furthermore, the costs must reflect the lost opportunity of using each link after routing the flow 😭.

3.6.6 Performance Objectives 🎯

One common objective during path computation is to improve total network throughput. In this regard, just routing a flow on any path that accommodates its QoS requirement is not a good strategy 🙅. In fact, this corresponds to uncontrolled alternate routing [SD95] and may adversely impact performance at higher traffic loads. So it's necessary to consider the total resource allocation for a flow along a path, in relation to available resources, to determine whether or not the flow should be routed on that path. This document refers to such a mechanism as "higher level admission control" ✨.

The goal is to ensure that the "cost" incurred by the network in routing a flow with a given QoS is never more than the revenue gained 💰. The routing cost here may be the lost revenue from potentially blocking other flows that contend for the same resources. Formulating the higher level admission control strategy, with suitable administrative hooks and with fairness to all flows that want entry to the network, is an issue 🙈. The fairness problem shows up because flows with smaller reservations tend to be routed more successfully than flows with large reservations, for a given engineered capacity. To guarantee a certain acceptance rate for "larger" flows without over-engineering the network, you need a fair higher level admission control mechanism 🌸. Applying higher level admission control to multicast routing is discussed later.

3.7 Administrative Control 🎀

There are several administrative control issues, bestie 💅. First, within an AS employing state-dependent routing, administrative control of routing behavior may be necessary. Higher level admission control was one example we talked about earlier, and some others are described in this section. Second, controlling interdomain routing based on policy is an issue. The discussion of interdomain routing is defered to Section 5 🦋.

Two areas that need administrative control, on top of appropriate routing mechanisms, are handling flow priority with preemption, and resource allocation for multiple service classes 👑.

3.7.1 Flow Priorities and Preemption 🔥

If there are critical flows that must be accorded higher priority than other flows, the network needs a mechanism to recognize flow priorities 🫶. There are two aspects to prioritizing flows. First, there must be a policy to decide how different users are allowed to set priorities for the flows they originate. The network must be able to verify that a given flow is allowed to claim the priority level signaled for it 🙈. Second, the routing scheme must ensure that a path with the requested QoS will be found for a flow with a probability that increases with the priority of the flow. In other words, for a given network load, a high priority flow should be more likely to get a certain QoS from the network than a lower priority flow requesting the same QoS 💖. Routing procedures for flow prioritization can be complex. Identifying and evaluating different procedures are areas that require investigation 🧐.

3.7.2 Resource Control 🍓

If there are multiple service classes, the network has to be engineered to carry the forecasted traffic demands of each class 📈. To do this, router and link resources may be logically partitioned among the various service classes. It's desirable to have dynamic partitioning, where unused resources in one partition get dynamically shifted to other partitions on demand [ACFH92] 🌈. Dynamic sharing, though, has to be done in a controlled fashion, so traffic in one service class can't take up more resources than were engineered for it for prolonged periods of time 🙅‍♀️. Designing such a resource sharing scheme, and fitting it into the QoS-based routing scheme, are significant issues 💀.

3.8 QoS-Based Routing for Multicast Flows 🧚

QoS-based multicast routing is an important problem, especially if the notion of higher level admission control is included 👑. The dynamism in the receiver set allowed by IP multicast, plus receiver heterogeneity, adds to the problem 😵. With a straightforward implementation of distributed heuristic algorithms for multicast path computation [W88, C91], the difficulty is essentially one of scalability 📉. To accommodate QoS, multicast path computation at a router must know not only the ids of the subnets where group members are present, but also the identity of the branches in the existing tree. In other words, routers must keep flow-specific state information 🫣. Also, computing optimal shared trees based on the shared reservation style [BZBH97] may require new algorithms 🔬. Multicast routing is discussed in some detail in Section 6 🦋.

3.9 Routing Overheads 🌸

The overheads a routing scheme incurs depend on the type of routing scheme, as well as on the implementation. There are three types of overheads to consider: computation, storage and communication. It's necessary to understand what choosing a routing mechanism implies for each of these overheads 🧠.

For example, with link state routing, the choice of update propagation mechanism is important, since network state is dynamic and changes relatively frequently. Specifically, a flooding mechanism would result in many unnecessary message transmissions and lots of processing 😩. Alternative techniques, such as tree-based forwarding [R96], have to be considered 🌳. A related issue is quantization of state information, to prevent frequent updating of dynamic state. Coarse quantization reduces updating overheads, but it may affect the performance of the routing scheme. The tradeoff has to be carefully evaluated ⚖️. QoS-based routing incurs certain overheads during flow establishment, for example, computing a source route. Whether this overhead is disproportionate compared to the length of the sessions is an issue 🤷‍♀️. In general, techniques for minimizing routing-related overheads during flow establishment must be investigated. Approaches that are useful include pre-computation of routes, caching recently used routes, and TOS routing based on hints in packets (e.g., the TOS field) 💜.

3.10 Scaling by Hierarchical Aggregation 🏰

QoS-based routing should be scalable, and hierarchical aggregation is a common technique for scaling (e.g., [PNNI96]) ✨. But this introduces problems with regard to the accuracy of the aggregated state information [L95]. Aggregating paths under multiple constraints is also difficult 😬. One of the difficulties is the risk of accepting a flow based on inaccurate information, and then not being able to support the QoS requirements of the flow, because the capabilities of the actual paths that were aggregated aren't known during route computation 😱. So performance impacts of aggregating path metric information must be understood 🔍. A way to compensate for inaccuracies is to use crankback, i.e., dynamic search for alternate paths as a flow is being routed. But crankback increases the time to set up a flow, and it may adversely affect the performance of the routing scheme under some circumstances ⏳. Thus, crankback must be used judiciously, if at all, along with a higher level admission control mechanism 🫶.

4. INTRADOMAIN ROUTING REQUIREMENTS ✨

At the intradomain level, the vibe is total freedom 💅. The goal is to give as much latitude as possible when it comes to QoS-based routing issues, because there are SO many ideas about how QoS-based routing services can be set up inside ASs 🧠. They range from on-demand path computation based on current state information all the way to statically provisioned paths that only support a few service classes.

Another thing that invites different solutions is performance optimization 🚀. Depending on the technique, intradomain routing could be super sophisticated or pretty simple, no cap. Then there's the service class stuff. The classes supported, and the specific QoS engineered for each class, can differ from AS to AS 🌈. Like, some ASs may not support guaranteed service, while others MAY. Some ASs that do support it might be engineered for a better delay bound than others 💜. So it takes real thought to figure out the high-level requirements for intradomain routing that fits the big-picture view of QoS-based routing across the Internet AND gives everyone maximum autonomy to build their own solutions 🫶.

Our view is that certain minimum requirements MUST be satisfied by intradomain routing before it can be qualified as "QoS-based" routing 🎀. These are:

• The routing scheme MUST route a flow along a path that can accommodate its QoS requirements, or indicate that the flow cannot be admitted with the QoS currently being requested.
• The routing scheme MUST indicate disruptions to the current route of a flow due to topological changes.
• The routing scheme MUST accommodate best-effort flows without any resource reservation requirements. That means present best-effort applications and protocol stacks need not have to change to run in a domain employing QoS-based routing 🙌.
• The routing scheme MAY optionally support QoS-based multicasting with receiver heterogeneity and shared reservation styles.

In addition, the following capabilities are also RECOMMENDED:

• Capabilities to optimize resource usage.
• Implementation of higher-level admission control procedures to limit the overall resource utilization by individual flows.

Further requirements along these lines MAY be specified. The requirements SHOULD capture the consensus view of QoS-based routing, but SHOULD NOT preclude particular approaches (e.g., TOS-based routing) from being implemented. So yeah, the intradomain requirements are expected to be rather broad 🦋.

---

5. INTERDOMAIN ROUTING 🌸

The fundamental requirement for interdomain QoS-based routing is scalability 📈. That means interdomain routing cannot be based on highly dynamic network state information. Instead, it MUST be aided by sound network engineering and relatively sparse information exchange between independent routing domains. The upside is that this approach can be realized through straightforward extensions of the present Internet interdomain routing model, which is giving real main character energy 👑. A number of issues still need to be addressed to make it work, and we'll get into those below ✨.

5.1 Interdomain QoS-Based Routing Model

Here's the interdomain QoS-based routing model 🗺️:

          AS1                   AS2             AS3
      ___________        _____________      ____________
     |           |      |             |    |            |
     |           B------B             B----B            |
     |           |      |             |    |            |
      -----B-----       B-------------      --B---------
            \         /                      /
             \       /                      /
          ____B_____B____         _________B______
         |               |       |                |
         |               B-------B                |
         |               |       |                |
         |               B-------B                |
          ---------------         ----------------
               AS4                           AS5

Here, ASs exchange standardized routing information via border nodes B. Under this model, each AS can itself be a set of interconnected ASs with standardized routing interaction, so the interdomain routing model is hierarchical 🪆. Each lowest-level AS also employs an intradomain QoS-based routing scheme, either proprietary or standardized by intradomain routing efforts such as QOSPF 🧚. Given this structure, some questions come up that we need to answer:

• What information is exchanged between ASs?
• What routing capabilities does the information exchange lead to? (E.g., source routing, on-demand path computation, etc.)
• How is the external routing information represented within an AS?
• How are interdomain paths computed?
• What sort of policy controls may be exerted on interdomain path computation and flow routing?, and
• How is interdomain QoS-based multicast routing accomplished?

At a high level, the answers to these questions depend on the routing paradigm 💭. Specifically, if we look at link state routing, the information exchanged between domains would consist of an abstract representation of the domains in the form of logical nodes and links, along with metrics that quantify their properties and resource availability. The hierarchical structure of the ASs MAY be handled by a hierarchical link state representation, with appropriate metric aggregation.

Link state routing may not necessarily be advantageous for interdomain routing, though, and here's why 🙈:

• One advantage of intradomain link state routing is that it would allow fairly detailed link state information to be used to compute paths on demand for flows requiring QoS. The state and metric aggregation used in interdomain routing, on the other hand, erodes this property to a great degree 😭.
• The usefulness of keeping track of the abstract topology and metrics of a remote domain, or of the interconnection between remote domains, is not obvious. This is especially true when the remote topology and metric encoding are lossy.
• ASs may not want to advertise any details of their internal topology or resource availability 🤐.
• Scalability in interdomain routing can be achieved only if information exchange between domains is relatively infrequent. So it seems practical to limit information flow between domains as much as possible 🍓.

Compact information flow is what allows QoS-enhanced versions of existing interdomain protocols, such as BGP-4, to be implemented 🎀. We look at the interdomain routing issues in this context.

5.2 Interdomain Information Flow

The information flow between routing domains MUST enable certain basic functions:

1. Determination of reachability to various destinations
2. Loop-free flow routes
3. Address aggregation whenever possible
4. Determination of the QoS that will be supported on the path to a destination. The QoS information SHOULD be relatively static, determined from the engineered topology and capacity of an AS rather than ephemeral fluctuations in traffic load through the AS. Ideally, the QoS supported in a transit AS SHOULD be allowed to vary significantly only under exceptional circumstances, such as failures or focused overload 🚨.
5. Determination, optionally, of multiple paths for a given destination, based on service classes.
6. Expression of routing policies, including monetary cost, as a function of flow parameters, usage, and administrative factors 💸.

Items 1-3 are already part of existing interdomain routing 💅. Item 5 is also a straightforward extension of the current model. So the main problem areas are items 4 and 6 🔍.

The QoS of an end-to-end path is obtained by composing the QoS available in each transit AS 🧩. So border routers MUST first determine what the locally available QoS is, in order to advertise routes to both internal and external destinations. The determination of local "AS metrics" (corresponding to link metrics in the intradomain case) SHOULD NOT be subject to too much dynamism. That makes the real question how to define such metrics and what triggers an occasional change that results in re-advertisements of routes 🤔.

The approach suggested in this document is NOT to compute paths based on residual or instantaneous values of AS metrics, since those can be dynamic. Instead, it uses only the QoS capabilities engineered for aggregate transit flows 🫶. Such engineering MAY be based on knowledge of the traffic to be expected from each neighboring AS and the corresponding QoS needs. This information MAY be obtained based on contracts agreed upon prior to the provisioning of services 📝. The AS metric then corresponds to the QoS capabilities of the "virtual path" engineered through the AS (for transit traffic), and a different metric MAY be used for different neighbors. Here's how that looks 👇:

          AS1                   AS2             AS3
      ___________        _____________      ____________
     |           |      |             |    |            |
     |           B------B1           B2----B            |
     |           |      |             |    |            |
      -----B-----       B3------------      --B---------
            \         /
             \       /
          ____B_____B____
         |               |
         |               |
         |               |
         |               |
          ---------------
               AS4

Here, B1 MAY utilize an AS metric specific to AS1 when computing path metrics to be advertised to AS1. This metric is based on the resources engineered in AS2 for transit traffic from AS1. Similarly, B3 MAY utilize a different metric when computing path metrics to be advertised to AS4. Now, it is assumed that as long as traffic flow into AS2 from AS1 or AS4 does not exceed the engineered values, these path metrics would hold 🌈. Excess traffic due to transient fluctuations, however, MAY be handled as best effort or marked with a discard bit ⚡.

So this model is different from the intradomain model, where end nodes pick a path dynamically based on the QoS needs of the flow to be routed 🔄. Here, paths within ASs are engineered based on presumed, measured, or declared traffic and QoS requirements 📊. Under this model, an AS can contract for routes via multiple transit ASs with different QoS requirements. For instance, AS4 above can use both AS1 and AS2 as transits for the same or different destinations 🙌. Also, a QoS contract between one AS and another may generate another contract between the second and a third AS, and so on, like a whole chain of contracts 🔗.

An issue is what triggers the recomputation of path metrics within an AS. Failures or other events that prevent engineered resource allocation SHOULD certainly trigger recomputation 🚨. Recomputation SHOULD NOT be triggered in response to the arrival of flows within the engineered limit 🙅‍♀️.

5.3 Path Computation

Path computation for an external destination at a border node is based on reachability, path metrics, and local policies of selection 🗺️. If there are multiple selection criteria (e.g., delay, bandwidth, cost, etc.), multiple alternatives MAY have to be maintained, as well as propagated, by border nodes 🧠. Selecting a path from among many alternatives depends on the QoS requests of flows, as well as policies 💖. Path computation MAY also utilize any heuristics for optimizing resource usage ✨.

5.4 Flow Aggregation ✨

okay so the big issue in interdomain routing is how much flow state transit ASs have to chew through 😩 which means reducing that state with aggregation techniques needs to be seriously considered 💅 flow aggregation basically means transit traffic through an AS gets sorted into a few aggregated streams instead of being routed flow by flow 🦋 an entry border router might sort the transit flows coming into an AS into a few coarse categories based on the egress node and the QoS requirements of each flow 🎀 then the aggregated stream for a given traffic class can be routed as a single flow inside the AS to the exit border router ✨ that router may then present individual flows to different neighboring ASs, and the process repeats at each entry border router 🌸

under this scenario, it is essential that entry border routers keep track of the resource requirements for each transit flow and apply admission control to figure out whether the aggregate requirement from any neighbor goes past the engineered limit 🫶 if it does, some policy has to be invoked to deal with the excess traffic 😭 otherwise, it may be assumed that aggregated flows are routed over paths with adequate resources to guarantee QoS for the member flows 💜 finally, it is possible that entry border routers at a transit AS may prefer not to aggregate flows, if finer grain routing within the AS would be more efficient (e.g., to aid load balancing within the AS) 🧚

5.5 Path Cost Determination 💸

it is hoped that the integrated services Internet architecture would let providers charge for IP flows based on their QoS requirements 🤑 a QoS-based routing architecture can help distribute information on expected costs of routing flows to various destinations via different domains 🌈 from a provider's point of view, there is a cost incurred in guaranteeing QoS to flows, and this cost could be a function of several parameters, some related to flow parameters and others based on policy 💅 from a user's point of view, the consequence of requesting a particular QoS for a flow is the cost incurred, so the selection of providers may be based on cost 👑 a routing scheme can help a provider distribute these costs in routing to various destinations, as a function of several parameters, to other providers or to end users ✨

in the interdomain routing model described earlier, the costs to a destination change as routing updates are passed through a transit domain 🦋 one of the goals of the routing scheme should be to maintain a uniform semantics for cost values (or functions) as they are handled by intermediate domains 💖 for example, look at the cost function generated by border node B1 in domain A and passed to node B2 in domain B below 🎀 the routing update may be injected into domain B by B2 and finally passed to B4 in domain C by router B3 🌸 domain B may interpret the cost value it received from domain A in any way it wants, for instance by adding a locally significant component to it 🙈 but when this cost value is passed on to domain C, its meaning must be what domain A intended, plus the incremental cost of transiting domain B, and not what domain B uses internally 😤

 Domain A                    Domain B           Domain C
  ____________          ___________      ____________
 |            |        |           |    |            |
 |            B1------B2          B3---B4            |
 |            |        |           |    |            |
  ------------          -----------      ------------

a problem with charging for a flow is figuring out the cost when the QoS promised for the flow was not actually delivered 💀 clearly, when a flow is routed via multiple domains, it must be determined whether each domain delivers the QoS it declares possible for traffic through it 🔍

---

6. QOS-BASED MULTICAST ROUTING 🌈

the goals of QoS-based multicast routing are as follows 🎯

• Scalability to large groups with dynamic membership
• Robustness in the presence of topological changes
• Support for receiver-initiated, heterogeneous reservations
• Support for shared reservation styles, and
• Support for "global" admission control, i.e., administrative control of resource consumption by the multicast flow.

the RSVP multicast flow model goes like this 🧠 the sender of a multicast flow advertises the traffic characteristics periodically to the receivers 📣 when a receiver gets an advertisement, it may generate a message to reserve resources along the flow path from the sender 💌 receiver reservations may be heterogeneous 🦋 other multicast models may be considered too ✨

the multicast routing scheme tries to find a path from the sender to each receiver that can accommodate the requested reservation 🛣️ the routing scheme may try to maximize network resource utilization by minimizing the total bandwidth allocated to the multicast flow, or by optimizing some other measure 📊

6.1 Scalability, Robustness and Heterogeneity 💫

when addressing scalability, two aspects must be considered 👀

1. The overheads associated with receiver discovery. This overhead is incurred when determining the multicast tree for forwarding best-effort sender traffic characterization to receivers.
2. The overheads associated with QoS-based multicast path computation. This overhead is incurred when flow-specific state information has to be collected by a router to determine QoS-accommodating paths to a receiver.

depending on the multicast routing scheme, one or both of these aspects become important 🙃 for instance, under the present RSVP model, reservations are established on the same path the sender traffic characterizations are sent over, so there is no path computation overhead 😌 on the other hand, under the proposed QOSPF model [ZSSC97] of multicast source routing, receiver discovery overheads are incurred by MOSPF [M94] receiver location broadcasts, and additional path computation overheads are incurred because existing flow paths need to be tracked 😵‍💫 scaling of QoS-based multicast depends on both of these scaling issues 💅 however, scalable best-effort multicasting is really not in the domain of QoS-based routing work (solutions for this are being devised by the IDMR WG [BCF94, DEFV94]) 🫠 QoS-based multicast routing may build on these solutions to achieve overall scalability 🌸

there are several options for QoS-based multicast routing 🎀 multicast source routing is one, where multicast trees are computed by the first-hop router from the source, based on sender traffic advertisements 🛰️ the advantage is that it blends nicely with the present RSVP signaling model 🫶 this scheme also works well when receiver reservations are homogeneous and the same as the maximum reservation derived from the sender advertisement 💖 the disadvantages are the extra effort needed to accommodate heterogeneous reservations and the difficulties in optimizing resource allocation based on shared reservations 😬

in these regards, a receiver-oriented multicast routing model seems to have some advantage over multicast source routing 👑 under this model:

1. Sender traffic advertisements are multicast over a best-effort tree, which can be different from the QoS-accommodating tree for sender data.
2. Receiver discovery overheads are minimized by utilizing a scalable scheme (e.g., PIM, CBT) to multicast sender traffic characterization.
3. Each receiver-side router independently computes a QoS-accommodating path from the source, based on the receiver reservation. This path can be computed based on unicast routing information only, or with additional multicast flow-specific state information. In any case, multicast path computation is broken up into multiple, concurrent unicast path computations 🧩
4. Routers processing unicast reserve messages from receivers aggregate resource reservations from multiple receivers 🍓

flow-specific state information may be limited in Step 3 to achieve scalability [RN98] 🙌 in general, limiting flow-specific information when making multicast routing decisions is important in any routing model 💭 the advantages of this model are the ease with which heterogeneous reservations can be accommodated, and the ability to handle shared reservations 🥰 the disadvantages are the incompatibility with the present RSVP signaling model, and the need to rely on reverse paths when link state routing is not used 😩 both multicast source routing and the receiver-oriented routing model described above use per-source trees to route multicast flows 🌳 another possibility is using shared, per-group trees for routing flows, but the computation and usage of such trees require further work 🔮

finally, scalability at the interdomain level may be achieved if QoS-based multicast paths are computed independently in each domain 🌍 this principle is illustrated by the QOSPF multicast source routing scheme, which allows independent path computation in different OSPF areas 🗺️ it is easy to incorporate this idea in the receiver-oriented model too 💕 an evaluation of multicast routing strategies must take into account the relative advantages and disadvantages of the various approaches, in terms of scalability features and functionality supported 📝

6.2 Multicast Admission Control 🚦

higher level admission control, as defined for unicast, prevents excessive resource consumption by flows when traffic load is high 🚧 such an admission control strategy must be applied to multicast flows when the flow path computation is receiver-oriented or sender-oriented 🙆‍♀️ in essence, a router computing a path for a receiver must determine whether the incremental resource allocation for that receiver is excessive under some administratively determined admission control policy 🧾 other admission control criteria, based on the total resource consumption of a tree, may be defined too 🌲

7. QOS-BASED ROUTING AND RESOURCE RESERVATION PROTOCOLS 🎀🔥

there must clearly be a well-defined interface between routing and resource reservation protocols 🤝 the nature of this interface, and the interaction between routing and resource reservation, has to be determined carefully to avoid incompatibilities 😤 the importance of this can be readily illustrated with RSVP 💅

RSVP was designed to operate independent of the underlying routing scheme ✨ under this model, RSVP PATH messages establish the reverse path for RESV messages 🛤️ in essence, this model is not compatible with QoS-based routing schemes that compute paths after receiver reservations are received 💀 while this incompatibility can be resolved in a simple manner for unicast flows, multicast with heterogeneous receiver requirements is a more difficult case 😵 for this, reconciliation between RSVP and QoS-based routing models is necessary 🫠 such a reconciliation, however, may require some changes to the RSVP model, depending on the QoS-based routing model [ZES97, ZSSC97, GOA97] 🛠️ on the other hand, QoS-based routing schemes may be designed with RSVP compatibility as a necessary goal 🥹 how this affects scalability and other performance measures must be considered 📈

8. SECURITY CONSIDERATIONS 🔒💖

okay bestie, security is serious but it's also kind of iconic. 💅 Routing security is all about keeping the routing protocol healthy even when someone accidentally (or on purpose) sneaks in bad information that breaks everything [P88]. 😭

QoS-based routing needs extra protection on top of that. First, it has to check that QoS requests for flows are legit. Second, it has to stop resource-depletion attacks, which happen when flows are allowed to ask for arbitrary resources along whatever paths they want. 🙈 One greedy or errant flow can eat up resources and leave legit flows with nothing. Not cute. 💀

Policy constraints, charging models, and policing at different points in the network can help prevent this. ✨ But actually formalizing that protection is still unfinished work, so the people who study this are still on it [BCCH94]. 🧚

---

9. RELATED WORK 🦋

Okay so adaptive routing, which means routing based on what the network is currently doing, has a long history. 📜 It's especially iconic in circuit-switched networks. It also showed up in early datagram and virtual circuit packet networks. 🌸

More recently, people have been studying it in ATM networks, where the traffic and the topology look really different from circuit-switched networks [MMR96]. It's super useful to look back at how adaptive routing has been done before, so we understand both the problems that came up and the solutions people tried. 💜

Fundamentally, adaptive, network-state-dependent routing has two parts:

1. Measuring and gathering network state information, and
2. Computing routes based on the information that's available.

Depending on how those two steps get done, you can end up with a bunch of different routing techniques. 🎀 They differ in ways like these:

• what state information is used
• whether local or global state is used
• what triggers the propagation of state information
• whether routes are computed in a distributed or centralized manner
• whether routes are computed on-demand, pre-computed, or in a hybrid manner
• what optimization criteria, if any, are used in computing routes
• whether source routing or hop by hop routing is used, and
• how alternate route choices are explored

Quick heads-up: most adaptive routing work has focused on unicast routing. ✨ Multicast routing is going to be a big deal for Internet QoS-based routing, so it gets handled separately, and this review only looks at unicast. 🦋 It's also not exhaustive, just a brief overview of some of the approaches. 🌈

9.1 Optimization Criteria 💅

The most common optimization criteria in adaptive routing are throughput maximization or delay minimization. ⭐ A general way to write the optimization problem is to maximize network revenue, given that routing a flow over a given path has a cost [MMR96, K88].

Here's the catch though: global optimization solutions are hard to implement, and they depend on a bunch of assumptions about the traffic being routed [MMR96]. 🙈 So the practical move has been to route each flow (a VC, a circuit, or a packet stream to a given destination) independently of the other flows. Lots of routing schemes like this have been implemented. 🫶

9.2 Circuit Switched Networks 📞

Many adaptive routing concepts have been proposed for circuit-switched networks. 💖 One simple example is sequential alternate routing [T88]. It's hop-by-hop and destination-based, and it only uses local state information. 🧚

Under this scheme, each node gets a routing table that lists multiple output link choices for each destination. When a call setup request shows up at a node, the node tries each output link choice in order until it finds one that can handle the call. Then it reserves resources on that link and forwards the setup to the next node. 🔥

The setup either makes it to the destination, or it gets blocked somewhere along the way. If it gets blocked, it can be cranked back to the previous node, or a failure can just be declared. ✨ Crankback is what lets the previous node try an alternate path. 🦋

The routing table can be computed either centrally or in a distributed way, and it's based only on the network's topology. For example, a k-shortest-path algorithm can find k alternate paths from a node that each start with a different initial link [T88]. And some mechanism has to be in place during path computation or call setup to stop loops. 🌀

Okay, now the cautionary tale. 🚨 Performance studies of this scheme show the pitfalls of alternate routing in general, and crankback in particular [A84, M86, YS87]. Alternate routing improves throughput when traffic is pretty light, but it actively hurts performance when traffic is heavy. Crankback can make things even worse under those conditions. 😭

In general, uncontrolled alternate routing (with or without crankback) can be harmful in a heavily used network, because circuits end up on longer paths and use up more capacity. It's an obvious result, but it's an important one, and it applies to QoS-based Internet routing too. 💀

Here's the drama: both direct routed calls (over shortest paths) and alternate routed calls compete for the same resources. ⚔️ At higher loads, giving those resources to alternate routed calls pushes out direct routed calls, which then get alternate routed too. 🙃 So a lot of approaches have been proposed to limit how many alternate routed calls flow through under heavy traffic.

These schemes are designed for the fully-connected logical topology of long-distance telephone networks, where there's a logical link between every pair of nodes. 💜 In that setup, direct routed calls always take a 1-hop path to the destination, and alternate routed calls take at most a 2-hop path. 🌸

"Trunk reservation" is a scheme where a certain amount of bandwidth on each link is reserved for direct routed calls [MS91]. Alternate routed calls are only allowed on a trunk as long as the remaining trunk bandwidth is greater than the reserved capacity. 🎀 So alternate routed calls can't totally displace direct routed ones on a trunk. This strategy has been shown to be super effective at preventing the bad effects of alternate routing. 👑

"Dynamic alternate routing" (DAR) is a strategy where alternate routing is controlled by limiting the number of choices, on top of trunk reservation [MS91]. ✨ Under DAR, the source first tries the direct link to the destination. If that's blocked, the source tries to alternate route the call through a pre-selected neighbor. If the call is still blocked, a different neighbor gets selected for alternate routing to that destination in the future, and the present call is dropped. 🥲 DAR only needs local state information. It also "learns" good alternate paths through random sampling, and then it sticks with them for as long as it can. 🌈

More recent circuit-switched routing schemes use global state to select routes for calls. 🌍 One example is AT&T's Real-Time Network Routing (RTNR) scheme [ACFH92]. Unlike schemes like DAR, RTNR handles multiple classes of service, including voice and data at fixed rates. 🧚 It uses a sophisticated per-class trunk reservation mechanism with dynamic bandwidth sharing between classes. When it's alternate routing a call, RTNR looks at the loading on all trunks in the network to pick a path. 🔍 And because the topology is fully connected, passing around status information is simple: each node just exchanges status information directly with all the others. 🫶

From the point of view of designing QoS-based Internet routing schemes, there's a lot to learn from circuit-switched routing. ✨ Take alternate routing and how to control it, or dynamic resource sharing among different classes of traffic, for example. Honestly though, it's not simple to apply some of these results to a general topology network with heterogeneous multirate traffic. 🌸 The ATM network routing work described next shows exactly why. 🦋

9.3. ATM Networks 🦋

okay so ATM networks have the VC routing problem, and it's giving circuit-switched drama 💅 Shocker, right? So people have proposed extensions of circuit-switched routing. The goal is to get higher throughput than plain old shortest-path routing. And the flows here usually have just one QoS requirement, i.e., bandwidth. Simple but iconic ✨

The first idea is to extend alternate routing with trunk reservation to general topologies [SD95]. Bestie, here's how it goes: a distance vector routing protocol builds routing tables at each node, with multiple choices of increasing hop count to each destination. A VC set-up is first routed along the primary ("direct") path. If there aren't enough resources on that path, alternate paths get tried in order of increasing hop count. A flag in the VC set-up message says whether it's primary or alternate routing, and bandwidth on alternate-path links is allocated subject to trunk reservation. The trunk reservation values are based on some assumptions about traffic characteristics. But the scheme only works for a single data rate, so its practical utility is limited 🥲

The next idea is to bring the notion of controlled alternate routing into traditional link state QoS-based routing [GKR96]. This one is so clever 🧠 First, each VC gets a maximum permissible routing cost. That cost can be set from expected revenues in carrying the VC, or simply from the length of the shortest path to the destination. Each link has a metric that increases exponentially with its utilization. When a switch computes a path for a VC, it just picks the least-cost feasible path based on the link metric and the VC's QoS requirement. The VC is admitted if the path cost is less than or equal to the maximum permissible routing cost. This limits how far a VC can "detour," which stops it from eating up excessive resources. It's a practical scheme, and the basic idea can be extended to hierarchical routing. The catch is that its performance hasn't been analyzed thoroughly yet 🤷‍♀️ A similar notion of admission control based on the connection route also showed up in a routing scheme presented in [ACG92].

Now let's talk about the ATM Forum PNNI protocol [PNNI96]! Here's a partial list of its stated characteristics:

• Scales to very large networks
• Supports hierarchical routing
• Supports QoS
• Uses source routed connection setup
• Supports multiple metrics and attributes
• Provides dynamic routing

PNNI is sub-divided into two protocols: a signaling protocol and a routing protocol. The PNNI signaling protocol establishes point-to-point and point-to-multipoint connections, and it supports source routing, crankback and alternate routing. PNNI source routing allows loop-free paths. It also lets each implementation use its own path computation algorithm. And source routing is expected to support incremental deployment of future enhancements, such as policy routing. Very flexible, very modular 💖

The PNNI routing protocol is a dynamic, hierarchical link state protocol that spreads topology information by flooding it through the network. The topology information is the set of resources (e.g., nodes, links and addresses) that define the network. Resources are qualified by defined sets of metrics and attributes (delay, available bandwidth, jitter, etc.), grouped by supported traffic class. Some metrics change frequently (available bandwidth, for example), so threshold algorithms decide whether a change is significant enough to require sending updated information. Other features include auto configuration of the routing hierarchy, connection admission control (as part of path calculation), and aggregation and summarization of topology and reachability information 🌈

Despite all that functionality, the PNNI routing protocol doesn't address multicast routing, policy routing or control of alternate routing. 😭 A general problem with link state QoS-based routing is efficient broadcasting of state information. Flooding is a reasonable choice with static link metrics, but it may hurt performance with dynamic metrics.

Finally, Integrated PNNI [I-PNNI] was designed from the start to take advantage of the QoS routing capabilities available in PNNI and integrate them with routing for layer 3. The idea is an integrated layer 2 and layer 3 routing protocol for networks that include PNNI in the ATM core, which is a whole mood 🎀 The I-PNNI specification has been under development in the ATM Forum, and at this time it has not yet incorporated QoS routing mechanisms for layer 3.

---

9.4. Packet Networks 🌸

Early attempts at adaptive routing in packet networks wanted to minimize delay by adapting dynamically to network congestion. Alternate routing based on k-shortest path tables, with route selection based on some local measure (e.g., shortest output queue), has been described [R76, YS81]. The original ARPAnet routing scheme was a distance vector protocol with a delay-based cost metric [MW77]. That scheme was shown to be prone to route oscillations [B82]. For this and other reasons, a link state delay-based routing scheme was developed later for the ARPAnet [MRR80]. It demonstrated techniques like triggered updates, flooding, etc., which OSPF and PNNI routing use today! Not one of these schemes can be called QoS-based routing, but they had features that are relevant to it 🧚

IBM's System Network Architecture (SNA) introduced the concept of Class of Service (COS)-based routing [A79, GM79]. There were several classes of service: interactive, batch, and network control. Users could also define other classes. When starting a data session, an application or device would request a COS. Routing would then map that COS to a statically configured route marking a path across the physical network. SNA is connection oriented, so a session got set up along that path, and the application's or device's data traversed the path for the life of the session. At first, the service a session got was based on network engineering and the current state of congestion. Later, transmission priority was added to subarea SNA. Transmission priority let more important traffic (e.g. interactive) go before less time-critical traffic (e.g. batch), and it improved link and network utilization. A session's transmission priority was based on its COS 💜

SNA later evolved to support multiple or alternate paths between nodes. But even with help from network design tools, the network administrator still had to statically configure routes 🙄 IBM later introduced SNA's Advanced Peer to Peer Networking (APPN) [B85]. APPN added new features to SNA, including dynamic routing based on a link state database. An application would use COS to indicate its traffic requirements, and APPN would calculate a path capable of meeting them. Each COS was mapped to a table of acceptable metrics and parameters that qualified the nodes and links in the APPN topology database. Metrics and parameters used in the APPN route calculation include, but are not limited to: delay, cost per minute, node congestion and security. Because APPN is dynamic, it could route around failures and reduce network configuration work ✨

The service APPN delivered was still based on network engineering, transmission priority and network congestion. IBM later introduced an extension to APPN, High Performance Routing (HPR) [IBM97]. HPR uses a congestion avoidance algorithm called adaptive rate based (ARB) congestion control. Using predictive feedback methods, the ARB algorithm prevents congestion and improves network utilization. Most recently, an extension to the COS table was defined so that HPR routing could recognize and take advantage of ATM QoS capabilities. Full circle moment 🦋

Now for IP routing. Both IDRP [R92] and OSPF support type of service (TOS)-based routing. The IP header has a TOS field, but there's no standardized way of using it for TOS specification and routing. It seems possible to use the IP TOS feature, along with TOS-based routing and proper network engineering, to do QoS-based routing. The emerging differentiated services model is generating renewed interest in TOS support 👀 Among the newer schemes, Source Demand Routing (SDR) [ELRV96] allows on-demand path computation by routers, and the implementation of strict and loose source routing. The Nimrod architecture [CCM96] has a number of concepts built in to handle scalability and specialized path computation. Recently, some work has been done on QoS-based routing schemes for the integrated services Internet. For example, in [M98], heuristic schemes for efficient routing of flows with bandwidth and/or delay constraints are described and evaluated 🔥

9. SUMMARY AND CONCLUSIONS ✨

In this document, a framework for QoS-based Internet routing was defined. 👑 This framework adopts the traditional separation between intra and interdomain routing. That approach is especially meaningful for QoS-based routing, since there are many views on how it should be accomplished and many different needs. The objective of this document was to encourage the development of different solution approaches for intradomain routing, subject to some broad requirements, while consensus on interdomain routing is achieved. To get there, the QoS-based routing issues were described, some broad intradomain routing requirements and an interdomain routing model were defined, QoS-based multicast routing was discussed, and a detailed review of related work was presented 🫶

Deploying QoS-based routing across multiple administrative domains requires both the development of intradomain routing schemes and a standard way for them to interact through a well-defined interdomain routing mechanism. This document outlined the issues that must be addressed, but it did not engage in specifying the actual features of the interdomain routing scheme. That would be the next step in the evolution of wide-area, multidomain QoS-based routing 🌈

REFERENCES 📚✨

okay bestie, here are all the receipts so you can check every single source yourself 🕵️‍♀️💅

• [A79] V. Ahuja, "Routing and Flow Control in SNA", IBM Systems Journal, 18 No. 2, pp. 298-314, 1979.
• [A84] J. M. Akinpelu, "The Overload Performance of Engineered Networks with Non-Hierarchical Routing", AT&T Technical Journal, Vol. 63, pp. 1261-1281, 1984.
• [ACFH92] G. R. Ash, J. S. Chen, A. E. Frey and B. D. Huang, "RealTime Network Routing in a Dynamic Class-of-Service Network", Proceedings of ITC 13, 1992.
• [ACG92] H. Ahmadi, J. Chen, and R. Guerin, "Dynamic Routing and Call Control in High-Speed Integrated Networks", Proceedings of ITC-13, pp. 397-403, 1992.
• [B82] D. P. Bertsekas, "Dynamic Behavior of Shortest Path Routing Algorithms for Communication Networks", IEEE Trans. Auto. Control, pp. 60-74, 1982.
• [B85] A. E. Baratz, "SNA Networks of Small Systems", IEEE JSAC, May, 1985.
• [BBCD98] Black, D., Blake, S., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Services", Work in Progress.
• [BCCH94] Braden, R., Clark, D., Crocker, D., and C. Huitema, "Report of IAB Workshop on Security in the Internet Architecture", RFC 1636, June 1994.
• [BCF94] A. Ballardie, J. Crowcroft and P. Francis, "Core-Based Trees: A Scalable Multicast Routing Protocol", Proceedings of SIGCOMM `94.
• [BCS94] Braden, R., Clark, D., and S. Shenker, "Integrated Services in the Internet Architecture: An Overview", RFC 1633, July 1994.
• [BZ92] S. Bahk and M. El Zarki, "Dynamic Multi-Path Routing and How it Compares with Other Dynamic Routing Algorithms for High Speed Wide Area Networks", Proc. SIGCOMM `92, pp. 53-64, 1992.
• [BZBH97] Braden, R., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Spec", RFC 2205, September 1997.
• [C91] C-H. Chow, "On Multicast Path Finding Algorithms", Proceedings of the IEEE INFOCOM `91, pp. 1274-1283, 1991.
• [CCM96] Castineyra, I., Chiappa, J., and M. Steenstrup, "The Nimrod Routing Architecture", RFC 1992, August 1996.
• [DEFV94] S. E. Deering, D. Estrin, D. Farinnacci, V. Jacobson, C-G. Liu, and L. Wei, "An Architecture for Wide-Area Multicast Routing", Technical Report, 94-565, ISI, University of Southern California, 1994.
• [ELRV96] Estrin, D., Li, T., Rekhter, Y., Varadhan, K., and D. Zappala, "Source Demand Routing: Packet Format and Forwarding Specification (Version 1)", RFC 1940, May 1996.
• [GKR96] R. Gawlick, C. R. Kalmanek, and K. G. Ramakrishnan, "On-Line Routing of Permanent Virtual Circuits", Computer Communications, March, 1996.
• [GPSS98] A. Ghanwani, J. W. Pace, V. Srinivasan, A. Smith and M. Seaman, "A Framework for Providing Integrated Services over Shared and Switched IEEE 802 LAN Technologies", Work in Progress.
• [GM79] J. P. Gray, T. B. McNeil, "SNA Multi-System Networking", IBM Systems Journal, 18 No. 2, pp. 263-297, 1979.
• [GOA97] Y. Goto, M. Ohta and K. Araki, "Path QoS Collection for Stable Hop-by-Hop QoS Routing", Proc. INET '97, June, 1997.
• [GKOP98] R. Guerin, S. Kamat, A. Orda, T. Przygienda, and D. Williams, "QoS Routing Mechanisms and OSPF extensions", work in progress, March, 1998.
• [IBM97] IBM Corp, SNA APPN - High Performance Routing Architecture Reference, Version 2.0, SV40-1018, February 1997.
• [IPNNI] ATM Forum Technical Committee. Integrated PNNI (I-PNNI) v1.0 Specification. af-96-0987r1, September 1996.
• [ISI81] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981.
• [JMW83] J. M. Jaffe, F. H. Moss, R. A. Weingarten, "SNA Routing: Past, Present, and Possible Future", IBM Systems Journal, pp. 417-435, 1983.
• [K88] F.P. Kelly, "Routing in Circuit-Switched Networks: Optimization, Shadow Prices and Decentralization", Adv. Applied Prob., pp. 112-144, March, 1988.
• [L95] W. C. Lee, "Topology Aggregation for Hierarchical Routing in ATM Networks", ACM SIGCOMM Computer Communication Review, 1995.
• [M86] L. G. Mason, "On the Stability of Circuit-Switched Networks with Non-hierarchical Routing", Proc. 25th Conf. On Decision and Control, pp. 1345-1347, 1986.
• [M98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
• [M94] Moy, J., "MOSPF: Analysis and Experience", RFC 1585, March 1994.
• [M98] Q. Ma, "Quality-of-Service Routing in Integrated Services Networks", PhD thesis, Computer Science Department, Carnegie Mellon University, 1998.
• [MMR96] D. Mitra, J. Morrison, and K. G. Ramakrishnan, "ATM Network Design and Optimization: A Multirate Loss Network Framework", Proceedings of IEEE INFOCOM `96, 1996.
• [MRR80] J. M. McQuillan, I. Richer and E. C. Rosen, "The New Routing Algorithm for the ARPANET", IEEE Trans. Communications, pp. 711-719, May, 1980.
• [MS91] D. Mitra and J. B. Seery, "Comparative Evaluations of Randomized and Dynamic Routing Strategies for Circuit Switched Networks", IEEE Trans. on Communications, pp. 102-116, January, 1991.
• [MW77] J. M. McQuillan and D. C. Walden, "The ARPANET Design Decisions", Computer Networks, August, 1977.
• [NC94] Nair, R. and Clemmensen, D. : "Routing in Integrated Services Networks", Proc. 2nd International Conference on Telecom. Systems Modeling and Analysis, March 1994.
• [P88] R. Perlman, "Network Layer Protocol with Byzantine Robustness", Ph.D. Thesis, Dept. of EE and CS, MIT, August, 1988.
• [PNNI96] ATM Forum PNNI subworking group, "Private Network-Network Interface Spec. v1.0 (PNNI 1.0)", afpnni-0055.00, March 1996.
• [R76] H. Rudin, "On Routing and "Delta Routing": A Taxonomy and Performance Comparison of Techniques for Packet-Switched Networks", IEEE Trans. Communications, pp. 43-59, January, 1996.
• [R92] Y. Rekhter, "IDRP Protocol Analysis: Storage Overhead", ACM Comp. Comm. Review, April, 1992.
• [R96] B. Rajagopalan, "Efficient Link State Routing", Work in Progress, available from braja@ccrl.nj.nec.com.
• [RN98] B. Rajagopalan and R. Nair, "Multicast Routing with Resource Reservation", to appear in J. of High Speed Networks, 1998.
• [SD95] S. Sibal and A. Desimone, "Controlling Alternate Routing in General-Mesh Packet Flow Networks", Proceedings of ACM SIGCOMM, 1995.
• [SPG97] Shenker, S., Partridge, C., and R. Guerin, "Specification of Guaranteed Quality of Service", RFC 2212, September 1997.
• [T88] D. M. Topkis, "A k-Shortest-Path Algorithm for Adaptive Routing in Communications Networks", IEEE Trans. Communications, pp. 855-859, July, 1988.
• [W88] B. M. Waxman, "Routing of Multipoint Connections", IEEE JSAC, pp. 1617-1622, December, 1988.
• [W97] Wroclawski, J., "Specification of the Controlled-Load Network Element Service", RFC 2211, September 1997.
• [WC96] Z. Wang and J. Crowcroft, "QoS Routing for Supporting Resource Reservation", IEEE JSAC, September, 1996.
• [YS81] T. P. Yum and M. Schwartz, "The Join-Based Queue Rule and its Application to Routing in Computer Communications Networks", IEEE Trans. Communications, pp. 505-511, 1981.
• [YS87] T. G. Yum and M. Schwartz, "Comparison of Routing Procedures for Circuit-Switched Traffic in Nonhierarchical Networks", IEEE Trans. Communications, pp. 535-544, May, 1987.
• [ZES97] Zappala, D., Estrin, D., and S. Shenker, "Alternate Path Routing and Pinning for Interdomain Multicast Routing", USC Computer Science Technical Report #97-655, USC, 1997.
• [ZSSC97] Zhang, Z., Sanchez, C., Salkewicz, B., and E. Crawley, "QoS Extensions to OSPF", Work in Progress.

---

AUTHORS' ADDRESSES 💌🌸

Bala Rajagopalan
NEC USA, C&C Research Labs
4 Independence Way
Princeton, NJ 08540
U.S.A

Phone: +1-609-951-2969
EMail: braja@ccrl.nj.nec.com


Raj Nair
Arrowpoint
235 Littleton Rd.
Westford, MA 01886
U.S.A

Phone: +1-508-692-5875, x29
EMail: nair@arrowpoint.com


Hal Sandick
Bay Networks, Inc.
1009 Slater Rd., Suite 220
Durham, NC 27703
U.S.A

Phone: +1-919-941-1739
EMail: Hsandick@baynetworks.com


Eric S. Crawley
Argon Networks, Inc.
25 Porter Rd.
Littelton, MA 01460
U.S.A

Phone: +1-508-486-0665
EMail: esc@argon.com

---

Full Copyright Statement 🎀⚖️

Copyright (C) The Internet Society (1998). All Rights Reserved.

okay so the legal bit, but make it cute 💅 You're allowed to copy this document and share it with others. You can also write derivative works that comment on or explain it or help people implement it, and you can publish and distribute those, in whole or in part, with no restrictions at all. The one condition is that the copyright notice and this paragraph stay on every copy and every derivative work.

BUT the document itself can't be changed in any way. That means no removing the copyright notice and no removing the references to the Internet Society or other Internet organizations. The only exceptions are developing Internet standards, where the copyright procedures in the Internet Standards process must be followed, and translating it into languages other than English. 🌈

The permissions above are perpetual and won't be revoked by the Internet Society or its successors or assigns 🫶

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.