RFC 1753

RFC 1753: IPng Technical Requirements of the Nimrod Routing and Addressing Architecture 💖

• Network Working Group 🌸: this one lives in the IETF's IPng world
• Author 🎀: N. Chiappa
• Date 📅: December 1994
• Category ✨: Informational
• Obsoletes / Updates 🙈: nothing is listed in its header, so it doesn't obsolete or update anything

Part 1 of 5! 🫶 This is a long one, so more parts are coming, bestie. Stay tuned. 💅

---

Status of This Memo 📜

This memo provides information for the Internet community. It is not a standard 🙅‍♀️, it does not specify an Internet standard of any kind. Distribution of this memo is unlimited, so share the vibes 🦋.

---

Abstract 🌈

Okay so this doc was submitted to the IETF IPng area as a reply to RFC 1550. Publishing it does not mean the IPng area has accepted any of the ideas inside 🙈. If you have thoughts, send them to the big-internet@munnari.oz.au mailing list 💌.

This document lays out the requirements that the Nimrod routing and addressing architecture puts on the internetwork layer protocol. If you want Nimrod to be happy, any protocol picked as the IPng should meet these requirements 👑. It also gives some background, which covers two things: i) the architectural and design principles that might apply to designing a new internetworking layer, and ii) the logic and reasoning behind particular requirements 🧠✨.

---

1. Introduction 🦋

Okay, first things first: this is not "IPng Requirements for Routing" 🙅‍♀️. Other routing and addressing designs might need different support, so this doc is specific to Nimrod and doesn't pretend to speak for anyone else 💅.

Now, I'm not assuming the Nimrod WG's designs will be widely adopted by the Internet 🤷‍♀️. For one thing, they haven't been deployed, tried, and tested in practise yet to see if they really work, and that is an absolutely necessary hurdle for any protocol 😭. Still, there are good reasons to believe that any routing architecture for a large, ubiquitous global Internet will share many of the same basic fundamental principles as Nimrod, and so will end up with similar requirements 🌍.

Current routing tech doesn't have the characteristics and capabilities that create these requirements yet, but it also doesn't seem totally suited to routing in the next-generation Internet 🤔. As routing tech moves toward what the next-generation Internet needs, the underlying fundamental laws and principles of routing will almost inevitably push the design, and therefore the requirements, toward stuff that looks like the material in this doc ✨.

So even if Nimrod isn't the routing architecture of the next-generation Internet, that Internet's basic routing architecture will have requirements that differ in detail but will almost inevitably be similar to these 💜.

In a similar but more general vibe, the general analysis in sections 3.1 ("Interaction Architectural Issues") and 3.2 ("State and Flows in the Internetwork Layer") will apply to other areas of a new internetwork layer, not just routing 🧚.

I'm going to tackle the internetwork packet format first, because it's simpler 🍓. After that I'll get into the whole interaction with the rest of the internetwork layer, which is a much more subtle topic 🌀.

---

2. Packet Format 🎀

2.1 Packet Format Issues 💅

So the big rule here is that Nimrod's design philosophy is "maximize the lifetime (and flexibility) of the architecture" 👑. Design tradeoffs (aka optimizations) that hurt the flexibility, adaptability, and lifetime of the design are not necessarily wise choices. They might cost more than they save 💸. Optimizations like that could be fine in a stand-alone system where replacement costs are small, but on the global communication network, replacement costs are way, way higher 😭.

Providing Nimrod functionality means carrying certain information in the packets. The design principle above has a few corollaries for how to specify the fields that hold that info 📦.

First, the design should be "simple and straightforward" 🧸. That means different functions should be handled by completely separate mechanisms and fields in the packets. It might seem like you could save space by overloading two functions onto one mechanism or field, but general experience says that over time this kind of "optimization" ends up costing more, because it restricts flexibility and adaptability 🙃.

Second, field lengths should be specified to be somewhat larger than anything that could conceivably be used 📏. System architecture history is full of examples (processor address size is the most notorious one) where fields became too short over the lifetime of the system 💀. The doc says what the smallest reasonable "adequate" lengths are, but that's more of a "critical floor" than a recommendation ⚠️. There's also a "recommended" length, which is the length that matches this principle, and a wise designer would pick it 🧠✨.

It's important to note that this does not mean implementations have to support the maximum value that fits in a field of that size 🙈. The author imagines that system-wide administrative limits will be placed on the maximum values that must be supported. Then, as the need arises, the administrative limit can go up 📈. This gives an easy, completely interoperable path (with no special mechanisms) for upgrading the network's capability 🦋. Say the maximum supported value of a field needs to go from M to N: an announcement goes out that this is coming. During the interim, the system keeps operating with M while new implementations get deployed. Interoperation is automatic the whole time, with no transition mechanisms of any kind needed 🫶. When things are "ready" (meaning the proportion of old equipment is small enough), use of the larger value starts 🚀.

Also, when you're talking about the packet format, you first have to tell apart the host-router part of the path from the router-router part 🛤️. A format that works fine for one may not work for the other 🙅‍♀️.

This is complicated by the fact that Nimrod can be made to work, though not in optimal form, with information or fields missing from the packet in the host to "first hop router" section of the path 🧩. The first hop router can then add the missing fields and information 🪄. (This capability is what lets Nimrod deploy and run with unmodified IPv4 hosts, though similar techniques could work with other internetworking protocols 🌸.) To get the full range of Nimrod capabilities, hosts need to be upgraded so they include the necessary information in the packets they exchange with routers 👑.

Second, Nimrod currently has three planned forwarding modes: flows, datagram, and source-routed packets 🧚. A format that works for one mode may not work for another, since some modes use fields that other modes don't 🙃. Whether those fields are present or absent makes a difference 💜.

2.2 Packet Format Fields ✨

okay so here's the big wishlist for what Nimrod wants to see in the internetworking packet 💅

• Source and destination endpoint identification. There are a few possibilities here 🦋

One is "UID"s. These are "shortish", fixed length fields that show up in every packet, in the internetwork header. They hold globally unique, topologically insensitive identifiers for either i) endpoints (if you're not sure what an endpoint is, think of it as a host) or ii) multicast groups. When it's the first one, the UID is an EID. When it's the second, it's a "set ID", or SID. An SID looks just like an EID, but it refers to a group of endpoints. The semantics of SIDs are not completely defined 🤷‍♀️ For each of these, 48 bits is adequate, but we would recommend 64 bits. (IPv4 will be able to run with smaller ones for a while, but eventually it'll either need a new packet format, or the difficult and not wholly satisfactory technique known as Network Address Translators, which lets the contents of these fields be only locally unique. 💀)

Another possibility is a shorter field called an "endpoint selector", or ESEL. It holds a value that isn't globally unique, only unique in the mapping tables on each end. Those tables map the small value to a globally unique value, such as a DNS name.

Finally, it's possible to imagine overall network designs that don't include any endpoint identification in the packet at all. Instead, the endpoint gets transferred at the start of a communication, and from then on it's inferred. This alternative would need some other way of telling which endpoint a given packet is for, if there are several endpoints at a given destination. Some coordination on allocation of flow-ids, or higher level port numbers, etc., might do this 🧚

• Flow identification. There are two basic approaches here, depending on whether flows are aggregated (in intermediate switches) or not. It should be emphasized at this point that it is not yet known whether flow aggregation will be needed. The only reason to do it is to control the growth of state in intermediate routers, but there is no hard case made that this growth will be unmanageable, or that aggregating flows will be feasible practically. So, yeah, still up in the air 🌸

For the non-aggregated case, a single flow-id field will suffice. This must not use one of the two previous UID fields, because in datagram mode (and probably source-routed mode as well) the flow-id will be over-written during transit of the network. It could most easily be built by adding a UID to a locally unique flow-id, which gives you a globally unique flow-id. It is possible to use non-globally unique flow-ids (which would let this field be shorter), but then collisions would happen and have to be dealt with. An adequate length for the local part of a globally unique flow-id would be 12 bits (which would be my "out of thin air" guess), but we recommend 32. For a non-globally unique flow-id, 24 bits would be adequate, but I recommend 32 🎀

For the aggregated case, three broad classes of mechanism are possible 👑

• Option 1: The packet contains a sequence (sort of like a source route) of flow-ids. Whenever you aggregate or deaggregate, you move along the list to the next one. This takes the most space, but is otherwise the least work for the routers.
• Option 2: The packet contains a stack of flow-ids, with the current one on the top. When you aggregate, you push a new one on. When you de-aggregate, you take one off. This takes more work, but less space in the packet than the complete "source-route". Encapsulating packets to do aggregation basically does this, but you're stacking entire headers, not just flow-ids. The clever way to do this flow-id stacking, without doing encapsulation, is to find out from flow-setup how deep the stack will get, and allocate the space in the packet when it's created. That way, all you ever have to do is insert a new flow-id, or "remove" one. You never have to make room for more flow-ids 🧠
• Option 3: The packet contains only the "base" flow-id (i.e., the one with the finest granularity), and the current flow-id. When you aggregate, you just bash the current flow-id. The tricky part comes when you de-aggregate, because you have to put the right value back. To do this, you have to have state in the router at the end of the aggregated flow, which tells you what the de-aggregated flow for each base flow is. The downside here is obvious: we get away without individual flow state for each of the constituent flows in all the routers along the path of that aggregated flow, except for the last one 🙈

Other than encapsulation, which has significant inefficiency in space overhead fairly quickly, after just a few layers of aggregation, there appears to be no way to do it with just one flow-id in the packet header. Even if you don't touch the packets, but do the aggregation by mapping some number of "base" flow-id's to a single aggregated flow in the routers along the path of the aggregated flow, the table that does the mapping is still going to need a number of entries directly proportional to the number of base flows going through the switch 😭

• A looping packet detector. This is any mechanism that will detect a packet which is "stuck" in the network. A timeout value in packets, together with a check in routers, is an example. If this is a hop-count, it has to be more than 8 bits. 12 bits would be adequate, and I recommend 16 (which also makes it easy to update). This is not to say that I think networks with diameters larger than 256 are good, or that we should design such nets. But I think limiting the maximum path through the network to 256 hops is likely to bite us down the road, the same way making "infinity" 16 in RIP did (as it did, eventually). When we hit that ceiling, it's going to hurt, and there won't be an easy fix. I will note in passing that we are already seeing path lengths of over 30 hops ⭐
• Optional source and destination locators. These are structured, variable length items. They're topologically sensitive identifiers for the place in the network from which the traffic originates, or to which the traffic is destined. The locator will probably contain internal separators which divide up the fields, so that a particular field can be enlarged without creating a great deal of upheaval. An adequate value for maximum length supported would be up to 32 bytes per locator, and longer would be even better. I would recommend up to 256 bytes per locator 🌈
• Perhaps (paired with the above), an optional pointer into the locators. This is optional "forwarding state" (i.e., state in the packet which records something about its progress across the network). It's used in the datagram forwarding mode to help ensure that the packet does not loop, and it can also improve the forwarding processing efficiency. It is thus not absolutely essential, but it is very desirable from a real-world engineering viewpoint. It needs to be large enough to identify locations in either locator. E.g., if locators can be up to 256 bytes, it would need to be 9 bits 🦋
• An optional source route. This is used to support the "source routed packet" forwarding mode. Although it's not designed in detail yet, we can discuss two possible approaches 💖

In the first one, used with "semi-strict" source routing (in which a contiguous series of entities is named, albeit perhaps at a high layer of abstraction), the syntax will likely look a lot like source routes in PIP. In Nimrod, they will be a sequence of Nimrod entity identifiers (i.e., locator elements, not complete locators), along with clues as to the context in which each identifier is to be interpreted (e.g., up, down, across, etc.). Since those identifiers are variable length themselves (although probably most will be two bytes or less, otherwise the routing overhead inside the named object would be excessive), and the hop count above contemplates the possibility of paths of over 256 hops, it would seem these might possibly someday exceed 512 bytes, if a lengthy path was specified in terms of the actual physical assets used. An adequate length would be 512 bytes. The recommended length would be 2^16 bytes (although this length would probably not be supported in practise; rather, the field length would allow it) 🫶

In the other, used with classical "loose" source routes, the source consists of a number of locators. It is not yet clear if this mode will be supported. If it is, the header would need to be able to store a sequence of locators (as described above). Space might be saved by not repeating locator prefixes that match that of the previous locator in the sequence. Nimrod will probably allow use of such "locally useful" locators. It is hard to determine what an adequate length would be for this case. The recommended length would be 2^16 bytes (again, with the previous caveat) 🔥

• Perhaps (paired with the above), an optional pointer into the source route. This is also optional "forwarding state". It needs to be large enough to identify locations anywhere in the source route. E.g., if the source route can be up to 1024 bytes, it would need to be 10 bits 💜
• An internetwork header length. I mention this because the above fields could easily exceed 256 bytes, if they are all to be carried in the internetwork header (see comments below as to where to carry all this information). So the header length field needs to be more than 8 bits. 12 bits would be adequate, and I recommend 16 bits. Putting some of the data items above into an interior header, to limit the size of the basic internetworking header, does not really seem optimal. This data is for use by the intermediate routers, and it needs to be easily accessible 🍓
• Authentication of some sort is needed. See the recent IAB document which was produced as a result of the IAB architecture retreat on security (draft-iab-sec-arch-workshop-00.txt), section 4, and especially section 4.3. There is currently no set way of doing "denial/theft of service" in Nimrod, but this topic is well explored in that document. Nimrod would use whatever mechanism(s) seem appropriate to those knowledgeable in this area 🔐
• A version number. Future forwarding mechanisms might need other information (i.e., fields) in the packet header. Use of a version number would allow it to be modified to contain what's needed. (This would not necessarily be information that is visible to the hosts, so this does not necessarily mean that the hosts would need to know about this new format.) 4 bits is adequate. It's not clear if a larger value needs to be recommended 🌸

2.3. Field Requirements and Addition Methods ✨

okay so, the Nimrod thing we talked about is that it can run in a limited mode where the first-hop router fills in the info/fields that are missing 🧚 which makes you ask: "which of the fields must be present in the host-router header, and which could be added by the router?" 🤔 The ONLY ones that are absolutely necessary in every packet are the endpoint identification (as long as there's some way to map them into locators; this would obviously be most useful on UID's which are EID's) 💖

as for the rest, if the user wants to use flows and wants a guarantee that their packets get assigned to the correct flows, then the flow-id field is needed 🦋 if the user wants efficient use of the datagram mode, it's probably necessary to include the locators in the packet sent to the router 💅 and if the user wants to specify the route for their packets and doesn't want to set up a flow, they need to include the source route 🗺️

now the big question: how would extra info/fields get added to the packet if it's sent from the host in an incomplete form? (and I mean the simple, mechanical how, not the messier where-does-the-info-come-from thing 🙈)

this is a complex one ngl, because all the IPng candidates (and honestly, any reasonable internetworking protocol) are extensible protocols, so those extension mechanisms could be used ✨ it would also be possible to carry some of the required info as user data inside the internetworking packet, with the original user's data encapsulated even further inside 🪆 and finally, a private inter-router packet format could be defined 🔐

it's not clear which path is the best one 😭 BUT we can totally talk about which fields the Nimrod routers need access to, and how often 👀 the less used ones could live in harder-to-get-to places (like inside an encapsulated header) 🎀 the fields the routers need on every hop are the flow-id and the looping packet detector 🔥 the locator/pointer fields are only needed at intervals (in what datagram forwarding mode calls "active" routers) 💜 and the source route is needed too, but only at every object that's named in the source route 🌸

depending on how access control gets done, and which forwarding mode is being used, the UID's and/or locators might get examined for access control purposes, wherever that function happens 🛡️

this isn't a complete exploration of the topic, but it should give you a rough idea of what's going on 💕

---

3. Architectural Issues 🦋

3.1. Interaction Architectural Issues 🧩

the topic of how this interacts with the rest of the internetwork layer is even more complex 😵‍💫 Nimrod partly springs from a design vision that sees the entire internetwork layer, spread across all the hosts and routers of the internetwork, as one single system. it's a distributed system, but it's still one system! 🌐

when you look at it from that angle, you naturally fall into a typical system designer mindset, where you start thinking about how to modularize the system: choosing the functional boundaries that split the system into functional units, and defining how those units interact 🧠 as we all know, modularization is the key part of the system design process 👑

it's rare for a group of completely independent modules to form a system; there's usually a pretty strong internal interaction 🫶 those interactions have to be thought about and understood as part of the modularization process, since they affect where the functional boundaries end up. poor placement leads to complex interactions, or to desired interactions that can't be realized at all 💀

these are even bigger issues for a system that's expected to have a long lifetime. putting the functional boundaries in the right places, so the system breaks up cleanly and simply into truly fundamental units, is a total necessity if the system is going to endure and serve well 🏆

3.1.1. The Internetwork Layer Service Model 🍓

okay so, to get back to the view of the internetwork layer as a system: that system provides certain services to its clients. in other words, it instantiates a service model 🎀 to begin with, if there's no shared view of the service model the internetwork layer is supposed to provide, it's reasonable to suppose it'll be impossible to agree on mechanisms at the internetwork level to provide that service 😭

to answer what the service model ought to be, you can view the internetwork layer itself as a subsystem of an even larger system: the entire internetwork itself 🌍 (that system is quite likely the largest and most complex system we will ever build, since it's the largest system we can possibly build. it's the system that will inevitably contain almost all other systems 🤯)

from that point of view, the service model question gets a lot clearer. the services the internetwork layer provides are no longer purely abstract. they can be thought of as the external module interface of the internetwork layer module 🔌 if we can agree on where to put the functional boundaries of the internetwork layer, and on what overall service the internet as a whole should provide, then the internetwork layer's service model should be easier to agree on ✨

in general terms, the unreliable packet ought to stay the fundamental building block of the internetwork layer 📦 the design principle here is that you can take any packet and throw it away with no warning or other action, or take any router and turn it off with no warning, and the system still works. that is SO powerful 🦋 the component design simplicity (routers don't have to stand on their heads to hold on to a packet they have the only copy of), and the overall system robustness, that come from these two assumptions are absolutely critical 💅

in detail, though, especially in areas that are still being researched and experimented on (like resource allocation, security, etc.), it's hard to come up with a finished definition of exactly what the service model of the internetwork layer ought to be 🧪

3.1.2. The Subsystems of the Internetwork Layer 🧁

anyway 💁‍♀️ by viewing the internetwork layer as one big system, you start thinking about what subsystems are needed, and what the interactions among them should look like. Nimrod is basically a bunch of subsystems of this larger system, the internetwork layer 🌸 it is not meant to be a purely standalone set of subsystems. it's supposed to work together in close concert with the other subsystems of the internetwork layer (resource allocation, security, charging, etc.) to provide the service model 🫶

one reason Nimrod isn't just one monolithic subsystem is that some of the interactions with the other subsystems, like the resource allocation subsystem, are way clearer and easier to manage if the routing is broken up into several subsystems with the interactions between them open 🔓

here's the thing: Nimrod was initially broken up into separate subsystems for purely internal reasons 🙈 but it just so happens that, looked at as a separate problem, the fundamental boundary lines for dividing routing into subsystems are the same boundaries that make interacting with other subsystems cleaner. that's added evidence these boundaries are the right ones 🏆

the subsystems that make up the functionality Nimrod covers are i) routing information distribution (for Nimrod, that's topology map distribution, along with the attributes [policy, QOS, etc.] of the topology elements), ii) route selection (strictly speaking, not part of the Nimrod spec per se, but functional examples will be produced), and iii) user traffic handling 🗺️

the first one can pretty much be defined without reference to other subsystems, but the second and third are way more involved 🥹 for example, route selection might mean finding out which links have the resources available to handle some required level of service. and for user traffic handling, if a particular application needs a resource reservation, getting that reservation to the routers is just as much a part of getting the routers ready as making sure they have the correct routing info. so routing is tied in with other subsystems here too 🕸️

so, even though we can talk about the relationship between the Nimrod subsystems and the other functional subsystems of the internetwork layer, until the service model of the internetwork layer is more clearly visible (along with the functional boundaries inside that layer), that discussion is necessarily kinda nebulous 🌫️

3.2. State and Flows in the Internetwork Layer 🌊

the internetwork layer as a whole holds a bunch of information with varying lifetimes 📚 we can refer to this information as the internetwork layer's "state". some of that state is stored in the routers, and some of it is stored in the packets 📦

inside the packet, I distinguish between what I call "forwarding state", which records something about how far this individual packet has gotten through the network (like the hop count, or the pointer into a source route), and other state, which is info about what service the user wants from the network (like the destination of the packet), etc. 💕

3.2.1. User and Service State 👑

I call state that reflects the desires and service requests of the user "user state" 💖 this is info that could be sent in each packet, or stored in the router and applied to multiple packets (whichever makes the most engineering sense) ✨ it's still called user state even when a copy is stored in the routers 🎀

user state can be split into two classes: "critical" (like destination addresses), without which the packets can't be forwarded at all 😱 and "non-critical" (like a resource allocation class), without which the packets can still get forwarded, just not quite the way the user would most prefer 🫠

there are a range of possible mechanisms for getting this user state to the routers. it can be put in every packet, or placed there by a setup 🛠️ in the latter case, you have a whole range of options for how to get it back when you lose it, like placing a copy in every Nth packet 🔁

BUT other state is needed that can't be stored in each packet 🙅‍♀️ it's state about the longer-term situation (across the life of many packets), so it's state that's inherently associated with a bunch of packets over some time-frame (e.g., a resource allocation). I call this state "server state" 🧚

this apparently changes the "stateless" model of routers a bit, but honestly the change is more apparent than real 🙃 routers already contain state, like routing table entries, and without that state it's virtually impossible to handle user traffic. all that's being changed is the amount, granularity, and lifetime of the state in the routers 🌈

some of this service state may need to be installed in a fairly reliable way. for example, if there's service state related to billing, or to allocating resources for a critical application, you more or less need to be able to guarantee that this service state has been correctly installed 🧾

and to the extent that you have state in the routers (either service state or user state), you have to be able to associate that state with the packets it goes with. the fields in the packets that let you do this are called "tags" 🏷️💜

3.2.2 Flows 🦋

okay so let's step back for a sec and think about all the traffic zooming around the network 🌸 some of it comes from apps that are basically transactions, meaning they only need a single packet or a very small number of them. (I personally use "datagram" for those apps and "packet" for the unit of transmission through the network 📦) But other packets are part of longer-lived communications, and those have been called "flows" ✨

From the user's point of view, a flow is a sequence of packets that are associated, usually because they come from a single application instance 💖 In an internetwork layer with a more complex service model (say it supports resource allocation, etc.), the flow would have service requirements to pass along to some or all of the subsystems that provide those services 👑

To the internetworking layer, a flow is a sequence of packets that share all the attributes the internetworking layer cares about 🧚 That includes, but isn't limited to: source/destination, path, resource allocation, accounting/authorization, authentication/security, etc., etc. 💅

There isn't necessarily a one-to-one mapping from flows to anything else 🙈 whether that's a TCP connection, an application instance, or whatever. One flow might hold several TCP connections (like with FTP, where you've got the control connection plus a bunch of data connections), or one application might have several flows (like multimedia conferencing, where you'd have one flow for the audio, another for a graphic window, etc., each with different resource needs in terms of bandwidth, delay, etc.) 🎀

Flows can also be multicast constructs, meaning they have multiple sources and destinations. They are not inherently unicast 🌈 Multicast flows are more complex than unicast (there's a big pool of state that has to be kept coherent), but the concepts are pretty similar.

Okay, here's an interesting architectural issue 🫶 Let's assume we've got all these internetwork level subsystems (routing, resource allocation, security/access-control, accounting), etc. Now we have two choices.

First choice: we could let each individual subsystem that uses the concept of flows define for itself what it thinks a "flow" is, and define which values in which fields in the packet make up a given "flow" for it. Then, presumably, we'd have to allow 2 flows for subsystem X to map onto 1 flow for subsystem Y to map onto 3 flows for subsystem Z. In other words, you can mix and match to your heart's content 💖

Second choice: we could define a standard "flow" mechanism for the internetwork layer, along with a way of identifying the flow in the packet, etc. Then, if two things want to differ in any subsystem, they have to have a separate flow for each one 🧐

The first choice is a little easier to deploy incrementally, since you don't have to agree on a common flow mechanism 🍓 It might also save on replicated state (if I have 3 flows that are the same for subsystem X and different for Y, I only need one set of X state), and it has a lot more flexibility. The second choice is simple and straightforward, and given how complex the proposal already is, any place we can make things simpler, we should 💜

The choice isn't trivial, and it all depends on things like "what percentage of flows will want to share the same state in certain subsystems with other flows". I don't know how to quantify that 🤷‍♀️ But as an architect, I prefer simple, straightforward things. This system is pretty complex already, and I'm not sure the benefits of being able to mix and match are worth the added complexity. So, for the moment, I'll assume a single, system-wide definition of flows 🔥

The packets that belong to a flow could be identified by a tag made up of a number of fields (such as addresses, ports, etc.), as opposed to a specialized field. However, it may be more straightforward, and foolproof, to simply identify which flow a packet belongs to with a specialized tag field (the flow-id) in the internetwork header. Given that you can always find situations where the existing fields alone don't do the job, and you still need a separate field to do it correctly, it seems best to take the simple, direct approach and say "the flow a packet belongs to is named by a flow-id in the packet header" 🎯

Globally unique flow-ids (or at least a flow-id that is unique along the path of the flow) are also simple, and that's desirable. They take more bits in the header, but then you don't have to worry about all the mechanism needed to remap locally-unique flow-ids, etc., etc. 🙌 For something with a long lifetime that's going to be deployed widely, simplicity and directness are the only way to go. For me, that means flows are named solely by globally unique flow-ids, rather than by some complex semantics on existing fields 👑

However, how to recognize which packets belong to flows is somewhat orthogonal to whether the internetwork level recognizes flows at all. Should it? 🤔

3.2.3 Flows and State 🌷

To the extent that you have service state in the routers, you have to be able to associate that state with the packets it goes with. This is a fundamental reason for flows 💫 Access to service state is one reason to explicitly recognize flows at the internetwork layer, but it's not the only one.

If the user has requirements in a number of areas (like routing and access control), they can theoretically communicate those to the routers by putting a copy of all the relevant information in each packet (in the internetwork header). If many subsystems of the internetwork are involved and the requirements are complex, that could be a lot of bits 😭

(As a final aside, there's clearly no point in storing in the routers any user state about packets that are providing datagram service. The datagram service has usually come and gone in the same packet, and this whole discussion is about state retention 💀)

There are two schools of thought on how to proceed 🧠 The first says that, for reasons of robustness and simplicity, all user state ought to be repeated in each packet. For efficiency reasons, the routers may cache that user state, probably along with precomputed data derived from it. (It makes sense to store such cached user state along with any applicable server state, of course.) 🍒

The second school says that if something is going to generate lots of packets, it makes engineering sense to give all this information to the routers once, and from then on have a tag (the flow-id) in the packet that tells the routers where to find that information 🦋 It's simply going to be too inefficient to carry all the user state around all the time. This is purely an engineering efficiency reason, but it's a significant one ✨

There's also a slightly deeper argument: the routers will inevitably come to contain more user state, and the only question is whether that state is installed by an explicit mechanism, or whether the routers infer it from watching the packets that pass through them 👀 To the extent that it's inevitable anyway, there are obvious benefits to recognizing that, and an explicit design of the installation is more likely to give satisfactory results (as opposed to an ad-hoc mechanism) 🫶

It's worth noting that although the term "flow" is often used to refer to this state in the routers along the path of the flow, it's important to tell apart i) a flow as a sequence of packets (the definition given in 3.2.2 above), and ii) a flow as the thing that gets set up in the routers. They're different, and although the particular meaning is usually clear from context, they are not the same thing at all 🙅‍♀️

I'm not sure how much use there is in any intermediate position, where one subsystem installs user state in the routers and another carries a copy of its user state in each packet 🤨

(There are other intermediate positions. First, one flow might use a given technique for all its subsystems, and another flow might use a different technique for its subsystems. There's potentially some use to this, although I'm not sure the added complexity of supporting both mechanisms is worth the benefits. Second, one flow might use one mechanism with one router along its path and a different one for another router. There are a number of reasons someone might do this, including the fact that not all routers may support the same mechanisms at the same time. 🧩)

It seems to me that having one internetwork layer subsystem (e.g., resource allocation) carry user state in all the packets (perhaps with a "hint" in the packets to find potentially cached copies in the router), while a second (e.g., routing) uses direct installation and a tag in the packets to find it, makes little sense 🙃 We should do one or the other, based on a consideration of the efficiency/robustness tradeoff.

Also, if there's a way to install flow-associated state, it makes sense to have only one, which all subsystems use, instead of building a separate one for each flow 💖

It's a little hard to choose between installation and carrying a copy in each packet without more information about exactly how much user state the network is likely to have in the future. (For instance, we might end up with 500 byte headers if we include the full source route, resource reservation, etc., in every header.) 📏

It's also hard without considering the actual mechanisms involved. As a general principle, we want recovery from a loss of state to be as local as possible, so that the number of entities that have to get involved is limited. (For instance, when a router crashes, traffic is rerouted around it without needing to open a new TCP connection.) The option of "installation" looks a lot more attractive if it's simple and relatively cheap to reinstall the user state when a router crashes, without causing a lot of hassle otherwise 🔄

However, given the likely growth in user state, the necessity for service state, the requirement for reliable installation, and a number of similar considerations, it seems that direct installation of user state, plus explicit recognition of flows through a unified definition and tag mechanism in the packets, is the way to go. This is the path that Nimrod has chosen 👑🌈

3.3. Specific Interaction Issues 🦋

okay bestie, here's the wishlist 💅 this is a very incomplete list of the things Nimrod would like to see from the internetwork layer as a whole 🫶

• One unified definition of flows in the internetwork layer, plus one unified way to use a separate flow-id field to figure out which packets belong to a given flow ✨
• One unified mechanism (it could be distributed 👑) for installing flow state in routers, multicast flows included 🌸
• A way to find out whether a given resource allocation request has failed along a given path 😭 this might just be part of that unified flow setup mechanism
• An interface to a mechanism (potentially distributed 🧚) for keeping track of membership in a multi-cast group 💜
• Support for multiple interfaces, aka multi-homing 🌈 Nimrod handles this by splitting transport identification (that's the EID's 🦋) from interface identification (that's the locators). E.g., a packet with any valid destination locator should be accepted by an endpoint's TCP, as long as the destination EID is the one assigned to that endpoint. iconic separation of concerns, no cap 👑
• Support for multiple locators ("addresses") per network interface 🌸 this is needed for a bunch of reasons, and one of them is making transitions in the locator abstraction hierarchy less painful when the topology changes 😭
• Support for multiple UID's ("addresses") per endpoint (roughly, per host) 🧚 this would definitely include multiple multicast SID's, plus at least one unicast EID (the need for multiple unicast EID's per endpoint isn't obvious 🤔)
• Support for telling apart a multicast group as a named entity 💖 from a multicast flow, which may not reach all the members 🌙
• A distributed, replicated, user name translation system (DNS?) 💀 that maps such user names into (EID, locator0, ... locatorN) bindings 🍓

---

Security Considerations 💀

okay spooky part 👻 security issues are discussed in section 2.2, so go check that out 🫶

---

Author's Address ✨

J. Noel Chiappa 💜

Phone: (804) 898-8183

EMail: jnc@lcs.mit.edu