RFC 4116
IPv4 Multihoming Practices and Limitations
RFC 4116: IPv4 Multihoming Practices and Limitations 🦋
Okay bestie, this one is an Informational RFC from July 2005 ✨ It came out of the Network Working Group, and the authors are J. Abley (ISC), K. Lindqvist (Netnod Internet Exchange), E. Davies (Independent Researcher), B. Black (Layer8 Networks), and V. Gill (AOL). The header doesn't list anything it obsoletes or updates, so it's a standalone doc 💅
Status of This Memo 📝
Quick vibe check: this memo gives information to the Internet community. It is not an Internet standard of any kind 🙈 Distribution of this memo is unlimited, so share it with everyone 🌸
Copyright Notice ⚖️
Copyright (C) The Internet Society (2005).
Abstract 🌸
Multihoming is a huge deal for lots of Internet sites, no cap 💖 This doc walks through some ways to do multihoming with IPv4 and lists features so you can compare it with other multihoming proposals, especially the IPv6 ones 🧚
Table of Contents 📚
1. Introduction ....................................................3
2. Terminology .....................................................3
3. IPv4 Multihoming Practices ......................................4
3.1. Multihoming with BGP .......................................4
3.1.1. Addressing Considerations ...........................4
3.1.2. AS Number Considerations ............................6
3.2. Multiple Attachments to a Single Transit Provider ..........6
3.3. NAT- or RFC2260-based Multihoming ..........................7
4. Features of IPv4 Multihoming ....................................7
4.1. Redundancy .................................................7
4.2. Load Sharing ...............................................8
4.3. Performance ................................................8
4.4. Policy .....................................................8
4.5. Simplicity .................................................9
4.6. Transport-Layer Survivability ..............................9
4.7. Impact on DNS ..............................................9
4.8. Packet Filtering ...........................................9
4.9. Scalability ................................................9
4.10. Impact on Routers ........................................10
4.11. Impact on Hosts ..........................................10
4.12. Interactions between Hosts and the Routing System ........10
4.13. Operations and Management ................................10
4.14. Cooperation between Transit Providers ....................10
5. Security Considerations ........................................10
6. Acknowledgements ...............................................10
7. Informative References .........................................111. Introduction ✨
Multihoming is an important part of service for many Internet sites 🍓 Current IPv4 multihoming practices got bolted onto the Classless Inter Domain Routing (CIDR) architecture [RFC1519], which assumes that routing table entries can be aggregated based on a hierarchy of customers and service providers.
Multihoming is how sites meet a bunch of high-level requirements, and it's widely used across the IPv4 Internet. There are some practical limitations though 😭 including concerns about how it scales as the Internet keeps growing. This document is here to record common IPv4 multihoming practices and list their features so you can compare them with other multihoming approaches 💜
There are a few different ways to route and manage traffic in and out of a multihomed site. Most of them lean on the routing policy features of the inter-domain routing protocol, the Border Gateway Protocol, version 4 (BGP) [RFC1771]. This document also covers a multi-homing strategy that doesn't depend on BGP at all 🦋
2. Terminology 📖
Let's get the vocab down first, bestie:
- "site": an entity that autonomously runs a network using IP. It's the one deciding the addressing plan and routing policy for that network. This definition is meant to be equivalent to 'enterprise' as defined in [RFC1918] 🏠
- "transit provider": a site that directly provides Internet connectivity to one or more external sites. That connectivity goes beyond the transit provider's own site and its own direct customer networks. A transit provider's site is directly connected to the sites it provides transit for 🚇
- "multihomed": a site with more than one transit provider. "Site-multihoming" is the practice of setting a site up to be multihomed 🌐
- "re-homing": a transition of a site between two states of connectedness, caused by a change in the connectivity between the site and its transit providers' sites 🔁
- "multi-attached": a site with more than one point of layer-3 interconnection to a single transit provider 🔗
- Provider-Independent (PI) addresses: globally-unique addresses that aren't assigned by a transit provider. They come from some other organisation, usually a Regional Internet Registry (RIR) 🌍
- Provider-Aggregatable (PA) addresses: globally-unique addresses assigned by a transit provider to a customer. They're considered "aggregatable" because the routes for PA addresses are usually covered by an aggregate route set for the address space the transit provider operates, which is where the assignment came from 📦
Heads up: the words "assign" and "allocate" have specific meanings in Regional Internet Registry (RIR) address management policies, but this document uses them more loosely 🙈
3. IPv4 Multihoming Practices 🎀
3.1. Multihoming with BGP 🌈
The basic move for multihoming with BGP is to announce a set of routes to two or more transit providers. That gives the rest of the Internet multiple paths back to the multihomed site, and each transit provider adds another possible path for the site's outbound traffic 🛣️✨
3.1.1. Addressing Considerations 🏡
3.1.1.1. PI Addresses 💎
The site uses PI addresses, and a set of routes covering those PI addresses is announced or propagated by two or more transit providers 🙌
Using PI addresses has long been the favourite approach for IPv4 multihoming 👑 Until the mid-1990s it was pretty easy to pull off, since the maximum generally accepted prefix length in the global routing table was a /24, and you didn't need much justification to get a /24 PI assignment. Then things got stricter. RIR address management policies became less liberal, and not every RIR hands out address blocks to small, multihomed end-users. The ones that do need justification even for blocks as big as a /24, which small sites usually can't meet. So PI addresses are simply out of reach for lots of sites that want to multihome 😭
Each site using PI addresses adds another prefix to the global routing system. If this scheme for multihoming became widespread, it would raise scaling concerns 📈💀
3.1.1.2. PA Addresses 🧩
The site uses PA addresses assigned by a single transit provider. The set of routes covering those PA addresses (the "site route set") is announced or propagated by one or more additional transit providers. The transit provider that assigned the PA addresses (the "primary transit provider") originates a set of routes that cover the site route set. The primary transit provider often originates or propagates the site route set too, along with the covering aggregates 🎁
Using PA addresses fits sites whose addressing needs aren't enough to qualify for PI assignments from RIRs. But if the site route set is going to be announced or propagated by two or more different transit providers, common operational practice still says minimum /24 prefixes. That can be bigger than what small sites get in their allocation 😩
There are well-documented cases of sites filtering long-prefix routes that sit under a transit provider's aggregate. If that practice got really widespread, it could limit how well multihoming with PA addresses works 🚫 Limited filtering like this can be tolerated, though, because the primary transit provider's aggregate announcements should still be enough to attract traffic from autonomous systems that don't accept the covered site route set. The more traffic that follows the primary transit provider's aggregate when the covered, more-specific route is missing, the more the site leans on that primary transit provider. In some cases, that reliance could turn into an effective single point of failure 🥺
Traffic following the primary transit provider's aggregate routes may still reach the multihomed site, even if the connection between the primary transit provider and the site has failed. The site route set keeps propagating through the site's other transit providers. If that route set reaches the primary transit provider and gets accepted there, connectivity for traffic following the aggregate route is preserved 🫶
Sites using PA addresses are usually obliged to renumber if they decide not to keep connectivity to the primary transit provider. That's a common requirement for every site using PA addresses, not just the multihomed ones. It can hit harder on sites that multihome mainly to make ISP changes easier. A multihomed site using PA addresses can still add or drop other service providers without renumbering 🔄💅
3.1.2. AS Number Considerations 🔢
3.1.2.1. Consistent Origin AS 🌟
A multihomed site may choose to announce routes to two or more transit providers from a globally-unique Autonomous System (AS) number assigned to the site. This makes the origin of the route look consistent from every part of the Internet 🌍💜
(Part 1 of 3 done! Part 2 is coming up next, bestie 🫶✨)
3.1.2.2. Inconsistent Origin AS 🦋
Okay so a multihomed site may choose to use a private-use AS number [RFC1930] to originate routes to transit providers 🫶 It's normal practice for those private-use AS numbers to get stripped from AS_PATH attributes before they're allowed to propagate from transit providers towards peers. So routes seen from other parts of the Internet may look like they have inconsistent origins 👀 Not a bug, just vibes!
Also, when you're using private-use AS numbers, collisions between the use of individual numbers by different transit providers are possible 💀 Arguably the best move is to not use private-use AS numbers for anything that involves routing across administrative domain boundaries. Bestie, avoid the drama 💅
A multihomed site may request that their transit providers each originate the site's routes from the transit providers' ASes ✨ Dynamic routing is still possible here, if you want to withdraw the site's route when connectivity to the site is lost. The trick is that it uses the transit providers' internal routing systems to trigger the externally-visible announcements 🧚
And operational troubleshooting gets so much easier with a consistent origin AS 🎀 It lets import policies be based on a route's true origin instead of intermediate routing details, which can change (for example, when transit providers get added and dropped by the multihomed site) 🔄
3.2. Multiple Attachments to a Single Transit Provider 🌸
Multihoming can be achieved through multiple connections to a single transit provider 🫶 This imposes no additional load on the global routing table beyond what's involved in the site being single-attached. A site that has solved its multihoming needs this way is commonly called "multi-attached" 💖
It is not a requirement that the multi-attached site exchange routing information with its transit provider using BGP. But, in the event of failure, some mechanism for re-routing inbound and outbound traffic over remaining circuits is required 😭 BGP is often used for exactly this purpose.
Multi-attached sites gain no advantages from using PI addresses or (where BGP is used) globally-unique AS numbers, and they have no need to be able to justify address assignments of a particular minimum size 🍓 However, multi-attachment does not protect a site from the failure of the single transit provider 💀
3.3. NAT- or RFC2260-based Multihoming 🌈
This method uses PA addresses assigned by each transit provider the site is connected to 🫧 The addresses are either allocated to individual hosts within the network according to [RFC2260], or the site uses Network Address Translation (NAT) to translate the various provider addresses into a single set of private-use addresses [RFC1918] within the site. The site is effectively singlehomed to more than one transit provider, and none of the transit providers need to make any accommodations beyond those typically made for a non-multihomed customer 👑
This approach accommodates a wide range of sites, from residential Internet users to very large enterprises. It requires no PI addresses or AS numbers, and it imposes no additional load on the Internet's global routing system ✨ However, it does not address several common motivations for multihoming, most notably transport-layer survivability 🙈
4. Features of IPv4 Multihoming ✨
The following sections describe some of the features of the approaches from Section 3, in the context of the general goals for multihoming architectures presented in [RFC3582]. Detailed descriptions and rationale for these goals can be found in that document 📚
4.1. Redundancy 🛟
All the methods described provide redundancy, which can protect a site from some single-point failures. How much protection you get depends on the choice of transit providers and the methods used to interconnect the site to those transit providers 💖
4.2. Load Sharing 🎀
All of the methods described provide some measure of load-sharing capability 🔥 Outbound traffic can be shared across ISPs using appropriate exit selection policies. Inbound traffic can be distributed using appropriate export policies designed to influence the exit selection of remote sites sending traffic back towards the multihomed site 🫶
In the case of RFC2260/NAT multihoming, distribution of inbound traffic is controlled by address selection on the host or NAT 🧚
4.3. Performance 🏎️
BGP-speaking sites can employ import policies that cause exit selection to avoid paths known to be problematic. For inbound traffic, sites can often employ route export policy, which gives different treatment to traffic going towards particular address ranges within their network 💅
It should be noted that this is not a comprehensive capability. In general, there are many traffic engineering goals which can only be loosely approximated using this approach 🙈
In the case of RFC2260/NAT multihoming in the absence of BGP routing information, management of outbound traffic is not possible 😭 The path taken by inbound traffic for a particular session can be controlled by source address selection on the host or NAT.
4.4. Policy 📜
In some circumstances, it's possible to route traffic of a particular type (for example, by protocol) via particular transit providers 🦋 This can be done if the devices in the site which source or sink that traffic can be isolated to a set of addresses that a special export policy can be applied to 🎯
An example of this capability is grouping budget, best-effort Internet customers into a particular range of addresses. That range is covered by a route which is announced preferentially over a single, low-quality transit path 💀
In the case of RFC2260/NAT multihoming, policies such as those described here can be accommodated by appropriate address selection on the host or NAT. More flexible implementations may be possible for sessions originated from the multihomed site, by selecting an appropriate source address on a host or NAT, according to criteria such as transport-layer protocols and addresses (ports) 🌟
4.5. Simplicity 🍰
The current methods used as multihoming solutions are not without their complexities. But they've proven to be sufficiently simple to be used, and they have the advantage of familiarity due to having been deployed extensively 💜
4.6. Transport-Layer Survivability 🌊
All BGP-based multihoming practices provide some degree of session survivability for transport-layer protocols. However, in cases where path convergence takes a long time following a re-homing event, sessions may time out ⏳
Transport-layer sessions will not, in general, survive over a re-homing event when using RFC2260/NAT multihoming. Transport protocols which support multiple volatile endpoint addresses may be able to provide session stability. However, these transport protocols are not in wide use 🥲
In all the methods described in this document, new transport-layer sessions are able to be created following a re-homing event 🫶
4.7. Impact on DNS 🌐
These multihoming strategies impose no new requirements on the DNS 💖
4.8. Packet Filtering 🛡️
These multihoming practices do not preclude filtering of packets with inappropriate source or destination addresses at the administrative boundary of the multihomed site 🧹
4.9. Scalability 📈
Current IPv4 multihoming practices are thought to contribute to significant observed growth in the amount of state held in the global inter-provider routing system 😬 This is a concern because of both the hardware requirements it imposes and its impact on the stability of the routing system. This issue is discussed in greater detail in [RFC3221].
Of the methods presented in this document, RFC2260/NAT multihoming and multi-attaching to a single transit provider provide no additional state to be held in the global routing system ✨ All other strategies contribute to routing system state bloat 🫠
Globally-unique AS numbers are a finite resource. Thus, widespread multihoming that uses strategies requiring assignment of AS numbers might lead to increased resource contention 🍓
4.10. Impact on Routers 🖥️
For some of the multihoming approaches described in this document, the routers at the boundary of the multihomed site are required to participate in BGP sessions with transit provider routers 🫶 Other routers within the site generally have no special requirements beyond those in singlehomed sites.
4.11. Impact on Hosts 💻
There are no requirements of hosts beyond those in singlehomed sites. Easy 🫶
4.12. Interactions between Hosts and the Routing System 🤝
There are no requirements for interaction between routers and hosts beyond those in singlehomed sites 💅
4.13. Operations and Management 🛠️
There is extensive operational experience in managing IPv4-multihomed sites 👑
4.14. Cooperation between Transit Providers 🤝
Transit providers who are asked to announce or propagate a PA prefix covered by some other (primary) transit provider usually obtain authorisation first 🦋 However, there is no technical requirement or common contractual policy which requires this coordination to take place 🙃
5. Security Considerations 🚨
This document discusses current IPv4 multihoming practices, but provides no analysis of the security implications of multihoming 😱 Bestie, this is a huge gap, so please don't assume the security side is covered here. It just isn't 💀
6. Acknowledgements 🫶
Special acknowledgement goes to John Loughney for proof-reading and corrections 🌷 Thanks also go to Pekka Savola and Iljitsch van Beijnum for providing feedback and contributing text 💜
This work was supported by the US National Science Foundation (research grant SCI-0427144) and DNS-OARC 🎓
7. Informative References 📚
[RFC1519] Fuller, V., Li, T., Yu, J., and K. Varadhan, "Classless
Inter-Domain Routing (CIDR): an Address Assignment and
Aggregation Strategy", RFC 1519, September 1993.
[RFC1771] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4
(BGP-4)", RFC 1771, March 1995.
[RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and
E. Lear, "Address Allocation for Private Internets",
BCP 5, RFC 1918, February 1996.
[RFC1930] Hawkinson, J. and T. Bates, "Guidelines for creation,
selection, and registration of an Autonomous System (AS)",
BCP 6, RFC 1930, March 1996.
[RFC2260] Bates, T. and Y. Rekhter, "Scalable Support for Multi-
homed Multi-provider Connectivity", RFC 2260,
January 1998.
[RFC3221] Huston, G., "Commentary on Inter-Domain Routing in the
Internet", RFC 3221, December 2001.
[RFC3582] Abley, J., Black, B., and V. Gill, "Goals for IPv6 Site-
Multihoming Architectures", RFC 3582, August 2003.Authors' Addresses 🧚
Joe Abley
Internet Systems Consortium, Inc.
950 Charter Street
Redwood City, CA 94063
USA
Phone: +1 650 423 1317
EMail: jabley@isc.org
Kurt Erik Lindqvist
Netnod Internet Exchange
Bellmansgatan 30
Stockholm S-118 47
Sweden
Phone: +46 8 615 85 70
EMail: kurtis@kurtis.pp.se
Elwyn B. Davies
Independent Researcher
Soham, Cambridgeshire CB7 5AW
UK
Phone: +44 7889 488 335
EMail: elwynd@dial.pipex.com
Benjamin Black
Layer8 Networks
EMail: ben@layer8.net
Vijay Gill
AOL
12100 Sunrise Valley Dr
Reston, VA 20191
US
Phone: +1 410 336 4796
EMail: vgill@vijaygill.comFull Copyright Statement 💅
okay bestie, the legal bit, but make it cute 🎀 The copyright belongs to The Internet Society, and it's dated 2005. This document is covered by the rights, licenses, and restrictions in BCP 78. Outside of what BCP 78 says, the authors keep all of their rights. ✨
THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN ARE PROVIDED ON AN "AS IS" BASIS AND THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🙈
Intellectual Property 🦋
the IETF is not picking a side on whether any Intellectual Property Rights (or other rights) might cover the technology in this document. It also doesn't say whether a license for those rights exists or is available, and it hasn't done its own digging to find any of them. 🧚 If you want the procedures for rights in RFC documents, check out BCP 78 and BCP 79. 💜
If you want copies of IPR disclosures that were sent to the IETF Secretariat, or assurances that licenses will be available, or the outcome of anyone trying to get a general license or permission to use proprietary rights for this specification, you can check the IETF's online IPR repository at http://www.ietf.org/ipr. 🌸
The IETF also invites anyone to tell it about copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org. 📬 ngl this is giving "please speak up, we're listening" energy 🫶
Acknowledgement 🌈
the RFC Editor function is currently funded by the Internet Society. So the people keeping these documents organized get their paycheck from them right now, and that's iconic 👑
📦 where this page comes from
- ietf: rfc4116 2005-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms