RFC 7814

Virtual Subnet: A BGP/MPLS IP VPN-Based Subnet Extension Solution

πŸ“… published March 2016, from ietf

RFC 7814: Virtual Subnet πŸ¦‹

Okay bestie, this one is RFC 7814 πŸŽ€, titled Virtual Subnet: A BGP/MPLS IP VPN-Based Subnet Extension Solution. It was written by X. Xu (Huawei Technologies), C. Jacquenet (Orange), R. Raszuk and T. Boyes (both at Bloomberg LP), and B. Fee (Extreme Networks). It was published by the IETF in March 2016 as an Informational document, with ISSN 2070-1721 🌸. The header doesn't list anything it obsoletes or updates, so no drama there ✨.

Abstract

okay so the abstract is short and sweet πŸ’–. It describes a BGP/MPLS IP VPN-based subnet extension solution called "Virtual Subnet," and its whole job is building Layer 3 network virtualization overlays inside and/or between data centers 🌈.

Status of This Memo

This doc is not an Internet Standards Track specification. It's published just for informational purposes πŸ“š.

It's a product of the Internet Engineering Task Force (IETF) 🫢. It represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved document is a candidate for any level of Internet Standard, though, so check Section 2 of RFC 5741 if you're curious about that πŸ€“.

For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc7814 πŸ”—.

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved ©️.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were in effect on the date of publication. Please read these carefully, bestie, because they spell out your rights and restrictions for this document πŸ“œ. Code Components pulled out of this document must include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes πŸ’….

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
3.  Solution Description  . . . . . . . . . . . . . . . . . . . .   5
  3.1.  Unicast . . . . . . . . . . . . . . . . . . . . . . . . .   5
    3.1.1.  Intra-subnet Unicast  . . . . . . . . . . . . . . . .   5
    3.1.2.  Inter-subnet Unicast  . . . . . . . . . . . . . . . .   6
  3.2.  Multicast . . . . . . . . . . . . . . . . . . . . . . . .   8
  3.3.  Host Discovery  . . . . . . . . . . . . . . . . . . . . .   9
  3.4.  ARP/ND Proxy  . . . . . . . . . . . . . . . . . . . . . .   9
  3.5.  Host Mobility . . . . . . . . . . . . . . . . . . . . . .   9
  3.6.  Forwarding Table Scalability on Data-Center Switches  . .  10
  3.7.  ARP/ND Cache Table Scalability on Default Gateways  . . .  10
  3.8.  ARP/ND and Unknown Unicast Flood Avoidance  . . . . . . .  10
  3.9.  Path Optimization . . . . . . . . . . . . . . . . . . . .  10
4.  Limitations . . . . . . . . . . . . . . . . . . . . . . . . .  11
  4.1.  Non-support of Non-IP Traffic . . . . . . . . . . . . . .  11
  4.2.  Non-support of IP Broadcast and Link-Local Multicast  . .  11
  4.3.  TTL and Traceroute  . . . . . . . . . . . . . . . . . . .  11
5.  Security Considerations . . . . . . . . . . . . . . . . . . .  12
6.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  12
  6.1.  Normative References  . . . . . . . . . . . . . . . . . .  12
  6.2.  Informative References  . . . . . . . . . . . . . . . . .  13
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  14
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  15

1. Introduction ✨

Okay, so business continuity is a big deal πŸ’Ό. That's why Virtual Machine (VM) migration across data centers is pretty common for things like data-center maintenance, migration, consolidation, expansion, or disaster avoidance 🌸. The IETF community has noticed that renumbering the IP addresses of servers (aka VMs) after a migration is usually complex and costly 😭. So to move a VM from one data center to another without renumbering it, the subnet the VM lives on has to get stretched across those data centers πŸ¦‹.

To pull off subnet extension across multiple cloud data centers in a scalable way, the following requirements and challenges must be considered πŸ‘‘:

  • a. VPN Instance Space Scalability: In a modern cloud data-center environment, thousands or even tens of thousands of tenants could be hosted over one shared network infrastructure 🏠. For security and performance isolation, these tenants need to be kept separate from each other πŸ”’.
  • b. Forwarding Table Scalability: With server virtualization taking off, it's not uncommon for one cloud data center to hold millions of VMs 🀯. That number is already a big challenge for the forwarding table scalability of data-center switches. If multiple data centers that big got interconnected at Layer 2, things would get even worse πŸ’€.
  • c. ARP/ND Cache Table Scalability: [RFC6820] points out that the Address Resolution Protocol (ARP) / Neighbor Discovery (ND) cache tables kept by default gateways inside cloud data centers can cause scalability issues πŸ™ˆ. So keeping the size of the ARP/ND cache tables under control is critical as the number of connected data centers goes up πŸ“ˆ.
  • d. ARP/ND and Unknown Unicast Flooding: It's well-known that flooding ARP/ND broadcast/multicast messages, plus unknown unicast traffic, across a big Layer 2 network is likely to hurt network and host performance 🌊. When multiple data centers that each host millions of VMs are interconnected at Layer 2, all that flooding gets even worse 😩. That's why it's more and more important to avoid flooding ARP/ND broadcast/multicast and unknown unicast traffic across data centers 🚫.
  • e. Path Optimization: A subnet usually points to a location in the network πŸ“. But once a subnet is stretched across multiple geographically dispersed data-center locations, it stops carrying that location meaning 🌍. So traffic between a specific user and a server sitting in different data centers may first get forwarded through a third data center 😡. That suboptimal routing wastes bandwidth between data centers for no reason πŸ’Έ. Also, when traditional Virtual Private LAN Service (VPLS) technology [RFC4761] [RFC4762] is used for data-center interconnect, return traffic from a server may end up going to a default gateway in a different data center because of how a virtual router redundancy group is configured πŸ™ƒ. That suboptimal routing also eats up bandwidth between data centers πŸ₯².

This document describes a BGP/MPLS IP VPN-based subnet extension solution called "Virtual Subnet," which can be used for data-center interconnection while addressing all of the requirements and challenges above πŸŽ€. Here, BGP/MPLS IP VPN means both BGP/MPLS IPv4 VPN [RFC4364] and BGP/MPLS IPv6 VPN [RFC4659] 🧚. Virtual Subnet is built mainly on proven tech like BGP/MPLS IP VPN and ARP/ND proxy [RFC925] [RFC1027] [RFC4389], so service providers offering Infrastructure as a Service (IaaS) cloud services can rely on the BGP/MPLS IP VPN infrastructure they already have and use their existing BGP/MPLS VPN operational experience to interconnect data centers πŸ’….

Although this document describes Virtual Subnet as an approach for data-center interconnection, it can be used within data centers as well 🌈.

Note that the approach here is not meant to be an exact emulation of Layer 2 connectivity πŸ™…β€β™€οΈ. It can only support a restricted Layer 2 connectivity service model, with the limitations discussed in Section 4 ⚠️. Where this service model can apply is outside the scope of this document πŸ€·β€β™€οΈ.

2. Terminology πŸ’…

This memo uses the terms defined in [RFC4364] πŸ“–.

3. Solution Description πŸ”₯

3.1. Unicast πŸ’–

3.1.1. Intra-subnet Unicast πŸ¦‹

Here's the picture, bestie πŸ‘€:

                       +--------------------+
+------------------+   |                    |   +------------------+
|VPN_A:192.0.2.1/24|   |                    |   |VPN_A:192.0.2.1/24|
|              \   |   |                    |   |  /               |
|    +------+   \ ++---+-+                +-+---++/   +------+     |
|    |Host A+-----+ PE-1 |                | PE-2 +----+Host B|     |
|    +------+\    ++-+-+-+                +-+-+-++   /+------+     |
|     192.0.2.2/24 | | |                    | | |  192.0.2.3/24    |
|                  | | |                    | | |                  |
|     DC West      | | |  IP/MPLS Backbone  | | |     DC East      |
+------------------+ | |                    | | +------------------+
                     | +--------------------+ |
                     |                        |
VRF_A :              V                VRF_A : V
+------------+---------+--------+      +------------+---------+--------+
|   Prefix   |Next hop |Protocol|      |   Prefix   |Next hop |Protocol|
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.1/32|127.0.0.1| Direct |      |192.0.2.1/32|127.0.0.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.2/32|192.0.2.2| Direct |      |192.0.2.2/32|   PE-1  |  IBGP  |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.3/32|   PE-2  |  IBGP  |      |192.0.2.3/32|192.0.2.3| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.0/24|192.0.2.1| Direct |      |192.0.2.0/24|192.0.2.1| Direct |
+------------+---------+--------+      +------------+---------+--------+

                  Figure 1: Intra-subnet Unicast Example

So in Figure 1, Host A and Host B are on the same subnet (192.0.2.0/24) but live in different data centers, DC West and DC East πŸ™οΈ. The PE routers that connect these two data centers (PE-1 and PE-2) each make host routes for their own local hosts, then advertise those routes using BGP/MPLS IP VPN signaling πŸ“‘. Meanwhile, an ARP proxy is turned on for the Virtual Routing and Forwarding (VRF) attachment circuits on these PE routers πŸ§šβ€β™€οΈ.

Let's say Host A sends an ARP request for Host B before it talks to Host B πŸ—£οΈ. When PE-1 gets that ARP request, it acts as an ARP proxy and replies with its own MAC address πŸͺž. Host A then sends IP packets meant for Host B over to PE-1 πŸ“¦. PE-1 tunnels those packets to PE-2, which forwards them on to Host B πŸš€. And just like that, Hosts A and B can talk to each other as if they were in the same subnet, no cap πŸ’―.

Next up is Section 3.1.2, and it's about to get good πŸ‘€βœ¨.

3.1.2. Inter-subnet Unicast πŸ¦‹

okay so now we're going across subnets, and it's giving multi-DC drama πŸ’… Here's Figure 2:

                           +--------------------+
    +------------------+   |                    |   +------------------+
    |VPN_A:192.0.2.1/24|   |                    |   |VPN_A:192.0.2.1/24|
    |              \   |   |                    |   |  /               |
    |  +------+     \ ++---+-+                +-+---++/     +------+   |
    |  |Host A+-------+ PE-1 |                | PE-2 +-+----+Host B|   |
    |  +------+\      ++-+-+-+                +-+-+-++ |   /+------+   |
    |   192.0.2.2/24   | | |                    | | |  | 192.0.2.3/24  |
    |   GW=192.0.2.4   | | |                    | | |  | GW=192.0.2.4  |
    |                  | | |                    | | |  |    +------+   |
    |                  | | |                    | | |  +----+  GW  +-- |
    |                  | | |                    | | |      /+------+   |
    |                  | | |                    | | |    192.0.2.4/24  |
    |                  | | |                    | | |                  |
    |     DC West      | | |  IP/MPLS Backbone  | | |      DC East     |
    +------------------+ | |                    | | +------------------+
                        | +--------------------+ |
                        |                        |
VRF_A :                 V                VRF_A : V
+------------+---------+--------+      +------------+---------+--------+
|   Prefix   |Next hop |Protocol|      |   Prefix   |Next hop |Protocol|
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.1/32|127.0.0.1| Direct |      |192.0.2.1/32|127.0.0.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.2/32|192.0.2.2| Direct |      |192.0.2.2/32|  PE-1   |  IBGP  |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.3/32|   PE-2  |  IBGP  |      |192.0.2.3/32|192.0.2.3| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.4/32|   PE-2  |  IBGP  |      |192.0.2.4/32|192.0.2.4| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.0/24|192.0.2.1| Direct |      |192.0.2.0/24|192.0.2.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
| 0.0.0.0/0  |   PE-2  |  IBGP  |      | 0.0.0.0/0  |192.0.2.4| Static |
+------------+---------+--------+      +------------+---------+--------+

                Figure 2: Inter-subnet Unicast Example (1)

so in Figure 2, only ONE data center (DC East) actually has a default gateway (GW) 🏰 PE-2, which is plugged into GW, either gets configured with a default route pointing at GW or learns one from GW, and the next hop is GW. Then that route gets passed along to the other PE routers (like PE-1) the normal way, as described in [RFC4364] ✨

Now say Host A sends an ARP request for its default gateway (192.0.2.4) before it talks to anything outside its subnet. PE-1 is acting as an ARP proxy, so it answers with its own MAC address πŸ™ˆ Host A then sends its packet for Host B over to PE-1. PE-1 tunnels that packet to PE-2 using the default route it learned from PE-2, and PE-2 forwards it on to GW. Main character routing, no cap 🧚

                           +--------------------+
    +------------------+   |                    |   +------------------+
    |VPN_A:192.0.2.1/24|   |                    |   |VPN_A:192.0.2.1/24|
    |              \   |   |                    |   |  /               |
    |  +------+     \ ++---+-+                +-+---++/     +------+   |
    |  |Host A+----+--+ PE-1 |                | PE-2 +-+----+Host B|   |
    |  +------+\   |  ++-+-+-+                +-+-+-++ |   /+------+   |
    |  192.0.2.2/24 |  | | |                    | | |  | 192.0.2.3/24  |
    |  GW=192.0.2.4 |  | | |                    | | |  | GW=192.0.2.4  |
    |  +------+    |   | | |                    | | |  |    +------+   |
    |--+ GW-1 +----+   | | |                    | | |  +----+ GW-2 +-- |
    |  +------+\       | | |                    | | |      /+------+   |
    |  192.0.2.4/24    | | |                    | | |    192.0.2.4/24  |
    |                  | | |                    | | |                  |
    |     DC West      | | |  IP/MPLS Backbone  | | |      DC East     |
    +------------------+ | |                    | | +------------------+
                        | +--------------------+ |
                        |                        |
VRF_A :                 V                VRF_A : V
+------------+---------+--------+      +------------+---------+--------+
|   Prefix   |Next hop |Protocol|      |   Prefix   |Next hop |Protocol|
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.1/32|127.0.0.1| Direct |      |192.0.2.1/32|127.0.0.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.2/32|192.0.2.2| Direct |      |192.0.2.2/32|  PE-1   |  IBGP  |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.3/32|   PE-2  |  IBGP  |      |192.0.2.3/32|192.0.2.3| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.4/32|192.0.2.4| Direct |      |192.0.2.4/32|192.0.2.4| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.0/24|192.0.2.1| Direct |      |192.0.2.0/24|192.0.2.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
| 0.0.0.0/0  |192.0.2.4| Static |      | 0.0.0.0/0  |192.0.2.4| Static |
+------------+---------+--------+      +------------+---------+--------+

                Figure 3: Inter-subnet Unicast Example (2)

Figure 3 is the glow-up 🌸 Here each data center has its own default gateway, so hosts get their ARP responses straight from their local gateways, not from their local PE routers when they ARP for their gateways. Iconic for locality πŸ’œ

                                  +------+
                           +------+ PE-3 +------+
    +------------------+   |      +------+      |   +------------------+
    |VPN_A:192.0.2.1/24|   |                    |   |VPN_A:192.0.2.1/24|
    |              \   |   |                    |   |  /               |
    |  +------+     \ ++---+-+                +-+---++/     +------+   |
    |  |Host A+-------+ PE-1 |                | PE-2 +------+Host B|   |
    |  +------+\      ++-+-+-+                +-+-+-++     /+------+   |
    |  192.0.2.2/24    | | |                    | | |    192.0.2.3/24  |
    |  GW=192.0.2.1    | | |                    | | |    GW=192.0.2.1  |
    |                  | | |                    | | |                  |
    |     DC West      | | |  IP/MPLS Backbone  | | |      DC East     |
    +------------------+ | |                    | | +------------------+
                         | +--------------------+ |
                         |                        |
VRF_A :                  V                VRF_A : V
+------------+---------+--------+      +------------+---------+--------+
|   Prefix   |Next hop |Protocol|      |   Prefix   |Next hop |Protocol|
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.1/32|127.0.0.1| Direct |      |192.0.2.1/32|127.0.0.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.2/32|192.0.2.2| Direct |      |192.0.2.2/32|  PE-1   |  IBGP  |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.3/32|   PE-2  |  IBGP  |      |192.0.2.3/32|192.0.2.3| Direct |
+------------+---------+--------+      +------------+---------+--------+
|192.0.2.0/24|192.0.2.1| Direct |      |192.0.2.0/24|192.0.2.1| Direct |
+------------+---------+--------+      +------------+---------+--------+
| 0.0.0.0/0  |   PE-3  |  IBGP  |      | 0.0.0.0/0  |   PE-3  |  IBGP  |
+------------+---------+--------+      +------------+---------+--------+

                Figure 4: Inter-subnet Unicast Example (3)

And in Figure 4, the PE routers themselves can be set up as default gateways for their locally connected hosts, as long as those PE routers have routes to reach outside networks 🫢


3.2. Multicast πŸ’–

bestie, multicast is covered too 😭 To get IP multicast working between hosts in the same Virtual Subnet, you can use Multicast VPN (MVPN) technologies [RFC6513] with no changes at all ✨ For example, PE routers attached to a given VPN join a default provider multicast distribution tree that's dedicated to that VPN. Ingress PE routers, when they get multicast packets from their local hosts, forward them to remote PE routers through that default provider multicast distribution tree. In this context, IP multicast doesn't include link-local multicast πŸŽ€

3.3. Host Discovery ✨

PE routers should be able to discover their local hosts dynamically πŸ” and keep that host list up-to-date in a timely manner, so the host routes they originate stay available and accurate πŸ’… PE routers could do local host discovery with some traditional host-discovery mechanisms that use ARP or ND protocols.

3.4. ARP/ND Proxy 🫢

okay the ARP/ND proxy rules are kind of a whole thing πŸ™ˆ Acting as an ARP or ND proxy, a PE router should only respond to an ARP request or Neighbor Solicitation (NS) message for a target host when it has a best route for that target host in the associated VRF, and the outgoing interface of that best route is different from the one the ARP request or NS message came in on. Now, if a VPN site (like a data center) is multihomed to more than one PE router through an Ethernet switch or Ethernet network, the Virtual Router Redundancy Protocol (VRRP) [RFC5798] is usually enabled on those PE routers. In that case, only the PE router that gets elected as the VRRP Master is allowed to do the ARP/ND proxy function πŸ‘‘

3.5. Host Mobility 🧚

during VM migration, the PE router the moving VM is now attached to creates a host route for that host once it gets a notification of the VM attachment (like a gratuitous ARP or an unsolicited NA message) πŸ¦‹ The PE router the VM was previously attached to withdraws the matching host route when it notices the VM has detached. Meanwhile, that latter PE router could optionally broadcast a gratuitous ARP or send an unsolicited NA message on behalf of the host, using one of its own MAC addresses as the source. This way, the ARP/ND entry for the host that moved, which is cached on any local host, gets updated too 🌈

If there's no explicit VM detachment notification mechanism, the PE router could use this trick to detect the detachment: when it learns a route update for a local host from a remote PE router for the first time, it could immediately check whether that local host is still attached to it by some means (like ARP/ND PING and/or ICMP PING) πŸ•΅οΈβ€β™€οΈ

It's super important that the same MAC and IP are tied to the default gateway that's active in each data center 😭 The VM will most likely keep sending packets to the same default gateway address after it migrates from one data center to another. One way to get there is to configure the same VRRP group in each location, so the default gateway that's active in each data center shares the same virtual MAC and virtual IP addresses πŸ’œ

3.6. Forwarding Table Scalability on Data-Center Switches ✨

okay so in a Virtual Subnet setup, the MAC learning domain for a Virtual Subnet that's stretched across multiple data centers gets chopped into segments, and each segment stays inside one single data center 🧩 which means data-center switches only have to learn local MAC addresses, not local and remote ones. no cap, that's so clean πŸ’…

3.7. ARP/ND Cache Table Scalability on Default Gateways πŸŽ€

when default gateway functions live on PE routers (see Figure 4 🌸), the ARP/ND cache table on each PE router only needs entries for its local hosts. so the table size would not grow as the number of data centers to be connected increases. it's giving scalable, periodt 🫢

3.8. ARP/ND and Unknown Unicast Flood Avoidance πŸ¦‹

in a Virtual Subnet environment, the flooding domain for a Virtual Subnet that got extended across multiple data centers is also split into segments, and each segment is confined within a single data center. the result? the performance hit on networks and servers from flooding ARP/ND broadcast/multicast and unknown unicast traffic is minimized πŸ™ˆ bestie, the servers can finally breathe 😭

3.9. Path Optimization 🧚

okay this one is my favorite πŸ“ to optimize the forwarding path between cloud users and cloud data centers, the PE routers sitting in the cloud data centers (i.e., PE-1 and PE-2), which are also acting as default gateways, advertise host routes for their own local hosts to remote PE routers attached to cloud user sites (i.e., PE-3). so traffic from cloud user sites to a given server on the Virtual Subnet that has been extended across data centers goes straight to the data-center location where that server lives 🎯 because traffic is now forwarded by the host route for that server, not the subnet route. the main character energy is real πŸ‘‘

going the other way, traffic from cloud data centers to cloud user sites is handled by each PE router acting as a default gateway, which forwards according to the longest-match route in the corresponding VRF. as a result, traffic from data centers to cloud user sites takes an optimal path too ✨

4. Limitations πŸ₯€

4.1. Non-support of Non-IP Traffic πŸ™ƒ

ngl, most traffic within and across data centers is IP traffic πŸ’» but there may still be a few legacy clustering applications that rely on non-IP communications (e.g., heartbeat messages between cluster nodes). since Virtual Subnet is strictly based on L3 forwarding, those non-IP communications cannot be supported in the Virtual Subnet solution 😬

so to support those few non-IP flows (if they exist in your environment), the approach following the idea of "route all IP traffic, bridge non-IP traffic" could be considered 🌈 basically, all IP traffic, both intra- and inter-subnet, gets processed according to the Virtual Subnet design, while non-IP traffic gets forwarded according to a particular Layer 2 VPN approach. this unified L2/L3 VPN approach requires ingress PE routers to classify packets received from hosts before distributing them to the corresponding L2 or L3 VPN forwarding processes. and fun fact: more and more cluster vendors are offering clustering applications based on Layer 3 interconnection 🧠

as we already covered, intra-subnet traffic across PE routers is forwarded at Layer 3 in the Virtual Subnet solution. which means IP broadcast and link-local multicast traffic cannot be forwarded across PE routers in this solution 🚫 to support them, the unified L2/L3 overlay approach from Section 4.1 could be considered as well. the idea is that IP broadcast and link-local multicast messages get forwarded at Layer 2, while routable IP traffic gets processed according to the Virtual Subnet design πŸ’–

4.3. TTL and Traceroute πŸ•΅οΈβ€β™€οΈ

as mentioned before, intra-subnet traffic is forwarded at Layer 3 in the Virtual Subnet context. since this doesn't require any change to the Time-To-Live (TTL) handling mechanism of the BGP/MPLS IP VPN, a traceroute from one host to another (assuming both hosts are in the same subnet but attached to different sites) will show the truth πŸ’ the traceroute output reflects the fact that these two hosts are actually connected via a Virtual Subnet, rather than a Layer 2 connection, because the PE routers that those two hosts connect to get displayed in the traceroute output.

heads up, bestie: for any other applications that generate intra-subnet traffic with TTL set to 1, those apps may not work properly in the Virtual Subnet context, unless special TTL processing and loop-prevention mechanisms for such context have been implemented 🚨

details about such special TTL processing and loop-prevention mechanisms are outside the scope of this document πŸ“š

5. Security Considerations πŸ”πŸ’€

okay, the security part is SERIOUS, so pay attention πŸ‘€ since the BGP/MPLS IP VPN signaling is reused without any change, the security considerations described in [RFC4364] apply to this document. meanwhile, since security issues associated with NDP are inherited through the use of NDP proxy, the security considerations and recommendations described in [RFC6583] apply here too πŸ”’

inter-data-center traffic often carries highly sensitive information at higher layers that isn't directly understood (parsed) within an egress or ingress PE. like, migrating a VM will often mean moving private keys and other sensitive configuration information 😱 for this reason, inter-data-center traffic should always be protected for both confidentiality and integrity using a strong security mechanism such as IPsec [RFC4301] πŸ›‘οΈ in the future, it may be feasible to protect that traffic within the MPLS layer [MPLS-SEC], though at the time of writing, the mechanism for that is not sufficiently mature to recommend πŸ™

exactly how such security mechanisms get deployed will vary from case to case. securing the inter-data-center traffic may or may not involve deploying security mechanisms on the ingress/egress PEs, or further "inside" the data centers concerned 🏒 but note, though: if security is not deployed on the egress/ingress PEs, there is a substantial risk that some sensitive traffic may be sent in the clear and will therefore be vulnerable to pervasive monitoring [RFC7258] or other attacks. that's a literal nightmare, so please don't skip this 😭πŸ”₯

6. References πŸ“–

6.1. Normative References πŸ’œ

[RFC925]   Postel, J., "Multi-LAN address resolution", RFC 925,
           DOI 10.17487/RFC0925, October 1984,
           <http://www.rfc-editor.org/info/rfc925>.

[RFC1027]  Carl-Mitchell, S. and J. Quarterman, "Using ARP to
           implement transparent subnet gateways", RFC 1027,
           DOI 10.17487/RFC1027, October 1987,
           <http://www.rfc-editor.org/info/rfc1027>.

[RFC4364]  Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
           Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
           2006, <http://www.rfc-editor.org/info/rfc4364>.

[RFC4389]  Thaler, D., Talwar, M., and C. Patel, "Neighbor Discovery
           Proxies (ND Proxy)", RFC 4389, DOI 10.17487/RFC4389, April
           2006, <http://www.rfc-editor.org/info/rfc4389>.

6.2. Informative References 🌸

[MPLS-SEC] Farrel, A. and S. Farrell, "Opportunistic Security in MPLS
           Networks", Work in Progress, draft-ietf-mpls-
           opportunistic-encrypt-01, March 2016.

[RFC4301]  Kent, S. and K. Seo, "Security Architecture for the
           Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
           December 2005, <http://www.rfc-editor.org/info/rfc4301>.

[RFC4659]  De Clercq, J., Ooms, D., Carugi, M., and F. Le Faucheur,
           "BGP-MPLS IP Virtual Private Network (VPN) Extension for
           IPv6 VPN", RFC 4659, DOI 10.17487/RFC4659, September 2006,
           <http://www.rfc-editor.org/info/rfc4659>.

[RFC4761]  Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private
           LAN Service (VPLS) Using BGP for Auto-Discovery and
           Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007,
           <http://www.rfc-editor.org/info/rfc4761>.

[RFC4762]  Lasserre, M., Ed. and V. Kompella, Ed., "Virtual Private
           LAN Service (VPLS) Using Label Distribution Protocol (LDP)
           Signaling", RFC 4762, DOI 10.17487/RFC4762, January 2007,
           <http://www.rfc-editor.org/info/rfc4762>.

[RFC5798]  Nadas, S., Ed., "Virtual Router Redundancy Protocol (VRRP)
           Version 3 for IPv4 and IPv6", RFC 5798,
           DOI 10.17487/RFC5798, March 2010,
           <http://www.rfc-editor.org/info/rfc5798>.

[RFC6513]  Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/
           BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February
           2012, <http://www.rfc-editor.org/info/rfc6513>.

[RFC6583]  Gashinsky, I., Jaeggli, J., and W. Kumari, "Operational
           Neighbor Discovery Problems", RFC 6583,
           DOI 10.17487/RFC6583, March 2012,
           <http://www.rfc-editor.org/info/rfc6583>.

[RFC6820]  Narten, T., Karir, M., and I. Foo, "Address Resolution
           Problems in Large Data Center Networks", RFC 6820,
           DOI 10.17487/RFC6820, January 2013,
           <http://www.rfc-editor.org/info/rfc6820>.

[RFC7258]  Farrell, S. and H. Tschofenig, "Pervasive Monitoring Is an
           Attack", BCP 188, RFC 7258, DOI 10.17487/RFC7258, May
           2014, <http://www.rfc-editor.org/info/rfc7258>.

Acknowledgements 🫢

big thanks to Susan Hares, Yongbing Fan, Dino Farinacci, Himanshu Shah, Nabil Bitar, Giles Heron, Ronald Bonica, Monique Morrow, Rajiv Asati, Eric Osborne, Thomas Morin, Martin Vigoureux, Pedro Roque Marque, Joe Touch, Wim Henderickx, Alia Atlas, and Stephen Farrell for their valuable comments and suggestions on this document ✨ thanks to Loa Andersson for his WG LC review on this document πŸ’… thanks to Alvaro Retana for his AD review on this document πŸ‘‘ thanks to Ronald Bonica for his RtgDir review 🌷 thanks to Donald Eastlake 3rd for his Sec-DIR review of this document πŸ” thanks to Jouni Korhonen for the OPS-Dir review of this document 🧚 thanks to Roni Even for the Gen-ART review of this document πŸŽ€ and thanks to Sabrina Tanamal for the IANA review of this document πŸ¦‹

Authors' Addresses πŸ’Œ

Xiaohu Xu
Huawei Technologies
No.156 Beiqing Rd
Beijing  100095
China

Email: xuxiaohu@huawei.com


Christian Jacquenet
Orange
4 rue du Clos Courtel
Cesson-Sevigne,  35512
France

Email: christian.jacquenet@orange.com


Robert Raszuk
Bloomberg LP
731 Lexington Avenue
New York City, NY  10022
United States

Email: robert@raszuk.net


Truman Boyes
Bloomberg LP

Email: tboyes@bloomberg.net


Brendan Fee
Extreme Networks

Email: bfee@extremenetworks.com
πŸ“¦ where this page comes from