RFC 8368

Using an Autonomic Control Plane for Stable Connectivity of Network Operations, Administration, and Maintenance (OAM)

📅 published May 2018, from ietf

Using an Autonomic Control Plane for Stable Connectivity of Network Operations, Administration, and Maintenance (OAM) 🎀

Okay bestie, let's get into this one! 💖 This is RFC 8368, an Informational doc (so it's not a Standards Track spec, it's just published for info 🌸) by T. Eckert, Ed. from Huawei and M. Behringer. It came out in May 2018, and its ISSN is 2070-1721. It doesn't obsolete or update any other RFCs. ✨

Abstract

Operations, Administration, and Maintenance (OAM), as per BCP 161, for data networks keeps running into circular dependencies when it relies on the very network it's supposed to manage. 🙈 Ugh, the worst!

Provisioning while bringing up devices and networks is way harder to automate than service provisioning later on. 😭 Changes to core network functions that affect reachability can't be automated, because the OAM equipment itself still needs connectivity the whole time. And the widely used OAM protocols aren't secure enough to be carried across the network without security concerns. 💀

So this doc explains how to plug OAM processes into an autonomic control plane, so they get stable and secure connectivity. 🦋 That connectivity isn't subject to those circular dependencies, so it's giving clean and reliable vibes. 💅

Status of This Memo

This document is not an Internet Standards Track specification; it is published for informational purposes. 🌈

This document is a product of the Internet Engineering Task Force (IETF). It has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for any level of Internet Standard; see Section 2 of RFC 7841. ✨

Info about the current status, any errata, and how to give feedback lives at https://www.rfc-editor.org/info/rfc8368. 🧚

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. 📜

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
  1.1.  Self-Dependent OAM Connectivity . . . . . . . . . . . . .   3
  1.2.  Data Communication Networks (DCNs)  . . . . . . . . . . .   3
  1.3.  Leveraging a Generalized Autonomic Control Plane  . . . .   4
2.  GACP Requirements . . . . . . . . . . . . . . . . . . . . . .   5
3.  Solutions . . . . . . . . . . . . . . . . . . . . . . . . . .   6
  3.1.  Stable Connectivity for Centralized OAM . . . . . . . . .   6
    3.1.1.  Simple Connectivity for Non-GACP-Capable
            NMS Hosts . . . . . . . . . . . . . . . . . . . . . .   7
    3.1.2.  Challenges and Limitations of Simple Connectivity . .   8
    3.1.3.  Simultaneous GACP and Data-Plane Connectivity . . . .   9
    3.1.4.  IPv4-Only NMS Hosts . . . . . . . . . . . . . . . . .  10
    3.1.5.  Path Selection Policies . . . . . . . . . . . . . . .  12
    3.1.6.  Autonomic NOC Device/Applications . . . . . . . . . .  16
    3.1.7.  Encryption of Data-Plane Connections  . . . . . . . .  16
    3.1.8.  Long-Term Direction of the Solution . . . . . . . . .  17
  3.2.  Stable Connectivity for Distributed       Network/OAM . .  18
4.  Architectural Considerations  . . . . . . . . . . . . . . . .  18
  4.1.  No IPv4 for GACP  . . . . . . . . . . . . . . . . . . . .  18
5.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  20
7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  21
  7.1.  Normative References  . . . . . . . . . . . . . . . . . .  21
  7.2.  Informative References  . . . . . . . . . . . . . . . . .  22
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  23
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  24

1. Introduction ✨

1.1. Self-Dependent OAM Connectivity

Operations, Administration, and Maintenance (OAM), as per BCP 161 [RFC6291], for data networks is often subject to the problem of circular dependencies when relying on the connectivity service provided by the network to be managed. OAM can easily but unintentionally break the connectivity it needs for its own operations. 😩 Avoiding these problems can lead to complexity in OAM. This document describes the problem and how to use an autonomic control plane to solve it without adding more OAM complexity. 🫶

The ability to perform OAM on a network device first requires running the OAM needed to create network connectivity to that device across all the intervening devices. 🌸 This typically leads to a sequential "expanding ring configuration" from a Network Operations Center (NOC). It also creates tight dependencies between provisioning tools and security enrollment of devices. So any process that wants to enroll multiple devices along a newly deployed network topology has to tightly interlock with the provisioning process that creates connectivity, before the enrollment can move on to the next device. Chaotic, honestly. 🙃

Likewise, when you're making changes to a network, you need to be sure at every step that there's no interruption of connectivity that could cut off remote devices. 😬 This especially includes provisioning changes to routing, forwarding, security, and addressing policies, which often happen during mergers and acquisitions, the introduction of IPv6, or other major overhauls of the infrastructure design. Examples include changing an IGP or area, changing from Provider Aggregatable (PA) to Provider Independent (PI) addressing, or systematic topology changes (such as Layer 2 to Layer 3 changes). 💅

All of these circular dependencies make OAM complex and potentially fragile. 💀 When automation is in use (for example, through provisioning systems), that complexity spills into the automation software too. 🎀

1.2. Data Communication Networks (DCNs)

In the late 1990s and early 2000s, IP networks became the go-to method for building separate OAM networks for the communications infrastructure within Network Providers. 🦋 This concept was standardized in ITU-T G.7712/Y.1703 [ITUT_G7712] and called "Data Communications Networks" (DCNs). These were (and still are) physically separate IP or IP/MPLS networks that provide access to the OAM interfaces of all the equipment that needed managing, from Public Switched Telephone Network (PSTN) switches, over optical equipment, to today's Ethernet and IP/MPLS production network equipment. 🌈

DCNs provide stable connectivity that isn't subject to the problems above, because they're a completely separate network. So changing the configuration of the production IP network is done via the DCN, but never affects the DCN configuration. 🔥 Of course, this approach costs money to buy and operate a separate network, and that cost isn't feasible for many providers. Most notably, smaller providers, most enterprises, and typical Internet of Things (IoT) networks. 🥺

1.3. Leveraging a Generalized Autonomic Control Plane

One of the goals of the IETF ANIMA (Autonomic Networking Integrated Model and Approach) Working Group is specifying a secure, automatically built in-band management plane that provides stable connectivity similar to a DCN, but without having to build a separate DCN. 👑 It's clear that such an "in-band" approach can never fully achieve the same level of separation, but the goal is to get as close to it as possible. 🧚

This document discusses how such an in-band management plane can support the DCN-like OAM use case, how to leverage its stable connectivity, and what the options are for deploying it incrementally in the short and long term. 🍓

The ANIMA Working Group's evolving specification [ACP] calls this in-band management plane the "Autonomic Control Plane" (ACP). The discussions in this document don't depend on the specification of that ACP. They only depend on a set of high-level constraints listed below, which were decided on early in the work on the ACP. 🌸 Except when being specific about ACP details, this document uses the term "Generalized ACP" (GACP), and it applies to any design that meets those high-level constraints, for example the variations of the ACP protocol choices. ✨

2. GACP Requirements 🎀

The high-level constraints of a GACP assumed and discussed in this document are as follows: 💖

VRF isolation: The GACP is a virtual network (Virtual Routing and Forwarding (VRF)) across network devices. Its routing and forwarding are separate from other routing and forwarding in the network devices. Non-GACP routing/forwarding is called the "data plane". 🦋

IPv6-only addressing: The GACP provides only IPv6 reachability. It uses Unique Local Addresses (ULAs) [RFC4193] that are routed in a location-independent fashion, for example through a subnet prefix for each network device. So automatic addressing in the GACP is simple and stable. It doesn't require allocation by address registries, addresses are identifiers, they don't change when devices move, and no engineering of the address space to the network topology is necessary. 🧚

NOC connectivity: NOC equipment (controlling OAM operations) either has access to the GACP directly or has an IP subnet connection to a GACP edge device. 🏠

Closed Group Security: GACP devices have cryptographic credentials to mutually authenticate each other as members of a GACP. Traffic across the GACP is authenticated with these credentials and then encrypted. 🔐

GACP connect (interface): The only traffic permitted in and out of the GACP that is not authenticated by GACP cryptographic credentials is traffic explicitly configured to and from the aforementioned non-GACP NOC equipment with subnet connections to a GACP edge device (as a transition method). 🌉

The GACP must be built to be autonomic, and its function must not be able to be disrupted by operator or automated configuration/provisioning actions (i.e., Network Management System (NMS) or Software-Defined Networking (SDN)). Those actions are allowed to impact only the data plane. This document doesn't cover those aspects. Instead, it focuses on the impact of the above constraints: IPv6 only, dual connectivity, and security. 👑

Heads up, bestie: the GACP's whole job is to stay untouchable by the config changes that break everything else. That's the vibe. 💅

Part 1 of 6 is done! 🎀 Next up is Section 3, "Solutions," so stay tuned for the next part. 🦋💖

3.1. Stable Connectivity for Centralized OAM ✨

okay so the ANI (Autonomic Networking Infrastructure) is the whole squad: secure zero-touch bootstrap ([BRSKI]), the GeneRic Autonomic Signaling Protocol ([GRASP]), and the Autonomic Control Plane ([ACP]) 💖. For an overview of how these pieces vibe together, check the reference model [REF_MODEL], and for the concepts and terminology of ANI and autonomic networks, look at [RFC7575] 📚.

This section is all about stable connectivity for centralized OAM through the GACP 🦋. That could be via the ACP with or without a complete ANI. We start with the option we expect to be the easiest to deploy in the short term 🌸. Then we get real about the limitations and challenges of that approach, plus the solutions and workarounds for them. It all wraps up with the preferred target option, autonomic NOC devices, in Section 3.1.6 👑.

This order is intentional 💅. It shows how simple a GACP can be to start with, and how messy workarounds can get (so you know what to avoid 🙈). Also, one very promising long-term solution is basically the easiest short-term one, just virtualized and automated. It's giving full-circle energy 🔥.

Most of the time, OAM is done by one or more applications running on a bunch of centralized NOC systems that talk to network devices 🖥️. This document covers ways to use a GACP for stable connectivity, going from simple to complex, depending on what the equipment can and can't do 🧚.

There are three stages to think about:

  • There are simple options in Sections 3.1.1 through 3.1.3 that we consider good starting points for using a GACP for stable connectivity today 🎀. These options only need network and OAM/NOC device configuration. Easy peasy 🍓.
  • There are workarounds for connecting a GACP to NOC devices that aren't IPv6-capable, using IPv4/IPv6 NAT (Network Address Translation), as described in Section 3.1.4 😬. These workarounds are not recommended. Still, if non-IPv6-capable NOC devices have to stay in use for the long haul, then these workarounds are the only way to hook them up to a GACP 💀.
  • Options for the near to long term can deliver all the desired operational, zero-touch, and security benefits of an autonomic network ✨. But a bunch of details still need to be worked out, and development work on NOC/OAM equipment is necessary 🛠️. These options are discussed in Sections 3.1.5 through 3.1.8 🌈.

3.1.1. Simple Connectivity for Non-GACP-Capable NMS Hosts

okay the most simple candidate deployment has the GACP extending all the way into the NOC through one or more GACP edge devices 🏠. See also Section 6.1 of [ACP]. These edge devices "leak" the GACP natively to NMS hosts. The GACP is otherwise encrypted, but they serve it up raw 🙊. They act as the default routers for those NMS hosts and give them IPv6 connectivity into the GACP 🦋. NMS hosts with this setup need to support IPv6 (e.g., see [RFC6434]), but they don't need any other changes to use the GACP 💜.

Note that even though the GACP only uses IPv6, it can totally support OAM for any type of network deployment, as long as the network devices support the GACP ✨. The data plane can be IPv4 only, dual stack, or IPv6 only 🌈. The data plane is always separate from the GACP, so there's no dependency between the GACP and whatever IP version(s) the data plane uses. No cap 🧢.

This setup is enough for troubleshooting stuff like SSH into network devices, NMS doing SNMP read operations for status checking, software downloads onto autonomic devices, provisioning devices via NETCONF, and so on 🔧. When you pair it with otherwise unmodified OAM through separate NMS hosts, this setup can cover a good chunk of the stable connectivity goals 💖. The limitations of this approach are coming up in the next section 👀.

Here's the thing: the GACP only does "only" IPv6 connectivity 🥺. And the addressing the GACP gives out has no topological structure, which NOCs usually rely on to figure out where devices are on the network 🗺️. So it's highly desirable to set up the Domain Name System (DNS; see [RFC1034]) so the GACP IPv6 addresses of autonomic devices are known by domain names that include the structure you want 🎀. For example, say DNS in the network has names for network devices like devicename.noc.example.com. And say operators use the well-known structure of the data-plane IPv4 address space to figure out the region a device is in. Then the GACP address of that device could be set up as devicename_<region>.acp.noc.example.com, and devicename.acp.noc.example.com could be a CNAME pointing to devicename_<region>.acp.noc.example.com 😍. Also, lots of networks already name their equipment with topological info, even without a GACP 💅.

3.1.2. Challenges and Limitations of Simple Connectivity

This simple connectivity for non-autonomic NMS hosts comes with a bunch of challenges (aka operators might not be able to do it this way 😩) and limitations (aka operators can't hit their goals with this setup 💀). The list below sums these up, and the sections after it cover extra mechanisms to fix them 🧩.

Quick note: these challenges and limitations exist because the GACP is mainly designed to support a distributed Autonomic Service Agent (ASA), which is a piece of autonomic software, in the lightest way possible 🪶. The GACP is not required to support the extra mechanisms that centralized NOC systems need. This document is the one that describes those extra (short-term) workarounds and (long-term) extensions 🌸.

  1. (Limitation) NMS hosts can't directly probe whether the desired so-called "data-plane" network connectivity works, because they don't have direct access to it 🙃. This is similar to probing connectivity for other services (like VPN services) that they don't have direct access to, so the NOC may already use suitable mechanisms for this (probing proxies) 🔍. See Section 3.1.3 for candidate solutions.
  2. (Challenge) NMS hosts need to support IPv6, and that's often still not possible in enterprise networks 😭. See Section 3.1.4 for some workarounds.
  3. (Limitation) Performance of the GACP may be limited compared to normal "data-plane" connectivity 🐢. The GACP setup often only supports forwarding that isn't hardware accelerated. Pushing a lot of traffic through the GACP, especially for tasks that don't need it, will reduce its performance and effectiveness for the operations where it is necessary or highly desirable 🚫. See Section 3.1.5 for candidate solutions.
  4. (Limitation) Security of the GACP is reduced by exposing the GACP natively (and unencrypted) in a NOC subnet where the NOC devices are attached 🚨. See Section 3.1.7 for candidate solutions.

These four problems can be tackled independently of each other through solution improvements 🌈. Combining some of these improvements can lead toward a candidate long-term solution 👑.

3.1.3. Simultaneous GACP and Data-Plane Connectivity

Being connected to the GACP and the data plane at the same time can be done in a variety of ways 🦋. If the data plane is IPv4 only, then any method for dual-stack attachment of the NOC device or application will do the job ✨. IPv6 connectivity from the NOC gives access via the GACP, and IPv4 gives access via the data plane 💖. And if, as explained in the simple case above, an autonomic device supports native attachment to the GACP, and the existing NOC setup is IPv4 only, then it could be enough to attach the GACP device(s) as the IPv6 default router on the NOC subnet and leave the existing IPv4 default router setup unchanged 🎀.

If the data plane is also running IPv6, then the most compatible setup for NOC devices is to have two IPv6 interfaces 🫶. One is a virtual (e.g., via IEEE 802.1Q [IEEE.802.1Q]) or physical interface connecting to a data-plane subnet, and the other connects into a GACP connect subnet. See Section 8.1 of [ACP] for more details 📖. That document also specifies how a NOC device can receive autoconfigured addressing and routes toward the ACP connect subnet, as long as it supports default address selection as specified in [RFC6724] and default router preferences as specified in [RFC4191] 🌟.

Configuring a second interface on a NOC host may be impossible, or seen as undesired complexity 😵. In that case, the GACP edge device needs to provide support for a "combined ACP and data-plane interface," as described in Section 8.1 of [ACP] 🧠. This setup may not work with autoconfiguration and all NOC host network stacks, because of limitations in those stacks 🥴. They need to be able to perform Rule 5.5 of [RFC6724] regarding source address selection, including caching of next-hop information 🔥.

For security reasons, it is not considered appropriate to connect a non-GACP router to a GACP connect interface 🚫🔒. The GACP is a secured network domain, and all NOC devices connecting via GACP connect interfaces are also part of that secure domain 💜. The main difference is that the physical links between the GACP edge device and the NOC devices are not authenticated or encrypted, so they need to be physically secured 🏰. If the secure GACP could be extended via untrusted routers, it would be a whole lot harder to verify the secure domain assertion 😱. So the GACP edge devices are not supposed to redistribute routes from non-GACP routers into the GACP 💅.

Part 2 done, bestie 🌈💖

3.1.4. IPv4-Only NMS Hosts ✨

okay so the big architectural vibe for the GACP from Section 1.3 is that every device that wants to use it (NMS hosts included) has to support IPv6 💖 but this doesn't mean the data plane needs IPv6 too 🫶 the data plane could be IPv4 only, IPv6 only, dual stack, or it might not even have an IP host stack on the network devices 🦋

that vibe can mean we need some short-term workarounds when the way a network is run doesn't match the target yet 🙈 this section explains when and why those workarounds might be operationally needed, and walks through them 💅 but the long-term goal is to upgrade all NMS hosts to native IPv6, so these workarounds are not supposed to be permanent, bestie 😭

ngl most network gear today supports IPv6, but NOC backend solutions (hardware or software) and enterprise products are still very far from ubiquitous support 💀 even when IPv6 is there, there are often extra limitations or issues in a dual-stack setup, or the operator just requires single stack for all operations to keep things simple ✨ so an IPv4-only management plane is still required and is common practice at many enterprises 🌸 if they're not using the GACP, this required and common practice isn't a problem for them, even when they run dual stack in the network 💜 we talk about these workarounds here because it's a short-term deployment challenge that's specific to operating a GACP 🎀

to connect IPv4-only management-plane devices and applications to a GACP, some form of IP/ICMP translation between IPv4 and IPv6 is necessary 🔥 the basic mechanisms for this are in [RFC7915], which describes the Stateless IP/ICMP Translation Algorithm (SIIT) 👑 there are multiple solutions that use this mechanism, so let's look at the requirements first to figure out which one fits 🧚

  1. NMS hosts need to be able to initiate connections to any GACP device for management purposes 💅 examples include provisioning via NETCONF, SNMP poll operations, or just diagnostics via SSH connections from operators 🫶 every GACP device or function that needs to be reachable from NMS hosts needs to have a separate IPv4 address 🍓
  2. GACP devices need to be able to initiate connections to NMS hosts 🌈 for example, to start NTP or RADIUS/Diameter connections, send syslog or SNMP traps, or initiate NETCONF Call Home connections after bootstrap ✨ every NMS host needs its own separate IPv6 address that's reachable from the GACP 🦋 when a connection from a GACP device is made to an NMS host, the IPv4 source address of that connection (as seen by the NMS host) must be unique per GACP device, and it must be the same address as in (1) so addressing stays simple, just like a native IPv4 deployment 💖 like in syslog, the source IP address of a logging device is what identifies it, and if the device starts showing problems, an operator might want to SSH into it to diagnose the issue 😭

because of these requirements, the necessary and sufficient set of solutions are the ones that provide 1:1 mapping of IPv6 GACP addresses into IPv4 space and 1:1 mapping of IPv4 NMS host space into IPv6 (for use in the GACP) 🔥 that means SIIT-based solutions are sufficient and preferred 👑

note that GACP devices may use multiple IPv6 addresses in the GACP 🧚 for example, Section 6.10 of [ACP] defines multiple useful addressing sub-schemes that support this option 🌸 all of those addresses may then need to be reachable through IPv6/IPv4 address translation 💜

needing one or more IPv4 addresses for every GACP device shouldn't be a problem if, as we assume, the NMS hosts can use private IPv4 address space ([RFC1918]) 🎀 still, even with private IPv4 space, it's important that GACP IPv6 addresses can be mapped efficiently into IPv4 space without too much waste 🫶

right now, the most flexible mapping scheme for this is [RFC7757], because it allows a configured IPv4 <-> IPv6 prefix mapping 🔥 let's say the GACP uses the ACP Zone Addressing Sub-Scheme and there are 3 registrars 🦋 in that sub-scheme, each registrar has a constant /112 prefix that can get an Explicit Address Mapping (EAM), as defined in RFC 7757, to a /16 prefix (for example, in the private IPv4 address space described in [RFC1918]) 💅 inside a registrar's /112 prefix, Device-Numbers are assigned sequentially for devices: with the V bit (Virtualization bit), effectively two numbers are assigned per GACP device 🧚 this also means that if IPv4 space is even more constrained, and it's known that a registrar will never need the full /15 extent of Device-Numbers, then a prefix longer than a /112 can be configured into the EAM to use less IPv4 space 🌈

when using the ACP Vlong Addressing Sub-Scheme, it's unlikely anyone wants or needs to translate the full /8 or /16 of addressing space per GACP device into IPv4 😭 in that case, the EAM rule of dropping trailing bits can be used to map only N bits of the V bits into IPv4 🍓 but heads up, this does imply that only addresses that differ in those high-order N V bits can be told apart on the IPv4 side 💀

likewise, the IPv4 address space used for NMS hosts can easily be mapped into an address prefix assigned to a GACP connect interface ✨

a full specification of a solution that does SIIT in conjunction with GACP connect, following the considerations below, is outside the scope of this document 🌸

to be in compliance with security expectations, SIIT has to happen on the GACP edge device itself so that GACP security considerations can be taken into account 🔥 for example, IPv4-only NMS hosts can be handled exactly like IPv6 hosts connected to a GACP connect interface 👑

prior solutions like NAT64 ([RFC6146]) may also be usable to translate between GACP IPv6 address space and NMS hosts' IPv4 address space 🧚 as a workaround, this can also be done on non-GACP Edge Devices connected to a GACP connect interface 🦋 the details vary by implementation, because the options for configuring address mappings differ widely 💅 outside of EAM, there are no standardized solutions that allow mapping of prefixes, so it will most likely be necessary to explicitly map every individual (/128) GACP device address to an IPv4 address 😭 that approach should use automation or scripting, where these address translation entries get created dynamically whenever a GACP device is enrolled or first connected to the GACP network 🫶

the NAT methods described here aren't specific to a GACP 💜 they're similar to what would be needed when some parts of a network only support IPv6, but the NOC equipment doesn't support IPv6 🎀 whether it's better to wait until the NOC equipment supports IPv6 or to use NAT in the meantime depends a lot on how long the former will take, and how easy the latter is to run with products that support the NAT options described, so you can operationalize the recommendations above 🌈✨

3.1.5. Path Selection Policies 🛣️✨

okay so bestie, let's talk about the GACP's performance situation 😭. As mentioned above, a GACP is NOT expected to have high performance, because its main goals are connectivity and security 🔐. For existing network device platforms, this usually means it's a LOT more effort to implement that extra connectivity with hardware acceleration than without, especially because we want full encryption across the GACP to get the security we need 💅.

Some of these issues may go away later as more devices adopt a GACP and device designs get better at handling a separate OAM plane 🌈. BUT it's wise to plan long-term designs that do NOT depend on the GACP being fast 🙈. This is the opposite of the expectation that future NMS hosts will have IPv6, and that any IPv4/NAT considerations in this solution are temporary 🦋.

To deal with the expected performance limits, we DO expect the dual connectivity we described earlier, via both the GACP and the data plane, between NOC application devices and devices with a GACP 🫶. The GACP connectivity is expected to always be there (as soon as a device is enrolled) 🎀, but the data-plane connectivity is only there during normal operations. It will NOT be present during things like early device bootstrap, failures, provisioning mistakes, or network configuration changes 💀.

So the desired policy is this 👑: as long as there are no further security considerations (see below), traffic between NMS hosts and GACP devices should PREFER the data plane and only fall back to the GACP when it's actually necessary 🍓. The exception is an operation we KNOW is covered by the use cases where the GACP is required, so it makes no sense to even try the data plane 🧚. An example is an SSH connection from the NOC to a network device to troubleshoot connectivity 🔥. This could easily ALWAYS rely on the GACP 💜. Also, if an NMS host is known to send large amounts of data and it uses the GACP, then its data rate needs to be controlled so it won't overload the GACP path ⭐. Typical examples are software downloads 📦.

There are lots of ways to build these policies 🌸. We describe a few below ✨.

Ideally, a NOC system would learn and track ALL the addresses of a device (the GACP ones and the various data-plane ones) 🧠. Every action the NOC system takes would indicate via a "path-policy" what kind of connection it needs (for example, only data-plane, GACP only, default to data plane, fall back to GACP, and so on) 💅. A connection policy manager would then build a connection to the target using the right address(es) 🦋. In the shorter term, a common practice is to identify different paths to a device via different names (for example, loopback vs. interface addresses) 🎀. This approach can be extended to GACP uses, whether it uses the DNS or names local to the NOC system 🌈. Below, we describe example schemes using DNS 🫶.

DNS can be used to set up names for the same network devices, but with different addresses assigned 💖:

  • One name (name.noc.example.com) with ONLY the data-plane address(es) (IPv4 and/or IPv6), to be used for probing connectivity or for routine software downloads that may stall or fail when there are connectivity issues 🛣️.
  • One name (name-acp.noc.example.com) with ONLY the GACP-reachable address of the device, for troubleshooting and probing/discovery that is desired to ALWAYS use only the GACP 🔥.
  • One name (name-both.noc.example.com) with data-plane and GACP addresses 👑.

Traffic policing and/or shaping at the GACP edge in the NOC can be used to throttle applications such as software downloads going into the GACP 🍓.

Using different names that map to different addresses (or subsets of addresses) can be tricky to set up and maintain 😵‍💫, especially because data-plane addresses may change due to reconfiguration or device relocation 🧚. The name-based approach alone CANNOT strongly support policies where existing applications and long-lived flows automatically switch between the ACP and the data plane when the data plane fails and recovers 💀. A solution would be host transport stacks on GACP nodes that support the following requirements 🎀:

  1. Only the GACP addresses of the responder MUST be required by the initiator for the initial setup of a connection/flow across the GACP ✨.
  2. Responder and initiator MUST be able to exchange their data-plane addresses through the GACP, and then, if policy requires it, build an additional flow across the data plane 🫶.
  3. For unmodified applications, the following policies SHOULD be configurable on at least a per-application basis for their TCP connections with GACP peers 🌈:

Fallback (to GACP): An additional data-plane flow is built and used EXCLUSIVELY to send data whenever the data plane is operational. When the additional flow cannot be built during connection setup, or when it fails later, traffic is sent across the GACP flow. This could be a default policy for most OAM applications using the GACP 💜.

Suspend/Fail: Like the Fallback policy, EXCEPT that traffic will NOT use the GACP flow. Instead, it will be suspended until a data-plane flow is operational, or until a policy-configurable timeout tells the application there's a connection failure 😭. This policy would be appropriate for large-volume background or scavenger-class OAM applications, such as firmware downloads or telemetry/diagnostic uploads 📦, applications that would otherwise easily overrun performance-limited GACP implementations 🔥.

GACP (only): NO additional data-plane flow is built; traffic is ONLY sent via the GACP flow. This can just be a TCP connection 🦋. This policy would be most appropriate for OAM operations known to change the data plane in a way that could impact connectivity through it (at least temporarily) 💀.

  1. In the presence of responders or initiators that do NOT support these host stack functions, the Fallback and GACP policies MUST result in a TCP connection across the GACP. For Suspend/Fail, the presence of TCP-only peers MUST result in failure during connection setup 🙈.
  2. In the case of Fallback and Suspend/Fail, a failed data-plane connection SHOULD automatically be rebuilt when the data plane recovers, including when the data-plane address of one side or both sides may have changed, for example because of reconfiguration or device repositioning 🍓.
  3. Additional data-plane flows created by these host transport stack functions MUST be end-to-end authenticated by these host transport stack functions with the GACP domain credentials, and encrypted 🔐. This maintains the expectation that connections from GACP addresses to GACP addresses are authenticated and encrypted 👑. This MAY be skipped if the application already provides for end-to-end encryption 🫶.
  4. For enhanced applications, the host stack MAY support application control to select the policy on a per-connection basis, or even more explicit control over building the flows and which flow should pass traffic 🧚.

Protocols like Multipath TCP (MPTCP; see [RFC6824]) and the Stream Control Transmission Protocol (SCTP; see [RFC4960]) can already support part of these requirements 💅. MPTCP, for example, supports signaling of addresses in a TCP-backward-compatible fashion 🌸, establishing additional flows (called subflows in MPTCP) ✨, and having primary and fallback subflows via MP_PRIO signaling 🦋. The details of how MPTCP, SCTP, and/or other approaches (potentially with extensions and/or (shim) layers on top of them) can best provide a complete solution for the above requirements still need further work and are OUTSIDE the scope of this document 🙈.

3.1.6. Autonomic NOC Device/Applications 🤖💖

Setting up connectivity between the NOC and autonomic devices when the NOC device itself is non-autonomic is a security issue, as mentioned at the beginning of this document 😬. It also brings a range of connectivity considerations (discussed in Section 3.1.5) 🛣️, some of which may be quite undesirable or complex to operationalize 💀.

So making NMS hosts autonomic and having them participate in the GACP is not only a highly desirable solution to the security issues 👑, it can also provide a likely easier operationalization of the GACP, because it minimizes special edge considerations for the NOC 🫶. The GACP is simply built all the way automatically, even inside the NOC 🦋, and it ONLY authorizes and authenticates the NOC devices/applications that will have access to it 🔐.

According to [ACP], supporting the ACP all the way into an application device requires implementing the following aspects in it: AN bootstrap/enrollment mechanisms, the secure channel for the ACP, and at least the host side of IPv6 routing setup for the ACP 🧚. Minimally, this could all be implemented as an application and made available to the host OS via, for example, a TAP driver, so the ACP shows up as another IPv6-enabled interface 🌈.

Having said this: if the structure of NMS hosts is transformed through virtualization anyway, then it may be considered equally secure and appropriate to build a (physical) NMS host system by combining a virtual GACP-enabled router with non-GACP-enabled Virtual Machines (VMs) for NOC applications via a hypervisor 🎀. This would leverage the configuration options described in the previous sections, but just virtualize them ✨.

3.1.7. Encryption of Data-Plane Connections 🔒🌸

When combining GACP and data-plane connectivity for availability and performance reasons, this ALSO has an impact on security 😭: when using the GACP, most traffic will be encryption-protected, especially when considering the above-described use of application devices with GACP 💜. If, instead, the data plane is used, then that is NOT the case anymore, unless the application does it itself 🙈.

The simplest solution for this problem exists when using GACP-capable NMS hosts, because in that case the communicating GACP-capable NMS host and the GACP network device have credentials they can mutually trust (same GACP domain) 🫶. As a result, data-plane connectivity that does support this can simply leverage TLS [RFC5246] or DTLS [RFC6347] with those GACP credentials for mutual authentication, and this does NOT incur new key management 🦋.

If this automatic security benefit is seen as most important, but a "full" GACP stack in the NMS host is unfeasible, then it would still be possible to design a stripped-down version of GACP functionality for such NOC hosts. It would only provide enrollment of the NOC host with the GACP cryptographic credentials and would NOT directly participate in the GACP encryption method 🧚. Instead, the host would just leverage TLS/DTLS using its GACP credentials via the data plane with GACP network devices, as well as indirectly via the GACP connect interface with the above-mentioned GACP connect interface into the GACP 💅.

When using the GACP itself, TLS/DTLS for the transport layer between NMS hosts and network devices is somewhat of a double price to pay (GACP also encrypts), and it could potentially be optimized away 🌈. However, given the assumed lower performance of the GACP, it seems that this is an unnecessary optimization 🍓.

3.1.8. Long-Term Direction of the Solution ✨

okay so if we think about what could be the most lightweight and autonomic long-term solution using the techs we just covered, here's the direction I'm obsessed with 💅🧚

  1. NMS hosts should at least support IPv6. IPv4/IPv6 NAT in the network to enable use of a GACP is undesirable in the long term. Having IPv4-only applications automatically leverage IPv6 connectivity via host-stack translation may be an option, but this has not been investigated yet. 🙈
  2. Build the GACP as a lightweight application for NMS hosts so GACP extends all the way into the actual NMS hosts. 🫶
  3. Leverage and (as necessary) enhance host transport stacks with automatic GACP with multipath connectivity and data plane as outlined in Section 3.1.5. 🔥
  4. Consider how to best map NMS host desires to underlying transport mechanisms. The three points above do not cover all options. Depending on the OAM, one may still want only GACP, want only data plane, automatically prefer one over the other, and/or want to use the GACP with low performance or high performance (for emergency OAM such as countering DDoS). As of today, it is not clear what the simplest set of tools is to explicitly enable the choice of desired behavior of each OAM. The use of the above-mentioned DNS and multipath mechanisms is a start, but this will require additional work. This is likely a specific case of the more generic scope of TAPS. this one is giving "to be continued" energy 😭

3.2. Stable Connectivity for Distributed Network/OAM

Today, many distributed protocols implement their own unique security mechanisms. It's giving a lot of reinventing the wheel 💀

Keying and Authentication for Routing Protocols (KARP; see [RFC6518]) has tried to start to provide common directions and therefore reduce the reinvention of at least some of the security aspects, but it only covers routing protocols and it is unclear how applicable it is to a wider range of network distributed agents such as those performing distributed OAM. The common security of a GACP can help in those cases. 🛡️✨

Furthermore, a GRASP instance ([GRASP]) can run on top of a GACP as a security and transport substrate and provide common local and remote neighbor discovery and peer negotiation mechanisms; this would allow unifying and reusing future protocol designs. main character energy for GRASP 👑

4. Architectural Considerations 🏗️

4.1. No IPv4 for GACP

The GACP is intended to be IPv6 only, and the prior explanations in this document show that this can lead to some complexity when having to connect IPv4-only NOC solutions, and that it will be impossible to leverage the GACP when the OAM agents on a GACP network device do not support IPv6. Therefore, the question was raised whether the GACP should optionally also support IPv4. 🤔

The decision not to include IPv4 for GACP in the use cases in this document was made for the following reasons:

In service provider networks that have started to support IPv6, often the next planned step is to consider moving IPv4 from a native transport to just a service on the edge. There is no benefit or need for multiple parallel transport families within the network, and standardizing on one reduces operating expenses and improves reliability. This evolution in the data plane makes it highly unlikely that investing development cycles into IPv4 support for GACP will have a longer term benefit or enough critical short-term use cases. Support for IPv6-only for GACP is purely a strategic choice to focus on the known important long-term goals. 💅

In other types of networks as well, we think that efforts to support autonomic networking are better spent in ensuring that one address family will be supported so all use cases will work with it in the long term, instead of duplicating effort with IPv4. Also, auto-addressing for the GACP with IPv4 would be more complex than in IPv6 due to the IPv4 addressing space. 🌈

5. Security Considerations 🔐😱

okay this section is the one you do NOT skip bestie, it's serious 🥺 We only discuss security considerations not covered in the appropriate subsections of the solutions described.

Even though GACPs are meant to be isolated, explicit operator misconfiguration to connect to insecure OAM equipment and/or bugs in GACP devices may cause leakage into places where it is not expected. Mergers and acquisitions and other complex network reconfigurations affecting the NOC are typical examples. 💀

GACP addresses are ULAs. Using these addresses also for NOC devices, as proposed in this document, is not only necessary for the simple routing functionality explained above, but it is also more secure than global IPv6 addresses. ULAs are not routed in the global Internet and will therefore be subject to more filtering even in places where specific ULAs are being used. Packets are therefore less likely to leak and less likely to be successfully injected into the isolated GACP environment. ULAs are literally the bouncers of IPv6 🚪💖

The random nature of a ULA prefix provides strong protection against address collision even though there is no central assignment authority. This is helped by the expectation that GACPs will never connect all together, and that only a few GACPs may ever need to connect together, e.g., when mergers and acquisitions occur. 🎲

Note that the GACP constraints demand that only packets from connected subnet prefixes are permitted from GACP connect interfaces, limiting the scope of non-cryptographically secured transport to a subnet within a NOC that instead has to rely on physical security (i.e., only connect trusted NOC devices to it). 🔒

To help diagnose packets that unexpectedly leaked, for example, from another GACP (that was meant to be deployed separately), it can be useful to voluntarily list your own ULA GACP prefixes on some sites on the Internet and hope that other users of GACPs do the same so that you can look up unknown ULA prefix packets seen in your network. Note that this does not constitute registration. <https://www.sixxs.net/tools/grh/ula/> was a site to list ULA prefixes, but it has not been open for new listings since mid-2017. The authors are not aware of other active Internet sites to list ULA use. 🕵️‍♀️

Note that there is a provision in [RFC4193] for address space that is not locally assigned (L bit = 0), but there is no existing standardization for this, so these ULA prefixes must not be used. 🚫

According to Section 4.4 of [RFC4193], PTR records for ULA addresses should not be installed into the global DNS (no guaranteed ownership). Hence, there is also the need to rely on voluntary lists (as mentioned above) to make the use of an ULA prefix globally known. 📒

Nevertheless, some legacy OAM applications running across the GACP may rely on reverse DNS lookup for authentication of requests (e.g., TFTP for download of network firmware, configuration, or software). Therefore, operators may need to use a private DNS setup for the GACP ULAs. This is the same setup that would be necessary for using RFC 1918 addresses in DNS. For example, see the last paragraph of Section 5 of [RFC1918]. In Section 4 of [RFC6950], these setups are discussed in more detail. 🧐

Any current and future protocols must rely on secure end-to-end communications (TLS/DTLS) and identification and authentication via the certificates assigned to both ends. This is enabled by the cryptographic credential mechanisms of the GACP. 🔑💜

If DNS and especially reverse DNS are set up, then they should be set up in an automated fashion when the GACP address for devices are assigned. In the case of the ACP, DNS resource record creation can be linked to the autonomic registrar backend so that the DNS and reverse DNS records are actually derived from the subject name elements of the ACP device certificates in the same way as the autonomic devices themselves will derive their ULAs from their certificates to ensure correct and consistent DNS entries. ⚙️✨

If an operator feels that reverse DNS records are beneficial to its own operations, but that they should not be made available publicly for "security" by concealment reasons, then GACP DNS entries are probably one of the least problematic use cases for split DNS. The GACP DNS names are only needed for the NMS hosts intending to use the GACP -- but not network wide across the enterprise. 🫣

6. IANA Considerations 🌸

This document has no IANA actions. Zero registry drama, all vibes ✨

7. References 📚

7.1. Normative References

Here are the normative references, every single one, exactly as listed in the original:

7.2. Informative References ✨

These are the extra reads that help you understand the bigger picture 📚💖

[ACP]      Eckert, T., Behringer, M., and S. Bjarnason, "An Autonomic
           Control Plane (ACP)", Work in Progress,
           draft-ietf-anima-autonomic-control-plane-13,
           December 2017.

[BRSKI]    Pritikin, M., Richardson, M., Behringer, M., Bjarnason,
           S., and K. Watsen, "Bootstrapping Remote Secure Key
           Infrastructures (BRSKI)", Work in Progress,
           draft-ietf-anima-bootstrapping-keyinfra-15, April 2018.

[GRASP]    Bormann, C., Carpenter, B., and B. Liu, "A Generic
           Autonomic Signaling Protocol (GRASP)", Work in Progress,
           draft-ietf-anima-grasp-15, July 2017.

[IEEE.802.1Q]
           IEEE, "IEEE Standard for Local and metropolitan area
           networks -- Bridges and Bridged Networks",
           IEEE 802.1Q-2014, DOI 10.1109/ieeestd.2014.6991462,
           December 2014, <http://ieeexplore.ieee.org/servlet/
           opac?punumber=6991460>.

[ITUT_G7712]
           ITU, "Architecture and specification of data communication
           network", ITU-T Recommendation G.7712/Y.1703, November
           2001, <https://www.itu.int/rec/T-REC-G.7712/en>.

[REF_MODEL]
           Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L.,
           and J. Nobre, "A Reference Model for Autonomic
           Networking", Work in Progress,
           draft-ietf-anima-reference-model-06, February 2018.

[RFC1034]  Mockapetris, P., "Domain names - concepts and facilities",
           STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987,
           <https://www.rfc-editor.org/info/rfc1034>.

[RFC4960]  Stewart, R., Ed., "Stream Control Transmission Protocol",
           RFC 4960, DOI 10.17487/RFC4960, September 2007,
           <https://www.rfc-editor.org/info/rfc4960>.

[RFC5246]  Dierks, T. and E. Rescorla, "The Transport Layer Security
           (TLS) Protocol Version 1.2", RFC 5246,
           DOI 10.17487/RFC5246, August 2008,
           <https://www.rfc-editor.org/info/rfc5246>.

[RFC6146]  Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful
           NAT64: Network Address and Protocol Translation from IPv6
           Clients to IPv4 Servers", RFC 6146, DOI 10.17487/RFC6146,
           April 2011, <https://www.rfc-editor.org/info/rfc6146>.

[RFC6291]  Andersson, L., van Helvoort, H., Bonica, R., Romascanu,
           D., and S. Mansfield, "Guidelines for the Use of the "OAM"
           Acronym in the IETF", BCP 161, RFC 6291,
           DOI 10.17487/RFC6291, June 2011,
           <https://www.rfc-editor.org/info/rfc6291>.

[RFC6347]  Rescorla, E. and N. Modadugu, "Datagram Transport Layer
           Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347,
           January 2012, <https://www.rfc-editor.org/info/rfc6347>.

[RFC6434]  Jankiewicz, E., Loughney, J., and T. Narten, "IPv6 Node
           Requirements", RFC 6434, DOI 10.17487/RFC6434, December
           2011, <https://www.rfc-editor.org/info/rfc6434>.

[RFC6518]  Lebovitz, G. and M. Bhatia, "Keying and Authentication for
           Routing Protocols (KARP) Design Guidelines", RFC 6518,
           DOI 10.17487/RFC6518, February 2012,
           <https://www.rfc-editor.org/info/rfc6518>.

[RFC6950]  Peterson, J., Kolkman, O., Tschofenig, H., and B. Aboba,
           "Architectural Considerations on Application Features in
           the DNS", RFC 6950, DOI 10.17487/RFC6950, October 2013,
           <https://www.rfc-editor.org/info/rfc6950>.

Acknowledgements 🫶

This whole thing started as an Autonomic Networking project at Cisco Systems way back in early 2010 🌸 Customers got involved in the design and early testing too, which is so cool. Tons of people helped shape the stuff in this document, and they're listed here in alphabetical order: BL Balaji, Steinthor Bjarnason, Yves Herthoghs, Sebastian Meissner, and Ravi Kumar Vadapalli 👑 The authors also want to thank Michael Richardson, James Woodyatt, and Brian Carpenter for their reviews and comments. Extra special thanks go to Sheng Jiang and Mohamed Boucadair for their thorough reviews 💜

Authors' Addresses 💌

Toerless Eckert (editor)
Huawei USA
2330 Central Expy
Santa Clara  95050
United States of America

Email: tte+ietf@cs.fau.de, toerless.eckert@huawei.com


Michael H. Behringer

Email: michael.h.behringer@gmail.com
📦 where this page comes from