RFC 8886

Secure Device Install

📅 published September 2020, from ietf

1. Introduction 🎀

okay so bestie, let's talk about the BIG problem 😭 a growing global network is basically a money-and-time black hole 🕳️. deploying new devices and "forklift" upgrading old ones costs a ridiculous amount of both. and in lots of cases the gear lives in shared facilities (think Internet Exchange Points (IXP) or "carrier-neutral data centers") where the staff on hand can be contracted to do physical installs, reboots, loading initial configs, and so on 🧰. there are also a bunch of (often proprietary) protocols for initial installs and configs 🔧. for example, lots of network devices will try DHCP [RFC2131] or DHCPv6 [RFC8415] to get an IP address and a configuration server, then fetch and install a config the first time they're powered on ⚡.

the configs on network devices are packed with security-sensitive and proprietary stuff, like RADIUS [RFC2865] or TACACS+ [TACACS] secrets 🤫. so exposing those to a third party so they can load them onto a new device (or using an auto-install trick that grabs an unencrypted config file over TFTP [RFC1350]) is an unacceptable security risk for lots of operators 🚨. that's why they keep flying employees out to remote sites to do the initial config by hand, which, yikes, costs time and money 💸.

there are some workarounds, like asking the vendor to preconfigure the device before it ships, asking remote support to install a terminal server, or providing a minimal, unsecured config and using it to bootstrap to the full one 🙃. but these are often clumsy and have security issues. for example, with the terminal server approach, the console port connection could be easily snooped 👀.

the ideal solution would protect BOTH the confidentiality of the device config in transit AND the authenticity (and authorization status) of the config the device uses 🏆. the mechanism in this document only covers the first one 💅. it makes no effort to do the second, so the device accepts any config file that comes its way, as long as it's encrypted to the device's key (or not encrypted, depending on the case) 🫣. other solutions, like Secure Zero Touch Provisioning (SZTP) [RFC8572], Bootstrapping Remote Secure Key Infrastructures (BRSKI) [BRSKI], and other voucher-based methods, are more fully featured, but they also need more complicated machinery 🤯. this document is something much simpler, and the trade-off is that it only gives limited protection 🦋.

this document layers security onto existing auto-install solutions (one example is [Cisco_AutoInstall]) so you can initially configure new devices while keeping (limited) confidentiality of the initial config 🔐. it's optimized for simplicity, for both the implementor AND the operator 🌸. it is explicitly NOT meant to be a fully featured system for managing installed devices, and it's not meant to solve every use case 🙅‍♀️. it's a simple, targeted fix for a common ops problem: the device has been delivered, the fiber has been laid (where needed), and there's no trusted staff member to do the initial config 📦. this solution only increases the confidentiality of the info in the config file. it does NOT protect the device from loading a malicious config 🚫.

this document describes a concept and some example ways to implement it. devices have different capabilities and use different config paradigms, so one method won't fit everyone, and vendors are expected to differ in exactly how they implement this 🧩.

this solution is specifically designed to be a simple method layered on top of existing device functionality. if devices don't support this new method, they can keep using the existing functionality 💜. operators can also choose to use it to protect their config info, or just keep using the existing functionality, totally their call ✨.

securely installing devices is not a new problem, and it's not limited to network devices. it shows up when deploying servers, PCs, Internet of Things (IoT) devices, and plenty of other stuff 🖥️. the solution here is kind of obvious (encrypt the config, then decrypt it with a device key), but this document only talks about network devices, like routers and switches 🛜.

2. Overview 🌈

most network devices already have some kind of initial bootstrapping logic (sometimes called 'autoboot' or 'autoinstall') 🥰. it usually works like this: a newly installed, unconfigured device gets an IP address for itself and finds the address of a configuration server (often called 'next-server', 'siaddr', or 'tftp-server-name') using DHCP or DHCPv6 (see [RFC2131] and [RFC8415]) 🔍. then the device contacts that config server to download its initial config, which is usually identified by the device's serial number, Media Access Control (MAC) address, or something similar 📇. this document extends that vendor-specific paradigm by letting the config file be encrypted 🔒.

this document uses the device's serial number as a unique identifier, just to keep things simple 🧃. some vendors may not want to use the serial number for business reasons, though, since a competitor could enumerate serial numbers and figure out how many devices have been manufactured 👀. implementors are free to pick another way to generate identifiers (for example, a Universally Unique Identifier (UUID) [RFC4122]), but that will likely make things a bit harder for operators, since the serial number is usually easy to find on the device 🏷️.

2.1. Example Scenario 🎬

Operator_A needs another peering router, so they order one from Vendor_B to be drop-shipped to the facility 📮. Vendor_B starts assembling the new device and tells Operator_A what its serial number will be (SN:17894321). when Vendor_B first installs the firmware and boots the device, the device generates a public-private key pair, and Vendor_B publishes the public key on its key server (in a public key certificate, for ease of use) 🔑.

while the device is being shipped, Operator_A generates the initial device config and fetches the certificate from Vendor_B's key servers by providing the device's serial number 🧾. then Operator_A encrypts the device config and puts it on a (local) TFTP server 📁.

when the device arrives at the Point of Presence (POP), it gets installed in Operator_A's rack and cabled as instructed 🗄️. the new device powers up and discovers it hasn't been configured yet 😮. it enters its autoboot state and starts the DHCP process 🚀. Operator_A's DHCP server gives it an IP address and the address of the configuration server 🎯. the router uses TFTP to fetch its config file. note that all of this is existing functionality 💅. the device tries to load the config file. as an added step, if the config file can't be parsed, the device tries to use its private key to decrypt the file and, assuming it validates, goes ahead and installs the new, decrypted config ✨.

only the "correct" device will have the required private key and be able to decrypt and use the config file (see Security Considerations (Section 7)) 🔐. an attacker could connect to the network and get an IP address 😈. they could also retrieve (encrypted) config files by guessing serial numbers (or maybe the server allows directory listing), but without the private keys, the attacker won't be able to decrypt the files 🙅‍♀️.

3. Vendor Role 💖

this section describes the vendor's roles and gives an overview of what the device needs to do 🎀.

3.1. Device Key Generation 🔑

okay bestie, every device needs its own public-private key pair 🗝️, and the public half has to be published somewhere the operator can grab it later ✨. The crypto algorithm and key lengths are out of scope for this doc 🙅‍♀️. This section shows one method, but like most of this doc, the exact mechanism may vary by vendor 💅. Enrollment over Secure Transport [RFC7030] and maybe the Simple Certificate Enrollment Protocol [RFC8894] are options vendors may want to consider 🌸.

During manufacturing, when the device gets powered on for the first time, it generates a public-private key pair 🎀. Then it sends its unique device identifier and the public key to the vendor's directory server [RFC5280] so it gets published 📤. The directory server should only accept certificates from the manufacturing facility that match vendor-defined policies (for example, extended key usage and extensions) 🛡️. Some devices are constrained, so they may send just the raw public key and unique device identifier to the publication server. More capable devices may generate and send self-signed certificates instead 🦋. This communication with the directory server should be integrity protected and should happen in a controlled environment 🔒.

This reference architecture needs a serialization format for the key material 📦. X.509 certificates are super convenient because network devices already have tons of tooling support for them 🛠️. BUT, most of the metadata that would normally be used for revocation and aging won't get used, and both the client and server should ignore it 🙈. In that case the signature on the certificate conveys no value, so the consumer of the certificate is expected to pin the end-entity key fingerprint (instead of using a PKI and signature chain) 📌.

3.2. Directory Server

The directory server holds a database of certificates 🗄️. If newly manufactured devices upload certificates, the directory server can just publish them 🚀. If the devices hand over raw public keys and a unique device identifier instead, the directory server will need to wrap those in a certificate 🎁.

Customers (e.g., Operator_A) query this server with the serial number (or other provided unique identifier) of a device and grab the matching certificate 🔍. Operators are expected to get the unique device identifier (serial number) when they buy the devices, then download and store the certificate 🧾. So there's no hard requirement that the directory server is always reachable 🥳.

                         +------------+
        +------+         |            |
        |Device|         | Directory  |
        +------+         |   Server   |
                         +------------+
   +----------------+   +--------------+
   |   +---------+  |   |              |
   |   | Initial |  |   |              |
   |   |  boot?  |  |   |              |
   |   +----+----+  |   |              |
   |        |       |   |              |
   | +------v-----+ |   |              |
   | |  Generate  | |   |              |
   | |Self-signed | |   |              |
   | |Certificate | |   |              |
   | +------------+ |   |              |
   |        |       |   |   +-------+  |
   |        +-------|---|-->|Receive|  |
   |                |   |   +---+---+  |
   |                |   |       |      |
   |                |   |   +---v---+  |
   |                |   |   |Publish|  |
   |                |   |   +-------+  |
   |                |   |              |
   +----------------+   +--------------+

Figure 1: Initial Certificate Generation and Publication 📊

4. Operator Role 👑

4.1. Administrative

When the operator buys a new device, the accounting department needs to get the unique device identifier (e.g., serial number) of that device and pass it along to the operations group 💸.

4.2. Technical

The operator contacts the vendor's publication server and downloads the certificate by providing the device's unique device identifier 📥. The operator fetches the certificate over a secure transport that authenticates the source of the certificate, such as HTTPS (confidentiality protection can give some privacy and metadata-leakage benefits, but it's not key to the primary mechanism of this document) 🔐. Then the operator encrypts the initial configuration (for example, using S/MIME [RFC8551]) with the key in the certificate and puts it on their configuration server 🎯.

See Appendix A for examples 📖.

                         +------------+
      +--------+         |            |
      |Operator|         | Directory  |
      +--------+         |   Server   |
                         +------------+
   +----------------+  +----------------+
   | +-----------+  |  |  +-----------+ |
   | |   Fetch   |  |  |  |           | |
   | |  Device   |<------>|Certificate| |
   | |Certificate|  |  |  |           | |
   | +-----+-----+  |  |  +-----------+ |
   |       |        |  |                |
   | +-----v------+ |  |                |
   | |  Encrypt   | |  |                |
   | |   Device   | |  |                |
   | |   Config   | |  |                |
   | +-----+------+ |  |                |
   |       |        |  |                |
   | +-----v------+ |  |                |
   | |  Publish   | |  |                |
   | |    TFTP    | |  |                |
   | |   Server   | |  |                |
   | +------------+ |  |                |
   |                |  |                |
   +----------------+  +----------------+

Figure 2: Fetching the Certificate, Encrypting the Configuration, and Publishing the Encrypted Configuration 🧁

4.3. Example Initial Customer Boot

When the customer first boots the device (and on every boot after that), if the device doesn't have a valid configuration, it uses its existing auto-install functionality 🚦. For example, it does DHCP Discovery until it gets a DHCP offer that includes DHCP option 66 (Server-Name) or 150 (TFTP server address). Then it contacts the server listed in those DHCP options and downloads its configuration file 📡. Heads up: this is already existing functionality. For example, Cisco devices fetch the config file named by the Bootfile-Name DHCP option (67) 🙌.

After grabbing the configuration file, the device has to figure out if it's encrypted 🕵️‍♀️. If it isn't, the existing behavior is used. If it is encrypted, the process continues as described in this document. If the device has been set up to only support encrypted configuration and finds that the file isn't encrypted, it should abort 🛑. How the device checks for encryption is implementation dependent. There are a bunch of (obvious) options, like a magic string in the file header, a file name extension (e.g., config.enc), or specific DHCP options 🧩.

If the file is encrypted, the device tries to decrypt and parse it 🔓. If that works, it installs the configuration and starts using it 🎉. If it can't decrypt the file, or parsing the configuration fails, the device will either abort the auto-install process or keep repeating this process until it succeeds 🔁. When retrying, be careful not to overwhelm the server hosting the encrypted configuration files 🫶. It is suggested that the device retry every 5 minutes for the first hour and then every hour after that ⏰. Since devices may get installed well before the configuration file is ready, a maximum number of retries is not specified ♾️.

Note that the device only needs to be able to download the configuration file. After the initial power on in the factory, it never needs to access the Internet, vendor, or directory server 🙅‍♀️. The device (and only the device) has the private key, so it's the only thing that can decrypt the configuration file 🔑💜.

             +--------+                +--------------+
             | Device |                |Config server |
             +--------+                |(e.g., TFTP)  |
                                       +--------------+
   +---------------------------+    +------------------+
   | +-----------+             |    |                  |
   | |           |             |    |                  |
   | |   DHCP    |             |    |                  |
   | |           |             |    |                  |
   | +-----+-----+             |    |                  |
   |       |                   |    |                  |
   | +-----v------+            |    |  +-----------+   |
   | |            |            |    |  | Encrypted |   |
   | |Fetch config|<------------------>|  config   |   |
   | |            |            |    |  |   file    |   |
   | +-----+------+            |    |  +-----------+   |
   |       |                   |    |                  |
   |       X                   |    |                  |
   |      / \                  |    |                  |
   |     /   \ N    +--------+ |    |                  |
   |    | Enc?|---->|Install,| |    |                  |
   |     \   /      |  Boot  | |    |                  |
   |      \ /       +--------+ |    |                  |
   |       V                   |    |                  |
   |       |Y                  |    |                  |
   |       |                   |    |                  |
   | +-----v------+            |    |                  |
   | |Decrypt with|            |    |                  |
   | |private key |            |    |                  |
   | +-----+------+            |    |                  |
   |       |                   |    |                  |
   |       v                   |    |                  |
   |      / \                  |    |                  |
   |     /   \ Y    +--------+ |    |                  |
   |    |Sane?|---->|Install,| |    |                  |
   |     \   /      |  Boot  | |    |                  |
   |      \ /       +--------+ |    |                  |
   |       V                   |    |                  |
   |       |N                  |    |                  |
   |       |                   |    |                  |
   |  +----v---+               |    |                  |
   |  |Retry or|               |    |                  |
   |  | abort  |               |    |                  |
   |  +--------+               |    |                  |
   |                           |    |                  |
   +---------------------------+    +------------------+

Figure 3: Device Boot, Fetch, and Install Configuration File 🚀

5. Additional Considerations ✨

5.1. Key Storage

Ideally, the key pair would live in a Trusted Platform Module (TPM) on whatever is identified as the "router" (for example, the chassis/backplane) 🔐. That way the key pair is bound to what humans think of as the "device", and not to something like (redundant) routing engines 🤖. Devices that implement IEEE 802.1AR [IEEE802-1AR] could choose to use the Initial Device Identifier (IDevID) for this purpose 🪪.

5.2. Key Replacement

It's anticipated that some operators will want to replace the vendor-provided keys after installing the device 🔄. There are two options for implementing this 🧐. Option one: the vendor lets the operator's key completely replace the initial device-generated key. That means if the device is ever sold, the new owner couldn't use this technique to install it 😬. Option two: the device prefers the operator's installed key 💖. This is an implementation decision left to the vendor 💅.

5.3. Device Reinstall ✨

Okay so, automation is basically taking over device management 🤖💅. Lots of operators now have configs that are programmatically generated, either a piece of the config or the whole thing. That means an operator might want to make a brand-new config after the device is already installed and tell the device to load it and use it 🔄. The spec doesn't define how that works, because it's vendor specific 🙈, but it's expected that vendors will let the operator copy a new, encrypted config (or part of one) onto the device, then ask the device to decrypt and install it, like load replace <filename> encrypted 🔐. The operator could also just factory reset the device so it acts like it's booting for the very first time (see Section 4.3) 🌸.

6. IANA Considerations 🎀

This document has no IANA actions. Nothing to register here, bestie 💖.

7. Security Considerations 🔥

Okay, this part is seriously important 😭. The whole point of this reference architecture is to level up the "auto-install" practices people already use, which can send configs totally unencrypted. Think unencrypted config files sitting on unprotected ports in data centers, configs mailed on flash drives 💾, or configs emailed to some third party who has to copy and paste them over a serial terminal 💀. Some of these also let anyone access the configs with no restrictions at all.

This document describes an object-level security design that gives confidentiality for the config while it's stored at rest on the configuration server, and also while it's in transit between the configuration server and the unprovisioned device 🔒. That's true even if the underlying transport doesn't provide this security service.

The architecture gives no assurances about where the encrypted config came from, and it doesn't protect against theft and reuse of devices 🚨.

An attacker (say, a malicious data center employee or someone sneaking into the supply chain 🕵️‍♀️) who has physical access to the device before it's connected to the network, or who manages to exploit it once it's installed, may be able to pull out the device private key (especially if it isn't stored in a TPM). With that, they could pretend to be the device when connecting to the network, then download and extract the (encrypted) config file 😱.

An attacker who controls the configuration server (or can route traffic to a configuration server under their control), and who also has the device's public key, could return a config of their own choosing to the unprovisioned device 👀.

This mechanism does not protect against a malicious vendor 😤. The key pair should be generated on the device and the private key should be stored securely, but the mechanism can't detect or protect against a vendor who claims to do that while actually generating the key pair off device and keeping a copy of the private key. The operator community pretty much agrees that a malicious vendor, or an attacker with physical access to the device, is a "Game Over" situation 🎮💀.

Even when you're using a secure bootstrap mechanism, security-conscious operators may want to bootstrap devices with a minimal or less-sensitive config, then swap in a more complete one after install 🧐✨.

8. Informative References 📚

Appendix A. Proof of Concept 🧪

This section has a proof of concept of the system. It's only for illustration and is not meant for production 🙅‍♀️.

It uses OpenSSL from the command line. In production, something more automated would be used 🤖. In this example, the unique device identifier is the serial number of the router, SN19842256 🔢.

A.1. Step 1: Generating the Certificate 🔑

This step is done by the router 🛜. It generates a key, then a Certificate Signing Request (CSR), and then a self-signed certificate.

A.1.1. Step 1.1: Generate the Private Key

$ openssl ecparam -out privatekey.key -name prime256v1 -genkey
$

A.1.2. Step 1.2: Generate the Certificate Signing Request

$ openssl req -new -key key.pem -out SN19842256.csr
Common Name (e.g., server FQDN or YOUR name) []:SN19842256

A.1.3. Step 1.3: Generate the (Self-Signed) Certificate Itself

$ openssl req -x509 -days 36500 -key key.pem -in SN19842256.csr
-out SN19842256.crt

The router then sends the key to the vendor's key server for publication (not shown) 📤.

A.2. Step 2: Generating the Encrypted Configuration 🔐

The operator now wants to deploy the new router 🚀.

They generate the initial config (using whatever magic tool generates router configs! ✨), fetch the router's certificate, and encrypt the config file to that key. The operator does all of this 🧙‍♀️.

A.2.1. Step 2.1: Fetch the Certificate

$ wget http://keyserv.example.net/certificates/SN19842256.crt

A.2.2. Step 2.2: Encrypt the Configuration File

S/MIME is used here because it's simple to demonstrate. This is almost definitely not the best way to do this 🫠.

$ openssl smime -encrypt -aes-256-cbc -in SN19842256.cfg\
   -out SN19842256.enc -outform PEM SN19842256.crt
$ more SN19842256.enc
-----BEGIN PKCS7-----
MIICigYJKoZIhvcNAQcDoIICezCCAncCAQAxggE+MIIBOgIBADAiMBUxEzARBgNV
BAMMClNOMTk4NDIyNTYCCQDJVuBlaTOb1DANBgkqhkiG9w0BAQEFAASCAQBABvM3
...
LZoq08jqlWhZZWhTKs4XPGHUdmnZRYIP8KXyEtHt
-----END PKCS7-----

A.2.3. Step 2.3: Copy Configuration to the Configuration Server

$ scp SN19842256.enc config.example.com:/tftpboot

A.3. Step 3: Decrypting and Using the Configuration 🎉

When the router connects to the operator's network, it'll notice it doesn't have a valid config file and will kick off the "autoboot" process 🥾. This is a well-documented process, but the high-level version is that it uses DHCP to get an IP address and a configuration server. Then it uses TFTP to download a config file based on its serial number (this document modifies the solution to fetch an encrypted config file, ending in .enc) 📥. After that, it decrypts the config file and installs it ✅.

A.3.1. Step 3.1: Fetch Encrypted Configuration File from Configuration Server

$ tftp 2001:0db8::23 -c get SN19842256.enc

A.3.2. Step 3.2: Decrypt and Use the Configuration

$ openssl smime -decrypt -in SN19842256.enc -inform pkcs7\
   -out config.cfg -inkey key.pem

If an attacker doesn't have the correct key, they won't be able to decrypt the config file 🙅‍♀️:

$ openssl smime -decrypt -in SN19842256.enc -inform pkcs7\
   -out config.cfg -inkey wrongkey.pem
Error decrypting PKCS#7 structure
140352450692760:error:06065064:digital envelope
 routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:592:
$ echo $?
4

Acknowledgments 💜

The authors wish to thank everyone who contributed, including Benoit Claise, Francis Dupont, Mirja Kuehlewind, Sam Ribeiro, Michael Richardson, Sean Turner, and Kent Watsen. Joe Clarke also gave significant comments and review, and Tom Petch gave significant editorial contributions to better describe the use cases and clarify the scope 🫶.

Roman Danyliw and Benjamin Kaduk also contributed helpful text, especially around certificate usage and security considerations 🙌.

Authors' Addresses 🦋

Warren Kumari
Google
1600 Amphitheatre Parkway
Mountain View, CA 94043
United States of America

Email: warren@kumari.net


Colin Doyle
Juniper Networks
1133 Innovation Way
Sunnyvale, CA 94089
United States of America

Email: cdoyle@juniper.net
📦 where this page comes from