RFC 8240

Report from the Internet of Things Software Update (IoTSU) Workshop 2016

📅 published September 2017, from ietf

Report from the Internet of Things Software Update (IoTSU) Workshop 2016 🎀

Okay bestie, this is RFC 8240 🦋 It was written by H. Tschofenig and S. Farrell, and it's published by the Internet Architecture Board (IAB) as Informational, dated September 2017, with ISSN 2070-1721. The header doesn't list anything it obsoletes or updates, so no drama there ✨

Abstract 🌸

This doc sums up the Internet of Things Software Update (IoTSU) Workshop, which went down at Trinity College Dublin, Ireland, on the 13th and 14th of June, 2016 🎓 The big goal was to get people talking about requirements, challenges, and solutions for getting software and firmware updates onto IoT devices. The report covers what was discussed and lists recommendations for the standards community 💜

Note that this document is a report on the proceedings of the workshop. The views and positions documented in this report are those of the workshop participants and do not necessarily reflect IAB views and positions. So these are the participants' takes, not the IAB's official stance 🙈

Status of This Memo 💅

This document is not an Internet Standards Track specification. It's published for informational purposes, so it's not a spec you have to implement 📚

This document is a product of the Internet Architecture Board (IAB). It's info the IAB decided was worth keeping on the permanent record, and it represents the consensus of the IAB. Documents the IAB approves for publication aren't a candidate for any level of Internet Standard. See Section 2 of RFC 7841 for the details 🔥

Info about the current status of this document, any errata, and how to give feedback on it lives at https://www.rfc-editor.org/info/rfc8240 🌈

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved ©️

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document. Don't skip it, bestie 🫶

Table of Contents 🧚

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   5
   3.  Requirements and Questions Raised . . . . . . . . . . . . . .   6
   4.  Authorizing a Software/Firmware Update  . . . . . . . . . . .  12
   5.  End-of-Support  . . . . . . . . . . . . . . . . . . . . . . .  13
   6.  Incentives  . . . . . . . . . . . . . . . . . . . . . . . . .  15
   7.  Measurements and Analysis . . . . . . . . . . . . . . . . . .  15
   8.  Firmware Distribution in Mesh Networks  . . . . . . . . . . .  16
   9.  Compromised Devices . . . . . . . . . . . . . . . . . . . . .  17
   10. Miscellaneous Points  . . . . . . . . . . . . . . . . . . . .  17
   11. Tentative Conclusions and Next Steps  . . . . . . . . . . . .  19
   12. Security Considerations . . . . . . . . . . . . . . . . . . .  20
   13. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  20
   14. Informative References  . . . . . . . . . . . . . . . . . . .  20
   Appendix A.  Program Committee  . . . . . . . . . . . . . . . . .  24
   Appendix B.  Accepted Position Papers . . . . . . . . . . . . . .  24
   Appendix C.  List of Participants . . . . . . . . . . . . . . . .  26
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  27
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  27

1. Introduction ✨

This doc sums up the Internet of Things Software Update (IoTSU) Workshop [IoTSU], which happened at Trinity College Dublin, Ireland, on the 13th and 14th of June, 2016. The main goal was to get a discussion going on requirements, challenges, and solutions for bringing software and firmware updates to IoT devices 💖

The views and positions in this report belong to the workshop participants. They don't necessarily reflect those of their employers or sponsors, the authors of this memo, or the Internet Architecture Board (IAB), under whose auspices the workshop was held 🙈

The IAB runs occasional workshops to think about long-term issues and strategies for the Internet, and to suggest where the Internet architecture should go next 🔭 The topics often need different organizations and industry bodies to work together on a problem. A big part of these workshops is helping relevant organizations, companies, and universities talk to each other, especially when a topic is partly outside the scope of the Internet Engineering Task Force (IETF). This long-term planning work from the IAB is complementary to the ongoing engineering work done by IETF working groups 🧠

In his essay "The Internet of Things Is Wildly Insecure -- And Often Unpatchable" [BS14], Bruce Schneier raised concerns about software and firmware updates for IoT devices 😭 IoT devices have a reputation for being insecure from the moment they're manufactured. They're often expected to stay active in the field for 10 or more years, and to run unattended with Internet connectivity 💀

Security experts recommend including a software update mechanism to fix vulnerabilities, update configuration settings, and add new functionality too 🔐 But software updates come with challenges, which are documented in the United States Federal Trade Commission (FTC) report titled "internet of things: Privacy & Security in a Connected World" [FTC], and in the Article 29 Data Protection Working Party document "Opinion 8/2014 on the on [sic] Recent Developments on the Internet of Things" [WP29].

Some of the challenges in designing a basic software/firmware update function are:

  • Implementations of software update mechanisms may contain vulnerabilities, which makes them an attractive attack target. See, for example, [OS14].
  • Operational challenges, like the case of an expired certificate in a hub device [BB14].
  • Privacy issues if devices "call home" often to check for updates 👀
  • A lack of incentives to distribute software updates along the value chain 💸
  • Questions such as: Who should be able to update device software after normal support stops? When should an alternate source of software updates take over? 🤔

There are lots of software update mechanisms out there today, and many of them are proprietary. How much they can do depends a lot on what the IoT device is being used for 🛠️ More powerful IoT devices, like ones running general purpose operating systems (such as Linux), can use sophisticated software update mechanisms from the desktop and mobile worlds. This workshop focused on more constrained IoT devices, which often run dedicated real-time operating systems or possibly no operating system at all 🤖

There's a real risk that many IoT devices will keep shipping without a solid software/firmware update mechanism in place 😬 Ideally, IoT software developers and product designers should be able to plug in standardized mechanisms that have had substantial review and where the documentation is available to the public 📖

So the IAB decided to host a workshop to reach out to relevant stakeholders, explore the state of the art, and identify requirements and gaps 🫧 In particular, the call for position papers asked for:

  • Protocol mechanisms for distributing software updates.
  • Mechanisms for securing software updates.
  • Metadata about software/firmware packages.
  • Implications of operating system and hardware design on the software update mechanisms.
  • Installation of software updates (in context of software and hardware security of IoT devices).
  • Privacy implications of software update mechanisms.
  • Implications of device ownership and control for software update.

The rest of the doc is laid out like this: basic terminology is in Section 2, followed by a longer section on requirements. Later sections dig into topics like incentives and measurements. Honestly, most of the write-up raises more questions than it answers 🙃 We still tried to pull together some possible conclusions and offer a few next steps 🎀

2. Terminology 🧚

As is typical with people from different backgrounds, the workshop participants kicked things off by talking through terminology. This section is meant to reflect those discussions rather than present canonical definitions of terms 📝

Device Classes: IoT devices come in different "sizes," like how much RAM or flash memory they have. Because of that, devices are limited in what they can support in terms of operating-system features, cryptographic algorithms, and protocol stacks. So the group split devices into two classes: ARM Cortex A-class/Intel Atom and Cortex M-class/Intel Quark types of devices 🧩 A-class devices have powerful processors, typically found in set-top boxes and home routers. The Raspberry Pi is an example of an A-class device that can run a regular desktop operating system, like Linux. There are differences between the Intel and ARM-based CPUs in terms of architecture, microcode, and who is allowed to update a Basic Input/Output System (BIOS) (if one is available). A detailed discussion of these hardware architectural differences was, however, outside the scope of the workshop 🙅‍♀️ The implication is that lower-end microcontrollers have constraints that limit how much software can be put on them. It's easy to require support for a wide range of features, but those may not necessarily fit on these devices 🥲

Software Update and Firmware Update: The group noticed that regular operating systems come with sophisticated software update mechanisms, such as Red Hat Package Manager (RPM) [RPM] or pacman [PACMAN]. These use the operating system to install and run each application in a compartmentalized way 📦 Firmware updates usually don't offer that kind of fine-grained control. Instead, they distribute the entire binary image, which includes the (often minimalistic) operating system and all applications. The line between what A-class and M-class devices use may get fuzzier over time, but most M-class devices use firmware updates, while A-class devices use a mix of firmware and software updates (with firmware updates happening less often) 🔄

Hitless Update: A hitless update means the user experience is not "hit," i.e., it isn't impacted. It's possible to impact the user experience when applying an update even if the device doesn't reboot (to obtain or apply the update). If the update is applied while a user isn't using a product and their service isn't impacted, the update is "hitless" 🌙

3. Requirements and Questions Raised 💅

Workshop participants discussed requirements, and several of these raised further questions. Just like in the previous section, we aim to present the discussion as it was 🎀

okay bestie, so there may be a need to support partial (differential) updates 🧩 where the whole firmware image doesn't have to be sent. That could mean using techniques like bsdiff [BSDIFF] and courgette [COURGETTE], but it might also mean devices that can download just applications and libraries on their own. That last part may need dynamic linking and position independent code. It was unclear whether position independent code should be recommended for low-end IoT devices 🤔

  • the relative importance of dynamic linkers for low-end IoT devices is unclear 🔗 some operating systems used with M-class devices, like Contiki, support a dynamic linker according to [OS-Support]. this could help minimize the amount of data sent during updates, since only the modified application or library needs to be transmitted 💖
  • how should dependencies among software updates be handled? these dependencies may include info about the hardware platform and configuration, plus other software components running on a system 🧠 for firmware updates, the dependency problem is often solved by the manufacturer or Original Equipment Manufacturer (OEM) instead of on the device itself 🏭
  • devices with multiple microcontrollers may need an architecture where one microcontroller talks to the update service and then sends software images to the attached microcontrollers within its local realm 🎀 letting each microcontroller talk to an update service seemed way less practical 🙅‍♀️
  • support may be required for devices with multiple owners/stakeholders, which raises the question of who is authorized to push a firmware/software update 👑
  • data origin authentication (DAO) was agreed to be required for software updates 🔐 without DAO, updates just become a perfect vulnerability 💀 it's not easy, though, to make sure the DAO mechanism actually models the real trust relationships that exist 🥲 for some devices and deployment scenarios, any DAO mechanism is onerous, possibly to the point where it may be hard to convince a device maker to include it 😭
  • should digital signatures and encryption for software updates be recommended as a best current practice? this especially raises the question of symmetric key cryptography, since not all low-end IoT devices currently use asymmetric crypto 🔑
  • DAO is most commonly provided through digital signature mechanisms 🖋️ symmetric schemes could also be developed, but IETF discussion of such mechanisms (for purposes less sensitive than software update) has proved significantly controversial 🙈 the main problem seems to be that simple symmetric schemes only ensure the sender is a member of a group, and they don't fully authenticate a specific sender 🫣 and with a software update, we do not want any (possibly compromised) device to be able to authenticate new software for all other similar devices 🚫
  • what are the firmware update signing key requirements? devices have a rather long lifetime, so there has to be a way to change the signing key during the lifetime of the device 🔄
  • should a firmware update mechanism support multiple signatures of firmware images? multiple signatures come in two different flavors, namely:
    • a single firmware image may be signed by multiple different parties 🧾 imagine an OEM signing the software it creates, and then the software is signed again by the enterprise that approves its distribution within the company 🏢 other examples include regulatory signatures, where software for a medical device may be signed as approved by a certification body 🩺
    • a software image may contain libraries that are each signed by their developers 📚

is a device expected to verify the different types of signatures, or is this a service provided by some unconstrained device? this raises questions about who the IoT device should trust for what, and whether transitive trust is acceptable for some types of devices 🤝

  • are applications from a range of sources allowed to run on a device, or only those from the OEM? if the device is a "closed" device that only supports/runs software from the OEM, then a single signature may be enough ✨ in a system that is more "open", third-party applications may require support of multiple signatures 🌈
  • there is a need for some form of secure storage, at least for IoT devices that are exposed to physical attacks 🛡️ this includes at least protecting the integrity of the public key of the update service on the device (if signature-based DAO is in use) 🗝️ using symmetric key cryptography requires improved confidentiality protection, in addition to integrity protection 💜
  • is there a need to let the update infrastructure side authenticate the IoT device before distributing an update? questions about the identifier used for that authentication were raised 🪪 the idea of reusing Media Access Control (MAC) addresses led to concerns about the significant privacy implications of identifier reuse 👀
  • it is important to minimize device/service downtime due to update processing, and to minimize user interaction (e.g., a car should not distract the driver) 🚗 (see "Hitless Update" in Section 2). while it may not be possible to avoid all downtime, there was agreement that one ought to strive for "no inappropriate" device downtime 🙌 this means minimal downtime impacting the user/operation of the device 💫 the definition of "downtime" also depends on the use case: with a smart light bulb, the device could be "up" if the light is still on, even if some advanced services are unavailable for a short time 💡 whether an update can be done without rebooting the device depends on the software being installed, on the OS architecture, and potentially even on the hardware architecture 🛠️ the cost/benefit ratio also plays a role 🧮
  • it is desirable to minimize the time taken from the start of the update to when it is finished ⏱️ in some systems with many devices (e.g., industrial lighting), this can be a challenge if updates need to be unicasted 💡
  • in some systems with multiple devices, it can be a challenge to make sure all devices are at the same release level, especially if some devices are sleepy 😴 there are some systems where having all relevant devices at the same release level is a hard requirement 🧱 in other cases, it's fine if devices converge much more slowly to the current release level 🐢
  • it ought not be possible for a factory worker to compromise the update process (e.g., copy signing keys or install unauthorized public keys/trust anchors) during manufacturing 🏭 there are typically two factories involved: the first produces microcontrollers and other components, and the second produces the complete product, such as a fridge 🧊 this fridge contains many of the components made earlier, so the firmware of components produced in the first stage may be six months old when the fridge leaves the factory 📦 one does not want to install a firmware update when the fridge boots the first time, so for that to happen, the firmware update already happens at the end of the manufacturing process 🎉
  • should devices have a recovery procedure when the device gets compromised? how is the compromise detected? 🚨
  • there was a bit of discussion about why it matters for IoT devices to know the current time, for the purpose of checking certificate validity ⏰ for example, what does "real-time clock" (RTC) actually mean? and what counts as "good enough" time? 🤷‍♀️ there are cost, power, size, and environmental constraints, though, that can make adding a real-time clock to an IoT device complicated 😩
    • Cost: battery- or supercap-backed RTC modules might cost several times the rest of the bill of materials 💸
    • Size: the battery and other components are often several times larger than the rest of the material 📏
    • Manufacturing: some modules require an extra assembly step, because the battery could be damaged or explode at high temperatures during the reflow process 🔥
    • Supply chain: devices with fitted batteries need additional supply-chain management to account for storage temperature and to avoid shipping aged devices 🚚
    • Environmental: real-time-clock modules are typically not rated for industrial temperature ranges, and those that are have extremely reduced lifetime at high temperatures 🌡️
    • Lifetime: some of these modules last only a few years at the top of their environmental range 🥀

while a good solution is needed, it's not clear whether there is one true solution 🌸 a recent proposal from Google called "Roughtime" [RT] may be worth exploring 💅

  • how do devices learn about a firmware update? push or pull? what should the required functionality be for a firmware update protocol? 📬
  • there is a need to find out whether a software update was successful ✅ in one discussed solution, the bootloader analyzes the performance of the running image to decide which image to run (rather than just verifying the integrity of the received image) 🧐 one of the key criteria is that the updated system is able to connect to the device management/software update infrastructure 🔌 as long as it can talk to the update infrastructure, it can receive another update 🔁 as an alternative perspective, the argument was made that one needs a way to update the system without having the full system running 🦋
  • gateway requirements 🌉 in some deployments, gateways terminate the IP-based protocol communication and use non-IP mechanisms to talk to other microcontrollers, for example within a car 🚙 the gateway in such a system is the endpoint of the IP communication 🎯 the group had mixed feelings about using gateways versus using IP communication to every microcontroller 🤔 participants argued that there is a lack of awareness of IPv6 header compression (with the IPv6 over Low-Power Wireless Personal Area Network (6LoWPAN) standards), and of the possible benefits of IPv6 in those environments for lowering the complexity of the overall system 🧚
  • the amount of energy consumed by software update needs to be minimized ⚡ for example, waking a sleepy device regularly only to check for new software would seem wasteful if the device cannot feasibly be used while asleep 😴 however, the trade-off is that once the device wakes up with old software, there may be a window of vulnerability if some relevant exploit has been discovered 😱
  • The amount of storage required for an update ought to be minimized, and sometimes it can be significant. 💾 But there are also perks to schemes that store two or three different software images for robustness. 🛡️ If you have space for a current last-known-good image and a being-updated image, devices can better survive the buggy occasional updates that are also inevitable 😭

Okay so which of the features in the list above are nice to have? 🤔 Which are required? Not all of these are required to achieve improvement. Which are most important? 💅

Among the participants, there was consensus that supporting signatures (for integrity and authentication) of the firmware image itself, and the need for partial updates, were seen as important. 🔥

Still, there were concerns about performance, since certain device categories may not use public key cryptography at all 🥺 So only a symmetric key approach seems viable, unless some other scheme, such as a hash-based signature, became practical (they currently aren't, due to signature size). This got people talking about device management infrastructure, similar to Kerberos, that manages keys and distributes them to the appropriate parties 🗝️ In this setup, there could be a unique key shared with the key distribution center, but for use with specific services (such as a software update service), a fresh and unique secret would be distributed. ✨

Infrastructure needs are a whole other vibe too 🏗️ The infrastructure may consist of servers in the local network and/or various servers deployed on the Internet. It may also consist of some application-layer gateways. The potential benefits of having a local server might include:

  • The local server acting for neighboring nodes. For example, in a vehicle, one microcontroller can process all firmware updates and redistribute the relevant parts to interconnected microcontrollers. 🚗
  • Local infrastructure could perform some digital signature checks on behalf of the devices, e.g., certificate-revocation checking. 🔐
  • Local multicast can enable transmission of the same update to many devices. 📡
  • Local servers can hide complexity associated with Network Address Translation (NAT) and firewalls from the device. 🧚

Another point about local infrastructure: since many IoT devices will not be (directly) connected to the Internet, but only through a gateway, there may in any case be a need to develop a software/firmware update mechanism that works in environments where no end-to-end Internet connectivity exists. 🌸

Some current firmware update schemes need to identify devices, and there are different design approaches for that. 🪪

  • In an extreme form in one case, the decision about updating a device is made by the infrastructure based on the unique device identification. The operator of the firmware update infrastructure knows about the hardware and software requirements for the IoT devices, knows about the policy for updating the device, etc. The device itself is provisioned with credentials so that it can verify a firmware update coming from an authorized device. 👑
  • In another extreme, the device has knowledge about the software and hardware configuration and possible dependencies. It consults software repositories to obtain the software packages that are most appropriate. Verifying the authenticity of the software packages/firmware images will still be required. 🦋

So in some deployed software update mechanisms, there is no desire for the device to be identified beyond the need to exchange information about the most recent software versions. For other devices, it is seen as important to identify the device itself, in order to provide the appropriate firmware image/software packages. 💜

Related to device identification, various privacy concerns arise, such as the need to determine what information is provided to whom, and the uses to which this information is put. 🙈 For IoT devices where there is a close relationship to an individual (see [RFC6973]), privacy concerns are likely higher than for devices where such a relationship does not exist (e.g., a sensor measuring concrete). The software/firmware update mechanism should, however, not make the privacy situation of IoT devices worse. The proposal from the group was to introduce a minimal requirement of not sending any new identifiers over an unencrypted channel as part of an update protocol. 🔒

Software updates will also be another venue where the tension between those advocating better privacy and those seeking to monetize information plays out. 💀 It is in the nature of software update that it requires devices to sometimes "call home", and those interactions provide fertile ground for monetization.

4. Authorizing a Software/Firmware Update ✨

There were quite a few points revolving around authorization:

  • Who can accept or reject an update? Is it the owner of the device, the user, or both? The user may not necessarily be the owner. 🤷‍♀️
  • With products that fall under a regulatory structure, such as healthcare, you don't want firmware other than what has been accredited. 🏥
  • In some cases, it will be very difficult for a firmware update system to communicate to users that an update is available. Doing so may require tracking the device and its status with regard to the installed firmware/software, with all the privacy downsides if such tracking is badly done. 👀
  • Not all updates are the same. Security updates are often treated differently compared to feature updates, and the authorization for these may differ. 🛡️
  • Some people may choose to decline updates, often on the basis that their system is currently stable, but also possibly due to concerns about unwanted changes, such as the HP printer firmware update pushed in March 2016 [HP-Firmware] that turned off features that end users liked. 🖨️😤

5. End-of-Support 🪦

There was quite a bit of discussion about end-of-support for products/devices and how to handle it. 💬

  • How should end-of-support or end-of-features be treated? Devices are often deployed for 10+ years (or even longer in some verticals). Device makers may not want or be able to support software and services for such an extended period of time. Will these devices stop working after a certain, previously unannounced period of time, such as Eye-Fi cards [EYEFI]? 😱
  • There will be a broad range of device makers involved in IoT, who may differ substantially in how well they can handle the full device life cycle. Some will be large commercial enterprises that are used to dealing with long device lifetimes, whereas others may be very small commercial entities where the device lifetime may be longer than the company lifetime. Yet other devices may be the result of open-source activities that prosper or flounder. The problem of end-of-support arises in all these cases, though feasible solutions for software update may substantially differ. In some cases, device makers may not be willing to continue to update devices, for example, due to a change in business strategy caused by a merger. In yet other cases, a company may have gone bankrupt. 🥀
  • While there are many legal, ethical, and business-related questions, can we technically enable transfer of device service to another provider? Could there even be business models for entities that take over device updates for original device makers that no longer wish to handle software update? 🤔
  • The release of code, as it was done with the Little Printer manufactured and developed by a company called "Berg" [LittlePrinter], could provide a useful example. While the community took over support in that case, this can hardly be assumed in all cases. Just releasing the source code for a device will not necessarily motivate others to work on the code, to fix bugs, or to maintain a service. Nevertheless, escrowing code so that the community can take it over if a company fails is one possible option. 🖨️🫶
  • The situation gets more complex when the device has security mechanisms to ensure that only selected parties are allowed to update the device (which is really a basic requirement for any secure software update). In this case, private signing keys (or similar) may need to be made available as well, which could introduce security problems for already-deployed software. In the best case, it changes assumptions made about the trust model and about who can submit updates. 🔑
  • How should deployed devices behave when they are end-of-support and support ends? Many of them may still function normally, but others may fail due to the absence of cloud infrastructure services. Some products are probably expected to fail safely, similarly to a smoke alarm that makes a loud noise when the battery becomes empty. Cell phones without a contract can, in some countries, still be used for emergency services (although at the expense of society due to untraceable hoax calls), as discussed in RFC 7406 [RFC7406]. 📱🚨

The recommendation that can be provided to device makers and users is to think about the end-of-support and end-of-support scenarios ahead of time and plan for those. 🗓️ While device makers rarely want to consider what happens if their business fails, it is definitely legitimate to consider scenarios where they are hugely successful and want to evolve a product line instead of supporting previously sold products forever. 🚀 Maybe there is also value in subscription-based models, where product and device support is only provided as long as the subscription is paid. Without a subscription, the product is deactivated and cannot pose a threat to the Internet at large. 🛑

6. Incentives 💰

Workshop participants also discussed how to create incentives for companies to ship software updates, which is particularly important for products that will be deployed in the market for a long time. It is further complicated by complex value chains. 🔗

  • Companies shipping software updates benefit from improved security. Their devices are less likely to be abused as a vector to launch other attacks, whether on their own networks or (as part of a botnet) on other Internet hosts. This clearly creates an incentive to support and use software updates. 🙌
  • On the other hand, updates can also break things. The negative customer experience can be due to service interruptions during or after the update process, but it can also result from bad experiences with deliberate changes introduced as part of an update -- such as a feature that is not available anymore, or a "bug" that another service has relied upon being fixed. 😩
  • For most classes of device, there does not seem to be a regulatory requirement to report or fix vulnerabilities, similar to data-breach-notification laws. 🫠
  • Subscription models for device management were suggested so that companies providing the service have an economic interest in keeping devices online (and updated for that). 🌈

7. Measurements and Analysis ✨

Okay so, from a security point of view, it is SO important to know what devices are out there and what software version they're running 🔍. One workshop paper [PLONKA] reported measurements on buggy devices first shipped in 2003, and those devices were still showing up in significant numbers just before the workshop, 13 years later 😭. That's why, on top of the firmware update mechanism, companies have been offering device management solutions so OEMs can keep track of their devices 🎀. Tracking these devices and their status is still really hard, though, because some devices only connect now and then, or only get turned on when they're needed. Think of a hockey alarm that's only switched on before a match 🏒.

Lots of stakeholders have a totally valid reason to want to know stuff about deployed devices. For example:

  • Manufacturers and other players in the supply chain want to know what devices are out there, how many have been sold, and what devices are out there but have not been sold 📦. This helps them figure out which firmware versions to support, and for how long.
  • Device users, owners, and customers may want to know what devices have been installed over a longer period of time, what software or firmware version each device is running, what the uptime of each one is, what types of faults have happened, and so on 🧚. Forgotten devices can cause problems, especially if they (have the potential to) behave badly 💀.
  • Network operators offering services to device owners and other actors may also need similar info, for example, to control botnets 🤖.
  • Researchers studying the state of the Internet ecosystem (like what protocols are being used, how much data IoT devices generate, etc.) need measurements for their work 📊.

Getting these measurements can get pretty invasive, ngl 🙈. So the challenge was put forward to find ways to build measurement infrastructures that are privacy preserving 🫶. Arnar Birgisson pointed out that there are privacy-preserving statistical techniques, such as RAPPOR [RAPPOR]. Ned Smith added that techniques like Intel's Enhanced Privacy ID (EPID) may play a role in keeping some level of anonymity for IoT device (owners) while still enabling measurement 🌸. It was pretty clear that naive approaches to measurement (e.g., where devices are willing to expose a unique identifier to anyone who asks) are unlikely to be sufficient 🙅‍♀️.

8. Firmware Distribution in Mesh Networks 🕸️

There was some discussion about the requirements for mesh-based networks, mainly around industrial lighting 💡. In these networks, software updates can impose unacceptable performance burdens, especially when there are many devices and some of them are sleepy 😴.

The workshop talked about whether some forms of multicast (maybe not IP multicast) would be needed to provide acceptable solutions for software updates in these cases 💜. It wasn't clear at which layer a multicast solution might actually be effective. There also didn't seem to be any clearly applicable standards-based approach available at the time of the workshop 🤷‍♀️.

9. Compromised Devices 🚨

There was recognition that there are, and maybe always will be, large numbers of devices that can be, or have been, compromised 😩. Updating them can help with the problems, but there will always be new devices added to networks that cannot be updated (for various reasons). So the group discussed what, if anything, to do about compromised devices 🧐.

  • There may be value in being able to single out a device that shows faulty behavior or has been compromised, and shut it down in some sense 🔒.
  • Prior IETF work on Network Endpoint Assessment (NEA) [NEA] allowed assessing the "posture" of devices. Posture refers to the hardware or software configuration of a device, and may include knowing that the installed software is up to date ✅. The information obtained can then be used by some network infrastructure to create a quarantined region network around the device 🚧.
  • RFC 6561 [RFC6561] describes one scheme for an ISP to send "signals" to customers about hosts (usually ones that are part of a botnet or generating spam) in their home network 📡.
  • Neither RFC 6561 nor NEA has found widespread deployment 😬. Whether such mechanisms can be more successful in the IoT environment has yet to be studied 🔬.

The conclusion of the discussion at the workshop itself was that there is some interest in identifying and stopping misbehaving devices, but the actual solution mechanisms are unclear 🌫️.

10. Miscellaneous Points 🗂️

There were a bunch of points discussed at the workshop that don't neatly fit under the headings above, but they're still worth writing down 📝. Those include:

  • Complex questions can come up when you think about the impact of not updating on other devices, other people, or the public in general 🌍. If I don't update my device, and it gets used to attack a random host on the Internet, but at no apparent cost to me, then what incentive do I have to do the updates that would have protected that random host? And what incentive does my device's vendor have to have provided those updates in advance? An example of this can be found in DDoS attacks from IoT devices, such as printers [SNMP-DDOS] and cameras [DDOS-KREBS] 📷🖨️.
  • With some IoT devices, there are many stakeholders contributing to the end product (e.g., contributing different subsystems) 🧩. Making sure vulnerabilities get fixed and software or firmware updates get communicated through the value chain is known to be difficult, as demonstrated in [OS14] 😤.
  • What about forgotten devices? There are many of them, and there will be more 🕸️. Even though they're forgotten, such devices may be useless consumers of electricity, or they may be part of some critical system ⚡.
  • Can we determine whether an update impacts other devices on the Internet? Updates to one device can have unintended impact on other devices that depend on it, and this can have cascading effects if we aren't careful 🌊. Changing the output format of a sensor could cause cascading impacts, e.g., if some actuator reacts to the presence or absence of that sensor's data 🎢.
  • How should a device behave when it's running out-of-date software? The example of a smoke alarm was mentioned 🔥. We don't want 100 devices in a living room to start beeping when their batteries run low or when they can't communicate with the cloud 🙉. But are devices supposed to simply stop working? 😭
  • The IETF has published a specification that uses the Cryptographic Message Syntax (CMS) to protect firmware packages, as described in RFC 4108 [RFC4108]. It also contains metadata describing the firmware image itself 📜. During the workshop, the question was raised whether a post-quantum secure solution will be needed in the future 🔐. A post-quantum cryptosystem is one that's secure against quantum computers with more than a trivial number of quantum bits 🧪. It's open to conjecture whether it's feasible to build such a machine, but current signature algorithms are known not to be post-quantum secure 👀. Building that would mean introducing technologies like the Hash-based Merkle Tree Signature (MTS) [HOUSLEY], which was presented and discussed at the workshop 🌳. The downsides of these solutions are their novelty and, for these use cases, the fairly large signature or key sizes involved. For example, depending on the parameters, a signature could easily be 5-10 KiB [HASHSIG] [XMSS] 😵. It's likely that post-quantum secure signature algorithms will be needed for software updates at some point, but such algorithms may be needed sooner for services that require long-term confidentiality (e.g., using Transport Layer Security (TLS)) 🔑. So it wasn't clear that this application would be a first-mover in terms of post-quantum security 🏃‍♀️.
  • Many devices that use certificates don't check the revocation status of those certificates 🙃. Extensions like Online Certificate Status Protocol (OCSP) stapling exist [RFC6961], and they're increasingly deployed with Web browsers 🌐. The workshop participants did not reach a conclusion on the recommendations for certificate revocation checking, although its importance has been recognized 🧾. The reluctance around deploying certificate revocation deserves further investigation 🕵️‍♀️.

11. Tentative Conclusions and Next Steps 🗺️

The workshop participants discussed some tentative conclusions and possible next steps 💭:

  • There was strong agreement that having some standardized secure (authorized and authenticated) software update would be an improvement over having none 🙌.
  • It would be valuable to find agreement on the right scope for a standardized software/firmware update mechanism 🎯. It's not clear that an entire update system can or should be standardized, but there may be some aspects where standards would be beneficial, e.g., (meta-)data formats and/or protocols for distributing firmware updates 📦. More discussion is needed to identify which parts of the problem space could benefit from standardization 💬.
  • It will be useful to investigate ways to install updates with no operational interruption, as well as ways to distribute software updates without disrupting network operations (specifically in low-power wireless networks) 🔋. That includes developing a multicast transfer mechanism (with appropriate security) 🛡️.
  • There will almost certainly be a need for a way to transfer authority/responsibility for updates, particularly considering end-of-support cases 🫠. This is very close to calling for a standard way to "root" devices as a feature of all devices 👑.
  • We would benefit from documentation of proofs-of-concept of software/firmware updates for constrained devices on different operating system architectures 🧑‍💻. The IETF Light-Weight Implementation Guidance (lwig) Working Group may be a good venue for such documents 🌈.

12. Security Considerations 🚨🔐

This is the part where things get serious, bestie 😭. This document summarizes an IAB workshop on software/firmware updates, and the entire content is, therefore, security related 💀. Standardizing and deploying a software/firmware update mechanism for use with IoT devices could help fix security vulnerabilities faster and, in some cases, be the only way to get a vulnerability patched at all 🛠️🔥.

13. IANA Considerations 🎀

This document does not require any IANA actions 💖.

14. Informative References ✨

okay bestie, this is the reading list 📚💅 Everything here is a informative ref, so it's stuff the doc leans on for background. Still, every entry is listed in full, with the tag, the authors, the title, the date and the link. 🦋

Appendix A. Program Committee 💖

The following individuals helped to organize the workshop: Jari Arkko, Arnar Birgisson, Carsten Bormann, Stephen Farrell, Russ Housley, Ned Smith, Robert Sparks, and Hannes Tschofenig. The whole dream team, honestly 🌟

Appendix B. Accepted Position Papers 📝

The list of accepted position papers is below. Links to these, and to the workshop agenda and raw minutes are accessible at: <https://down.dsg.cs.tcd.ie/iotsu/>.

  • R. Housley, "Position Paper for Internet of Things Software Update Workshop (IoTSU)"
  • D. Thomas and A. Beresford, "Incentivising software updates"
  • L. Zappaterra and E. Dijk, "Software Updates for Wireless Connected Lighting Systems: requirements, challenges and recommendations"
  • M. Orehek and A. Zugenmaier, "Updates in IoT are more than just one iota"
  • D. Plonka and E. Boschi, "The Internet of Things Old and Unmanaged"
  • D. Bosschaert, "Using OSGi for an extensible, updatable and future proof IoT"
  • A. Padilla, E. Baccelli, T. Eichinger, and K. Schleiser, "The Future of IoT Software Must be Updated"
  • T. Hardie, "Software Update in a multi-system Internet of Things"
  • R. Sparks and B. Campbell, "Avoiding the Obsolete-Thing Event Horizon"
  • J. Karkov, "SW update for Long lived products"
  • S. Farrell, "Some Software Update Requirements"
  • S. Chakrabarti, "Internet Of Things Software Update Challenges: Ownership, Software Security & Services"
  • M. Kovatsch, A. Scholz, and J. Hund, "Why Software Updates Are More Than a Security Issue: Challenges for IETF CoRE and the W3C Web of Things"
  • A. Grau, "Secure Software Updates for IoT Devices"
  • Birr-Pixton, "Electric Imp's experiences of upgrading half a million embedded devices"
  • Y. Zhang, J. Yin, C. Groves, and M. Patel, "oneM2M device management and software/firmware update"
  • E. Smith, M. Stitt, R. Ensink, and K. Jager, "User Experience (UX) Centric IoT Software Update"
  • J.-P. Fassino, E.A. Moktad, and J.-M. Brun, "Secure Firmware Update in Schneider Electric IOT-enabled offers"
  • M. Orehek, "Summary of existing firmware update strategies for deeply embedded systems"
  • N. Smith, "Toward A Common Modeling Standard for Software Update and IoT Objects"
  • S. Schmidt, M. Tausig, M. Hudler, and G. Simhandl, "Secure Firmware Update Over the Air in the Internet of Things Focusing on Flexibility and Feasibility"
  • A. Adomnicai, J. Fournier, L. Masson, and A. Tria, "How careful should we be when implementing cryptography for software update mechanisms in the IoT?"
  • V. Prevelakis and H. Hamad, "Controlling Change via Policy Contracts"
  • H. Birkholz, N. Cam-Winget, and C. Bormann, "IoT Software Updates need Security Automation"
  • R. Bisewski, "Comparative Analysis of Distributed Repository Update Methodology and How CoAP-like Update Methods Could Alleviate Internet Strain for Devices that Constitute the Internet of Things"
  • J. Arrko, "Architectural Considerations with Smart Objects and Software Updates"
  • J. Jimenez and M. Ocak, "Software Update Experiences for IoT"
  • H. Tschofenig, "Software and Firmware Updates with the OMA LWM2M Protocol"

Appendix C. List of Participants 🧚

  • Arnar Birgisson, Google
  • Alan Grau, IconLabs
  • Alexandre Adomnicai, Trusted Objects
  • Alf Zugenmaier, Munich University of Applied Science
  • Ben Campbell, Oracle
  • Carsten Bormann, TZI University Bremen
  • Daniel Thomas, University of Cambridge
  • David Bosschaert, Adobe
  • David Malone, Maynooth University
  • David Plonka, Akamai
  • Doug Leith, Trinity College Dublin
  • Emmanuel Baccelli, Inria
  • Eric Smith, SpinDance
  • Jean-Philippe Fassino, Schneider Electric
  • Joergen Karkov, Grundfos
  • Jonathon Dukes, Trinity College Dublin
  • Joseph Birr-Pixton, Electric Imp
  • Kaspar Schleiser, Freie Universitaet Berlin
  • Luca Zappaterra, Philips Lighting Research
  • Martin Orehek, Munich University of Applied Science
  • Mathias Tausig, FH Campus Wien
  • Matthias Kovatsch, Siemens
  • Milan Patel, Huawei
  • Ned Smith, Intel
  • Robert Ensink, SpinDance
  • Robert Sparks, Oracle
  • Russ Housley, Vigil Security
  • Samita Chakrabarti, Ericsson
  • Stephen Farrell, Trinity College Dublin
  • Vassilis Prevelakis, TU Braunschweig
  • Hannes Tschofenig, ARM Ltd.

Acknowledgements 🫶

We would like to thank all paper authors and participants for their contributions. The IoTSU workshop is co-sponsored by the Internet Architecture Board and the Science Foundation Ireland funded CONNECT Centre for future networks and communications. The program committee would like to express their thanks to Comcast for sponsoring the social event. Such a good crew 🌈

Authors' Addresses 🦋

Hannes Tschofenig
ARM Limited

Email: hannes.tschofenig@gmx.net


Stephen Farrell
Trinity College Dublin
Dublin  2
Ireland

Phone: +353-1-896-2354
Email: stephen.farrell@cs.tcd.ie
📦 where this page comes from