SWUPDATE(1)
💖 SWUPDATE(1) 💖 — Embedded Software Update Documentation
okay bestie, welcome to the swupdate manual page, literally the glow-up your embedded system has been waiting for 👑✨
✨ NAME ✨
swupdate — SWUpdate Documentation [image]
SWUpdate gives you a reliable way to update the software on an embedded system. The sources live at https://github.com/sbabic/swupdate. And no pressure or anything, but do not forget to star SWUpdate ⭐ fr fr.
💖 SOFTWARE MANAGEMENT ON EMBEDDED SYSTEMS 💖
so listen. As Embedded Systems get more and more complex, their software is out here reflecting all that extra complexity 🦋. Which means it is vital that the software on an embedded system can be updated in an absolutely reliable way, because new features and fixes keep getting added, periodt.
On a Linux-based system, most of the time you'll find these elements:
- the boot loader 🥾
- the kernel and the DT (Device Tree) file 🌳
- the root file system 📁
- other file systems, mounted at a later point 🗂️
- customer data, in raw format or on a file system 💾
- application specific software. For example, firmware to be downloaded on connected micro-controllers, and so on 🔌
Generally speaking, in most cases you need to update the kernel and root file system while preserving user data — but honestly, cases vary 🙈.
In only a few cases is it required to update the boot loader too. And ngl, updating the boot loader is quite always risky, because a failure in the update breaks the board 💀. Restoring a broken board is possible in some cases, but in most cases this is not left to the end user, and the system has to be sent back to the manufacturer 😭.
There are a LOT of different concepts about updating the software. I like to expose some of them, and then explain why I have implemented this project 💅.
Updating through the boot loader
Boot loaders do much more than simply start the kernel. They have their own shell and can be managed using a processor's peripheral — in most cases a serial line. They are often script-able, which lets you implement some kind of software update mechanism ✨.
However, I found some drawbacks in this approach, and those are what made me search for another solution, based on an application running on Linux 🐧:
Boot loaders have limited access to peripherals
Not all peripherals supported by the kernel are available with the boot loader. When it makes sense to add support to the kernel — because then the peripheral is available to the main application — it does not always make sense to duplicate the effort to port the driver to the boot loader 🙃.
Boot loader's drivers are not updated
Boot loader's drivers are mostly ported from the Linux kernel, but due to adaptations they are not later fixed or synchronized with the kernel, while bug fixes flow regularly in the Linux kernel. Some peripherals can then work in not-reliable ways, and fixing the issues can be not easy 😬. Drivers in boot loaders are more or less a fork of the respective drivers in the kernel.
As an example, the UBI / UBIFS for NAND devices contains a lot of fixes in the kernel that are not ported back to the boot loaders. Same energy for the USB stack. The effort to support new peripherals or protocols is better spent on the kernel than on the boot loaders 💜.
Reduced file systems
The number of supported file systems is limited, and porting a file system to the boot loader requires high effort 😮💨.
Network support is limited
Network stack is limited — generally an update is possible via UDP but not via TCP 📡.
Interaction with the operator
It is difficult to expose an interface to the operator, like a GUI with a browser or on a display 🖥️.
A complex logic can be easier to implement inside an application than in the boot loader. Extending the boot loader becomes complicated because the whole range of services and libraries are not available 😵💫.
Boot loader's update advantages
That said, this approach has some advantages too, bestie 🫶:
- software for update is generally simpler. - smaller footprint: a stand-alone application only for software management requires an own kernel and a root file system. Even if their size can be trimmed by dropping what is not required for updating the software, their size is not negligible 🌸
Updating through a package manager
All Linux distributions are updating with a package manager. So why is it not suitable for embedded? 🤔
I cannot say it cannot be used, but there is an important drawback to using this approach. Embedded systems are well tested with a specific software. Using a package manager can put weirdness in the mix, because the software itself is not anymore atomic, but split into a long list of packages 📦📦📦. How can we be assured that an application with library version x.y works, and also with different versions of the same library? How can it be successfully tested? 💀
For a manufacturer, it is generally better to say that a new release of software (well tested by its test engineers) is released, and the new software (or firmware) is available for updating. Splitting into packages can generate nightmare and high effort for the testers 😭.
The ease of replacing single files can speed up the development, but it is a software-versions nightmare at the customer site. If a customer reports a bug, how can it possibly be that software is "version 2.5" when a patch for some files was sent previously to the customer? 🙈
An atomic update is generally a must-have feature for an embedded system ✨.
Strategies for an application doing software upgrade
Instead of using the boot loader, an application can take charge of upgrading the system. The application can use all services provided by the OS 👑. The proposed solution is a stand-alone software that follows customer rules and performs checks to determine if a software is installable, and then installs the software on the desired storage 💾.
The application can detect if the provided new software is suitable for the hardware, and it can also check if the software is released by a verified authority 🔐. The range of features can grow from a small system to a complex one, including the possibility to have pre- and post- install scripts, and so on 🎀.
Different strategies can be used, depending on the system's resources. I am listing some of them 💅.
Double copy with fall-back
If there is enough space on the storage to save two copies of the whole software, it is possible to guarantee that there is always a working copy even if the software update is interrupted or a power off occurs 🔋.
Each copy must contain the kernel, the root file system, and each further component that can be updated. It requires a mechanism to identify which version is running 🧐.
SWUpdate should be inserted in the application software, and the application software will trigger it when an update is required. The duty of SWUpdate is to update the stand-by copy, leaving the running copy of the software untouched 🙅♀️.
A synergy with the boot loader is often necessary, because the boot loader must decide which copy should be started. Again, it must be possible to switch between the two copies. After a reboot, the boot loader decides which copy should run. [image]
Check the chapter about boot loader to see which mechanisms can be implemented to guarantee that the target is not broken after an update 👀.
The most evident drawback is the amount of required space. The available space for each copy is less than half the size of the storage. However, an update is always safe even in case of power off 💖.
This project supports this strategy ✨. The application as part of this project should be installed in the root file system and started or triggered as required. There is no need for an own kernel, because the two copies guarantee that it is always possible to upgrade the not-running copy 🔥.
SWUpdate will set the bootloader's variable to signal that a new image is successfully installed ✅.
Single copy - running as standalone image
The software upgrade application consists of a kernel (maybe reduced by dropping not required drivers) and a small root file system, with the application and its libraries 🧚. The whole size is much less than a single copy of the system software. Depending on the set up, I get sizes from 2.5 until 8 MB for the stand-alone root file system 📏. If the size is very important on small systems, it becomes negligible on systems with a lot of storage or big NANDs 🌈.
The system can be put in "upgrade" mode, simply signaling to the boot loader that the upgrading software must be started 📣. The way can differ — for example setting a boot loader environment or using an external GPIO 🔌.
The boot loader starts "SWUpdate", booting the SWUpdate kernel and the initrd image as root file system 🚀. Because it runs in RAM, it is possible to upgrade the whole storage. Differently as in the double-copy strategy, the systems must reboot to put themselves in update mode 🔄.
This concept consumes less space in storage than having two copies, but it does not guarantee a fall-back without updating again the software 😬. However, it can be guaranteed that the system goes automatically in upgrade mode when the productivity software is not found or corrupted, as well as when the upgrade process is interrupted for some reason 💅. [image]
In fact, it is possible to consider the upgrade procedure as a transaction, and only after the successful upgrade is the new software set as "boot-able" 🎯. With these considerations, an upgrade with this strategy is safe: it is always guaranteed that the system boots and it is ready to get a new software, if the old one is corrupted or cannot run 🫶. With U-Boot as boot loader, SWUpdate is able to manage U-Boot's environment setting variables to indicate the start and the end of a transaction and that the storage contains a valid software 👑. A similar feature for GRUB environment block modification as well as for EFI Boot Guard has been introduced ✨.
SWUpdate is mainly used in this configuration 💖. The recipes for Yocto generate an initrd image containing the SWUpdate application, that is automatically started after mounting the root file system 🌸. [image]
Something went wrong ?
Many things can go wrong, and it must be guaranteed that the system is able to run again and maybe able to reload a new software to fix a damaged image 😭. SWUpdate works together with the boot loader to identify the possible causes of failures 🔍. Currently U-Boot, GRUB, and EFI Boot Guard are supported ✅.
We can at least group some of the common causes:
- damage / corrupted image during installing. SWUpdate is able to recognize it and the update process is interrupted. The old software is preserved and nothing is really copied into the target's storage 💀
- corrupted image in the storage (flash) 🧨
- remote update interrupted due to communication problem 📡
- power-failure 🔋💥
Okay so SWUpdate moves like a whole transaction process, bestie. The boot loader environment variable recovery_status gets set to signal how the update is going to the boot loader. And obvi, you can add more variables on top for fine tuning and for reporting error causes. recovery_status can have the values "progress", "failed", or it can be unset. That's it, that's the whole mood board. 💅
The second SWUpdate starts up, it sets recovery_status to "progress". Then if the update ends in success? The variable gets erased — gone, deleted, no trace, she's ✨healed✨. But if the update ends in an error, recovery_status has the value "failed". 😭
Now if an update gets interrupted — and honestly it does not matter why, could be anything — the boot loader clocks it immediately because recovery_status is sitting there in "progress" or "failed". So the boot loader can then start SWUpdate again to load the software all over again (that's the single-copy case, lowkey stressful) or it can just run the old copy of the application (double-copy case, self-care behavior honestly 🫶).
Power Failure
If a power off happens, you have to guarantee the system can work again — either by starting SWUpdate again or by restoring an old copy of the software. Like, that's non-negotiable, that's the assignment.
Generally the behavior splits according to the scenario you picked:
- single copy: SWUpdate gets interrupted and the update transaction did not end with a success. The boot loader is able to start SWUpdate again, and it has the possibility to update the software again. 🔁
- double copy: SWUpdate did not switch between the stand-by copy and the current copy. The same version of software — the one that was not touched by the update at all — is started again. Untouched, unbothered, thriving. 💅
To be completely safe, SWUpdate and the bootloader need to exchange some information, like they're passing notes in class. The bootloader has to detect if an update was interrupted because of a power-off, and then restart SWUpdate until an update is successful.
SWUpdate supports the U-Boot, GRUB, and EFI Boot Guard bootloaders. U-Boot and EFI Boot Guard both have a power-safe environment that SWUpdate is able to read from and change, so it can communicate with them. In the case of GRUB, a fixed 1024-byte environment block file is used instead. SWUpdate sets a variable as a flag when it starts updating the system, and it resets that same variable after completion. The bootloader can read this flag to check if an update was running before a power-off. [image]
What about upgrading SWUpdate itself ?
So SWUpdate is thought to be used in the whole development process, replacing customized processes to update the software during development. And before going into production, SWUpdate gets well tested for a project. Standards, basically. 🎀
But here's the thing: if SWUpdate itself should be updated, the update cannot be safe if there's only one copy of SWUpdate in the storage. Safe update can only be guaranteed if SWUpdate is duplicated. That's just the deal, no cap. ⚠️
There are some ways to circumvent this issue if SWUpdate is part of the upgraded image:
- have two copies of SWUpdate 👯
- take the risk, but have a rescue procedure using the boot loader 🫣
What about upgrading the Boot loader ?
Updating the boot loader is in most cases a one-way process. On most SOCs there is no possibility to have multiple copies of the boot loader, and when the boot loader is broken, the board does not simply boot. Like, that's it, game over, no respawn. 💀
Some SOCs do allow you to have multiple copies of the boot loader. But again, there is no general solution for this because it is very hardware specific. Hardware-dependent behavior, iykyk.
In my experience, most targets do not allow you to update the boot loader. And it is very uncommon that the boot loader must be updated when the product is ready for production. She's already employed, leave her alone. 💅
It's a whole different story if the U-Boot environment must be updated — that's a common practice, that's normal. U-Boot provides a double copy of the whole environment, and updating the environment from SWUpdate is power-off safe. Other boot loaders can or cannot have this feature. Depends on the bootloader's vibe check, honestly.
💖 LICENSE 💖
SWUpdate is Free Software. It is copyrighted by Stefano Babic and many others who contributed code (see the actual source code and the git commit messages for details). You can redistribute SWUpdate and/or modify it under the terms of version 2 of the GNU General Public License as published by the Free Software Foundation. Some files can also be distributed, at your option, under any later version of the GNU General Public License — see individual files for exceptions. 👑
To make this easier, license headers in the source files will be replaced with a single line reference to Unique License Identifiers as defined by the Linux Foundation's SPDX project [1]. Ideally, the license terms of all files in the source tree should be defined by such License Identifiers; in no case can a file contain more than one such License Identifier list.
We use the SPDX Unique License Identifiers (SPDX-Identifiers) — http://spdx.org/ and http://spdx.org/licenses/ 🦋
Licenses 🎀
Every single one, listed out, because we don't skip rows here:
- GNU General Public License v2.0 — SPDX Identifier:
GPL-2.0-only— OSI Approved: Y —http://www.gnu.org/licenses/gpl-2.0.txtonly - GNU Lesser General Public License v2.1 — SPDX Identifier:
LGPL-2.1-or-later— OSI Approved: Y —http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txtor later - BSD 1-Clause License — SPDX Identifier:
BSD-1-Clause— OSI Approved: Y —http://spdx.org/licenses/BSD-1-Clause - BSD 2-Clause License — SPDX Identifier:
BSD-2-Clause— OSI Approved: Y —http://spdx.org/licenses/BSD-2-Clause - BSD 3-Clause "New" or "Revised" License — SPDX Identifier:
BSD-3-Clause— OSI Approved: Y —http://spdx.org/licenses/BSD-3-Clause - MIT License — SPDX Identifier:
MIT— OSI Approved: Y —https://spdx.org/licenses/MIT.html - Creative Commons Zero 1.0 Universal (CC0) — SPDX Identifier:
CC0-1.0— OSI Approved: N —https://spdx.org/licenses/CC0-1.0.html - Creative Commons Attribution Share Alike 4.0 — SPDX Identifier:
CC-BY-SA-4.0— OSI Approved: Y —https://spdx.org/licenses/CC-BY-SA-4.0.html - ISC License (ISC) — SPDX Identifier:
ISC— OSI Approved: Y —https://spdx.org/licenses/ISC.html
🌈 SWUPDATE: SOFTWARE UPDATE FOR EMBEDDED SYSTEM 🌈
✨ Overview ✨
This project is thought to help update an embedded system from a storage media or from the network. But bestie, honestly? It should mainly be considered as a framework — one where further protocols or installers (in SWUpdate they're called handlers) can be easily added to the application. It's giving platform, it's giving extensible, it's giving main character architecture. 👑
One use case is updating from an external local media, like a USB-Pen or an SD-Card. In this case the update is done with zero intervention by an operator — it's thought of as "one-key-update", and the software just gets started at reset by simply pressing a key (or in any way that can be recognized by the target), and it makes all the checks automatically. And at the end, the whole updating process reports only the status to the operator: successful or failed. That's it. Short king behavior. 💅
The output can be displayed on an LCD using the frame-buffer device, or it can be directed to a serial line (Linux console).
It's generally used in the single copy approach, running in an initrd (recipes are provided to generate with Yocto). ✨ However, it is possible to use it in a double-copy approach by use of Software collections <# collections>. 🦋
If we're doing a remote update, SWUpdate rolls up with its own little embedded Web-server and just... waits for requests 💅 very main character of it. The operator (hey bestie, that's you) uploads a suitable image, SWUpdate checks it, and then installs it. And everything it has to say gets pushed straight to the operator's browser via AJAX notifications — no dead silence, no guessing, we always know what's going on. ✨
🦋 Features 🦋
🌸 General Overview 🌸
- Install on embedded Media (eMMC, SD, Raw NAND, NOR and SPI-NOR flashes) — the whole flash family, no exceptions 🎀
- It checks if an image is available. That image is built in a specified format (
cpio) and it must contain a file describing the software that has to be updated. - SWUpdate's whole thing is updating UBI volumes (mainly for NAND, but not only NAND 👀) and images on devices. You can still pass a whole image and have it updated as a partition on the SD card, or as a MTD partition.
- New partition schema. This one is bound with UBI volume. SWUpdate can recreate UBI volumes, resize them, and copy the new software in. Iconic.
- Support for compressed images, using the
xz,zlib,zstdandlz4libraries. tarballs (tgz files) are supported too. 🫶 - Support for a partitioned USB-pen, or an unpartitioned one (mainly the kind Windows leaves lying around).
- Support for updating a single file inside a filesystem. You have to describe which filesystem the file goes in. One file, no drama.
- Checksum for the single components of an image ✅
- Use a structured language to describe the image. The default parser is the libconfig <http://www.hyperrealm.com/libconfig/> library, which uses a JSON-like description.
- Your choice of description format, actually — you can write your own parser using the Lua language. There's an example using an XML description in Lua sitting in the examples directory, go look 👑
- Support for setting / erasing U-Boot variables
- Support for setting / erasing GRUB <https://www.gnu.org/software/grub/manual/html_node/Environment-block.html> environment block variables
- Support for setting / erasing EFI Boot Guard <https://github.com/siemens/efibootguard> variables
- Support for pre and post update commands — they run before the update starts processing data, and after the update has finished successfully.
- Support for lua hooks, executed before any handler runs.
- Support for preinstall scripts. They run after streamed handlers have handled their data, and before regular handlers.
- Support for postinstall scripts. They run after updating the images.
- Network installer using an embedded Web-server (Mongoose Server was chosen, in the version under Lua license). A different Web-server can be used, no pressure.
- Multiple interfaces for getting software
- local Storage: USB, SD, UART,..
- OTA / Remote
- integrated Web-Server
- pulling from a remote Server (HTTP, HTTPS, ..)
- using a Backend. SWUpdate is open to talk with back end servers for rolling out software updates. Current version supports the hawkBit server, but other backends can be added.
- Can be configured to check for compatibility between software and hardware revisions. The software image must contain an entry declaring which HW revision the software is allowed to run on. If that compatibility isn't verified, SWUpdate refuses to install. Periodt. 🙅
- Support for image extraction. A manufacturer can require a single image that contains the software for more than one device. This simplifies the manufacturer's management and reduces their administrative costs by having a single software product. SWUpdate receives the software as a stream without temporary storing, and extracts only the required components for the device that's being installed. Efficiency slay 💅
- Allow custom handlers for installing FPGA firmware and micro-controller firmware via custom protocols.
- Features are enabled / disabled using "make menuconfig". (Kbuild is inherited from the busybox project, iykyk.)
- Images are authenticated and verified before installing 🔥
- Power-Off safe 💀 (in the good way — a power cut won't ruin your day)
📦 Single image delivery 📦
The main concept is that the manufacturer delivers one big image. All the single images get packed together (cpio was chosen for its simplicity and because it can be streamed) together with an additional file (sw-description), which contains meta information about each single image. Everything in one little bundle, chef's kiss.
The format of sw-description can be customized: SWUpdate can be configured to use its internal parser (based on libconfig), or to call an external parser in Lua. [image]
Changing the rules to accept images with an external parser let it extend to new image types and new ways of installing them. The scope of the parser, honestly, is just to retrieve which single images must be installed and how. SWUpdate implements "handlers" to install a single image: there are handlers to install images into UBI volumes, or to an SD card, a CFI Flash, and so on. So adding your own handler when you need some very special installer is easy. ✨
For example, think of a project with a main processor and one or several micro-controllers. Let's say (for simplicity) that the main processor talks to the micro-controllers over UARTs using a proprietary protocol. The software on the micro-controllers can be updated using that proprietary protocol.
So it's totally possible to extend SWUpdate by writing a handler that implements the part of the proprietary protocol needed to perform the upgrade on the micro-controller. The parser must recognize which image must be installed with the new handler, and SWUpdate will call the handler during the installation process. Boom. 🎀
🌊 Streaming feature 🌊
The whole point is that SWUpdate can stream the received image directly into the target, with no temporary copy at all. The single installer (handler) receives as input the file descriptor set at the beginning of the image that must be installed. Very direct, very efficient.
This feature is set on an image basis — meaning you get to decide which partial images should be streamed. If an image is not streamed (see the installed-directly flag), the files are temporarily extracted into the directory pointed to by the environment variable TMPDIR, with /tmp as the fall-back if TMPDIR isn't set.
Of course, by streaming it is not possible to make checks on the whole delivered software before installing. That's the trade-off, bestie — know it going in. 😭
The temporary copy happens only when you update from the network. When the image is already stored on an external storage, there's no need for that copy.
💫 Images fully streamed 💫
In the case of a remote update, SWUpdate extracts the relevant images from the stream and copies them into the directory pointed to by the environment variable TMPDIR (if it's unset, then /tmp) before calling the handlers. That guarantees an update is only initiated if all the parts are present and correct. 🫶
But some systems have less resources, and the amount of RAM needed to copy those images might not be enough — for example if the filesystem on an attached SD Card has to be updated. In that case, it helps if the images are installed directly as a stream by the corresponding handler, with no temporary copies in between. Heads up though: not all handlers support streaming directly into the target. Zero-copy streaming is enabled by setting the flag "installed-directly" in the description of the single image.
🛠️ Configuration and Build 🛠️
🌈 Requirements 🌈
There are only a few libraries strictly required to compile SWUpdate:
zlib(<https://www.zlib.net>)libubootenvfor U-Boot environment support (<https://github.com/sbabic/libubootenv>)json-cfor parsing JSON (<https://github.com/json-c/json-c>)
Further library dependencies may be required when you activate more SWUpdate features — check the next section on configuring SWUpdate for that. 👑
⚙️ Configuring SWUpdate ⚙️
SWUpdate is configurable via make menuconfig. A small footprint is realized, e.g., by using the internal parser and disabling the web server. Every option has a small help describing its usage. In the default configuration, lots of options are already activated — so you're not starting from zero. 💅
To configure the options:
make menuconfig🔨 Building 🔨
To cross-compile, set the CC and CXX variables before running make. You can also set the cross-compiler prefix as an option with make menuconfig. Then generate the code by running
makeThe result is the binary swupdate. And notably, the tools/swupdate-progress binary gets built as well — it's an example of how to build your own interface to SWUpdate to, e.g., show a progress bar on an HMI. This example simply prints the current status of the update on the console and, more importantly, reboots the machine after a successful installation. 🔥
🐧 Building with Yocto 🐧
See the corresponding chapter meta-swupdate: building with Yocto <> on how to build SWUpdate in Yocto.
📦 Distribution Packages 📦
SWUpdate is made for Embedded Systems, and building inside an embedded distribution is the primary use case. But apart from the most used build systems for embedded systems like Yocto or Buildroot, sometimes a standard Linux distro is what you've got. SWUpdate is currently packaged for
- Debian <https://packages.debian.org/search?searchon=names&keywords=swupdate>
- Ubuntu <https://packages.ubuntu.com/search?keywords=swupdate>
Refer to the distribution source package and build system for how to build a package from source. Kbyeee. 🎀
🚀 Running SWUpdate 🚀
✨ What is expected from a SWUpdate run ✨
The whole update process can be seen as a set of pipelines. The incoming stream (the SWU file) is processed by each pipe and passed on to the next step. First, the SWU is streamed from one of the interfaces: local (USB, filesystem), Webserver, suricatta (one of the backends), etc. Then the incoming SWU is forwarded to the installer to be examined and installed.
A run of SWUpdate consists mainly of the following steps:
- extracts
sw-descriptionfrom the stream and verifies it. It parsessw-description, creating a raw description in RAM about the activities that must be performed. - if Signed Images is activated, extracts
sw-description.sigand validatessw-description. - checks for hardware-software compatibility, if any, by reading the hardware revision from hardware and matching it against the table in
sw-description. - parses
sw-descriptionto determine which artefacts in the incoming SWU are required. Not-required artifacts are simply skipped. If an"embedded-script"is defined, it is executed at this point, before parsing files. If"hooks"are defined, they are executed as each file is parsed — even if those files will be skipped. At the end of the parsing, SWUpdate builds an internal mapping for each artifact to recognize which handler should be called for each of them. - runs the pre update command, if set
- runs partition handlers, if required.
- reads through the cpio archive one file at a time and either:
- execute handlers directly consuming the data stream for each file marked as
"installed-directly". lowkey important: in this case a cpio checksum is not verified 😭 while a sha256 checksum (CONFIG_HASH_VERIFY) is — but only if the file has it set insw-description, or when signed images are used. iykyk. - copy other files to a temporary location while checking checksums, and stop immediately if there was a mismatch. no pushing through, bestie 💅
- iterates through all scripts and calls the corresponding handler for pre-install scripts. ok real talk though: if artifacts are streamed, they get extracted before this runs. so if you need your code to run earlier, use the
"embedded-script"feature or the hooks features to make sure it happens before installation takes place. plan accordingly ✨ - iterates through all images and calls the corresponding handler to install them on the target. go off 👑
- iterates through all files and calls the corresponding handler to install them on the target.
- iterates through all scripts and calls the corresponding handler for post-install scripts 🎀
- iterates through all bootenv and updates the bootloader environment.
- reports the status to the operator through the notification interface (logging, traces) and through the progress interface. keeping everyone in the loop fr fr 🫶
- runs the post update command, if it's set.
🚨 the first step that fails stops the entire procedure and an error is reported. periodt. no partial vibes.
so, how do you actually run this thing? 💖
wanna start SWUpdate expecting the image to come from a file?
swupdate -i <filename>wanna start it with the embedded web server instead?
swupdate -w "<web server options>"the main important parameters for the web server are "document-root" and "port":
swupdate -w "--document-root ./www --port 8080"the embedded web server is taken from the Mongoose project. cute collab 🌸
the list of available options (which depends on which features you activated) shows up with:
swupdate -hthis uses as website the pages delivered with the code — and yes, of course they can be customized and replaced. the website uses AJAX to talk to SWUpdate and to show the progress of the update to the operator 🦋
the default port of the Web-server is 8080. so you can connect to the target with:
http://<target_ip>:8080if it works, the start page should be displayed as in the next figure. [image] ⭐
if a correct image is downloaded, SWUpdate starts to process the received image. all notifications are sent back to the browser. SWUpdate provides a mechanism to send to a receiver the progress of the installation — in fact, SWUpdate takes a list of objects that register themselves with the application, and they get informed any time the application calls the notify() function. 😌
this is also how your own self-written handlers can tell the upper layers about error conditions, or just return the status. so it's genuinely easy to add your own receivers and implement a customized way to display the results: showing on an LCD (if the target has one), or sending back to another device over the network 🌈 an example of the notifications sent back to the browser is in the next figure: [image]
software collections can be specified by passing the --select command line option. say your sw-description file contains a collection named stable, with an alt installation location — SWUpdate can be called like this:
swupdate --select stable,alt✨ Command line parameters ✨
ok bestie, this is the part where you get to tell SWUpdate exactly what you want. every flag, every option, all the tea 🍵 no skips, no shortcuts, we're doing all of them 💅
-f <file>— the SWUpdate configuration file to use. the deets live inexamples/configuration/swupdate.cfgin the source code, go peek 👀-b <string>— only a thing ifCONFIG_UBIATTACHis set. it lets you blacklist MTDs while SWUpdate is out here searching for UBI volumes. example: U-Boot and environment in MTD0-1, so you goswupdate -b "0 1". iconic.-B <loader>— override the default bootloader interface and use loader instead. done and dusted.-e <sel>— sel comes in the format<software>,<mode>. this finds a subset of rules inside thesw-descriptionfile, and yes, multiple rules are allowed, we love options 🫶. the classic use case is the dual copy approach. so like:-e "stable, copy1"==> install on copy1, and-e "stable, copy2"==> install on copy2. periodt.--excluded <sel>— sel is in the format<software>,<mode>again. this one sets a blacklist of selections that cannot be used for an update. lowkey important: selections can be activated not only with-e, but also via IPC. and yeah, multiple--excludedare allowed. stack 'em up 🌈-h— run usage with help. the lil emergency exit for when you're lost 🙈-k <file>— available ifCONFIG_SIGNEDis set. filename with the public key.-K <file>— available onCONFIG_ENCRYPTED_IMAGESset. filename with the symmetric key to be used for decryption.--cert-purpose <purpose>— available ifCONFIG_SIGNED_IMAGESis set. set the expected certificate purpose.--forced-signer-name <cn>— available ifCONFIG_SIGNED_IMAGESis set. set the expected common name of the signer certificate.--ca-path <file>— available ifCONFIG_SIGNED_IMAGESis set. path to the Certificate Authority (PEM).--get-root— detect and print the root device, then exit. bye bestie ✌️--get-emmc-boot <device>— if the device is an eMMC, this reads the CSD register and detects which boot device it booted from (mmcblkXboot0ormmxblkXboot1). slay 💅-l <level>— set loglevel. it's anint, bestie.-L— send LOG output to syslog (local).-i <file>— run SWUpdate with a local.swufile.-n— run SWUpdate in dry-run mode 🍃-N <version>— the minimum required version of software. this gets checked against the version of the new software, and it forbids downgrading 🚫. the version is either 4 numbers (major.minor.rev.build, with each field in the range0..65535) or it's a semantic version.-max-version <version>— the maximum required version of software. this gets checked against the version of the new software too. version is either 4 numbers (major.minor.rev.build, with each field in the range0..65535) or it's a semantic version.-R <version>— string. The version that's installed right now. swupdate peeks at this, compares it against whatever version the new software is bringing, and if they match? It forbids reinstalling, bestie. No double-dipping 💅-o <file>— string. Saves the stream (that's the SWU, the whole thing) into a file. Your little souvenir 🎀-s <file>— string. Writes the installed version info out to a file, so you've got receipts 📝-v— turns on verbose output. She's talkative now and honestly? We love that for her ✨-M— disables setting the bootloader transaction marker. Nobody's marking anything, the drama is cancelled 🚫-m— disables setting the update state in the bootloader. Different switch, similar vibe, don't mix them up 🙈-w <parms>— string. Only available ifCONFIG_WEBSERVERis set. Starts the internal webserver and hands it a command line string. Web girl era 🌐-d <parms>— string. Only available ifCONFIG_DOWNLOADis set. Starts the internal downloader client and passes it a command line string. There's a whole list of internal command line arguments for the downloader waiting for you below 👀-u <parms>— string. Only available ifCONFIG_SURICATTAis set. Starts the internal suricatta client daemon and passes it a command line string. The internal command line arguments for suricatta? Also below. Patience, queen 🧚-H <board:rev>— string. Available whenCONFIG_HW_COMPATIBILITYis set. Sets the board name and the hardware revision, together, as a duo. Main character pairing fr fr 👑-c— checks the*.swufile. It makes sure every file referenced insw-descriptionis actually present, no ghost references, no fake friends 💀 Usage:swupdate -c -i <file>-P <cmd>— string. Executes a pre-update command. Do the thing before the thing 🔥-p <cmd>— string. Executes a post-update command. Do the thing after the thing 🌈-q <sel>— string. A list for software images set and source that get accepted via IPC. Example:stable,main. And get this — it can be set multiple times. Stack it up, babe 🍓
🔐 Crypto selection parameters 🔐
Picking your crypto implementation is like picking an outfit, and bestie, you get options:
--digest-provider <provider>— string. Selects the implementation used for signature verification. The full list of implementations gets written at start, so you can browse. Examples:opensslCMS,opensslRSA, etc. ✨--decrpyt-provider <provider>— string. Selects the implementation used for decryption. Again, the list of implementations is written at start. Examples:openssl,wolfssl, etc. (yes the flag is spelled like that, we don't fix it, we respect it 🫶)--hash-provider <provider>— string. Selects the implementation used for hash computation. Same deal, list written at start. Examples:openssl. Simple, clean, iconic 💖
📥 Downloader command line parameters 📥
Example, because examples ate and left no crumbs:
swupdate -d "-u example.com"okay so first things first, bestie 🎀 — the mandatory args are marked with a *, so keep an eye out for the star or you're gonna be out here fighting for your life wondering why nothing downloads.
-u <url>— (that's a string, mwah) * MANDATORY, bestie, no skipping — this is literally the URL where your new software gets pulled from. And not just any lil link okay, it has to point at a valid.swuimage or it's giving nothing. 💅-r <retries>— (integer energy) how many times it's gonna retry before the download gets written off as broken. And listen, the lore here is real: with-r 0, SWUpdate will not stop until a valid software is loaded. zero chill, zero surrender, she's committed fr fr. 👑-w <retrywait>— (integer) how long it waits before it retries and resumes a download. default is 5s, iconic of her honestly. 🕐-t <timeout>— (integer) the timeout for when the downloader or the Webserver loses the connection. nobody likes a ghosted connection, so this is your lil boundary. 🚫-a <usr:pwd>— (string) sends your user and password for Basic Auth. username and password in one, separated by that colon, that's the whole vibe. 🔐-a... no wait, it's not, moving on, I'm not doing that again 😭 —-n <value>— (string) the max download speed you're allowed to use. the value gets specified inkB/s,B/s,MB/sorGB/s. and here's the receipts, in her own words:-n 100k: Set limit to 100 kB/s.-n 500: Set limit to 500 B/s.-n 2M: Set limit to 1 M/s.-n 1G: Set limit to 1 G/s.
yes bestie, that-n 2Mreally does say1 M/s, I am not the one who wrote it, I am simply the messenger and I keep the receipts exactly as they came. 💀 (also lowkey: no suffix on-n 500means it's reading it asB/s, and thek/M/Gshorthand goeskB/s,MB/s,GB/s. that's not me guessing, that's just reading the list.)
🐦✨ Suricatta command line parameters ✨🐦
so THIS is where it gets a lil spicy. check it:
swupdate -u "-t default -u localhost:8080 -i 1B7"that's the whole move, that's the example, you're welcome. 🫶
now the tea you absolutely need to know before you start copy-pasting: different suricatta modules may have different parameters. translation: what works for one module might be a whole nothing-burger for another, so don't assume, okay? 🙈
and specifically, the options listed below that point are the ones for SWUpdate's hawkBit support. so if hawkBit is your module, you're in the right place and you're gonna eat. if it's not... keep scrolling, bestie, your section is somewhere else. 🌸
okay so before we get into it — mandatory arguments are marked with *, and you literally cannot skip those ones, bestie, the whole thing falls apart without them 😭
Every row below has a type and a vibe (that's the description, iykyk), and I've written them all out for you one by one because I love you 💖
-t <tenant>(string) — mandatory ✨ Set the hawkBit tenant ID for this device.-u <url>(string) — mandatory ✨ Host and port of the hawkBit instance, e.g.localhost:8080.-i <id>(integer) — mandatory ✨ The device ID to communicate to hawkBit. That's you. That's your name in the group chat 🫶-c <confirm>(integer) — Confirm update status to server:1=AGAIN,2=SUCCESS,3=FAILED. Lowkey the most important gossip you'll ever send tbh.-x— Do not abort on flawed server certificates. Living on the edge, no cap.-p <polldelay>(integer) — Delay in seconds between two hawkBit poll operations (default:45s). She's patient, she's chill 🌸-r <retry>(integer) — Resume and retry interrupted downloads (default:5tries). Not me giving up after zero tries — we retry, periodt.-w <retrywait>(integer) — Time to wait prior to retry and resume a download (default:5s). A lil breather before we go again 🧚-y <proxy>(string) — Use proxy. Either give the proxy URL, else{http,all}_proxyenv is tried.-k <targettoken>(string) — Set target token.-g <gatewaytoken>(string) — Set gateway token.-f <interface>(string) — Set the network interface to connect to hawkBit.-e— Daemon enabled at startup (default). ✨ main character energy, on by default ✨-d— Daemon disabled at startup.--disable-token-for-dwl— Do not send the authentication header when downloading SWU. 🙈--cache-file(string) — This allows one to resume an update after a power cut. If the SWU is saved in a file, SWUpdate can reuse the file and download just the remaining part of the SWU. power-cut-proof queen behavior 💅-m <seconds>(integer) — Delay in seconds between re-trying to send initial feedback specified with"-c"option. Default value is10seconds. If Suricatta is started with initial state ofSTATE_WAIT("-c 6"), this value is ignored.-s <seconds>(integer) — Connection timeout to use in seconds. If the user doesn't set this option, the default libcurl connection timeout value of300seconds is used.
NOTE: it is not possible for Suricatta to respond to external program API requests during this period — adapt this value to your use case! 🚨 Like fr fr, don't just copy-paste the default and then act surprised.
-a <name> <value>(strings) — Custom HTTP header with given name and value to be sent with every HTTP request made. It's giving signature scent but for HTTP 💜-n <value>(string) — Maximum download speed to be used. Value be specified in kB/s, B/s, MB/s or GB/s. Examples:-n 100k: Set limit to 100 kB/s. `-n 500
Okay so quick catch-up for the row that got clipped at the very top of this chunk — it reads exactly like this, no edits: : Set limit to 500 B/s. — then -n 2M : Set limit to 1 M/s. — and -n 1G : Set limit to 1 G/s. ✨ And that's the whole thing, bestie, word for word. Carry on. 🫶
🌐 Webserver command line parameters 🌐
Example, look how cute: swupdate -w "-r /www -p 8080"
Head's up on the little marks: arguments that are only available with Mongoose are flagged with #. Mandatory arguments are flagged with *. 💅
-r <document root>— string.*mandatory, so do not skip her. Path where the web app is stored. 🎀-p <port>— integer.*mandatory. TCP port to be listened; if not set,8080is used.-s <ssl>—*mandatory. Enable SSL support. Note: it must be configured withCONFIG_MONGOOSESSL. No config, no SSL, no cap. 🔥--ssl-cert <cert>— string. Path to the certificate to present to clients.-K <key>— string. Path to key corresponding to ssl certificate. 🔑-t <timeout>— integer. Timeout to consider a connection lost if clients stops to send data. If hit, an update is aborted. Default=0(unlimited).--auth-domain <string>— string.#Mongoose-only. Set authentication domain Default: none.--global-auth-file <string>— string.#Mongoose-only. Set authentication file if any Default: none.
🧚 systemd Integration 🧚
SWUpdate has optional systemd support via the compile-time configuration switch CONFIG_SYSTEMD. If enabled, SWUpdate signals systemd about start-up completion and can make optional use of systemd's socket-based activation feature. Bestie it literally taps systemd on the shoulder like "hey, I'm up". 👑
A sample systemd service unit file /etc/systemd/system/swupdate.service may look like the following, starting SWUpdate in suricatta daemon mode:
[Unit]
Description=SWUpdate daemon
Documentation=https://github.com/sbabic/swupdate
Documentation=https://sbabic.github.io/swupdate
[Service]
Type=notify
RuntimeDirectory=swupdate
RuntimeDirectoryPreserve=yes
ExecStart=/usr/bin/swupdate -u '-t default -u http://localhost -i 25'
[Install]
WantedBy=multi-user.targetStarted via systemctl start swupdate.service, SWUpdate (re)creates its sockets on startup. 🌸 For using socket-based activation, an accompanying systemd socket unit file /etc/systemd/system/swupdate.socket is required:
[Unit]
Description=SWUpdate socket listener
Documentation=https://github.com/sbabic/swupdate
Documentation=https://sbabic.github.io/swupdate
[Socket]
ListenStream=/run/swupdate/sockinstctrl
ListenStream=/run/swupdate/swupdateprog
[Install]
WantedBy=sockets.targetOn swupdate.socket being started, systemd creates the socket files and hands them over to SWUpdate when it starts. So, for example, when talking to /run/swupdate/swupdateprog, systemd starts swupdate.service and hands-over the socket files. The socket files are also handed over on a "regular" start of SWUpdate via systemctl start swupdate.service. ✨
Note, that all dependent services need to access the swupdate sockets via the paths specified in the swupdate.socket systemd unit. Like, every single one of them. Don't be the friend who hardcodes a different path. 🙈
🥾 Changes in boot-loader code 🥾
The SWUpdate consists of kernel and a root filesystem (image) that must be started by the boot-loader. In case using U-Boot, the following mechanism can be implemented:
- U-Boot checks if a sw update is required (check gpio, serial console, etc.).
- the script
"altbootcmd"sets the rules to start SWUpdate - in case SWUpdate is required, U-boot run the script
"altbootcmd"
Is it safe to change U-Boot environment ? Well, it is, but U-Boot must be configured correctly. U-Boot supports two copies of the environment to be power-off safe during an environment update. The board's configuration file must have defined CONFIG_ENV_OFFSET_REDUND or CONFIG_ENV_ADDR_REDUND. Check in U-Boot documentation for these constants and how to use them. 📚
There are a further enhancement that can be optionally integrated into U-boot to make the system safer. The most important I will suggest is to add support for boot counter in U-boot (documentation is in U-Boot docs). This allows U-Boot to track for attempts to successfully run the application, and if the boot counter is greater as a limit, can start automatically SWUpdate to replace a corrupt software. Slay, self-healing. 💖
GRUB by default does not support double copies of environment as in case of U-Boot. This means that there is possibility that environment block get's corrupted when power-off occurs during environment update. To minimize the risk, we are not modifying original environment block. Variables are written into temporary file and after successful operation rename instruction is called. 🫶
📦 Image File Format 📦
SWUpdate uses cpio as image file format because it is a simple, well-established, and streamable format. More specifically, the New ASCII format (header magic number 070701) and the New CRC format (header magic number 070702) are supported. Both formats are essentially equivalent with the New CRC format additionally having set the cpio header field check to the least-significant 32 bits of the sum of all (unsigned) data bytes. This checksum is verified by SWUpdate for artifacts copied to a temporary location as well as for skipped artifacts. It is not verified for artifacts marked installed-directly, which are directly streamed to their handler — use sha256 verification (CONFIG_HASH_VERIFY) for ensuring the integrity of those. 👀 If this verification fails, SWUpdate yields an error like the following:
Checksum WRONG ! Computed 0xfa11ed00, it should be 0xffffffffNote that there's artifact sha256 verification available (see CONFIG_HASH_VERIFY) which is recommended over relying on cpio's checksum facility, in particular for artifacts marked installed-directly for which cpio checksums are not verified. Like, lowkey just use sha256, she's the moment. ⭐
For both cpio formats, the New ASCII as well as the New CRC format, the cpio file size is limited to 32 Bit, i.e., 4 GB. 🎀
🛠️ Building a single image 🛠️
cpio is used as container format because of its simplicity and its ability to be streamed. The meta information file sw-description (default, see CONFIG_SETSWDESCRIPTION) describing the images in the container must be the first file in the cpio archive. The images follow it, in any order. No cutting the line, bestie. 💅
To produce an image, a script like the following can be used:
CONTAINER_VER="1.0"
PRODUCT_NAME="my-software"
FILES="sw-description image1.ubifs \
image2.gz.u-boot uImage.bin myfile sdcard.img"
for i in $FILES;do
echo $i;done | cpio -ov -H crc > ${PRODUCT_NAME}_${CONTAINER_VER}.swuAlternatively, swugenerator may be used to generate the image. ✨
The generated image can be checked by running the following command:
swupdate -c -i my-software_1.0.swu🌈 Support of compound image 🌈
A single image can be built automatically inside Yocto. meta-swupdate extends the classes with the swupdate class. A recipe should inherit it and add an own sw-description file to generate the image. So iconic. 👑
Alternatively, swugenerator may be used to generate compound images outside Yocto. 🦋
UPDATE STRATEGY EXAMPLES ✨
SWUpdate is literally a building block, bestie — it hands you the pieces and says "you go design your own update strategy, I believe in you." And yeah, even though tons of projects end up updating stuff in the same few ways, you can highkey customize the update for each individual project. The most common strategies (single-copy and dual-copy) were already described at the beginning of this documentation and obvi they're well supported in SWUpdate. 💅
Single copy - running as standalone image
See Single copy - running as standalone image <#single-copy>. 👑
Double copy with fall-back
See Double copy with fall-back <#double-copy>. 🫶
Combine double-copy with rescue system
Okay so this one gives you a recovery procedure to cover update failure in severe cases when the software is damaged. If none of the copies can be started, the bootloader will start the rescue system (possibly stored on another storage than the main system) and try to rescue the board. [image]
The rescue system can be updated as well during a standard update. 🌈
Split system update with application update
Updating a whole image is quite straightforward, bestie, but that means transferring a bigger amount of data even if just a few files are being updated. 😭 It is possible to split the update into several smaller parts to reduce the transfer size. This requires a special care to take care of compatibility between system and application, and that can be solved with customized Lua scripts in the sw-description file. SWUpdate supports versioning for each artefact, and anyone can add their own rules to verify compatibility between components. [image]
Configuration update
Thought it was only for updating the software? Nope — SWUpdate can be used to install configuration data as well. 🎀 The build system can create configuration SWU with files / data for the configuration of the system. There is no requirement on what these SWU should contain — it is the duty of the integrator to build them and make them suitable for their own project. Again, configuration data can be updated as a separate process using one of the above scenarios. 💖
SWUPDATE: SYNTAX AND TAGS WITH THE DEFAULT PARSER 🦋
Introduction
SWUpdate uses the library "libconfig" as the default parser for the image description. But here's the slay part — it is possible to extend SWUpdate and add your own parser, based on a different syntax and language than the one supported by libconfig. In the examples directory there is the code for a parser written in Lua, with the description in XML. 👀
Using the default parser, sw-description follows the syntax rules described in the libconfig manual. Please take a look at <http://www.hyperrealm.com/libconfig/libconfig_manual.html> for an explanation of basic types. The whole description must be contained in the sw-description file itself: using of the #include directive is not allowed by SWUpdate. 🚫
The following example explains the implemented tags:
software =
{
version = "0.1.0";
description = "Firmware update for XXXXX Project";
hardware-compatibility: [ "1.0", "1.2", "1.3"];
/* partitions tag is used to resize UBI partitions */
partitions: ( /* UBI Volumes */
{
name = "rootfs";
device = "mtd4";
size = 104896512; /* in bytes */
},
{
name = "data";
device = "mtd5";
size = 50448384; /* in bytes */
}
);
images: (
{
filename = "rootfs.ubifs";
volume = "rootfs";
},
{
filename = "swupdate.ext3.gz.u-boot";
volume = "fs_recovery";
},
{
filename = "sdcard.ext3.gz";
device = "/dev/mmcblk0p1";
compressed = "zlib";
},
{
filename = "bootlogo.bmp";
volume = "splash";
},
{
filename = "uImage.bin";
volume = "kernel";
},
{
filename = "fpga.txt";
type = "fpga";
},
{
filename = "bootloader-env";
type = "bootloader";
}
);
files: (
{
filename = "README";
path = "/README";
device = "/dev/mmcblk0p1";
filesystem = "vfat";
}
);
scripts: (
{
filename = "erase_at_end";
type = "lua";
},
{
filename = "display_info";
type = "lua";
}
);
bootenv: (
{
name = "vram";
value = "4M";
},
{
name = "addfb";
value = "setenv bootargs ${bootargs} omapfb.vram=1:2M,2:2M,3:2M omapdss.def_disp=lcd";
}
);
}The first tag is "software". The whole description is contained in this tag. It is possible to group settings per device by using Board specific settings. 💅
Handling configuration differences
The concept can be extended to deliver a single image containing the release for multiple devices. Each device has its own kernel, dtb, and root filesystem, or they can share some parts. 🌸
Currently this is managed (and already used in a real project) by writing an own parser, that checks which images must be installed after recognizing which is the device where software is running.
Because the external parser can be written in Lua and it is completely customizable, everybody can set their own rules. For this specific example, the sw-description is written in XML format, with tags identifying the images for each device. To run it, the liblxp library is needed.
<?xml version="1.0" encoding="UTF-8"?>
<software version="1.0">
<name>Update Image</name>
<version>1.0.0</version>
<description>Firmware for XXXXX Project</description>
<images>
<image device="firstdevice" version="0.9">
<stream name="dev1-uImage" type="ubivol" volume="kernel" />
<stream name="dev1.dtb" type="ubivol" volume="dtb" />
<stream name="dev1-rootfs.ubifs" type="ubivol" volume="rootfs"/>
<stream name="dev1-uboot-env" type="uboot" />
<stream name="raw_vfat" type="raw" dest="/dev/mmcblk0p4" />
<stream name="sdcard.lua" type="lua" />
</image>
<image device="seconddevice" version="0.9">
<stream name="dev2-uImage" type="ubivol" volume="kernel" />
<stream name="dev2.dtb" rev="0.9" type="ubivol" volume="dtb" />
<stream name="dev2-rootfs.ubifs" type="ubivol" volume="rootfs"/>
</image>
</images>
</software>The parser for this is in the /examples directory. By identifying which is the running device, the parser returns a table containing the images that must be installed and their associated handlers. By reading the delivered image, SWUpdate will ignore all images that are not in the list processed by the parser. This way, it is possible to have a single delivered image for the update of multiple devices. 🦋
Multiple devices are supported by the default parser, too. 👑
software =
{
version = "0.1.0";
target-1 = {
images: (
{
...
}
);
};
target-2 = {
images: (
{
...
}
);
};
}This way, it is possible to have a single image providing software for each device you have. 💖
By default, the hardware information is extracted from /etc/hwrevision file. The file should contain a single line in the following format:
<boardname> <revision>Where:
<revision>will be used for matching with hardware compatibility list<boardname>can be used for grouping board specific settings
Software collections
Software collections and operation modes can be used to implement a dual copy strategy. The simplest case is to define two installation locations for the firmware image and call SWUpdate selecting the appropriate image. ✨
software =
{
version = "0.1.0";
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
};
}This way it is possible to specify that copy-1 gets installed to /dev/mtd4, while copy-2 to /dev/mtd5. By properly selecting the installation locations, SWUpdate will update the firmware in the other slot. 💅
The method of image selection is out of the scope of SWUpdate and the user is responsible for calling SWUpdate passing proper settings. 🫶
Priority finding the elements in the file
SWUpdate searches for entries in the sw-description file according to the following priority:
- Try
<boardname>.<selection>.<mode>.<entry> - Try
<selection>.<mode>.<entry> - Try
<boardname>.<entry> - Try
<entry>
Take an example. The following sw-description describes the release for a set of boards. 🌈
software =
{
version = "0.1.0"; myboard = {
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
}
}
stable = {
copy-1: {
images: (
{
device = "/dev/mtd6"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd7"
...
}
);
}
}
}On myboard, SWUpdate searches and finds myboard.stable.copy1(2). But when it's running on some other board, SWUpdate does not find an entry corresponding to the boardname, and it falls back to the version without boardname. That's the whole point bestie — it lets you realize the same release for different boards with completely different hardware. Your myboard could have an eMMC and an ext4 filesystem, while another device has raw flash and installs a UBI filesystem. Still the same release, just a different format, and they can be described together in sw-description. 🫶 It is important to understand the priorities of how SWUpdate scans for entries during the parsing. No cap.
🦋 Using links 🦋
sw-description can become very complex. Imagine having just one board, but in multiple hw revisions that differ in Hardware. Some of them can be grouped together, some of them need a dedicated section. One way (but not the only one!!) could be to add mode and select the section with -e stable,<rev number>.
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.3", "3.0", "3.1"];
rev-1.0: {
images: (
...
);
scripts: (
...
);
}
rev-1.2: {
hardware-compatibility: ["1.2"];
images: (
...
);
scripts: (
...
);
}
rev-2.0: {
hardware-compatibility: ["2.0"];
images: (
...
);
scripts: (
...
);
}
rev-1.3: {
hardware-compatibility: ["1.3"];
images: (
...
);
scripts: (
...
);
}
rev-3.0:
{
hardware-compatibility: ["3.0"];
images: (
...
);
scripts: (
...
);
}
rev-3.1:
{
hardware-compatibility: ["3.1"];
images: (
...
);
scripts: (
...
);
}
}
}
}If each of them requires its own section, then yeah, that's the way to do it. But honestly it's way more probable that revisions can be grouped together — like, boards with the same major revision number probably have the same installation instructions. In the example that gives us 3 groups: rev1.X, rev2.X, and rev3.X. Links let you group sections together. When a "ref" is found while SWUpdate searches for a group (images, files, script, bootenv), it replaces the current path in the tree with the value of the string. So the example above can be written like this 👇
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.3", "3.0", "3.1"];
rev-1x: {
images: (
...
);
scripts: (
...
);
}
rev1.0 = {
ref = "#./rev-1x";
}
rev1.2 = {
ref = "#./rev-1x";
}
rev1.3 = {
ref = "#./rev-1x";
}
rev-2x: {
images: (
...
);
scripts: (
...
);
}
rev2.0 = {
ref = "#./rev-2x";
}
rev-3x: {
images: (
...
);
scripts: (
...
);
}
rev3.0 = {
ref = "#./rev-3x";
}
rev3.1 = {
ref = "#./rev-3x";
}
}
}
}A link can be absolute or relative. The keyword "ref" is what marks a link. When SWUpdate finds one, it traverses the tree and replaces the current path with the values it finds in the string pointed to by "ref". The rules for a link are simple, bestie ✨
- it must start with the character
# .points to the current level in the tree, meaning the parent ofref..points to the parent level in the tree/is used as filed separator in the link
A relative path has a number of leading ../ to move the current cursor to the parent leaf of the tree. In the following example, rev40 sets a link to a "common" section, where images is found. And that one is set via a link too, to a section in the parent node. So the path software.myboard.stable.common.images gets replaced by software.myboard.stable.trythis. 🌈
software =
{
version = {
ref = "#./commonversion";
}
hardware-compatibility = ["rev10", "rev11", "rev20"];
commonversion = "0.7-linked";
pc:{
stable:{
common:{
images =
{
ref = "#./../trythis";
}
};
trythis:(
{
filename = "rootfs1.ext4";
device = "/dev/mmcblk0p8";
type = "raw";
} ,
{
filename = "rootfs5.ext4";
device = "/dev/mmcblk0p7";
type = "raw";
}
);
pdm3rev10:
{
images:(
{
filename = "rootfs.ext3"; device = "/dev/mmcblk0p2";}
);
uboot:(
{ name = "bootpart";
value = "0:2";}
);
};
pdm3rev11 =
{
ref = "#./pdm3rev10";
}
pdm3rev20 =
{
ref = "#./pdm3rev10";
}
pdm3rev40 =
{
ref = "#./common";
}
};
};
}Each entry in sw-description can be redirected by a link, exactly like in the example above for the "version" attribute. 💅
💖 hardware-compatibility 💖
hardware-compatibility: [ "major.minor", "major.minor", ... ]This entry lists the hardware revisions that are compatible with this software image. Obsessed with how clear that is.
Example:
hardware-compatibility: [ "1.0", "1.2", "1.3"];That defines that the software is compatible with HW-Revisions 1.0, 1.2, and 1.3, but not with 1.1 or any other version that isn't explicitly listed here. In the example above, compatibility is checked by means of string comparison. If your software is compatible with a large number of hardware revisions, enumerating them all gets cumbersome 💀. To allow more compact specifications, regular expressions (POSIX extended) can be used by adding a prefix #RE: to the entry. Rewriting the above example would give:
hardware-compatibility: [ "#RE:^1\.[023]$" ];It is in the responsibility of the respective project to find the revision of the board on which SWUpdate is running. No assumptions are made about how the revision can be obtained (GPIOs, EEPROM,..) and each project is free to select the most appropriate way. In the end the result must be written to the file /etc/hwrevision (or in another file if specified as configuration option) before SWUpdate is started. 🫶
🎀 partitions : UBI layout 🎀
This tag allows one to change the layout of UBI volumes. Please take care that MTDs are not touched — they are configured by the Device Tree or in another way directly in kernel. 🌸
partitions: (
{
name = <volume name>;
size = <size in bytes>;
device = <MTD device>;
}
);Okay bestie, listen 💖 — every single field here is mandatory, no skips, no shortcuts, fr fr.
So here's the vibe: SWUpdate goes looking for a volume with the name you gave it, and if it needs to, it'll adjust the size or the type (more on that below 👇). But if no volume with that name exists? It just creates a whole new volume on the UBI device that's attached to the MTD device you specified in device. And that device can be either a number like "mtd4" or a name — the name of the MTD device, like "ubi_partition". Either one, your call 🫶. The UBI device gets attached automatically, so don't stress about that part.
By default, swupdate creates a dynamic UBI volume. Want a static one instead? Just add this line to that partition entry:
data = "static";And here's a fun one 🎀 — if you give a size of 0, the volume gets deleted if it exists. Which is actually so useful: it's how you yeet orphan volumes that some older software version left lying around and nobody needs anymore. Spring cleaning energy 🧹✨
🌸 images 🌸
The images tag is the one that collects all the images getting installed onto the system. Its syntax is:
images: (
{
filename[mandatory] = <Name in CPIO Archive>;
volume[optional] = <destination volume>;
device[optional] = <destination volume>;
mtdname[optional] = <destination mtd name>;
type[optional] = <handler>;
/* optionally, the image can be copied at a specific offset */
offset[optional] = <offset>;
/* optionally, the image can be compressed if it is in raw mode */
compressed;
},
/* Next Image */
.....
);So, volume — that one is only for installing the image into a UBI volume. And important: volume and device cannot be used at the same time, it's giving toxic relationship, pick one 💀. If you set device, then the raw handler gets selected automatically for you. Slay.
Wanna update a UBI volume? Look:
{
filename = "core-image-base.ubifs";
volume = "rootfs";
}Wanna update an image in raw mode instead? Here you go:
{
filename = "core-image-base.ext3";
device = "/dev/mmcblk0p1";
}And to flash an image at a specific offset, the syntax is:
{
filename = "u-boot.bin";
device = "/dev/mmcblk0p1";
offset = "16K";
}That offset understands these multiplicative suffixes: K=1024 and M=1024*1024. Math era 🔥
⚠️ Warning time bestie: writing to flash in raw mode has to be handled in a whole special way. Flashes must be erased before copying, and writing into NAND has to take care of bad blocks and ECC errors. Because of all that, you gotta select the "flash" handler. No cap.
So, for example, to copy the kernel into the MTD7 of a NAND flash:
{
filename = "uImage";
device = "mtd7";
type = "flash";
}The filename is mandatory — it's the Name of the file extracted by the stream. And volume is only mandatory when you're doing UBI volumes; otherwise, don't use it, it doesn't belong there 🙅♀️.
Alternatively, for the handler "flash", you can specify the mtdname instead of the device name, like this:
{
filename = "uImage";
mtdname = "kernel";
type = "flash";
}And the flash handler also supports some optional properties for NAND devices, which is honestly such a moment 🌈:
noecc = "true"— write in raw mode without ECC (equivalent tonandwrite --noecc).oob = "true"— the input image contains OOB data, and SWUpdate writes it alongside page data (equivalent tonandwrite --oob). The input layout is[page data][oob]repeated for each page.
Example, because seeing it hits different:
{
filename = "raw-nand.bin";
device = "mtd7";
type = "flash";
properties = {
noecc = "true";
oob = "true";
};
}🦋 Files 🦋
Okay so — you can copy single files instead of whole images. Lowkey not the preferred way, but it exists for debugging or special purposes, so iykyk 🙈
files: (
{
filename = <Name in CPIO Archive>;
path = <path in filesystem>;
device[optional] = <device node >;
filesystem[optional] = <filesystem for mount>;
properties[optional] = {create-destination = "true";}
}
);Entries in the "files" section are managed as single files. The attributes "filename" and "path" are mandatory. The attributes "device" and "filesystem" are optional — and what they do is tell SWUpdate to mount device (of the given filesystem type, like "ext4") before copying "filename" over to "path". If you don't give "device" and "filesystem", then "filename" just gets copied to "path" in the current rootfs. Simple 💅
General rule numero uno: swupdate doesn't copy out a file if the destination path doesn't exist. But you can change that behavior with the special property "create-destination". Iconic.
General rule numero dos: the raw file handler installs the file directly to the specified path. So if the target file already exists and the raw file handler gets interrupted, that existing file may end up replaced by an empty or partially written file 😭. And like, there are absolutely use cases where having an empty or corrupted file is worse than just keeping the existing one. That's exactly why the raw file handler supports an "atomic-install" property. Set it to "true" and the file gets installed to the specified path with ".tmp" appended to the filename. Once the contents are written and the buffer is flushed, the ".tmp" file gets renamed to the target file. Which minimizes the chances of an interrupted raw file handler creating an empty or corrupted file. Chef's kiss, honestly 👑✨
🎀 Scripts 🎀
Scripts run in the order they get put into the sw-description file, periodt. And SWUpdate valuates the result of a script — if the result is <> 0, the update stops with an error 🛑.
They get copied into a temporary directory before execution, and their name must be unique inside the same cpio archive. No duplicate names bestie, it's giving chaos otherwise.
If no type is given, SWUpdate defaults to "lua". But heads up — running a shell script opens up a whole set of different security issues, so go check the chapter "Best practice" too. Safety first 🫶
🌙 Lua
scripts: (
{
filename = <Name in CPIO Archive>;
type = "lua";
}
);Lua scripts get run using the internal interpreter. And they must have at least one of the following functions:
function preinst()SWUpdate scans for all scripts and checks for a preinst function. It's called before installing the images.
function postinst()SWUpdate scans for all scripts and checks for a postinst function. It's called after installing the images.
function postfailure()Only in case an update fails, SWUpdate scans for all scripts and checks for a postfailure function. Which is genuinely so useful when you need to restore a previous state — like, say the application got stopped and it should run again. Cleanup arc 🌸
🐚 shellscript
SWUpdate will run the binary shell /bin/sh to execute the script.
scripts: (
{
filename = <Name in CPIO Archive>;
type = "shellscript";
}
);Shell scripts are called by forking the process and running the shell as /bin/sh. SWUpdate scans for all scripts and calls them before and after installing the images. And it passes 'preinst', 'postinst' or 'postfailure' as the first argument to the script. If the data attribute is defined, its value gets passed as the last argument(s) to the script. So much handing-off 🫠
🌟 preinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "preinstall";
}
);These are shell scripts, and they're called via system command. SWUpdate scans for all scripts and calls them before installing the images. If the data attribute is defined, its value is passed as the last argument(s) to the script.
But lowkey be careful: it cannot be ensured thatpreinstallscripts run before an artifact is installed in streaming mode. Fr fr — if streaming is activated, the artifact must be installed as soon as it's received from the network, because there's no temporary copy to hang onto. And since there's no fixed order in the SWU, an artifact can be packed before any script in the SWU. The right way to do this is to write an"embedded-script"in Lua insidesw-description— because it becomes part ofsw-description, it runs whensw-descriptionis parsed and before any handler runs, even before a partition handler. That's the move 👑
💜 postinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "postinstall";
}
);postinstall are shell scripts too, called via system command. SWUpdate scans for all scripts and calls them after installing the images. If the data attribute is defined, its value is passed as the last argument(s) to the script.
✨ Update Transaction and Status Marker ✨
So by default, SWUpdate sets the bootloader environment variable "recovery_status" to "in_progress" prior to an update operation, and then either unsets it or sets it to "failed" after the update operation. This is essentially an interface for SWUpdate-external tooling, and here's how you read it:
- If there's no
"recovery_status"variable in the bootloader's environment → the update operation has been successful 🎉 - Else, if there is a
"recovery_status"variable with the value"failed"→ the update operation has not been successful 💀
While this is in general essential behavior for firmware updates, it doesn't have to be for less critical update operations. So whether or not the update transaction marker gets set by SWUpdate can be controlled by the boolean switch "bootloader_transaction_marker", which is global per sw-description file. It defaults to true.
This example snippet disables the update transaction marker:
software =
{
version = "0.1.0";
bootloader_transaction_marker = false;
...It's also possible to disable setting of the transaction marker entirely — and independently of whatever's set in sw-description — by starting SWUpdate with the -M option. Options for days, we love to see it 💅
The same thing applies to setting the update state in the bootloader via its environment variable "ustate" (that's the default) to STATE_INSTALLED=1 or STATE_FAILED=3 after an installation. This behavior can be turned off globally via the -m option to SWUpdate, or per sw-description via the boolean switch "bootloader_state_marker".
That's the marker lore bestie, don't skip it 🌈
🦋 reboot flag 🦋
okay so listen — you can signal that a specific update doesn't need a reboot. like, it's possible 💅. SWUpdate evaluates this info just to let a backend know about the transaction result, that's it.
ngl here's the catch: if a postinstall script (that's the command line parameter -p) gets passed at startup to perform a reboot, it's gonna run anyway, because SWUpdate literally cannot know the nature of this script. no cap. 🫠
SWUpdate sends this info to the progress interface, and it's duty of the listeners to interpret it. The attribute is a boolean:
reboot = false;This attribute lives in the general section, same place where version hangs out. And bestie, you do not need to activate the flag with reboot = true because that's literally the default behavior — so only disabling makes sense. ✨
The tool swupdate-progress interprets the flag: if it was started with reboot support (the -r parameter), it checks if a "no-reboot" message comes in and disables rebooting the device for this specific update. When the transaction completes, the reboot feature is activated again in case a new update needs a reboot. This is how you get on the fly updates, where not the whole software gets updated and a reboot isn't required. slay. 💖
🎀 bootloader 🎀
There are two ways to update the bootloader (currently U-Boot, GRUB, and EFI Boot Guard) environment. 👑
First way: add a file with the list of variables to be changed and set "bootloader" as type of the image. This informs SWUpdate to call the bootloader handler to manage the file (requires enabling bootloader handler in configuration). There is one bootloader handler for all supported bootloaders. The appropriate bootloader must be chosen from the bootloader selection menu in menuconfig.
images: (
{
filename = "bootloader-env";
type = "bootloader";
}
)The format of the file is described in U-boot documentation. Each line is in the format:
<name of variable>=<value>if value is missing, the variable is unset.
The format is compatible with U-Boot "env import" command. It is possible to produce the file from target as result of "env export".
Comments are allowed in the file to improve readability, see this example:
# Default variables
bootslot=0
board_name=myboard
baudrate=115200
## Board Revision dependent
board_revision=1.0Second way: define in a group setting the variables that must be changed:
bootenv: (
{
name = <Variable name>;
value = <Variable value>;
}
)SWUpdate will internally generate a script that will be passed to the bootloader handler for adjusting the environment. 🪄
For backward compatibility with previously built.swuimages, the"uboot"group name is still supported as an alias. However, its usage is deprecated. 💀
🧚 SWUpdate persistent variables 🧚
Not all updates require to inform the bootloader about the update, and in many cases a reboot is not required. There are also cases where changing bootloader's environment is unwanted due to restriction for security. So SWUpdate needs some info after new software is running to understand if everything is fine or if some actions like a fallback are needed.
SWUpdate can store such as information in variables (like shell variables), that can be stored persistently. The library libubootenv provide a way for saving such kind as database in a power-cut safe mode. 💾 It uses the algorythm originally implemented in the U-Boot bootloader. It is then guaranteed that the system will always have a valid instance of the environment. The library supports multiple environment databases at the same time, identifies with namespaces.
SWUpdate should be configured to set the namespace used for own variables. This is done by setting the attribute namespace-vars in the runtime configuration file (swupdate.cfg). See also example/configuration/swupdate.cfg for details. 📝
The format is the same used with bootloader for single variable:
vars: (
{
name = <Variable name>;
value = <Variable value>;
}
)SWUpdate will set these variables all at once like the bootloader variables. These environment is stored just before writing the bootloader environment, that is always the last step in an update. ⭐
🌸 Board specific settings 🌸
Each setting can be placed under a custom tag matching the board name. This mechanism can be used to override particular setting in board specific fashion.
Assuming that the hardware information file /etc/hwrevision contains the following entry:
my-board 0.1.0and the following description:
software =
{
version = "0.1.0";
my-board = {
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
}SWUpdate will set bootpart to 0:2 in bootloader's environment for this board. For all other boards, bootpart will be set to 0:1. Board specific settings take precedence over default scoped settings. 💅
🦋 Software collections and operation modes 🦋
Software collections and operations modes extend the description file syntax to provide an overlay grouping all previous configuration tags. The mechanism is similar to Board specific settings and can be used for implementing a dual copy strategy or delivering both stable and unstable images within a single update file. no cap, iconic fr fr.
The mechanism uses a custom user-defined tags placed within software scope. The tag names must not be any of: version, hardware-compatibility, uboot, bootenv, files, scripts, partitions, images
An example description file:
software =
{
version = "0.1";
hardware-compatibility = [ "revA" ];
/* differentiate running image modes/sets */
stable:
{
main:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p2";
}
);
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
alt:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p1";
}
);
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
};
};
}The configuration describes a single software collection named stable. Two distinct image locations are specified for this collection: /dev/mmcblk0p1 and /dev/mmcblk0p2 for main mode and alt mode respectively.
This feature can be used to implement a dual copy strategy by specifying the collection and mode explicitly. 🎀
🧚 Versioning schemas in SWUpdate 🧚
SWUpdate can perform version comparisons for the whole Software by checking the version attribute in the common part of sw-description and / or for single artifacts. SWUpdate supports two different version schemas, and they must be followed if version comparison is requested.
✨ Numbering schema (default) ✨
SWUpdate supports a version based on the schema:
<major>.<minor>.<revision>.<build>where each field is a plain number (no alphanumeric) in the range 0..65535. User can add further fields using the dot separator, but they are not considered for version comparison. SWUpdate will check if a version number is set according to this rule and fall back to semantic version upon failure. The version is converted to a 64 bit number (each field is 16 bit) and compared against the running version of the same artifact.
Please consider that, because additional fields are descriptive only, for the comparison they are not considered. This example contains version numbers that are interpreted as the same version number by SWUpdate:
1.2.3.4
1.2.3.4.5
1.2.3.4.5.6But the following is different:
1.2.3.4-alphaAnd it is treated as semantic version.
🌈 Semantic version 🌈
SWUpdate supports semantic <https://semver.org/> version. See official documentation for more details. iykyk 💖
👑 Checking version of installed software 👑
SWUpdate can optionally verify if a sub-image is already installed and, if the version to be installed is exactly the same, it can skip to install it. This is very useful in case some high risky image should be installed or to speed up the upgrade process.
One case is if the bootloader needs to be updated. In most time, there is no need to upgrade the bootloader, but practice showed that there are some cases where an upgrade is strictly required — the project manager should take the risk. 🫠 However, it is nicer to have always the bootloader image as part of the .swu file, allowing to get the whole distro for the device in a single file, but the device should install it just when needed.
SWUpdate searches for a file (/etc/sw-versions is the default location) containing all versions of the installed images. This must be generated before running SWUpdate. The file must contain pairs with the name of image and version, as:
<name of component> <version>In sw-description, the optional attributes "name", "version", and "install-if-different" provide the connection. Name and version are then compared with the data in the versions file. install-if-different is a boolean that enables the check for this image. It is then possible to check the version just for a subset of the images to be installed. ✨
If used with "install-if-different", then version can be any string. For example:
bootloader 2015.01-rc3-00456-gd4978d
kernel 3.17.0-00215-g2e876afSo yeah, there's also this attribute called install-if-higher 😌 and what it does is literally just: is the version of the new software higher than the version of the software that's already installed? If that's false, bestie, the new software does not get installed. The whole point of this bad boy is to avoid installing an older version of software over a newer one. Protecting you from yourself, honestly 💅
Now about that version thing — it can come in 2 formats. Either it's up to 4 numbers in the range 0..65535 separated by dots, like <major>.<minor>.<rev>.<build>, or it's a semantic version (https://semver.org). Let's see some real ones:
bootloader 2018.03.01
kernel 3.17.0-pre1+g2e876af
rfs 0.17-foo3.bar5+2020.07.01
app 1.7Real talk, bestie: do NOT mix version formats! Semantic versions only support 3 numbers (major, minor, patch) and the fourth number will be silently dropped if present. Silently. It won't even tell you. It's giving ghosting 😭
✨ Update Types ✨
Okay so sometimes you genuinely want distinct updates, right? Like an OS update vs. an Application update — two totally different vibes. SWUpdate is unaware of this, and the rules get defined globally into the configuration file (swupdate.cfg) or in the description file as part of the SWU.
But here's the thing 🫶 sometimes you need to overwrite some of those global rules, because specific updates require different setups. Like, an application update could require a different pre- or post- update script, and the accepted versions generally differ from the ones for the OS. That's exactly why update-types were introduced.
They must be listed in swupdate.cfg (go peek the example for this file), and sw-description will identify which is the type of the SWU. For example:
update-type = "application";To make it valid, swupdate.cfg must have an entry identifying it — otherwise the update will fail, because that specific type was not found. Then in swupdate.cfg there will be something like:
update-types: (
{
name = "OS";
postupdatecmd = "/usr/bin/echo Finished !";
no-downgrading = "5.0";
preupdatecmd = "/usr/bin/echo Hello";
max-version = "8.99";
no-reinstalling = "5.5";
reboot = "enabled";
},
{
name = "Application";
postupdatecmd = "/usr/bin/echo Finished !";
no-downgrading = "3.0";
preupdatecmd = "/usr/bin/echo Hello";
max-version = "5.99";
no-reinstalling = "4.5";
reboot = "disabled";
}
)Further releases could add new parameters that are specific for an update, so watch this space 👀
If a type is specified, SWUpdate will overwrite the rules set in the "global" section of swupdate.cfg with the specific one. And values in the "globals" are still identified with a type — one that SWUpdate calls "default".
There's also a flag in swupdate.cfg (update-type-required, globals section) that will force "update-type" to be mandatory — so any SWU without the type set will be simply rejected. No type, no entry, sorry babe 🙈
And the flag reboot identifies whether the update will be followed by a reboot, or if it's marked as "no rebootable". This can still be overwritten by the global attribute "reboot" in sw-description. Layers on layers, iconic.
✨ Embedded Script ✨
Bestie you can literally embed a script inside sw-description. It's a lifesaver in a lot of conditions where some parameters are only known by the target at runtime. The script is global to all sections, but it can contain several functions that can be specific for each entry in the sw-description file. Versatile king 🌈
These attributes are used for an embedded-script:
embedded-script = "<Lua code>"Now, must be taken into account: the parser has already run, and usage of double quotes can interfere with the parser. So for this reason, each double quote in the script must be escaped.
That means a simple Lua code like:
print ("Test")must be changed to:
print (\"Test\")If not, the parser thinks it's found the closure of the script and this generates an error 💀 See the examples directory for examples how to use it.
Any entry in files or images can trigger one function in the script. The "hook" attribute tells the parser to load the script and to search for the function pointed to by the hook attribute. For example:
files: (
{
filename = "examples.tar";
type = "archive";
path = "/tmp/test";
hook = "set_version";
preserve-attributes = true;
}
);After the entry is parsed, the parser runs the Lua function pointed to by hook. And if Lua is not activated, the parser raises an error — because a sw-description with an embedded script must be parsed, but the interpreter just isn't available 😭
Each Lua function receives as parameter a table with the setup for the current entry. A hook in Lua is in the format:
function lua_hook(image)image is a table where the keys are the list of available attributes. If an attribute contains a "-", it is replaced with "_", because "-" cannot be used in Lua. This means, for example, that:
install-if-different ==> install_if_different
installed-directly ==> installed_directlyAttributes can be changed in the Lua script and values are taken over on return. The Lua function must return 2 values:
- a boolean, to indicate whether the parsing was correct
- the
imagetable ornilto indicate that the image should be skipped
Example:
function set_version(image)
print (\"RECOVERY_STATUS.RUN: \".. swupdate.RECOVERY_STATUS.RUN)
for k,l in pairs(image) do
swupdate.trace(\"image[\" .. tostring(k) .. \"] = \" .. tostring(l))
end
image.version = \"1.0\"
image.install_if_different = true
return true, image
endThe example sets a version for the installed image. Generally, this is detected at runtime reading from the target. Main character energy, fr fr 👑
✨ Attribute reference ✨
There are 4 main sections inside sw-description, and yes, you're gonna need to know all of them:
images: entries are images and SWUpdate has no knowledge about them. It's just carrying the box, bestie 📦files: entries are files, and SWUpdate needs a filesystem for them. This is generally used to expand from a tar-ball or to update single files.scripts: all entries are treated as executables, and they will be run twice — as pre- and post- install scripts. Twice. Overachiever 💅bootenv: entries are a pair with bootloader environment variable name and its value.
✨ Attributes in sw-description ✨
ok so this is the part where we learn what every little attribute inside sw-description actually does. this is the main character table of the whole page bestie, so pay attention 💅 every entry has a Name, a Type, which things it Applies to, and a Description. let's go one by one, no skipping, no cap 🫶
filename— string — applies to: images, files, scripts — the filename exactly as it's found in the cpio archive 🎀volume— string — applies to: images — literally only iftype = "ubivol". it's the UBI volume where the image must be installed 🦋ubipartition— string — applies to: images — again, only iftype = "ubivol". it's the volume to be created or adjusted with a new size ✨device— string — applies to: images, files — the devicenode as found in/devor a symlink to it. you can give it as an absolute path OR as a name inside the/devfolder. for example, if/dev/mtd-dtbis a link to/dev/mtd3, then"mtd3","mtd-dtb","/dev/mtd3"and"/dev/mtd-dtb"are all valid names 💖. how it gets used depends on the handler — for files, it tells you which device the"filesystem"must be mounted on. and if you don't specify it at all, the current rootfs will be used 🌸filesystem— string — applies to: files — tells you the filesystem type where the file must be installed. heads up tho, it's only used if the"device"attribute is set 👑path— string — applies to: files — for files, this is the absolute path where the file must be installed. if"device"and"filesystem"are both set, SWUpdate will install the file after mounting"device"with the"filesystem"type. (and yes,pathis always relative to the mount point — periodt 😌)preserve-attributes— bool — applies to: files — a flag controlling whether the following attributes get preserved when files are unpacked from an archive (assuming the destination filesystem supports them, obvs 💁). the list is:- timestamp
- uid/gid (numeric)
- perms
- file attributes
- extended attributes
type— string — applies to: images, files, scripts — a string identifier for the handler, exactly as it's set by the handler when it registers itself. examples:"ubivol","raw","rawfile", 🌈compressed— string — applies to: images, files — a string to indicate the"filename"is compressed and must be decompressed before being installed. the value denotes
…and we're picking right back up mid-table bestie, no time to waste 💅
- the compression-type row, still going from the last part — it's the compression type, and the currently supported values are
"xz","zlib","zstd"and"lz4". that's the whole vibe ✨ compressed— bool (deprecated 🚩) — applies to: images files — deprecated, so use the string form instead. if you passtrue, it's literally the same thing as saying'compressed = "zlib"'. no cap.installed-directly— bool — applies to: images — this flag says the image gets streamed into the target with no temporary copy involved at all. it's giving zero-waste. but heads up: not all handlers support streaming, so don't assume it'll just work everywhere 😭name— string — applies to: bootenv — the name of the bootloader variable you wanna set 🎀value— string — applies to: bootenv — the value that gets assigned to that bootloader variable ✨
name— string — applies to: images files — the name that identifies thesw-component. it can be any string, and it gets compared with the entries insw-versions👑version— string — applies to: images files — the version for thesw-component. again, any string you want, and it's compared with the entries insw-versions🌸description— string — a user-friendly description of the swupdate archive (any string, go wild) 💖reboot— bool — this one lets you disable the reboot for the update that's currently running. iconic.install-if-different— bool — applies to: images files — flag, and if it's set,nameandversionget compared against the entries insw-versions🦋install-if-higher— bool — applies to: images files — flag, and if it's set,nameandversionget compared against the entries insw-versions(yes bestie it's the same comparison, it's just a different flavour of the same idea) ⭐install-if-hash-different— bool — applies to: images (raw only) — flag, and if it's set, the hash of the image sitting on the board gets compared with the hash insw-description🔥encrypted— string — applies to: images files scripts — a string that says this artefact is encrypted with this specific cipher. for example:'encrypted = "aes-cbc"'. peepswupdate_aes.hfor the supported values 👀encrypted— bool — applies to: images files scripts — use the string form instead if the key is provided inside thesw-description. passingtrueis equal to'encrypted = "aes-cbc"'💜aes-key— string — applies to: images files scripts — the AES key for encrypted artifacts. 📌 Note: this key should only be used ifsw-descriptionis encrypted (symmetrically or- (okay so this is the tail end of the row that got started above — it carries on from the previous line, no new field name here) — …
asymmetrically). It must be provided as an ASCII hex string of16,24, or32characters. ivt(typestring, fitsimages,files,scripts) — Optional IVT for encrypted artefacts. It has no effect if"encrypted"is not set. Each artefact can have an own IVT so that an attacker can't guess the the key. It is an ASCII string of32chars.data(typestring, fitsimages,files,scripts) — this one is literally just a little pocket you can stuff arbitrary data into and hand it to a handler. 🫶sha256(typestring, fitsimages,files,scripts) — the sha256 hash of the image, file or script. This is what gets used for verification of signed images. 💅embedded-script(typestring, no handler column action) — Lua code that is embedded right inside thesw-descriptionfile. Sneaky-cute ngl.offset(typestring, fitsimages) — optional destination offset. That's it, that's the whole vibe. ✨hook(typestring, fitsimages,files) — the name of the function (Lua) to be called when the entry is parsed.mtdname(typestring, fitsimages) — name of the MTD to update. Used only by theflashhandler to identify the the mtd to update, instead of specifying the devicenode. (yes, "the the" — that's the original's typo, we keep it, no cap 🎀)size(typeint64, fitsimages,files,scripts) — size of the file as it is expected in the SWU. If you set it and the cpio size does not match for some reason, the update will fail with an error. So don't be sloppy with this one bestie. 💀
🌈 UPDATE IMAGES FROM VERIFIED SOURCE 🌈
Okay so this part is important, like genuinely. It is becoming very very important that a device must not only be safely updated, but also that it can verify if the delivered image is coming from a known source and it was not corrupted introducing some malware. Two things: who sent this and is it still intact. 👑
To achieve this goal, SWUpdate must verify the incoming images. There are several ways to do this, and the drama starts immediately: should the compound image be signed? Or some parts of it? 🤔
Advantages and disadvantages are described in the following chapter. Grab a snack. 🍓
🦋 Signing the compound image
It looks quite straightforward if the whole compound image is signed. And like… sure, aesthetically? But this has some heavy drawbacks. It is not possible to know if the image is verified until the whole image is loaded. This means that verification can be done after installing the single images instead of doing it before touching the device. Which leads to having some uninstall procedure if part of a not verified image is already installed — procedures that cannot be safe in case of power off, letting some unwanted piece of software on the device. Yikes. Not the main character move. 💀
🎀 Signing the sub-images
If each sub-image is signed, the verification is done before calling the corresponding hardware. Only signed images can be installed. Slay, right? But anyway — this remains unbound with the description of the release in sw-description. Even if sw-description is signed, an attacker can mix signed images together generating a new compound image that can be installed as well, because all sub-images are verified. 🙈
💖 Combining signing sw-description with hash verification
To avoid the described drawbacks, SWUpdate combines signed sw-description with the verification of hashes for each single image. So: only sw-description generated by a verified source can be accepted by the installer, and sw-description contains hashes for each sub-image to verify that each delivered sub-image really belongs to the release. This is the iconic combo, periodt. ✨
🌸 Choice of algorithm
The algorithm chosen to sign and verify the sw-description file can be selected via menuconfig. Currently, the following mechanisms are implemented:
- RSA Public / private key. The private key belongs to the build system, while the public key must be installed on the target.
- CMS using certificates.
- GPG key signing.
For RSA and CMS algorithms, the key or certificate is passed to SWUpdate with the -k parameter. 🗝️
🧚 Architecture of crypto subsystem
The subsystem consists of crypto "providers", services and modules. A provider is a library that allows to build services — it is an external library like openSSL. Modules will use one of the providers to implement one service. SWUpdate defines three type of services: Verification, Hashing and Decryption.
The modules are the specific implementation of one of the services with an algorithm supported by the provider. It can be CMS, AES, pkcs#11, etc. Multiple modules can be installed, and the choice can be done at runtime. 👑
[image]
💜 Tool to generate keys / certificates
For RSA and CMS signing, the openssl tool is used to generate the keys. This is part of the OpenSSL project. A complete documentation can be found at the openSSL Website https://www.openssl.org/docs/manmaster/man1/openssl.html.
For GPG, gpg can be used to generate the keys and to sign the images. A complete documentation can be found at the GnuPG Website https://www.gnupg.org.
⭐ Usage with RSA PKCS#1.5 or RSA PSS
🍓 Generating private and public key
First, the private key must be created:
openssl genrsa -aes256 -out priv.pemThis asks for a passphrase. It is possible to retrieve the passphrase from a file — of course, this must be protected against intrusion. 🔒
openssl genrsa -aes256 -passout file:passout -out priv.pemThe private key is used to export the public key with:
openssl rsa -in priv.pem -out public.pem -outform PEM -pubout"public.pem" is holding your key in a format that swupdate is actually compatible with, and you can just hand that file over to swupdate on the command line with the -k parameter. bestie. that's it. that's the whole vibe. 💅
✨ How to sign with RSA ✨
Signing the image with rsa-pkcs#1.5 is genuinely so simple it's almost suspicious:
openssl dgst -sha256 -sign priv.pem sw-description > sw-description.sigAnd signing with rsa-pss? Equally low-drama, don't stress:
openssl dgst -sha256 -sign priv.pem -sigopt rsa_padding_mode:pss \
-sigopt rsa_pss_saltlen:-2 sw-description > sw-description.sig🎀 Usage with certificates and CMS 🎀
🌸 Generating self-signed certificates 🌸
okay so first you're gonna make yourself a little self-signed cert, and honestly the command is a whole meal:
openssl req -x509 -newkey rsa:4096 -noenc -keyout mycert.key.pem \
-out mycert.cert.pem -subj "/O=SWUpdate/CN=target" \
-addext extendedKeyUsage=1.3.6.1.5.5.7.3.4 -addext keyUsage=digitalSignatureCheck the docs for more tea about all the parameters involved. 🌸
"mycert.key.pem" is where your private key lives, and it's the thing you sign with. Do not ship it to the target — that key stays home, periodt. 🙅♀️
The target, however, must have "mycert.cert.pem" installed, because that's the one SWUpdate uses to verify. 🫶
Note: thatextendedKeyUsagevalue of"1.3.6.1.5.5.7.3.4"means"emailProtection". If you want"codeSigning"instead, use"1.3.6.1.5.5.7.3.3". iykyk 💜
🔥 Using PKI issued certificates 🔥
You can also use PKI issued code signing certificates, which is very main character of you. However — SWUpdate uses the OpenSSL library to handle CMS signatures, and by default that library requires the following attributes to be set on the signing certificate:
keyUsage=digitalSignature
extendedKeyUsage=emailProtectionNot vibing with the default extendedKeyUsage? You can change it by setting the cert-purpose parameter in swupdate.cfg. 💅
And if you literally cannot satisfy the key usage requirement at all, you can completely disable signing certificate key usage checking. That's controlled by the CONFIG_CMS_IGNORE_CERTIFICATE_PURPOSE configuration option. 🦋
🦋 How to sign with CMS 🦋
Signing is simple, same energy as the RSA case above, no cap:
openssl cms -sign -in sw-description -out sw-description.sig -signer mycert.cert.pem \
-inkey mycert.key.pem -outform DER -nosmimecap -binary🌈 Usage with GNU PG 🌈
⭐ Generating a new keypair ⭐
First things first, a primary keypair needs to be generated:
gpg --gen-keyAnd you can list all the keys you just generated like this:
gpg -kCheck the documentation for more info about the parameters, bestie. 📖
💜 How to sign with gpg 💜
Signing the image is very simple:
gpg --batch --output sw-description.sig
--detach-sig sw-descriptionNow — if you've got an alternative GnuPG home directory, and/or you're juggling multiple keypairs, here's how you specify exactly which is which. In this little example, the GnuPG home directory lives in GPG_HOMEDIR and the signing key is found in GPG_KEY:
gpg --batch --homedir "${GPG_HOMEDIR}" --default-key "${GPG_KEY}" --output sw-description.sig
--detach-sig sw-description👑 Building a signed SWU image 👑
So you've got two files: sw-description and its signature sw-description.sig. And this part is critical, babes — the signature file must always directly follow the description file. the way that order is non-negotiable... 😭
Every single image inside sw-description must carry the attribute "sha256", holding the SHA256 sum of that image. And if an image doesn't have the sha256 attribute? The whole compound image comes out as not verified and SWUpdate stops with an error before it even starts installing. 💀 no installation for you.
A simple script to create a signed image can be:
#!/bin/bash
MODE="RSA-PKCS-1.5"
PRODUCT_NAME="myproduct"
CONTAINER_VER="1.0"
IMAGES="rootfs kernel"
FILES="sw-description sw-description.sig $IMAGES"
#if you use RSA
if [ x"$MODE" == "xRSA-PKCS-1.5" ]; then
openssl dgst -sha256 -sign priv.pem sw-description > sw-description.sig
elif if [ x"$MODE" == "xRSA-PSS" ]; then
openssl dgst -sha256 -sign priv.pem -sigopt rsa_padding_mode:pss \
-sigopt rsa_pss_saltlen:-2 sw-description > sw-description.sig
elif if [ x"$MODE" == "xGPG" ]; then
gpg --batch --homedir "${GPG_HOME_DIR}" --default-key "${GPG_KEY}" \
--output sw-description.sig --detach-sig sw-description
else
openssl cms -sign -in sw-description -out sw-description.sig -signer mycert.cert.pem \
-inkey mycert.key.pem -outform DER -nosmimecap -binary
fi
for i in $FILES;do
echo $i;done | cpio -ov -H crc > ${PRODUCT_NAME}_${CONTAINER_VER}.swu🍓 Example for sw-description with signed image 🍓
This example is for a Beaglebone, installing Yocto images:
software =
{
version = "0.1.0";
hardware-compatibility: [ "revC"];
images: (
{
filename = "core-image-full-cmdline-beaglebone.ext3";
device = "/dev/mmcblk0p2";
type = "raw";
sha256 = "43cdedde429d1ee379a7d91e3e7c4b0b9ff952543a91a55bb2221e5c72cb342b";
}
);
scripts: (
{
filename = "test.lua";
type = "lua";
sha256 = "f53e0b271af4c2896f56a6adffa79a1ffa3e373c9ac96e00c4cfc577b9bea5f1";
}
);
}🧚 Running SWUpdate with signed images 🧚
Verification gets activated by setting CONFIG_SIGNED_IMAGES in SWUpdate's configuration. And once it's on, SWUpdate will always check the compound image. For security reasons, it is not possible to disable the check at runtime. that's the whole point, bestie, no takesies backsies. 🔒
For RSA and CMS signing, the -k parameter (public key file) is mandatory, and the program stops if the public key isn't passed. For CMS signing, CONFIG_SIGALG_CMS needs to be enabled.
When you're using CMS with OpenSSL, revocation checking can be enabled by passing --crl-path to SWUpdate. That file may contain one or more CRLs in PEM format, or a single CRL in DER format. And if the signer certificate is revoked, verification fails. 💀
swupdate -i image.swu -k /etc/swupdate/ca-chain.pem \
--crl-path /etc/swupdate/revocations.pemFor a DER-encoded CRL, just pass the binary file directly:
swupdate -i image.swu -k /etc/swupdate/ca-chain.pem \
--crl-path /etc/swupdate/revocations.derHeads up though: revocation is only checked against the signer certificate. Revoking an intermediate CA somewhere in the trust chain requires enabling full-chain checking with X509_V_FLAG_CRL_CHECK_ALL. 👑
By default, SWUpdate sets only X509_V_FLAG_CRL_CHECK. To enable full-chain checks, pass --crl-check-all (or set crl-check-all = true in the globals section of swupdate.cfg).
swupdate -i image.swu -k /etc/swupdate/ca-chain.pem \
--crl-path /etc/swupdate/revocations.pem \
--crl-check-allFor GPG signing, CONFIG_SIGALG_GPG needs to be enabled. The GPG key will need to be imported to the device's GnuPG home directory, and to do that, the key will need to be exported first:
gpg --export <keyid> --output <public key>Then you can copy it onto the device and import it into your public keyring:
gpg --import <public key>And to verify that the key has been imported successfully:
gpg --list-keysSWUpdate will need to be configured with the following parameters:
GnuPG Home directory: gpg-home-dir in swupdate.cfg
GPGME Protocol: gpgme-protocol in swupdate.cfg: openpgp or cms💖 SYMMETRICALLY ENCRYPTED UPDATE IMAGES 💖
okay so bestie, SWUpdate can also do symmetric encryption of your update images — that's the AES block cipher, running in CBC mode 🔐. The walkthrough below uses a 256-bit key length but honestly you can use other key lengths too, no cap.
✨ Building an Encrypted SWU Image ✨
First things first: you gotta make a key. For aes-256-cbc we need 32 bytes of key and 16 bytes for an initialisation vector (that's the IV, iykyk). The full documentation lives over at the OpenSSL Website <https://www.openssl.org/docs/manmaster/man1/openssl.html> if you wanna go deep 📚.
openssl rand -hex 32
# key, for example 390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040
openssl rand -hex 16
# IV, for example d5d601bacfe13100b149177318ebc7a4Then you encrypt an image with that info, like this:
openssl enc -aes-256-cbc -in <INFILE> -out <OUTFILE> -K <KEY> -iv <IV>where <INFILE> is the unencrypted source image file, and <OUTFILE> is the encrypted output image file that you're gonna reference in sw-description. <KEY> is the hex value part of the 2nd line of output from that key-generation command above, and <IV> is the hex value part of the 3rd line. literally just copy-paste them over 🫶
Then — and this part matters — you make a key file to hand to SWUpdate through the -K switch. Put the key and the initialization vector hex values on one line, separated by whitespace. For the example values above that looks like:
390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040 d5d601bacfe13100b149177318ebc7a4ngl, previous versions of SWUpdate allowed a salt as a third word in the key file. That was never actually used for aes, and it's been removed. bye 👋
bestie you should change the IV with every single encryption, see CWE-329 <http://cwe.mitre.org/data/definitions/329.html>. Also fyi: theivtsw-descriptionattribute overrides the key file's IV for one specific image. so it's giving "per-image override" energy ✨
🦋 Encryption of UBI volumes 🦋
So there's a limit in the Linux kernel API for UBI volumes — kinda annoying — which means the size reserved to be written on disk has to be declared before you actually write anything.
See the property decrypted-size in UBI Volume Handler's documentation. 📝
🎀 Example sw-description with Encrypted Image 🎀
Here's a (minimal) sw-description for installing a Yocto image onto a Beaglebone. Eyes on the encrypted = true; setting, that's the moment 👀
software =
{
version = "0.0.1";
images: ( {
filename = "core-image-full-cmdline-beaglebone.ext3.enc";
device = "/dev/mmcblk0p3";
encrypted = true;
ivt = "65D793B87B6724BB27954C7664F15FF3";
}
);
}💅 Running SWUpdate with Encrypted Images 💅
Symmetric encryption support wakes up when you set the ENCRYPTED_IMAGES option in SWUpdate's configuration. Then use the -K parameter to hand SWUpdate the symmetric key file you generated above. that's it, that's the whole vibe 🔥
🌸 Decrypting with a PKCS#11 token 🌸
PKCS#11 support is activated by setting the PKCS11 option in SWUpdate's configuration. In that case the key file has to have a PKCS#11 URL instead of the key — and it needs at least the elements from this example:
pkcs11:slot-id=42;id=%CA%FE%BA%BE?pin-value=1234&module-path=/usr/lib/libsofthsm2.so 65D793B87B6724BB27954C7664F15FF3You can import the encryption key into the PKCS#11 token using pkcs11-tool, like so:
echo -n "390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040" | xxd -p -r > swupdate-aes-key.bin
pkcs11-tool --module /usr/lib/libsofthsm2.so --slot 0x42 --login --write-object swupdate-aes-key.bin --id CAFEBABE --label swupdate-aes-key --type secrkey --key-type AES:32👑 Backing the PKCS#11 Token with a TPM2 (tpm2-pkcs11) 👑
The tpm2-pkcs11 <https://github.com/tpm2-software/tpm2-pkcs11> project provides a PKCS#11 library that stores all key material inside the TPM2 chip. Which means the AES image-decryption key never appears in plaintext outside the TPM at runtime. main character security behavior honestly 🔐✨
Prerequisites
The TPM 2 chip must support AES — and heads up, that's optional on the chip side. 🙈
Install the runtime library and the management tool (package names vary by distribution):
apt-get install libtpm2-pkcs11-tools libtpm2-pkcs11-1Setting up the TPM2-backed Token
The recommended workflow is:
- Generate the AES key on a trusted offline build host.
- Use it to encrypt the update images with OpenSSL (see above).
- Determine the slot-id of the TPM-backed token.
- Import the key into the TPM-backed token on the target device.
- Delete the plaintext key file from the target.
Step 1: Initialize the persistent store
Run once; creates a primary key inside the TPM:
tpm2_ptool initBy default the store is created in ~/.tpm2_pkcs11. For a system-wide deployment, set the environment variable TPM2_PKCS11_STORE to a suitable path before running all tpm2_ptool commands, for example:
export TPM2_PKCS11_STORE=/var/lib/tpm2_pkcs11
mkdir -p $TPM2_PKCS11_STORE
tpm2_ptool init --path=$TPM2_PKCS11_STOREStep 2: Create a new PKCS#11 token:
tpm2_ptool addtoken --pid=1 --sopin=<sopin> --userpin=<userpin> --label=swupdateStep 3: Determine the slot-id assigned to the token:
pkcs11-tool --module /usr/lib/libtpm2_pkcs11.so --list-slotsThe output lists all available slots with their slot IDs, for example:
Available slots:
Slot 0 (0x1): swupdate
token label : swupdate
...Step 4: Import the AES image-encryption key into the TPM-backed token:
echo -n "390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040" | xxd -p -r > swupdate-aes-key.bin
pkcs11-tool --module /usr/lib/libtpm2_pkcs11.so --slot <slot-id> --login --pin <userpin> --write-object swupdate-aes-key.bin --id CAFEBABE --label swupdate-aes-key --type secrkey --key-type AES:32
rm swupdate-aes-key.binStep 5: Verify the key is accessible:
pkcs11-tool --module /usr/lib/libtpm2_pkcs11.so --slot <slot-id> --login --pin <userpin> --list-objectsConfiguring SWUpdate
Create the key file with a PKCS#11 URL pointing to the TPM-backed slot, and append the IV as the second field (replace 1 with the actual slot ID you determined above, bestie):
pkcs11:slot-id=1;id=%CA%FE%BA%BE?pin-value=<userpin>&module-path=/usr/lib/libtpm2_pkcs11.so 65D793B87B6724BB27954C7664F15FF3Pass the key file to SWUpdate with the -K parameter as usual. And here's the slay part: because the decryption key is sealed inside the TPM, access is subject to the TPM's own authorization policy in addition to the PKCS#11 user PIN. double lock energy, obsessed 💜
more coming, don't go anywhere 🧚
ASYMMETRICALLY ENCRYPTED UPDATE IMAGES 🔐✨
okay bestie, so asymmetrically encrypted update images are a whole thing, and here's the tea: they're realized by an asymmetrically encrypted sw-description — which is what makes it possible to decrypt images device specific 👑. Like, per-device. The artifacts themselves? Still encrypted symmetrically, no change there. An AES key must be provided in the sw-description. And rn, at this very moment, only Cryptographic Message Syntax (CMS) is available for asymmetic decryption. That's it, that's the option set. 💅
Use Cases</p> 🦋
- Asymmetrically encrypted update images, with individual device key pairs, are inherently more secure than a purely symmetrical solution, because one compromised private device key does not affect the security of the others. slay, compartmentalization.
- If
CONFIG_SIGNED_IMAGESis enabled too and a device's private key gets compromised 😭, the key pair can be excluded from the list of eligible devices for receiving new update images. bye felicia. - The AES key can be securely exchanged with each new update image, since it's part of the
sw-description— even in the absence of direct access to the device. no cap, that's the good part.
Create a Self-Signed Device Key Pair 🎀
For the example, an elliptic curve key pair (PEM) is generated for a single device. Bestie, you must repeat these steps for all other devices. An RSA key pair could be used in the same way, if that's your vibe.
# Create a private key and a self-signed certificate
openssl ecparam -name secp521r1 -genkey -noout -out device-key-001.pem
openssl req -new -x509 -key device-key-001.pem -out device-cert-001.pem -subj "/O=SWUpdate /CN=target"
# Combine the private key and the certificate into a single file
# Note: The certificate is optional, but it speeds up key matching,
# especially when there are many different device keys.
cat device-key-001.pem device-cert-001.pem > device-001.pemSymmetric Encryption of Artifacts 💖
Generate an AES key and IV, exactly like you'd do for symmetric image encryption (that's the <#sym-encrypted-images> section, iykyk). The encryption process for the artifacts remains unchanged — we're not touching that part. periodt.
Encryption of sw-description for Multiple Devices 🌸
All device certificates togther are used for encryption. (yes, "togther", I'm quoting the manpage, don't come for me 🙈)
# Encrypt sw-description for multiple devices
openssl cms -encrypt -aes-256-cbc -in sw-description.in -out sw-description -outform DER $(printf ' -recip %q' certs/device-cert-*.pem)Assume sw-description.in is the plaintext input and sw-description is the encrypted output. After -recip, a list of all device certificates is expected. One for each device you want to encrypt for. 🫶
Decryption of sw-description for a Single Device 🧚
The combined key pair (private key + certificate) is used for decryption. SWUpdate handles the decryption process autonomously — so manually executing this step is not necessary, and it's provided here solely for development purposes. Like, dev only. Don't ship this as your pipeline, bestie. 💀
# Decrypt sw-description for a single device
openssl cms -decrypt -in sw-description -out sw-description.out -inform DER -inkey device-privatekey-001.pem -recip device-cert-001.pemAssume sw-description is the encrypted sw-description and sw-description.out is the plaintext output. The device private key is required for decryption, while the certificate itself is optional — but it helps speed up the key matching process. ⭐
Example Asymmetrically Encrypted Image 🔥
The image artifacts should be symmetrically encrypted and signed in advance. Now, create a plain sw-description.in file. The attributes: encrypted, aes-key and ivt are necessary for artifact decryption. It is totally okay that the AES key is included here, because the sw-description file will be encrypted afterwards. genius, honestly. 💅
software =
{
version = "1.0.0";
images: ({
filename = "rootfs.ext4.enc";
device = "/dev/mmcblk0p3";
sha256 = "131159df3a4efaa890ff80173664a125c496c458dd432a8a6acae18872e35822";
encrypted = "aes-cbc"; // former: encrypted = true;
aes-key = "ed73b9d3bf9c655d5a0b04836d8be48660a4a4bb6f4aa07c6778e00e342881ac";
ivt = "ea34a55a0c3476ed78f238ac87a7970c";
});
}Sign sw-description.in (the <#signed-images> situation) and create the signature file sw-description.sig:
openssl cms -sign -in sw-description.in -out sw-description.sig -signer signing-cert.pem -inkey signing-key.pem -outform DER -nosmimecap -binaryAsymmetrically encrypt the sw-description for multiple devices:
openssl cms -encrypt -aes-256-cbc -in sw-description.in -out sw-description -outform DER $(printf ' -recip %q' certs/device-cert-*.pem)Create the new update image (SWU):
#!/bin/sh
FILES="sw-description sw-description.sig rootfs.ext4.enc"
for i in $FILES; do
echo $i;done | cpio -ov -H crc > firmware.swuRunning SWUpdate with Asymmetrically Encrypted Images 🌈
Asymmetric encryption support can be enabled by configuring the compile-time option CONFIG_ASYM_ENCRYPTED_SW_DESCRIPTION, which depends on CONFIG_ENCRYPTED_SW_DESCRIPTION. To pass the combined key pair (PEM) you generated earlier to SWUpdate, use the -K argument. Or, alternatively, use the decryption-key parameter in the swupdate.cfg. your choice, main character. 👑
Security Considerations ⚠️✨
- Ideally, generate the private key on the device during factory provisioning, ensuring it never leaves the device. Only the public certificate leaves the device, for encrypting future update packages. 🤫
- This feature should be used in conjunction with signature verification (
CONFIG_SIGNED_IMAGES) to ensure data integrity. In principle, anyone with the corresponding device certificate can create update packages. so like, be aware of that. - As a side effect, the size of the update package may significantly increase in a large-scale deployment. To enhance scalability, consider using group keys. Smaller groups should be preferred over larger ones. For example, 1000 device keys (using
secp521r1) increase thesw-descriptionsize to0.35 MB. This means that by forming groups of 1000, it is possible to support 1 million devices. Alternatively, groups of 100 increase thesw-descriptionsize to3.5 MBaccordingly. math is mathing 📐 - Exchange the AES key in the
sw-descriptionwith each update package. - Avoid encrypting new update packages for compromised devices, if there is no direct access to the device or if unauthorized users have access to new update packages.
HANDLERS 🛠️💖
Overview 🦋
It is quite difficult to foresee all possible installation cases, bestie. Truly impossible. So instead of trying to find every use case ever, SWUpdate lets the developer be free to add his own installer — that is, a new handler — that must be responsible to install an image of a certain type. An image is marked to be of a defined type, to be installed with a specific handler. 🎯
The parser make the connection between 'image type' and 'handler'. It fills a table containing the list of images to be installed with the required handler to execute the installation. Each image can have a different installer. no cap.
Supplied handlers 🌸
In mainline there are the handlers for the most common cases. They include:
- flash devices in raw mode (both NOR and NAND)
- UBI volumes
- UBI volumes partitioner
- raw flashes handler (NAND, NOR, SPI-NOR, CFI interface)
- disk partitioner
- raw devices, such as a SD Card partition
- bootloader (U-Boot, GRUB, EFI Boot Guard) environment
- Lua scripts handler
- shell scripts handler
- rdiff handler
- readback handler
- archive (zo, tarballs) handler
- remote handler
- microcontroller update handler
For example, if an image is marked to be updated into a UBI volume, the parser must fill a supplied table setting "ubi" as required handler, and filling the other fields required for this handler: name of volume, size, and so on. 📝
Creating own handlers 🧚
SWUpdate can be extended with new handlers. The user needs to register his own handler with the core, and he must provide the callback that SWUpdate uses when an image is required to be installed with the new handler.
The prototype for the callback is:
int my_handler(struct img_type *img,
void __attribute__ ((__unused__)) *data)The most important parameter is the pointer to a struct img_type. It describes a single image and informs the handler where the image must be installed. The file descriptor of the incoming stream set to the start of the image to be installed is also part of the structure. 💖
The structure img_type contains the file descriptor of the stream pointing to the first byte of the image to be installed. The handler must read the whole image, and when it returns back SWUpdate can go on with the next image in the stream. don't leave that stream half-read, bestie 😭
The data parameter is usually a pointer that was registered with the handler. For script handlers it is instead a pointer to a struct script_handler_data which contains a script_fn enum value, indicating the current installation phase, and the registered data pointer.
SWUpdate provides a general function to extract data from the stream and copy to somewhere else:
int copyfile(int fdin, int fdout, int nbytes, unsigned long *offs,
int skip_file, int compressed, uint32_t *checksum, unsigned char *hash);fdin is the input stream, that is img->fdin from the callback. The hash, in case of signed images, is simply passed to copyfile() to perform the check, exactly as the checksum parameter. copyfile() will return an error if checksum or hash do not match. The handler does not need to bother with them. How the handler manages the copied data is specific to the handler itself. See supplied handlers code for a better understanding. 🔍
The handler's developer registers his own handler with a call to:
__attribute__((constructor))
void my_handler_init(void)
{
register_handler("mytype", my_handler, my_mask, data);
}SWUpdate uses the gcc constructors, and all supplied handlers are registered when SWUpdate is initialized. ✨
register_handler has the syntax:
register_handler(my_image_type, my_handler, my_mask, data);Where:
my_image_type: string identifying the own new image type.my_handler: pointer to the installer to be registered.my_mask:HANDLER_MASKenum value(s) specifying what input type(s)my_handlercan process.data: an optional pointer to an own structure, that SWUpdate saves in the handlers' list and pass to the handler when it will be executed.
Dummy Handler 💅
The always built-in and aptly named dummy handler copies the artifact to /dev/null, effectively skipping it and advancing to the next artifact, if any. it's giving "not my problem" energy and honestly? iconic. 👑
UBI Volume Handler 🔥
The UBI volume handler will update UBI volumes without changing the layout on the storage. Therefore, volumes must be created/adjusted beforehand — this can be done using the partitions tag (see partitions : UBI layout <#partitions-ubi-layout>). so set up your vibes first, then let the handler cook. 🍓
The UBI volume handler is obsessed with hunting down your volumes. It goes rummaging through every single MTD device (unless you've blacklisted them, see UBIBLACKLIST 💅) to find the volume where the image is supposed to land. Which means — volume names have to be unique across the whole system, bestie. Two volumes rocking the exact same name? Not supported, and it's giving unpredictable results 😭. SWUpdate will just install the image into the first volume with that name it stumbles across, and that might not be the right one at all!
But the good news is so good: when volumes get updated, the erase counters are preserved and never lost. Every time. The update behaviour is literally identical to what ubiupdatevol(1) from mtd-utils does — and actually, SWUpdate is straight up reusing the very same library from mtd-utils (libubi). Reuse queen behaviour 👑.
✨ atomic volume renaming ✨
The UBI volume handler has basic support for doing atomic volume renames, unlocked by defining the replaces property — and that property has to contain a valid UBI volume name. Once the image has been successfully installed into volume, an atomic swap of the names of volume and replaces happens. Look at this lil example:
{
filename ="u-boot.img";
volume ="u-boot_r";
properties: {
replaces = "u-boot";
}
}So after u-boot.img is successfully installed into the volume "u-boot_r", the volume "u-boot_r" gets renamed to "u-boot" and "u-boot" gets renamed to "u-boot_r". Switcheroo 🦋.
This mechanism lets you implement a simple double copy update approach without needing any shared state with the bootloader. For example, the U-Boot SPL can be configured to always load U-Boot from the volume u-boot, with zero need to access the environment. The volume replace functionality guarantees that this volume name always points at the currently valid volume. Iconic, honestly.
But bestie, please note the following limitations:
- Currently the rename takes place after each image was installed successfully. Which means it only makes sense to use this feature for images that are independent of the other installed images. The typical example is the bootloader. This behaviour may be modified in the future so that only one atomic rename is carried out, after all images were installed successfully.
- Atomic renames are only possible and permitted for volumes residing on the same UBI device.
There's also a handler called ubiswap that lets you do an atomic swap for several ubi volumes after all the images were flashed. From SWUpdate's point of view, this handler is a script, so the node that feeds it the data should be added in the scripts section:
scripts: (
{
type = "ubiswap";
properties: {
swap-0 = [ "boot" , " boot_r" ];
swap-1 = [ "kernel" , "kernel_r" ];
swap-2 = [ "rootfs" , "rootfs_r" ];
},
},
);WARNING: if you use the propertyreplaceson an ubi volume that is also used with the handlerubiswap, that ubi volume will get swapped twice. Yeah, that's probably not what you want... 💀
✨ volume auto resize ✨
The UBI volume handler can auto resize before flashing an image, using the property auto-resize. When this property is set on an image, the ubi volume gets resized to fit the image exactly:
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
auto-resize = "true";
}
}WARNING: when this property is used, the device must be defined.
✨ volume always remove ✨
The UBI volume handler can also always remove the ubi volume before flashing, using the property always-remove. When this property is set on an image, the ubi volume is always removed. And this property should be used together with the property auto-resize:
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
always-remove = "true";
auto-resize = "true";
}
}✨ size properties ✨
Okay so, due to a limit in the Linux kernel API for UBI volumes, the size reserved to be written on disk has to be declared before anything is actually written. Which is lowkey a problem, because the size of an encrypted or compressed image isn't known until the decryption or decompression has finished. So there's no way to correctly declare the file size that'll be written to disk 😭.
For this reason UBI images can declare the special properties "decrypted-size" or "decompressed-size", like this:
images: ( {
filename = "rootfs.ubifs.enc";
volume = "rootfs";
encrypted = true;
properties: {
decrypted-size = "104857600";
}
},
{
filename = "homefs.ubifs.gz";
volume = "homefs";
compressed = "zlib";
properties: {
decompressed-size = "420000000";
}
}
);The real size of the image should be calculated and written into the sw-description before assembling the cpio archive. In this example, 104857600 is the size of the rootfs after the decryption — the encrypted size is larger, btw. And the decompressed size of the homefs is 420000000.
The sizes are bytes in decimal notation. No vibe, just numbers 💅.
✨ Lua Handlers ✨
On top of the handlers written in C, you can extend SWUpdate with handlers written in Lua that get loaded at SWUpdate startup. The Lua handler source code file can either be embedded straight into the SWUpdate binary via the CONFIG_EMBEDDED_LUA_HANDLER config option, or it has to be installed on the target system in Lua's search path as swupdate_handlers.lua, so the embedded Lua interpreter can load it at run-time. Pick your lane, both slay.
Just like with C handlers, the prototype for a Lua handler is:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
...
endHere image is a Lua table (with attributes according to sw-description's attribute reference <# sw-description-attribute-reference>) that describes a single artifact to be processed by the handler (also see the Lua Handler Interface Specification in handlers/swupdate.lua, reproduced for convenience in Section Lua Handler Interface Specification).
Note that dashes in the attributes' names get replaced with underscores for the Lua domain, to make them idiomatic — e.g., installed-directly becomes installed_directly in the Lua domain. Clean ✨.
For a script handler written in Lua, the prototype is:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @param scriptfn string Type, one of `preinst` or `postinst`
--- @return number # 0 on success, 1 on error
function lua_handler(image, scriptfn)
...
endwhere scriptfn is either "preinst" or "postinst".
To register a Lua handler, the swupdate module provides the swupdate.register_handler() method, which takes the handler's name, the Lua handler function to be registered under that name, and — optionally — the types of artifacts for which the handler may be called. If that last bit isn't given, the Lua handler gets registered for all types of artifacts. This call registers the lua_handler function above as my_handler, which may be called for images:
swupdate.register_handler("my_handler", lua_handler, swupdate.HANDLER_MASK.IMAGE_HANDLER)A Lua handler can also call C handlers ("chaining") via the swupdate.call_handler() method. The callable and registered C handlers are available (as keys) in the table swupdate.handler. Here's some Lua code that's an example of a simple handler chain-calling the rawfile C handler:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
if not swupdate.handler["rawfile"] then
swupdate.error("rawfile handler not available")
return 1
end
image.path = "/tmp/destination.path"
local err, msg = swupdate.call_handler("rawfile", image)
if err ~= 0 then
swupdate.error(string.format("Error chaining handlers: %s", msg))
return 1
end
return 0
endNote that when you're chaining handlers and calling a C handler for a different type of artifact than the Lua handler is registered for, the image table's values have to satisfy the called C handler's expectations. Take the above Lua handler, registered for "images" (swupdate.HANDLER_MASK.IMAGE_HANDLER) via the swupdate.register_handler() call shown above. As per the sw-description's attribute reference <# sw-description-attribute-reference>, the "images" artifact type doesn't have the path attribute, but the "file" artifact type does. So for calling the rawfile handler, image.path has to be set before chain-calling the rawfile handler, exactly like in the example above. Usually though, no such adaptation is necessary if the Lua handler is registered for handling the type of artifact that image represents.
On top of calling C handlers, the image table passed as a parameter to a Lua handler has an image:copy2file() method that implements the super common use case of writing the input stream's data to a file — which is passed as this method's argument. On success, image:copy2file() returns 0, or -1 plus an error message on failure. Here's some Lua code that's an example of a simple handler calling image:copy2file():
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
local err, msg = image:copy2file("/tmp/destination.path")
if err ~= 0 then
swupdate.error(string.format("Error calling copy2file: %s", msg))
return 1
end
return 0
endAnd beyond using image:copy2file() or chain-calling C handlers, the image table passed as a parameter to a Lua handler has an image:read(<callback()>) method that reads from the input stream and calls the Lua callback function <callback()> for every chunk read, passing that chunk as the parameter. On success, 0 is returned by image:read(). On error, -1 plus an error message is returned. The following Lua code is an example of a simple handler printing the artifact's content:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
err, msg = image:read(function(data) print(data) end)
if err ~= 0 then
swupdate.error(string.format("Error reading image: %s", msg))
return 1
end
return 0
endso basically — the image:read() method is the moment 💅 it lets an artifact's contents get (post-)processed in Lua, and you get to use the full power of Lua while doing it, with zero reliance on preexisting C handlers built for whatever the purpose is. no cap, you just... write the thing you need.
and here's the rule you cannot skip, bestie: just like C handlers, a Lua handler must consume the artifact described in its image parameter, so that SWUpdate can keep going with the next artifact in the stream once your Lua handler returns. and look, satisfying that is not hard — chaining handlers, calling image:copy2file(), or using image:read() all do it. pick your fave. 🫶
the swupdate Lua module interface specification — literally the whole rundown of what functionality SWUpdate's corelib/lua_interface.c hands over to Lua handlers — is sitting in handlers/swupdate.lua. it's your reference, it works for mocking purposes, and it does type checking for you, all thanks to those EmmyLua-inspired annotations. iconic behavior, honestly. ✨
and a lil heads-up note 🎀 yes, Lua is dynamic, and technically that means somebody could embed Lua handlers straight into a .swu image that's about to be processed — but that's not implemented, because it carries some security implications, since the behavior of SWUpdate would be changed dynamically. so. no. we don't do that. 💀
📜 Lua Handler Interface Specification 📜
--[[
SWUpdate Lua Handler Interface.
Interface specification for the Lua module
provided by SWUpdate to Lua Handlers.
See: corelib/lua_interface.c
Copyright (C) 2022, Siemens AG
Author: Christian Storm <christian.storm@siemens.com>
SPDX-License-Identifier: GPL-2.0-or-later
--]]
---@diagnostic disable: missing-return
---@diagnostic disable: unused-local
-- luacheck: no unused args
--- SWUpdate Lua Handler Module.
--- @class swupdate
local swupdate = {}
--- @enum swupdate.RECOVERY_STATUS
--- Lua equivalent of `RECOVERY_STATUS` as in `include/swupdate_status.h`.
swupdate.RECOVERY_STATUS = {
IDLE = 0,
START = 1,
RUN = 2,
SUCCESS = 3,
FAILURE = 4,
DOWNLOAD = 5,
DONE = 6,
SUBPROCESS = 7,
PROGRESS = 8
}
--- Lua equivalent of `ERROR(format, ...)`.
--- @param format string Format string
--- @param ... any Varargs as referenced in format string
swupdate.error = function(format, ...) end
--- Lua equivalent of `TRACE(format, ...)`.
--- @param format string Format string
--- @param ... any Varargs as referenced in format string
swupdate.trace = function(format, ...) end
--- Lua equivalent of `INFO(format, ...)`.
--- @param format string Format string
--- @param ... any Varargs as referenced in format string
swupdate.info = function(format, ...) end
--- Lua equivalent of `WARN(format, ...)`.
--- @param format string Format string
--- @param ... any Varargs as referenced in format string
swupdate.warn = function(format, ...) end
--- Lua equivalent of `DEBUG(format, ...)`.
--- @param format string Format string
--- @param ... any Varargs as referenced in format string
swupdate.debug = function(format, ...) end
--- Lua equivalent of `notify(PROGRESS, ..., msg)`.
--- @param msg string Message to send to progress interface
--- @param cause number | nil `progress_cause_t` value as defined in `include/progress_ipc.h`
swupdate.progress = function(msg, cause) end
--- Lua equivalent of `notify(status, error, INFOLEVEL, msg)`.
--- @param status swupdate.RECOVERY_STATUS Current status, one of `swupdate.RECOVERY_STATUS`'s values
--- @param error number Error code
--- @param msg string Message
swupdate.notify = function(status, error, msg) end
--- Report whether dry-run installation is performed.
--
--- @return boolean # `true` if dry-run installation is performed, `false` otherwise
swupdate.is_dryrun = function() end
--- Update progress.
--
--- @param percent number Progress percent to set
swupdate.progress_update = function(percent) end
--- Mount a filesystem to a temporary mountpoint.
--
--- @param device string Device to mount
--- @param filesystem string Device's filesystem
--- @return string | nil # Mountpoint for use with `swupdate.umount(target)`, `nil` on error
swupdate.mount = function(device, filesystem) end
--- Unmount a mountpoint returned by `swupdate.mount()`.
--
--- @param target string Mountpoint to unmount
--- @return boolean | nil # `true` if successful, `nil` on error
swupdate.umount = function(target) end
--- @enum swupdate.ROOT_DEVICE
--- Lua equivalent of `root_dev_type` as in `include/lua_util.h`.
swupdate.ROOT_DEVICE = {
PATH = 0,
UUID = 1,
PARTUUID = 2,
PARTLABEL = 3
}
--- Current root device information.
--- @class swupdate.rootdev
--- @field type swupdate.ROOT_DEVICE Root device type, one of `swupdate.ROOT_DEVICE`'s values
--- @field value string Root device path
--- @field path string Full root device path, if identified, else nil
--- Get current root device.
--
--- @return swupdate.rootdev # Table containing type, and (full) path to root device
swupdate.getroot = function() end
--- Get the HW boot device of an eMMC.
--
--- @param devicepath string Fully qualified device path
--- @return integer # eMMC boot device number or `-1` on error
swupdate.emmcbootpart = function(devicepath) end
--- Table with major/minor numbers of the device on which the swupdate.stat()'d file resides.
--- @class swupdate.stat_dev
--- @field major number Major device number
--- @field minor number Minor device number
--- `struct stat`-alike Table.
--- @class swupdate.stat_info
--- @field mode "directory"|"named pipe"|"link"|"regular file"|"socket"|"block device"|"char device"|"unknown" File type and mode
--- @field dev swupdate.stat_dev ID of device containing file
--- @field ino number Inode number
--- @field nlink number Number of hard links
--- @field uid number User ID of owner
--- @field gid number Group ID of owner
--- @field rdev swupdate.stat_dev Device ID (if special file)
--- @field access string Time of last access, e.g., "Wed Jun 30 21:49:08 1993"
--- @field modification string Time of last modification, e.g., "Wed Jun 30 21:49:08 1993"
--- @field change string Time of last status change, e.g., "Wed Jun 30 21:49:08 1993"
--- @field size number Total size, in bytes
--- @field permissions string Unix file permissions string, e.g., "rwxr-xr-x"
--- @field blocks number Number of 512B blocks allocated
--- @field blksize number Block size for filesystem I/O
--- STAT(2) Wrapper.
--
--- @param pathname string Retrieve information about the file `pathname`
--- @return swupdate.stat_info | nil # `struct stat`-alike Table if successful, nil on error
swupdate.stat = function(pathname) end
--- Get SWUpdate's TMPDIRSCRIPT directory.
--
--- @return string # TMPDIRSCRIPT directory
swupdate.tmpdirscripts = function() end
--- Get SWUpdate's TMPDIR directory.
--
--- @return string # TMPDIR directory
swupdate.tmpdir = function() end
--- SWUpdate hardware information.
--- @class swupdate.hardware
--- @field boardname string SWUpdate's boardname
--- @field revision string SWUpdate's revision
--- Get SWUpdate hardware.
--
--- @return swupdate.hardware # Table with 'boardname' and 'revision' fields
swupdate.get_hw = function() end
--- SWUpdate version information.
--- @class swupdate.version
--- @field [1] number SWUpdate's version
--- @field [2] number SWUpdate's patch level
--- @field version number SWUpdate's version
--- @field patchlevel number SWUpdate's patch level
--- Get SWUpdate version.
--
--- @return swupdate.version # Table with 'version' and 'patchlevel' fields
swupdate.getversion = function() end
--- Get software Selection and Mode.
--
--- @return string # Selection
--- @return string # Mode
swupdate.get_selection = function() end
--- Get Linux Command Line Parameters
--
--- @return table # Table with keys and values fields
swupdate.get_cmdline = function() end
--- Set Bootloader environment key=value.
--
--- @param key string Bootloader environment key to set
--- @param value string Value to set `key` to in bootloader environment
swupdate.set_bootenv = function(key, value) end
--- Get Bootloader environment key's value.
--
--- @param key string Bootloader environment's key to get value from
--- @return string # Bootloader environment key's value, empty if key absent
swupdate.get_bootenv = function(key) end
--- Get table with linux kernel command line
--
--- @return table # Table with keys and value, for example t[root] return root device
swupdate.get_cmdline = function() end
--- @enum swupdate.HANDLER_MASK
--- Lua equivalent of `HANDLER_MASK` as in `include/handler.h`.
swupdate.HANDLER_MASK = {
IMAGE_HANDLER = 1,
FILE_HANDLER = 2,
SCRIPT_HANDLER = 4,
BOOTLOADER_HANDLER = 8,
PARTITION_HANDLER = 16,
NO_DATA_HANDLER = 32,
ANY_HANDLER = 1 + 2 + 4 + 8 + 16 + 32
}okay so picking right back up in the Lua docs ✨ — here's the whole shape of registering your own Handler and what an image actually hands you, and yes it is a lot, but every line matters so here we go:
--- Register a Lua function as Handler implementation.
--
-- The signature of the function to be registered as Handler implementation is:
--
-- --- @param image img_type Lua equivalent of `struct img_type`
-- --- @param fn | nil string `preinst` or `postinst` for `SCRIPT_HANDLER`s, else nil
-- --- @return number # 0 on success, 1 on error
-- function lua_handler(image, fn)
-- ...
-- end
--
--- @param name string Registered Handler's name
--- @param funcptr function Function to register as Handler implementation
--- @param mask number Type(s) for which to register the Handler, one or more ORed values of `swupdate.HANDLER_MASK`
swupdate.register_handler = function(name, funcptr, mask) end
--- Lua equivalent of `struct img_type {...}` as in `include/swupdate.h` plus `read()` and `copy2file()` functions.
--- @class img_type
--- @field name string `sw-component` name to check with `sw-versions`
--- @field version string `sw-component` version to check with `sw-versions`
--- @field filename string File name in CPIO archive
--- @field volume string If handler is "ubivol", the UBI volume name
--- @field type string The Handler name
--- @field device string Device node to operate on for mounting, flashing, ...
--- @field path string For FILE_HANDLERs: the relative destination path
--- @field mtdname string MTD device where image must be installed
--- @field data string Handler-specific arbitrary data
--- @field filesystem string For FILE_HANDLERs: the filesystem to mount
--- @field ivt string IVT of an encrypted artifact
--- @field installed_directly boolean Whether to stream image to `device` w/o temporary copy
--- @field install_if_different boolean Whether to compare `name` and `version` accordingly
--- @field install_if_higher boolean Whether to compare `name` and `version` accordingly
--- @field encrypted boolean Whether the artifact is encrypted
--- @field partition boolean Whether the artifact is a partitioner
--- @field script boolean Whether the artifact is a script
--- @field preserve_attributes boolean Whether to preserve attributes in archives
--- @field offset number Offset to seek to in artifact
--- @field size number Artifact size
--- @field checksum number Computed checksum, read-only
--- @field skip number `skip_t` enum number as in `include/swupdate.h`
--- @field compressed string `zlib`, `xz`, `zstd` or `lz4` (boolean value is deprecated)
--- @field properties table Properties Table equivalent as specified in `sw-description`
--- @field sha256 string sha256 hash of the image, file, or script
local img_type = {
--- Store the current image artifact to disk.
--
--- @param self img_type This `img_type` instance
--- @param path string Path to store the current image artifact to
--- @return number # 0 on success, -1 on error
--- @return string | nil # nil on success, error message on failure
['copy2file'] = function(self, path) end,
--- Process the current image artifact in Lua.
--
-- The `callback = function(chunk) ... end` is repeatedly called with
-- chunked artifact data of type `string` in its `chunk` parameter as
-- long as there is data available.
-- The callback function must completely consume the artifact data so
-- that SWUpdate can continue with the stream's next artifact after
-- the Lua Handler returns.
--
--- @param self img_type This `img_type` instance
--- @param callback fun(chunk: string):number Callback that is fed the image artifact in chunks; return < 0 on error.
--- @return number # 0 on success, -1 on error
--- @return string | nil # nil on success, error message on failure
['read'] = function(self, callback) end,
}
--- @class swupdate.handler
--- Chain-callable SWUpdate Handlers (Non-exhaustive).
--
-- Note: This list of built-in Handlers is non-exhaustive and
-- purely illustrative. At run-time, SWUpdate populates this
-- Table with the actually available Handlers according to its
-- compile-time configuration and Lua Handlers loaded.
--
--- @type table<string, number>
swupdate.handler = {
--- Archive Handler (See: `handlers/archive_handler.c`).
["archive"] = 1,
--- `tar` Archive Handler (See: `handlers/archive_handler.c`).
["tar"] = 1,
--- Bootloader Handler (See: `handlers/boot_handler.c`).
["bootloader"] = 1,
--- "dummy" Handler (See: `handlers/dummy_handler.c`).
["dummy"] = 1,
--- Delta Handler (See: `handlers/delta_handler.c`).
["delta"] = 1,
--- Disk Format Handler (See: `handlers/diskformat_handler.c`).
["diskformat"] = 1,
--- Disk Partition Handler (See: `handlers/diskpart_handler.c`).
["diskpart"] = 1,
--- Toggle Boot Flag Handler (See: `handlers/diskpart_handler.c`).
["toggleboot"] = 1,
--- Flash Hamming1 Handler (See: `handlers/flash_hamming1_handler.c`).
["flash-hamming1"] = 1,
--- Flash Handler (See: `handlers/flash_handler.c`).
["flash"] = 1,
--- Raw Image Handler (See: `handlers/raw_handler.c`).
["raw"] = 1,
--- Raw File Handler (See: `handlers/raw_handler.c`).
["rawfile"] = 1,
--- Raw Copy Handler (See: `handlers/raw_handler.c`).
["rawcopy"] = 1,
--- rdiff Image Handler (See: `handlers/rdiff_handler.c`).
["rdiff_image"] = 1,
--- rdiff File Handler (See: `handlers/rdiff_handler.c`).
["rdiff_file"] = 1,
--- Readback Handler (See: `handlers/readback_handler.c`).
["readback"] = 1,
--- Remote Handler (See: `handlers/remote_handler.c`).
["remote"] = 1,
--- SSBL Handler (See: `handlers/ssbl_handler.c`).
["ssblswitch"] = 1,
--- SWU Forward Handler (See: `handlers/swuforward_handler.c`).
["swuforward"] = 1,
--- UBI Volume Handler (See: `handlers/ubivol_handler.c`).
["ubivol"] = 1,
--- UBI Partition Handler (See: `handlers/ubivol_handler.c`).
["ubipartition"] = 1,
--- UBI Swap Handler (See: `handlers/ubivol_handler.c`).
["ubiswap"] = 1,
--- ucfw Handler (See: `handlers/ucfw_handler.c`).
["ucfw"] = 1,
--- Unique UUID Handler (See: `handlers/uniqueuuid_handler.c`).
["uniqueuuid"] = 1,
}
--- Chain-call another Handler.
--
--- @param handler string Chain-called Handler's name
--- @param image img_type Lua equivalent of `struct img_type` as in `include/swupdate.h`
--- @return number # 0 on success, 1 on failure
--- @return string | nil # nil on success, error message on failure
swupdate.call_handler = function(handler, image) end
return swupdatebestie that's the whole Lua surface area and it's giving main character API 💅 lowkey memorize register_handler, call_handler, and the img_type fields and you're unstoppable, fr fr.
🐚 Shell script handler 🐚
okay so this one runs a <pink>shell script</pink> that's packed straight into the SWU. and like — real talk — running a shell script opens a whole set of different security issues. Shell scripts are supported because everyone and their mom accepts them, but you should prefer Lua Scripts. (safety girlie behavior, we stan 🌸)
SWUpdate will run the binary shell /bin/sh to execute the script.
🧚 Lua script handler 🧚
a Lua script handler runs a script in the Lua language, and there are two possible ways to run it, so pay attention bestie 👑
- local — the script runs in its own (isolated) Lua state that is created for the script. The script has access only to function defined inside the script or functions provided by external libraries, like the internal SWUpdate library called via
"require(swupdate)". - global — SWUpdate create a Lua state at the beginning of an Update and this is valid until the update is terminated. In this case, the script has access to any function and structure that was defined during the update. For example, a function can be defined inside
sw-description, and the script can call it.
As default, each script runs in isolated / local Lua state. If the property "global-state" is set, then the common Lua state used for each Update transaction is taken. 🫶
Scripts ran in isolated context in previous versions. SWUpdate allocates a new Lua state, and import the basic libraries before loading the script. A script is then isolated, but it cannot access to function already loaded, or it is not possible to reuse functions from 2 or more scripts.
With the introduction of a per installation Lua state, Lua scripts can call functions already defined in previous scripts, or defined in sw-description. Because when a script is loaded, existing functions with the same name are overwritten, it was decided that functions in scripts must be unique, that is each function should be declared just once during an installation process. (no duplicate function names, periodt 💀)
This means that for global state, sw-description should contain the name of the function for each step (pre- , postinstall or postfailure) that should be called: the names preinst, postinst and postfailure are still valid in case the script runs with isolated state.
This allows also to load a script without executing if no functions are defined, and functions in the script can be called by later scripts. iconic, honestly.
Note that the handler will load the script in case of global state just once during the "preinstall" call. Later, it is assumed that functions will be already available.
Example:
scripts: (
{
filename = "testscript.lua";
type = "lua";
properties: {
global-state = "true";
preinstall = "pretest1";
}
},
{
filename = "test2script.lua";
type = "lua";
properties: {
global-state = "true";
postinstall = "posttest2";
postfailure = "failure";
}
}Okay so real quick, there are two scripts in play here, and they're both chilling in the global Lua state. That means functions over in test2script can totally find and run functions defined in testscript.lua — because bestie, they belong to the same context. 💅 But when preinstall scripts get called? Only the function "pretest1" from the first script runs, because no function name is defined for this step with the following scripts. Not me being picky, that's just how it works. 🫶
🌸 Remote handler 🌸
Okay so remote handlers were literally made for binding legacy installers without having to rewrite them all in Lua. Iconic behaviour. The remote handler forwards the image that needs installing to another process, then waits for an acknowledge to be sure the image actually got installed correctly. Lowkey the whole point. ✨
It makes use of the zeromq library — which is to simplify the IPC with Unix Domain Socket. The remote handler is pretty general: you describe in sw-description, using the "data" attribute, how to communicate with the external process. And the remote handler always acts as client — it tries a connect() using the socket identified by that "data" attribute. For example, a possible setup using a remote handler could be:
images: (
{
filename = "myimage"";
type = "remote";
data = "test_remote";
}
)The connection is instantiated using the socket test_remote (according to the "data" field's value, iykyk) in the directory pointed to by the environment variable TMPDIR, with /tmp as fall-back if TMPDIR is not set. If connect() fails, the remote handler signals that the update is not successful. 😭
Each zeromq message from SWUpdate is a multi-part message split into two frames:
- first frame contains a string with a command.
- second frame contains data and can be of
0bytes.
There are currently just two possible commands: INIT and DATA. After a successful connect, SWUpdate sends the initialization string in the format:
INIT:<size of image to be installed>The external installer is informed about the size of the image to be installed, so it can assign resources if it needs to. It will answer with the string ACK or NACK. Heads up bestie: the first NACK received by SWUpdate will interrupt the update. 🚨
After sending the INIT command, the remote handler will send a sequence of DATA commands, where the second frame in the message will contain chunks of the image to be installed. It is duty of the external process to take care of the amount of data transferred and to release resources when the last chunk is received. For each DATA message, the external process answers with a ACK or NACK message. No skipping, no ghosting the ACKs. 🎀
📡 SWU forwarder 📡
The SWU forwarder handler can be used to update other systems where SWUpdate is running. It's giving master/slaves energy — you know, where the master is connected to the network and the "slaves" are hidden to the external world. 👑 The master is then the only interface to the world, main character fr.
A general SWU can contain embedded SWU images as single artifacts, and the SWU handler will forward it to the devices listed in the description of the artifact. The handler can have a single "url" properties entry with an array of urls. Each url is the address of a secondary board where SWUpdate is running with webserver activated. The SWU handler expects to talk with SWUpdate's embedded webserver. This helps to update systems where an old version of SWUpdate is running, because the embedded webserver is a common feature present in all versions. History says slay. 💖
The handler will send the embedded SWU to all URLs at the same time, and setting installed-directly is supported by this handler.
[image]
The following example shows how to set a SWU as artifact and enables the SWU forwarder:
images: (
{
filename = "image.swu";
type = "swuforward";
properties: {
url = ["http://192.168.178.41:8080", "http://192.168.178.42:8080"];
};
});The SWU forwarder can be used as a generic uploader to an URL. This requires to enable Lua support. The back channel should still run via Websocket, if the connected server should communicate a progress status.
The handler allows to link an own Lua code that is able to parse the incoming data, and report an error. The properties field should contain the name of the Lua function that should be called to parse the answer from the remote system. Note that the data passed to Lua is converted to a string (and null terminated).
images: (
{
filename = "image.swu";
type = "swuforward";
properties: {
url = ["http://192.168.178.41", "http://192.168.178.42];
parser-function = "parse_answer";
};
});The parser function should be already loaded. The Lua function receives as input a string with the data received via the Websocket back channel. The function will read the data and return a single scalar value of swupdate_status, that is one of swupdate.RECOVERY_STATUS.*. Example:
function parse_answer(s)
if string.match (s, 'SUCCESS') then
return swupdate.RECOVERY_STATUS.SUCCESS
end
if string.match (s, 'FAILURE') then
return swupdate.RECOVERY_STATUS.FAILURE
end
return swupdate.RECOVERY_STATUS.RUN
end";The handler evaluates the return value, and stops when one of SUCCESS or FAILURE is returned. ✋
If no parser-function is passed, the handler will run the internal parser used to connect remote SWUpdate systems, and expects to see a JSON message sent by the Mongoose Webserver. 🦋
🔥 rdiff handler 🔥
The rdiff handler adds support for applying binary delta patches generated by librsync's <http://librsync.sourcefrog.net/> rdiff tool. Glow up for your patch sizes. ✨
Naturally, the smaller the difference between the diff's source and target, the more effective is using this handler rather than shipping the full target, e.g., via the image handler. Hence, the most prominent use case for the rdiff handler is when having a read-only root filesystem and applying a small update like security fixes or feature additions. But bestie, careful: if the sweet spot is crossed, an rdiff patch may even exceed the full target's size due to necessary patch metadata. 😭 Also note that in order to be most effective, an image to be processed with rdiff should be built deterministic (see reproducible-builds.org <https://reproducible-builds.org>).
The rdiff algorithm requires no resources whatsoever on the device as the patch is fully computed in the backend. Consequently, the backend has to have knowledge of the current software running on the device in order to compute a sensible patch. Alike, the patch has to be applied on the device to an unmodified source as used in the backend for patch computation. This property is in particular useful for resource-constrained devices as there's no need for the device to, e.g., aid in the difference computation. Iconic, honestly. 💅
First, create the signature of the original (base) file via rdiff signature <basefile> <signaturefile>. Then, create the delta file (i.e., patch) from the original base file to the target file via rdiff delta <signaturefile> <targetfile> <deltafile>. The <deltafile> is the artifact to be applied via this handler on the device. Essentially, it mimics running rdiff patch <basefile> <deltafile> <targetfile> on the device. Naturally for patches, the very same <basefile> has to be used for creating as well as for applying the patch to. No switching up the base file, okay?? 🫶
This handler registers itself for handling files and images. An exemplary sw-description fragment for the files section is
files: (
{
type = "rdiff_file"
filename = "file.rdiff.delta";
path = "/usr/bin/file";
}
);Note that the file referenced to by path serves as <basefile> and gets replaced by a temporary file serving as <targetfile> while the rdiff patch processing. 🌸
An exemplary sw-description fragment for the images section is
images: (
{
type = "rdiff_image";
filename = "image.rdiff.delta";
device = "/dev/mmcblk0p2";
properties: {
rdiffbase = ["/dev/mmcblk0p1"];
};
}
);Here, the property rdiffbase qualifies the <basefile> while the device attribute designates the <targetfile>. Note that there's no support for the optional offset attribute in the rdiff_image handler as there's currently no apparent use case for it and skipping over unchanged content is handled well by the rdiff algorithm. Periodt. 💜
🎀 ucfw handler 🎀
This handler allows one to update the firmware on a microcontroller connected to the main controller via UART. Parameters for setup are passed via sw-description file. Its behavior can be extended to be more general. The protocol is ASCII based. There is a sequence to be done to put the microcontroller in programming mode, after that the handler sends the data and waits for an ACK from the microcontroller. 🦋
The programming of the firmware shall be:
- Enter firmware update mode (bootloader)
- Set
"reset line"to logical"low" - Set
"update line"to logical"low" - Set
"reset line"to logical"high"
- Set
- Send programming message
$PROG;<<CS>><CR><LF>to the microcontroller. (microcontroller will remain in programming state)
- microcontroller confirms with
$READY;<<CS>><CR><LF>- Data transmissions package based from mainboard to microcontroller package definition:
- within a package the records are sent one after another without the end of line marker
<CR><LF> - the package is completed with
<CR><LF>
- within a package the records are sent one after another without the end of line marker
- The microcontroller requests the next package with
$READY;<<CS>><CR><LF> - Repeat step 4 and 5 until the complete firmware is transmitted.
- The keypad confirms the firmware completion with
$COMPLETED;<<CS>><CR><LF> - Leave firmware update mode
- Set
"Update line"to logical"high" - Perform a reset over the
"reset line"
- Set
<<CS>> : checksum. The checksum is calculated as the two's complement of the modulo-256 sum over all bytes of the message string except for the start marker "$". The handler expects to get in the properties the setup for the reset and prog gpios. They should be in this format:
properties = {
reset = "<gpiodevice>:<gpionumber>:<activelow>";
prog = "<gpiodevice>:<gpionumber>:<activelow>";
}Example:
…finishing that block off, here's the rest of it 💖
images: (
{
filename = "microcontroller-image";
type = "ucfw";
device = "/dev/ttymxc5";
properties: {
reset = "/dev/gpiochip0:38:false";
prog = "/dev/gpiochip0:39:false";
};
}
);🌀 SSBL Handler
Okay bestie, this one is so cool. It's the whole "two software sets, which one am I booting tho" situation, and it works by keeping a duplicated structure saved on the flash. And like, heads up: currently, only NOR flash is supported. No cap. 🙈
So check it: each of the two structures holds the address and the size of the image that a first loader is gonna load. And then there's a field holding the "age" — and that age gets incremented after every single switch, so you can literally just look at it and be like "oh, that's the active set." Main character field, honestly. 👑
📐 Structure of SSBL Admin
┌───────────────────────┬─────────────┐
│ SSBL Magic Number (29 │ Age (3 bit) │
│ bit)Name │ │
├───────────────────────┼─────────────┤
│ Image Address Offset │ │
├───────────────────────┼─────────────┤
│ Image Size │ │
└───────────────────────┴─────────────┘The handler implements a post install script. First thing it does is check the two structures for consistency, and it figures out which one is active by reading that first 32 bit value — the one with the magic number and the age chillin' in it. Then it increments the age and saves the brand new structure into the inactive copy. And after a reboot, the loader checks it and switches the software set. Literally a glow-up between boots. ✨
scripts: (
{
type = "ssblswitch";
properties: {
device = ["mtdX", "mtdY"];
offset = ["0", "0"];
imageoffs = ["0x780000", "0xA40000"];
imagesize = ["0x800000", "0x800000"];
}
}All the properties in sw-description are mandatory, okay? They define where the SSBL Administration data lives for both sets. And each property is an array of two entries — one value for each of the two SSBL administrations. Two sets, two entries, periodt. 💅
🎀 Properties for SSBL handler
device— string — the MTD device where the SSBL Admin Header is stored 📍offset— hex — the offset of the SSBL header inside that MTD deviceimageoffset— hex — the offset of the image that a bootloader is gonna load when this SSBL is the one that's setimagesize— hex — the size of the image that a bootloader is gonna load when this SSBL is set
🔍 Readback Handler
So you wanna know that the image actually got written properly? Yeah. Same. 🫶 That's exactly what this one's for — it calculates the sha256 hash of a partition (or, like, part of a partition, we're flexible) and compares it against a hash value you gave it. Vibe check for your flash. ✅
Here's how you use it, bestie:
scripts: (
{
device = "/dev/mmcblk2p1";
type = "readback";
properties: {
sha256 = "e7afc9bd98afd4eb7d8325196d21f1ecc0c8864d6342bfc6b6b6c84eac86eb42";
size = "184728576";
offset = "0";
};
}
);So size and offset are optional — everything else? mandatory, no exceptions, no skipping. 😤
🧾 Properties for readback handler
device— string — the partition which shall be verified 💽type— string — the identifier for the handlersha256— string — the expected sha256 hash of the partition 🔐size— string — data size (in bytes) to be verified. If it's0or simply not set, the handler will get the partition size from the device instead. Iconic behaviour honestly.offset— string — offset (in bytes) to the start of the partition. If it's not set, the default value0gets used. Chill default.
📋 Copy handler
This one copies one source to a destination. Simple. Elegant. No notes. 💅
It's a script handler, and there's no artifact in the SWU associated with it — so don't go looking for one. Its whole thing is copying configuration data, or parts that should be taken by the current installation.
It needs the mandatory property copyfrom, while device is the one holding the destination path. The handler does a byte copy, and honestly it does not care what the source is — file or partition, doesn't matter, it's built different. 🌈
There's also an optional type field you can set if you want the handler to be active as a preinstall or postinstall script. And if you don't set it? Then the handler gets called twice. Yeah. Twice. She's thorough like that. 😭
🏷️ Attributes for copy handler
device— string — if it's set, it's the destination 📥type— string — one of"preinstall"or"postinstall"
✨ Properties for copy handler ✨
Bestie, here's the whole vibe check for the copy handler's props, no cap:
size(string) — the data size (in bytes) to be copied. If it's0or just… not set at all, the handler will try to find the size from the device itself. She's resourceful like that 💅offset(string) — a value in dec or hex with the offset to be set for the source file, so you can copy just a part of it. Slicing, but make it embedded 🍰chain(string) — the handler to be called to install the data read from the"copyfrom"source. Passing the baton, iconic behaviour 🎀recursive(string) — recursive copy ifcopyfromis a directory ("true"or"false")create-destination(string) — create the destination path if it does not exist ("true"or"false"). Manifesting the folder into existence ✨
And here's how it actually looks in the wild:
scripts : (
{
device = "/dev/mmcblk2p1";
type = "copy";
properties : {
copyfrom = "/dev/mmcblk2p2";
type = "postinstall";
chain = "raw";
}
}🥾 Bootloader handler
Okay so the bootloader handler lets you set the bootloader's environment using a file. The file needs to have this format, periodt:
# Comments are allowed using the hash char
varname=valueEmpty lines get skipped, like they're not even there 🙈. This whole thing simplifies updating the entire environment instead of setting each variable one by one inside the "bootenv" section in sw-description. So much less admin, we love to see it.
The property nooverride allows you to skip variables that are already set in sw-description. If it's not set, variables set in bootenv are overwritten. Choose wisely bestie 💀
images: (
{
filename = "uEnv.txt";
type = "bootloader";
properties: {
nooverride = "true";
}
}
);
bootenv: (
{
name = "bootenv01key";
value = "SOME VALUE";
});In the example above, bootenv01key is not overwritten by a value in uEnv.txt, because the flag "nooverride" is set. She said "I'm not changing for anyone" 💅
💾 eMMC handler
This one is a script handler used to manage the eMMC boot partitions. Serving partition-management realness.
The emmc_boot selects the eMMC hardware boot partition based on the device name. The number following boot in the device path is what determines the boot partition to be selected:
/dev/mmcblkXboot0selects boot partition 0/dev/mmcblkXboot1selects boot partition 1
Then emmc_boot_toggle toggles the active eMMC hardware boot partition for the specified device. Flip-flopping, but on purpose 🔥
Example selecting boot partition 1:
scripts: (
{
type = "emmc_boot";
device = "/dev/mmcblk1boot1";
}Example toggling the active boot partition:
scripts: (
{
type = "emmc_boot_toggle";
device = "/dev/mmcblk1";
}📦 Archive handler
The archive handler extracts an archive to a destination path. It supports whatever format libarchive has been compiled to support — like, for example, even if SWUpdate itself has no direct support for xz, it can be possible to extract tar.xz files with it. Sneaky? No, just versatile 🦋
The attribute preserve-attributes must be set to preserve timestamps, uid/gid (numeric) and extended attributes. File modes (such as chmod 0755/0600) are always preserved — those stay no matter what, she doesn't play about permissions.
The property create-destination can be set to the string true to have SWUpdate create the destination path before extraction.
files: (
{
filename = "examples.tar.zst";
type = "archive";
path = "/extract/here";
preserve-attributes = true;
installed-directly = true;
properties: {
create-destination = "true";
}
}
);🗂️ Disk partitioner
This handler creates or modifies partitions using the library libfdisk. Handler must be put into the partitions section of sw-description — no exceptions bestie. Setup for each partition is put into the properties field of sw-description. After writing the partition table, it may create a file system on selected partitions. (Available only if CONFIG_DISKFORMAT is set.) 👑
✨ Properties for diskpart handler ✨
labeltype(string) —"gpt"or"dos"nolock(string) —"true"or"false"(default=false). This is like a force. If it is set, a lock failure will be ignored (lock will still be attempted).noinuse(string) —"true"or"false"(default=false). If set, it does not require the device to be not in use (mounted, etc.)partition-X(array) — array of values belonging to the partition number X
For each partition, an array of couples key=value must be given. The following keys are supported:
✨ Setup for a disk partition ✨
okay bestie, if you're laying down a diskpart you get to hand it a little bag of properties, and here's the whole menu — no cap, this is every single one 💅
size— string — how big the partition is. you can throwK,MandGon there for Kilobytes, Megabytes and Gigabytes, so go off with your units 🎀start— integer — the first sector of the partition. where it all begins, literally ✨name— string — the name of the partition, for when you want to call it something cutetype— string — the type of partition, and heads up because it has two whole different meanings. it's the hex code for a DOS (MBR) partition table, or it's the string identifier if you're on GPT. dual meaning icon behaviour 🔥dostype— string — the type of the DOS (MBR) partition entry when you're using a table with a"gpt"labeltype. using this option creates a hybrid MBR table, slay. it's the hex code for DOS (MBR) partition table. this is typically the move when you want a GPT formatted disk on a board that still demands a dos table entry for initial bootstrapping 🙈
👑 Note: a maximum of 3 partitions can have adostypespecified. this limit only applies to dos table entries and does not affect partitions without adostypespecified. so know your budget, bestie.
fstype— string — optional filesystem type to be created on the partition. if nofstypekey is given, no file will be created on the corresponding partition.vfat/ext2/ext3/ext4/btrfsfile system is supported 💖partuuid— string — the partition UUID (GPT only). if omitted, a UUID will be generated automatically. it's giving "the universe handles it" 🦋flag— string — The following flags are supported: Dos Partition :"boot"set bootflag 🌸force— string — if set to"true", existing file-system shall be overwritten uncoditionally. Default value is"false". so like, it will not ask, it will not text back, it will simply replace 🫶
GPT example:
partitions: (
{
type = "diskpart";
device = "/dev/sde";
properties: {
labeltype = "gpt";
partition-1 = [ "size=64M", "start=2048",
"name=bigrootfs", "type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B"];
partition-2 = ["size=256M", "start=133120",
"name=ldata", "type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7",
"fstype=vfat"];
partition-3 = ["size=512M", "start=657408",
"name=log", "fstype =ext4", 63DAF-8483-4772-8E79-3D69D8477DE4"];
partition-4 = ["size=4G", "start=1705984",
"name=system", "type=0FC63DAF-8483-4772-8E79-3D69D8477DE4"];
partition-5 = ["size=512M", "start=10094592",
"name=part5", "type=0FC63DAF-8483-4772-8E79-3D69D8477DE4"];
}
}MBR Example:
partitions: (
{
type = "diskpart";
device = "/dev/sde";
properties: {
labeltype = "dos";
partition-1 = [ "size=64M", "start=2048", "name=bigrootfs", "type=0x83"];
partition-2 = ["size=256M", "start=133120", "name=ldata", "type=0x83"];
partition-3 = ["size=256M", "start=657408", "name=log", "type=0x83"];
partition-4 = ["size=6G", "start=1181696", "name=system", "type=0x5"];
partition-5 = ["size=512M", "start=1183744", "name=part5", "type=0x83"];
partition-6 = ["size=512M", "start=2234368", "name=part6", "type=0x83"];
partition-7 = ["size=16M", "start=3284992", "name=part7", "type=0x6",
"fstype=vfat"];
}
}✨ Toggleboot Handler ✨
this one is a script handler, okay? it turns on the bootflag for one of a disk partition if the partition table is DOS. and if the table is GPT? it straight up reports an error 😭 — it's not flexible like that, it's a one-trick pony and honestly, respect.
script: (
{
type = "toggleboot";
device = "/dev/sde";
properties: {
partition = "1";
}
}✨ GPT Partition Handler ✨
✨ GPT Partition Installer ✨
okay so there's a handler called gptpart and it lets you write an image into a gpt partition that you select by the name. and listen — this handler does not modify the gpt partition (type, size, ...), it just writes the image in the GPT partition. it minds its own business, periodt 💅
images: (
{
filename = "u-boot.bin";
type = "gptpart";
device = "/dev/vdb";
volume = "u-boot-1";
offset = "1024";
},
{
filename = "kernel.bin";
type = "gptpart";
device = "/dev/vdb";
volume = "kernel-1";
},
);✨ GPT Partition Swap ✨
meet gptswap — it lets you swap gpt partitions after all the images were flashed 🌈. and here's the tea: this handler only swaps the name of the partition. it could be useful for a dual bank strategy, which is very main character of it. and for SWUpdate's point of view this handler is a script, so the node that provides it should be added in the section scripts. iykyk 🦋
Simple example:
scripts: (
{
type = "gptswap";
device = "/dev/vdb";
properties =
{
swap-0 = [ "u-boot-0" , "u-boot-1" ];
swap-1 = [ "kernel-0" , "kernel-1" ];
};
},
);✨ Diskformat Handler ✨
this handler checks if the device already has a file system of the specified type. (Available only if CONFIG_DISKFORMAT is set.) and if the file system does not yet exist? it will be created. manifesting a filesystem into being ✨. in case an existing file system shall be overwritten, this can be achieved by setting the property force to true 🔥
partitions: (
{
type = "diskformat";
device = "/dev/loop0p1";
properties: {
fstype = "vfat";
force = "true";
}
})✨ Unique UUID Handler ✨
this handler checks if the device already has a filesystems with a provide UUID 🎀. this is helpful in case the bootloader chooses the device to boot from the UUID and not from the partition number — because sometimes the bootloader is picky like that. one use case is with the GRUB bootloader when GRUB_DISABLE_LINUX_UUID is not set, as usual on Linux Distro as Debian or Ubuntu. so if you've ever wondered why your distro cares so much about a UUID instead of a plain old number, bestie, this is it 💜
The handler iterates all UUIDs given in sw-description and raises error if one of them is found on the device. It is a partition handler and it runs before any image is installed.
partitions: (
{
type = "uniqueuuid";
properties: {
fs-uuid = ["21f16cae-612f-4bc6-8ef5-e68cc9dc4380",
"18e12df1-d8e1-4283-8727-37727eb4261d"];
}
});BTRFS Handler 🦋
BTRFS Partition Handler
This handler wakes up if support for BTRFS is turned on, bestie. It lets you create and delete subvolumes while an update is happening. Very multitasking of it, honestly. 💅
partitions: (
{
type = "btrfs";
device = "/dev/loop0p1";
properties: {
command = < one of create" or "delete" >
path = <path for the subvolume>;
mount = "true" or missing;
create-destination = "true" or missing;
}
})If mount is set, SWUpdate will mount the device and the path is appended to the mountpoint used with mount. If device is already mounted, then path is the absolute path. No cap. ✨
BTRFS Snapshot Handler
Ooh okay so this one lets you install a BTRFS snapshot created with the btrfs send command. 🫶 SWUpdate is using the external btrfs utility — like, it has to be installed on the target, that's not optional — and btrfs receive is executed by sending the stream to that command.
All the generic features are available, which means an artifact can be streamed by using the "installed-directly" attribute. Iconic behaviour fr fr.
images: (
{
filename = "btrfs-snapshot";
type = "btrfs-receive";
device = <optional, device with BTRFS fs, must be set if tomount is "true">;
properties: {
path = <mandatory, path where to install subvolume>;
btrfs-cmd = <optional, path to btrfs command>;
tomount = <boolean, optional, "true" / "false" >;
}
})If tomount is set, SWUpdate will temporarily mount device as a BTRFS filesystem and will try to install the snapshot to path. The btrfs-cmd is optional, and the fallback is /usr/bin/btrfs. 💖
Generic Executor Handler
So the BTRFS snapshot handler needs to stream an artifact — after all the normal handling (decompression, decryption, etc.) — straight to the external btrfs command with no temporary copy. And the same infrastructure can be used to stream any artifact to any arbitrary external command that accepts the stream as stdin. That's what the "executor" handler does. Main character energy, honestly. 👑
{
filename = "test";
type = "executor";
properties: {
cmd = <mandatory, command to be executed in a pipe>;
}
}If create-destination is set, SWUpdate will create the destination path of the subvolume before creating it. 🌸
Delta Update Handler
The handler processes a ZCHUNK header and figures out which chunks should be downloaded after generating the corresponding header of the running artifact that needs updating. ✨ It uses just a couple of attributes from the main setup, and gets more info from the properties. Those attributes are then passed to a secondary handler that will install the artifact after the delta handler has assembled it. The handler requires ZST because, bestie, that's the compression format for Zchunk.
The SWU must contain just the ZCK's header, while the ZCK file is put as it is on the server. The utilities in the Zchunk project are used to build the ZCK file.
zck -u -h sha256 <artifact>This will generate a file <arifact>.zck. To extract the header, use the zck_read_header utility:
HSIZE=`zck_read_header -v <artifact>.zck | grep "Header size" | cut -d':' -f2`
dd if=<artifact>.zck of=<artifact>.header bs=1 count=$((HSIZE))The resulting header file must be packed inside the SWU. Periodt. 💯
Properties for delta update handler
url— type:string. This is the URL from where the handler will download the missing chunks. The server must support byte range header. 🔥source— type:string. Name of the device or file to be used for the comparison.chain— type:string. This is the name (type) of the handler that is called after reassembling the artifact. 🦋zckfile— type:string. Name of the external ZCK file. This is used only if URL is not present or it is set to"dynamic".max-ranges— type:string. Max number of ranges that a server can accept. Default value (150) should be ok for most servers. 💅zckloglevel— type:string. This sets the log level of the zcklib. Logs are intercepted by SWupdate and appear in SWUpdate's log. Value is one ofdebug,info warn,error,none.debug-chunks— type:string."true", default is not set. This activates more verbose debugging output and the list of all chunks is printed, and it reports if a chunk is downloaded or copied from the source. 🙈source-size— type:string. This limits the index of the source. It is helpful in case of filesystem in much bigger partition. It has the value for the size or it can be set to"detect"and the handler will try to find the effective size of fs. ✨
Example:
{
filename = "software.header";
type = "delta";
device = "/dev/mmcblk0p2";
properties: {
url = "http://examples.com/software.zck";
chain = "raw";
source = "/dev/mmcblk0p3";
zckloglevel = "error";
/* debug-chunks = "true"; */
};
}It is not always possible to set the URL into sw-description, okay? Hawkbit for example generates a URL when an artifact is uploaded, and the URL is just... not available during build. 😭 The Hawkbit connector will send the URL to SWUpdate, which adds it to its own list. If the URL is not present as a property, or it is set to "dynamic", the handler asks the core for the URL for the "zckfile" property. Note that the URL is then not part of sw-description and for this reason it is not verified.
Anyway, this is not a security issue, because all hashes related to the ZCK file are part of sw-description (as ZCK header), and each mismatch will stop the update. 🛑
The steps to use Delta Update are:
Hawkbit 🐦
- create a distribution
- add a software module to the distribution
- in the
"Upload", adds both the SWU and the ZCK file. If you have multiple artifacts that will be installed as delta update, upload all required ZCK.
sw-description</p> 🌸
Bestie, use the dynamic keyword to set the URL in the properties. SWUpdate will then go check if there's a URL for the value of zckfile. And the value of zckfile must be the filename of the ZCK file you uploaded to the Hawkbit server. No pressure or anything 💅 Example:
images: (
{
filename = "software.zck.header";
type = "delta";
device = "/dev/sde4";
properties: {
url = "dynamic";
chain = "raw";
source = "/dev/sde5";
zckloglevel = "info";
source-size = "detect";
zckfile = "software.rootfs.ext4.zck";
};
}Hawkbit also has some DOS countermeasures that should be deactivated, otherwise Hawkbit thinks it's literally under attack and refuses to deliver the chunks you need 😭 so do this:
hawkbit.server.security.dos.filter.enabled=false
hawkbit.server.security.dos.maxStatusEntriesPerAction=-1Memory issue with zchunk</p> 🫠
SWUpdate will create the header from the current version, often from a block partition. By default, Zchunk creates a temporary file with all chunks in /tmp, and that gets concatenated to the header at the end and written to the destination file. Which means — okay so — an amount of memory equal to the partition is required (SWUpdate does not compress the chunks). This was solved with later versions of Zchunk, so check inside the zchunk code if ZCK_NO_WRITE is supported. iykyk ✨
Docker Handlers 🐳
To improve containers update, a docker set of handlers implements part of the API to communicate with the docker daemon. Podman (another container solution) has a compatibility layer for the docker REST API and could be used as well, but SWUpdate is currently not checking if a podman daemon must be started. Just so you know 🫶
Goal of these handlers is just to provide the few API calls to update images and start containers — it does not implement the full API. Lowkey minimal, highkey useful 💅
Docker Load Image 📦
This handler allows you to load an image without copying it temporarily and push it to the docker daemon. It implements the /images/load entry point, and it is implemented as "image" handler. The file should be in a format accepted by docker.
images: (
{
filename = "docker-image.tar"
type = "docker_imageload";
installed-directly = true;
});The handler checks the return value (JSON message) from the daemon, and returns with success if the image is added. Slay ✅
In case the file must be decompressed, SWUpdate requires the size of the decompressed image to be passed to the daemon:
images: (
{
filename = "alpine.tar.gz";
type = "docker_imageload";
installed-directly = true;
compressed = "zlib";
properties: {
decompressed-size = "5069312";
};
});Docker Remove Image 🗑️
It is implemented as a script (post install). Example:
scripts: ( {
type = "docker_imagedelete";
properties: {
name = "alpine:3.4";
};
});Docker Delete Unused Images 🧹
It is implemented as a script (post install). Example:
scripts: ( {
type = "docker_imageprune";
});Docker: container create 🎀
It is implemented as a post-install script. The script itself is the json file passed to the daemon to configure and set the container. The container is just created, not started. Noted 📝
For example, having this hello-world.json file:
{
"Image": "hello-world",
"HostConfig": {
"RestartPolicy": {
"Name": "always"
},
"AutoRemove": false
}
}Creating the container can be done in sw-description with:
scripts: ( {
filename = "hello-world.json";
type = "docker_containercreate";
properties: {
name = "helloworld"; /* Name of container */
}
});Docker Remove Container 💀
It is implemented as a script (post install). Example:
scripts: ( {
type = "docker_containerdelete";
properties: {
name = "helloworld";
};
});Docker : Start / Stop containers 🔥
Examples:
scripts: ( {
type = "docker_containerstart";
properties: {
name = "helloworld";
};
});scripts: ( {
type = "docker_containerstop";
properties: {
name = "helloworld";
};
});MONGOOSE DAEMON MODE 🦦
Introduction ✨
Mongoose is a daemon mode of SWUpdate that provides a web server, web interface and web application. It's giving full-stack energy fr fr 👑
The web application in web-app uses the Node.js (https://nodejs.org/en/) package manager and gulp (https://gulpjs.com/) as build tool. It depends on Bootstrap 4 (https://getbootstrap.com/), Font Awesome 5 (https://fontawesome.com/) and Dropzone.js (http://www.dropzonejs.com/).
Startup 🚀
After having configured and compiled SWUpdate with enabled mongoose web server:
./swupdate --helplists the mandatory and optional arguments to be provided to mongoose. As an example,
./swupdate -l 5 -w '-r ./examples/www/v2 -p 8080' -p 'reboot'runs SWUpdate in mongoose daemon mode with log-level TRACE and a web server at http://localhost:8080. Iconic 🎀
Example 🌈
The ready to use example of the web application in the examples/www/v2 directory uses a Public Domain background.jpg image from pixabay (https://pixabay.com/de/leiterbahn-platine-technologie-3157431/) which is released under the Creative Commons CC0 license. The used favicon.png and logo.png images are made from the SWUpdate logo and therefore subject to the GNU General Public License version 2. You must comply to this license or replace the images with your own files. No skipping that part bestie 😭
Customize 💅
You could customize the web application inside the web-app directory. Beside replacing the favicon.png, logo.png and background.jpg images inside the images directory, you could customize the Bootstrap colors and settings inside the scss/bootstrap.scss style sheet. The style sheet changes need a rebuild of the web application source code. Ate 💖
Develop 🧚
The development requires Node.js version 18 or greater and a prebuilt SWUpdate project with enabled mongoose web server and web application interface version 2 support. Let's gooo 🦋
- Enter the web application directory:
cd ./web-app- Install the dependencies:
npm install- Build the web application:
npm run build- Start the web application:
../swupdate -w '-r ./dist -p 8080' -p 'echo reboot'- Test the web application:
http://localhost:8080/ - Pack the web application (optional):
npm run package -- --output swupdate-www.tar.gzContribute 🫶
Please run the linter before any commit
npm run lintSURICATTA DAEMON MODE 🐾
Introduction 💜
Suricatta is — like mongoose — a daemon mode of SWUpdate, hence the name suricatta (engl. meerkat) as it belongs to the mongoose family. Family reunion energy fr fr 🥺
Suricatta regularly polls a remote server for updates, downloads them, and installs them. Thereafter, it reboots the system and reports the update status to the server, based on an update state variable currently stored in bootloader's environment ensuring persistent storage across reboots. Some U-Boot script logics or U-Boot's bootcount feature may be utilized to alter this update state variable, e.g., by setting it to reflect failure in case booting the newly flashed root file system has failed and a switchback had to be performed. So much lore 😭
Suricatta is designed to be extensible in terms of the servers supported as described in Section The Suricatta Interface. Currently, support for the hawkBit (https://projects.eclipse.org/projects/iot.hawkbit) server is implemented via the hawkBit Direct Device Integration API (http://sp.apps.bosch-iot-cloud.com/documentation/developerguide/apispecifications/directdeviceintegrationapi.html) alongside a simple general purpose HTTP server. The support for suricatta modules written in Lua is not a particular server support implementation but rather an option for writing such in Lua instead of C. 💅
Running suricatta 🏃♀️
After having configured and compiled SWUpdate with enabled suricatta support for hawkBit,
./swupdate --helplists the mandatory and optional arguments to be provided to suricatta when using hawkBit as server. As an example,
./swupdate -l 5 -u '-t default -u http://10.0.0.2:8080 -i 25'runs SWUpdate in suricatta daemon mode with log-level TRACE, polling a hawkBit instance at http://10.0.0.2:8080 with tenant default and device ID 25. Periodt 🔥
If multiple server support is compiled in, the -S / --server option or a server entry in the configuration file's [suricatta] section selects the one to use at run-time. For convenience, when having support for just one server compiled-in, this is chosen automatically. Autopilot 🎀
Note that on startup when having installed an update, suricatta tries to report the update status to its upstream server, e.g., hawkBit, prior to entering the main loop awaiting further updates. If this initial report fails, e.g., because of a not (yet) configured network or a currently unavailable hawkBit server, SWUpdate may exit with an according error code. This behavior allows one to, for example, try several upstream servers sequentially. If suricatta should keep retrying until the update status is reported to its upstream server irrespective of the error conditions, this has to be realized externally in terms of restarting SWUpdate on exit. Yes, that's on you 😅
After an update has been performed, an agent listening on the progress interface may execute post-update actions, e.g., a reboot, on receiving DONE. Additionally, a post-update command specified in the configuration file or given by the -p command line option can be executed.
Note that at least a restart of SWUpdate has to be performed as post-update action since only then suricatta tries to report the update status to its upstream server. Otherwise, succinct update actions announced by the upstream server are skipped with an according message until a restart of SWUpdate has happened in order to not install the same update again. So like, actually restart it bestie, don't just vibe 💀
✨ The Suricatta Interface ✨
So you wanna talk to a server that is not hawkBit and not the general purpose HTTP one? Bestie, you absolutely can 💅. You just implement the "interfaces" that live in include/channel.h and include/suricatta/server.h — and the server one you can write in C or in Lua, your call 🦋.
The channel interface is the whole "how do I actually talk to this thing" layer — it abstracts one particular connection to the server, like HTTP-based when we're dealing with hawkBit. The server interface is where the vibes get logical: it defines the logics to poll and install updates. Want a real example in C that's targeted towards hawkBit? Go peek at corelib/channel_curl.c / include/channel_curl.h and suricatta/server_hawkbit.{c,h} ✨
include/channel.h describes the functionality a channel has to implement:
typedef struct channel channel_t;
struct channel {
...
};
channel_t *channel_new(void);and that channel_new() is the one that sets up and returns a channel_t struct stuffed with pointers to functions for opening, closing, fetching, and sending data over the channel. Iconic behavior, honestly 👑
Meanwhile include/suricatta/server.h describes what a server has to implement:
typedef struct {
server_op_res_t has_pending_action(int *action_id);
server_op_res_t install_update(void);
server_op_res_t send_target_data(void);
unsigned int get_polling_interval(void);
server_op_res_t start(const char *cfgfname, int argc, char *argv[]);
server_op_res_t stop(void);
server_op_res_t ipc(int fd);
void (*help)(void);
} server_t;These functions — the ones that make up a particular suricatta server implementation — have to be registered so they can be selected at run-time. You do that by calling register_server() (see include/suricatta/server.h) with a name and a server_t struct pointer, implemented inside a function marked with __attribute__((constructor)). suricatta/server_hawkbit.c is your example, go look 🔍
The type server_op_res_t is defined in include/suricatta/suricatta.h. It represents the valid function return codes for a server's implementation. That's the whole alphabet of "did it work or not" for your server, fr fr.
And listen — beyond implementing the particular channel and server, you also gotta adapt the suricatta/Kconfig file to include a new option, so that your new implementation actually becomes selectable in SWUpdate's configuration. In the simplest case, adding an option like the following one for hawkBit into the menu "Server" section is sufficient:
config SURICATTA_HAWKBIT
bool "hawkBit support"
default y
select JSON
help
Support for hawkBit server.
https://projects.eclipse.org/projects/iot.hawkbitOnce your new server implementation is in the configuration, go edit suricatta/Makefile to specify how the implementation gets linked into the SWUpdate binary. For the hawkBit example implementation, for instance, these lines add server_hawkbit.o to the resulting SWUpdate binary if SURICATTA_HAWKBIT was selected while configuring SWUpdate:
ifneq ($(CONFIG_SURICATTA_HAWKBIT),)
obj-$(CONFIG_SURICATTA) += server_hawkbit.o
endif🌈 Support for wfx 🌈
The wfx server — https://github.com/siemens/wfx — is supported by the Lua Suricatta module suricatta/server_wfx.lua (cf. Section Support for Suricatta Modules in Lua). Specifically, it implements a binding to the Device Artifact Update (https://github.com/siemens/wfx/tree/main/workflow/dau) (DAU) workflow family. Obsessed with the naming, ngl.
If it's enabled via CONFIG_SURICATTA_WFX, the wfx Lua Suricatta module gets embedded straight into the SWUpdate binary so that no extra deployment steps are required 😌. Note that this is purely a convenience shortcut for the installation of a Lua Suricatta module as described in Support for Suricatta Modules in Lua — that's the real, general mechanism, this is just the easy button.
📋 Job Definition 📋
Since wfx is a general purpose workflow executor, it doesn't impose a particular job definition nor schema — the only requirement is that it's in JSON format. Instead, the job definition is a contract between the operator creating jobs (each possibly following a different workflow) and the client(s) executing those jobs in lock-step with the wfx. Very "we agree to agree" of them 🫶
The wfx Lua Suricatta module understands job definitions as in the following example (see job.definition.json_schema in suricatta/server_wfx.lua):
{
"version": "1.0",
"type": ["firmware", "dummy"],
"artifacts": [
{
"name": "Example Device Firmware Artifact",
"version": "1.1",
"uri": "http://localhost:8080/download/example_artifact.swu"
}
]
}The type list field allows you to label update jobs. Labels are sent :-concatenated to the progress interface on Update Activation. The only predefined label is firmware, and it instructs the wfx Lua Suricatta module to record an installed update to the bootloader environment (see Bootloader Interface <>). 🎀
Within the artifacts list, only the uri field is strictly required for each artifact, while the fields name and version are used for informational purposes, if provided. Further fields, including top-level fields, are ignored on purpose and may be freely used — e.g., to enrich the job definition with metadata for update management. Go off with your metadata, bestie ✨
Since wfx is not concerned with the job definition except for conveying it to the client (i.e. SWUpdate), it can be adapted to specific needs by feeding a different job definition into the wfx on job creation and adapting the verification and job handling methods in the wfx Lua Suricatta module's job.definition = {...} Table. That's the customization window, use it 🪟
🚦 Workflows 🚦
The two Device Artifact Update (DAU) workflows wfx.workflow.dau.direct (https://github.com/siemens/wfx/blob/main/workflow/dau/wfx.workflow.dau.direct.yml) and wfx.workflow.dau.phased (https://github.com/siemens/wfx/blob/main/workflow/dau/wfx.workflow.dau.phased.yml) are supported by the wfx Lua Suricatta module. Hint: use wfx's wfx-viewer command line tool to visualize the YAML workflows in SVG or PlantUML format. It's giving "flowchart but make it fashion" 📊✨
For each transition in a workflow for which the eligible field contains CLIENT, e.g.,
transitions:
- from: <FROM_STATE>
to: <TO_STATE>
eligible: CLIENTthere has to be a matching transition execution function defined in the wfx Lua Suricatta module. It executes the client actions to go from state <FROM_STATE> to state <TO_STATE> and finally sends the new job status to the wfx, updating it:
job.workflow.dispatch:set(
"<FROM_STATE>",
"<TO_STATE>",
--- @param self job.workflow.transition
--- @param job job
--- @return transition.result
function(self, job)
if not job.status
:set({
state = self.to.name, -- resolves to `<TO_STATE>`
message = ("[%s] <TO_STATE> reached"):format(self.to.name),
progress = 100,
})
:send() then
-- Do not try to execute further transitions, yield to wfx.
return transition.result.FAIL_YIELD
end
return transition.result.COMPLETED
end
)See suricatta/server_wfx.lua for examples of such transition execution functions — there are real ones in there, not just vibes 💜
New or adapted workflows are supported by appropriately defining/modifying the transition execution functions in suricatta/server_wfx.lua — or taking it as inspiration and creating a new wfx Lua Suricatta module as described in Support for Suricatta Modules in Lua. Remix culture but for workflows 🦋
⚡ Update Activation ⚡
The Device Artifact Update (DAU) workflows offer a dedicated activation step in the update steps sequence, and the whole point is to decouple artifact installation and activation times — so that, e.g., upon a power-cut, it doesn't prematurely test-boot into the new firmware after installation until the activation is actually due. Safety queen behavior, we love her 👑
When the activation step is executed, the wfx Lua Suricatta module sends a progress message (see Getting information on running update <>), upon which a progress client executes or schedules activation measures. For example, the following JSON is sent as the progress message's .info field on activation of the Job Definition example given above:
{
"state": "ACTIVATING",
"progress": 100,
"message": "firmware:dummy"
}The progress message's .status is PROGRESS, see tools/swupdate-progress.c for details on how a progress client can be implemented. Steal the blueprint, bestie 📎
Note: the activation message may be sent multiple times if the update activation is pending — namely on each wfx poll operation, and if a new update job is enqueued while the current one is not activated. Not a bug, just how the polling life goes 💀
Because of the (predefined) firmware label being present, the progress client should initiate or schedule a reboot of the device in order to test-boot the new firmware. And also because of the firmware label being present, the wfx Lua Suricatta module records the installed update to the bootloader environment. If this label was missing, no such recording would've been made — which is exactly what you want for, e.g., configuration or application updates 🌸
In order for this mechanism to work, SWUpdate must not record the update to the bootloader environment after it has installed it, or — in the case of configuration or application updates — must not touch the bootloader environment at all (see the Sections Update Transaction and Status Marker and bootloader in SWUpdate: syntax and tags with the default parser <>).
Hence, for firmware updates requiring a test-boot into the new firmware, the following properties should be set in the .swu file's sw-description:
software =
{
bootloader_transaction_marker = true;
bootloader_state_marker = false;
...And for configuration or application updates, the following properties apply:
software =
{
bootloader_transaction_marker = false;
bootloader_state_marker = false;
...🌐 Support for general purpose HTTP server 🌐
Okay so this one is a very simple backend that uses standard HTTP response codes to signal if an update is available. There are closed source backends implementing this interface, but because the interface is a very simple interface, this server type is also suitable for implementing an own backend server. For inspiration, there's a simple (mock) server implementation available in examples/suricatta/server_general.py 🐍✨
The API consists of a GET with Query parameters to inform the server about the installed version. The query string has the format:
http(s)://<base URL>?param1=val1¶m2=value2...As examples for parameters, the device can send its serial number, MAC address and the running version of the software. It is duty of the backend to interpret this — SWUpdate just takes them from the "identify" section of the configuration file and encodes the URL. So you bring the parameters, the backend brings the meaning, and everybody wins 🫶💖
okay so when the device asks the server for an update, the server claps back with one of these HTTP codes 👇 bestie memorize them, they're the whole mood:
302Found — a new software is available at the URL that's sitting in theLocationheader 💖400Bad Request — some query parameters are missing or straight-up in the wrong format 🙈403Forbidden — the client certificate is not valid, so no update for you 🚫404Not found — there is literally no update available for this device 😭503Unavailable — an update is available but the server can't handle another update process right now 💀
and the server's answer can also carry these headers, they're like the little side notes on the message:
Retry-after— shows up with code503— contains a number telling the device how long to wait before asking again for updates. unit is seconds ⏳Content-MD5— shows up with code302— contains the checksum of the update file that's available under the url from theLocationheader 🔐Location— shows up with code302— the URL where the update file can be downloaded 🌐
🦋 logging, aka the device yapping back to the server 🦋
The device can send logging data to the server, yes queen, it talks back. Any information gets transmitted in a HTTP PUT request with the data as a plain string in the message body. The Content-Type Header needs to be set to text/plain, no cap.
The URL for the logging can be set as a separate URL in the configuration file, or via the --logurl command line parameter 💅
The device sends data in a CSV format (Comma Separated Values). the format is:
value1,value2,...The format can be specified in the configuration file. A format for each event can be set ✨ The supported events are:
check— dummy. It could send an event each time the server is polled 👀started— a new software is found and SWUpdate starts to install it 🚀success— a new software was successfully installed, slay 🎉fail— failure by installing the new software 💔
👑 the general server config 👑
The general server has its own section inside the configuration file. All channel-specific options (SSL, proxy, rate limiting, and custom headers) are supported. as an example:
gservice =
{
url = ....;
logurl = ;
polldelay = 30;
nocheckcert = false;
retry = 3;
retrywait = 10;
max-download-speed = "512k";
/*
cafile = "/etc/ssl/cafile";
sslkey = "/etc/ssl/sslkey";
sslcert = "/etc/ssl/sslcert";
*/
logevent : (
{event = "check"; format="#2,date,fw,hw,sp"},
{event = "started"; format="#12,date,fw,hw,sp"},
{event = "success"; format="#13,date,fw,hw,sp"},
{event = "fail"; format="#14,date,fw,hw,sp"}
);
}notice thecafile,sslkeyandsslcertlines are all commented out in that example, so they're just chillin there like optional fits you can unlock later 🎀
The available config options for the gservice section are:
url— string. The URL of the general suricatta server. ✨logurl— string. The URL where log/progress events get sent (and yeah, they go out viaPUT).polldelay— integer. Polling cycle, in seconds. 🕐retry— integer. How many times we're allowed to retry connecting to the server.retrywait— integer. Time in seconds to chill between connection retries.nocheckcert— boolean. Set it to true and we skip SSL certificate validation entirely. 🙈cafile— string. Path to the Certificate Authority (PEM) bundle file.sslkey— string. Path of the private key file for the SSL connection.sslcert— string. Path of the certificate file for the SSL connection.sslkeypassword— string. The password for that SSL private key, bestie.ciphers— string. The allowed ciphers suite list, written in CURL format.proxy— string. HTTP proxy address used to reach the server.interface— string. Network interface or IP address to bind communication to.max-download-speed— string. Maximum download speed limit (like"512k","1M"). 🏃♀️💨api_key_header— string. The HTTP custom header name used for API Key authentication.api_key— string. The custom API Key authentication value.logevent— array. Array of tables for defining event reporting formats.
Okay so date is a little special field, and ngl it's interpreted as localtime in RFC 2822 format. Every Comma Separated field gets looked up inside the identify section of the config file, and if there's a match, the substitution happens. No match? Then the field just gets sent exactly as it is. Periodt.
For example, if the identify section has these values:
identify : (
{ name = "sp"; value = "333"; },
{ name = "hw"; value = "ipse"; },
{ name = "fw"; value = "1.0"; }
);and the events are set like above, then the formatted text in the case of "success" will be:
Formatted log: #13,Mon, 17 Sep 2018 10:55:18 CEST,1.0,ipse,333🦋 Support for Suricatta Modules in Lua 🦋
The server_lua.c C-to-Lua bridge is literally the moment — it enables writing suricatta modules in Lua. It hands the Lua realm the infrastructure, like the interface to the SWUpdate "core", so the "business logic" — handling update flows, talking to backend server APIs, all of it — can be modeled in Lua. To the Lua realm, the server_lua.c C-to-Lua bridge provides the same functionality that the other suricatta modules written in C get, which is the whole separation of means and control thing. Basically, it lifts the interface laid out in Section The Suricatta Interface up into the Lua realm. 👑
As an example server implementation, go look at examples/suricatta/server_general.py — it's a simple (mock) server of a backend modeled after the "General Purpose HTTP Server" (cf. Section Support for general purpose HTTP server). The matching Lua suricatta module lives in examples/suricatta/swupdate_suricatta.lua. Drop it in Lua's path so a require("swupdate_suricatta") can load it, or embed it straight into the SWUpdate binary by enabling CONFIG_EMBEDDED_SURICATTA_LUA and setting CONFIG_EMBEDDED_SURICATTA_LUA_SOURCE accordingly.
The interface specification, in terms of a Lua (suricatta) module, is found in suricatta/suricatta.lua, reproduced for convenience in Section Lua Suricatta Interface Specification.
💅 suricatta 💅
The suricatta table is the module's main table, and it houses all the exposed functions and definitions via the sub-tables described below. On top of that, the main functions suricatta.install() and suricatta.download(), plus the convenience functions suricatta.getversion(), suricatta.sleep(), and suricatta.get_tmpdir(), are all exposed. So generous of them. 🫶
The function suricatta.install(install_channel) installs an update artifact from a remote server or a local file. The install_channel table parameter designates the channel to be used for accessing the artifact, plus channel options that diverge from the defaults set at channel creation time. For example, an install_channel table may look like this:
{ channel = chn, url = "https://artifacts.io/update.swu" }where chn is the return value of a call to channel.open(). The other table attributes, like url in this example, are channel options that diverge from — or were omitted at — channel creation time, see suricatta.channel. For installing a local file, an install_channel table may look like this:
{ channel = chn, url = "file:///path/to/file.swu" }The function suricatta.download(download_channel, localpath) just downloads an update artifact, that's it, nothing extra. The parameter download_channel is exactly as for suricatta.install(). The parameter localpath designates the output path for the artifact. The suricatta.get_tmpdir() function (see below) is especially useful here, since it hands you a temporary download location to use as localpath. A just-downloaded artifact may then be installed later using suricata.install() with an appropriate file:// URL — and that, bestie, is a deferred installation. ✨
Both suricatta.install() and suricatta.download() return true, or, in the case of an error, nil, a suricatta.status value, and a table with messages in case of errors — otherwise an empty table.
The function suricatta.getversion() returns a table with SWUpdate's version and patchlevel fields. This info can be used to determine API (in-)compatibility of the Lua suricatta module with the SWUpdate version that's running it. 🔥
Okay so, suricatta.sleep(seconds) is literally just a glow-up wrapper around SLEEP(3) — you'd use it for stuff like, idk, implementing a REST API call retry mechanism after a number of seconds have passed. Very "wait a sec bestie, try again" energy. ⏰
And suricatta.get_tmpdir() hands you the path to SWUpdate's temporary working directory — that's the spot where, say, the suricatta.download() function drops the downloaded artifacts. It's giving "here's where we keep the goods" 🎀
🦋 suricatta.status 🦋
The suricatta.status table is the moment where the server_op_res_t enum values (defined over in include/util.h) get exposed to the Lua realm. Lowkey iconic of it 💅
📣 suricatta.notify 📣
The suricatta.notify table serves up all the usual logging functions to the Lua suricatta module, matching their uppercase-named pendants over in the C realm, fr fr.
One notable exception bestie: suricatta.notify.progress(message, cause) — that one dispatches the message to the progress interface (see Getting information on running update <>) instead. Custom progress client implementations listening and acting on custom progress messages can totes be realized using this function. So if you want your own little progress-obsessed listener, this is your girl ✨
All notify functions return nil. No cap.
🎀 suricatta.pstate 🎀
The suricatta.pstate table gives you a binding to SWUpdate's (persistent) state handling functions defined in include/state.h — but bestie, it's limited to the bootloader environment variable STATE_KEY defined by CONFIG_UPDATE_STATE_BOOTLOADER, which defaults to ustate. In addition, it captures the update_state_t enum values. Very "we keep the receipts" of it 💜
The function suricatta.pstate.save(state) requires one of suricatta.pstate's "enum" values as parameter and returns true, or, in case of error, nil. And the function suricatta.pstate.get() returns one of suricatta.pstate's "enum" values or, in case of error, STATE_ERROR.
👑 suricatta.server 👑
The suricatta.server table provides literally just the one function: suricatta.server.register(function_p, purpose). It registers a Lua function "pointed" to by function_p for the purpose purpose — and that purpose is defined by suricatta.server's "enum" values. Those enum values correspond to the functions defined in the interface outlined in the Section on The Suricatta Interface, iykyk.
On top of those functions, the two callback functions CALLBACK_PROGRESS and CALLBACK_CHECK_CANCEL can be registered optionally. The former can be used to upload progress information to the server, while the latter serves as dwlwrdata function (see include/channel_curl.h) to decide on whether an installation should be aborted during the download phase. 🛑
For details on the (callback) functions and their signatures, check the interface specification suricatta/suricatta.lua and the documented example Lua suricatta module found in examples/suricatta/swupdate_suricatta.lua.
The suricatta.server.register() function returns true, or, in case of error, nil. Periodt. 💅
🌈 suricatta.channel 🌈
The suricatta.channel table captures channel handling for suricatta Lua modules. The single function suricatta.channel.open(options) creates and opens a channel to a server. Its single parameter options is a table specifying the channel's default options — such as proxy, retries, usessl, strictssl, or headers_to_send. For convenience, options that may change per request — like url, content-type, or headers_to_send — may be set as defaults on channel creation time while being selectively overruled on a per request basis. The channel options currently supported to be set are listed in the suricatta.channel.options table. In essence, the options parameter table is the Lua table equivalent of include/channel_curl.h's channel_data_t. 🦋
The suricatta.channel.open(options) function returns a channel table, which is either passed to the suricatta.install() and suricatta.download() functions or used directly for communication with a server. More specifically, it has the three functions:
get(options)for retrieving information from the server,put(options)for sending information to the server, andclose()for closing the channel.
The get() and put() functions' single parameter options is a per-request channel option table as described above. 💖
The functions get() and put() return true, or, in case of error, nil, a suricatta.status value, and an operation result table. The latter contains the fields:
http_response_codecarrying the HTTP error code,formatas one ofsuricatta.channel.content's options,raw_replyifoptionscontainedformat = suricatta.channel.content.RAW,json_replyifoptionscontainedformat = suricatta.channel.content.JSON, and- the HTTP headers received in the
received_headerstable, if any.
The suricatta.channel.content "enum" table defines the "format", i.e., the response body content type and whether to parse it or not:
NONEmeans the response body is discarded.RAWmeans the raw server's reply is available asraw_reply.JSONmeans the server's JSON reply is parsed into a Lua table and available asjson_reply.
The suricatta.channel.method "enum" table defines the HTTP method to use for a request issued with the put(options) function, i.e., POST, PATCH, or PUT as specified in the options parameter table via the method attribute. In addition to the HTTP method, the request body's content is set with the request_body attribute in the options parameter table. 🔥
As a contrived example, consider the following call to a channel's put() function:
...
local res, _, data = channel.put({
url = string.format("%s/%s", base_url, device_id),
content_type = "application/json",
method = suricatta.channel.method.PATCH,
format = suricatta.channel.content.NONE,
request_body = "{ ... }"
})
...which issues a HTTP PATCH to some URL with a JSON content without having interest in the response body. Very "send it and don't care" energy 😌
More examples of how to use a channel can be found in the example suricatta Lua module examples/suricatta/swupdate_suricatta.lua.
💫 suricatta.bootloader 💫
The suricatta.bootloader table exposes SWUpdate's bootloader environment modification functions to suricatta Lua modules.
The enum-like table suricatta.bootloader.bootloaders holds the bootloaders SWUpdate supports, i.e.
suricatta.bootloader.bootloaders = {
EBG = "ebg",
NONE = "none",
GRUB = "grub",
UBOOT = "uboot",
},The function suricatta.bootloader.get() returns the currently selected bootloader in terms of a suricatta.bootloader.bootloaders field value. 👑
The function suricatta.bootloader.is(name) takes one of suricatta.bootloader.bootloaders's field values as name and returns true if it's the currently selected bootloader, false otherwise.
The functions in the suricatta.bootloader.env table interact with the currently selected bootloader's environment:
The function suricatta.bootloader.env.get(variable) retrieves the value associated to variable from the bootloader's environment.
The function suricatta.bootloader.env.set(variable, value) sets the bootloader environment's key variable to value.
The function suricatta.bootloader.env.unset(variable) deletes the bootloader environment's key variable.
The function suricatta.bootloader.env.apply(filename) applies all key=value lines of a local file filename to the currently selected bootloader's environment. 🎀
✨💖 Lua Suricatta Interface Specification 💖✨
--[[
SWUpdate Suricatta Lua Module.
Interface specification for the Lua module provided by the
"Suricatta Lua module" suricatta "server" (suricatta/server_lua.c).
Author: Christian Storm <christian.storm@siemens.com>
Copyright (C) 2022, Siemens AG
SPDX-License-Identifier: GPL-2.0-or-later
--]]
---@diagnostic disable: missing-return
---@diagnostic disable: unused-local
-- luacheck: no max line length
-- luacheck: no unused args
-- Note: Definitions prefixed with `---` are not part of the SWUpdate functionality
-- exposed to the Lua realm but instead are `typedef`-alikes returned by a function
-- of the SWUpdate Suricatta Lua Module. Nonetheless, they are in the suricatta
-- "namespace" for avoiding name clashes and, not least, convenience.
--- SWUpdate Suricatta Binding.
--
--- @class suricatta
local suricatta = {}
--- @enum suricatta.status
--- Lua equivalent of `server_op_res_t` enum as in `include/util.h`.
suricatta.status = {
OK = 0,
EERR = 1,
EBADMSG = 2,
EINIT = 3,
EACCES = 4,
EAGAIN = 5,
UPDATE_AVAILABLE = 6,
NO_UPDATE_AVAILABLE = 7,
UPDATE_CANCELED = 8,
ID_REQUESTED = 9,
}
--- SWUpdate notify function bindings.
--
-- Translates to `notify(string.format(message, ...))`,
-- @see `corelib/lua_interface.c`
-- except for `suricatta.notify.progress()`.
--
--- @class suricatta.notify
suricatta.notify = {
--- @type fun(message: string, ...: any)
error = function(message, ...) end,
--- @type fun(message: string, ...: any)
trace = function(message, ...) end,
--- @type fun(message: string, ...: any)
debug = function(message, ...) end,
--- @type fun(message: string, ...: any)
info = function(message, ...) end,
--- @type fun(message: string, ...: any)
warn = function(message, ...) end,
--- @type fun(message: string, cause:suricatta.ipc.progress_cause?)
progress = function(message, cause) end,
}
--- SWUpdate's bootloader interface as in `include/bootloader.h`.
--
--- @class suricatta.bootloader
suricatta.bootloader = {
--- @enum suricatta.bootloader.bootloaders
--- Bootloaders supported by SWUpdate.
bootloaders = {
EBG = "ebg",
NONE = "none",
GRUB = "grub",
UBOOT = "uboot",
},
}
--- Get currently set bootloader's name.
--
--- @return suricatta.bootloader.bootloaders | nil # Name of currently set bootloader
suricatta.bootloader.get = function() end
--- Test whether bootloader `name` is currently set.
--
--- @param name suricatta.bootloader.bootloaders Name of bootloader to test for being currently selected
--- @return boolean # True if `name` is currently set bootloader, false otherwise
suricatta.bootloader.is = function(name) end
--- Operations on the currently set bootloader's environment.
--
--- @class suricatta.bootloader.env
suricatta.bootloader.env = {}Okay so bestie, the Lua side of suricatta is chef's kiss — and we're deep in it now. Let's talk bootloader env vars, because yes, you can absolutely touch them from Lua. 💅
🦋 bootloader env, but make it Lua
Want the value of a bootloader environment variable? Read it:
--- Get value of a bootloader environment variable.
--
--- @param variable string Name of the bootloader environment variable to get value for
--- @return string | nil # Value of the bootloader environment variable or nil if non-existent
suricatta.bootloader.env.get = function(variable) endYou get back the value as a string, or nil if that variable literally doesn't exist. No drama.
Want to set one? Serve:
--- Set value of a bootloader environment variable.
--
--- @param variable string Name of the bootloader environment variable to set
--- @param value string Value to set the bootloader environment variable `variable` to
--- @return boolean | nil # True on success, nil on error
suricatta.bootloader.env.set = function(variable, value) endReturns true when it works, nil when it doesn't. That's it, that's the whole vibe check.
And when a variable has got to go:
--- Drop a bootloader environment variable.
--
--- @param variable string Name of the bootloader environment variable to drop
--- @return boolean | nil # True on success, nil on error
suricatta.bootloader.env.unset = function(variable) endSame deal — true on success, nil on error. ✨
Now for the ✨bulk era✨: if you've got a whole little local file of variables in <variable>=<value> format, you do not have to loop. You just:
--- Set multiple bootloader environment variables from local file.
--
--- @param filename string Path to local file in format `<variable>=<value>`
--- @return boolean | nil # True on success, nil on error
suricatta.bootloader.env.apply = function(filename) endPoint it at filename, it reads that <variable>=<value> file and sets everything. true on success, nil on error. Obsessed. 🎀
👑 persistent state (pstate) has entered the chat
SWUpdate keeps a persistent state, and the Lua table below hands you both directions of the lookup at once — number ➡️ name and name ➡️ number. These IDs match include/state.h:
--- SWUpdate's persistent state IDs as in `include/state.h` and reverse-lookup.
--
--- @class suricatta.pstate
suricatta.pstate = {
OK = string.byte('0'), [string.byte('0')] = "OK",
INSTALLED = string.byte('1'), [string.byte('1')] = "INSTALLED",
TESTING = string.byte('2'), [string.byte('2')] = "TESTING",
FAILED = string.byte('3'), [string.byte('3')] = "FAILED",
NOT_AVAILABLE = string.byte('4'), [string.byte('4')] = "NOT_AVAILABLE",
ERROR = string.byte('5'), [string.byte('5')] = "ERROR",
WAIT = string.byte('6'), [string.byte('6')] = "WAIT",
IN_PROGRESS = string.byte('7'), [string.byte('7')] = "IN_PROGRESS",
}Note they're string.byte('0') through string.byte('7'), bestie — not literal 0–7. Iykyk, but don't freestyle it.
Grabbing the state that's stored right now:
--- Get the current stored persistent state.
--
--- @return number # Persistent state ID number, suricatta.pstate.ERROR if unsuccessful
suricatta.pstate.get = function() endYou get a number back. If it didn't work? You get suricatta.pstate.ERROR. 😭
And to save it:
--- Save persistent state information.
--
--- @param state number Persistent state ID number
--- @return boolean | nil # True on success, nil on error
suricatta.pstate.save = function(state) endHand it the state ID number. true on success, nil on error. Slay.
⭐ the server registry (a.k.a. plug yourself in)
So you want Lua to be the Suricatta interface implementation? Bet. First, here are the function registry IDs — this is the enum you pass in as purpose:
--- Function registry IDs for Lua suricatta functions.
--
--- @class suricatta.server
suricatta.server = {
HAS_PENDING_ACTION = 0,
INSTALL_UPDATE = 1,
SEND_TARGET_DATA = 2,
GET_POLLING_INTERVAL = 3,
SERVER_START = 4,
SERVER_STOP = 5,
IPC = 6,
PRINT_HELP = 7,
CALLBACK_PROGRESS = 8,
CALLBACK_CHECK_CANCEL = 9,
}Now the actual registration move:
--- Register a Lua function as Suricatta interface implementation.
--
--- @param function_p function Function to register for `purpose`
--- @param purpose number Suricatta interface function implemented (one of suricatta.server's enums)
--- @return boolean # Whether operation was successful or not
suricatta.server.register = function(function_p, purpose) endYou give it your function_p and a purpose (one of those suricatta.server enums from up there ☝️), and you get back a boolean — whether the operation was successful or not. Main character energy, fr fr. 🔥
🌈 channel stuff (the networking glow-up)
When a channel operation comes back, it can carry this with it:
--- Channel result table.
--
-- Result in return of a channel operation.
--
--- @class suricatta.channel_operation_result
--- @field http_response_code number
--- @field format suricatta.channel.content
--- @field json_reply table | nil Table if `format` was `suricatta.channel.content.JSON`
--- @field raw_reply string | nil Table if `format` was `suricatta.channel.content.RAW`
--- @field received_headers table<string, string> | nilHeads up: json_reply is a table if format was suricatta.channel.content.JSON, and raw_reply is documented as a string — written as Table if — if format was suricatta.channel.content.RAW. Yes, exactly like the original says it, I'm not touching that. 🙈
Then there's what you actually get handed when you open a channel — a suricatta.open_channel, with its options plus the get, put and close operations:
--- Options and methods on the opened channel.
--
-- Table as returned by `suricatta.channel.open()`.
--
--- @class suricatta.open_channel
--- @field options suricatta.channel.options Channel creation-time set options as in `include/channel_curl.h`.
--- @field get fun(options: suricatta.channel.options): boolean, suricatta.status, suricatta.channel_operation_result Channel get operation
--- @field put fun(options: suricatta.channel.options): boolean, suricatta.status, suricatta.channel_operation_result Channel put operation
--- @field close fun() Channel close operationThat options field is holding channel creation-time set options as in include/channel_curl.h. 📌
Now the big one — the suricatta.channel table itself. And it is giving:
--- @class suricatta.channel
suricatta.channel = {
-- Lua-alike of proxy environment variable usage marker as in `include/channel_curl.h`.
-- An empty `proxy` string means to use proxy environment variables.
-- @type string
USE_PROXY_ENV = "",
--- @enum suricatta.channel.content
--- Content type passed over the channel as in `include/channel_curl.h`.
content = {
NONE = 0,
JSON = 1,
RAW = 2,
},
--- @enum suricatta.channel.method
--- Transfer method to use over channel as in `include/channel_curl.h`.
method = {
GET = 0,
POST = 1,
PUT = 2,
PATCH = 3,
DELETE = 4,
HEAD = 5,
},
--- Channel options as in `include/channel_curl.h`.
--
--- @class suricatta.channel.options
--- @field url string | nil `CURLOPT_URL` - URL for this transfer
--- @field cached_file string | nil Resume download from cached file at path
--- @field auth string | nil `CURLOPT_USERPWD` - user name and password to use in authentication
--- @field request_body string | nil Data to send to server for `PUT` and `POST`
--- @field iface string | nil `CURLOPT_INTERFACE` - source interface for outgoing traffic
--- @field dry_run boolean | nil `swupdate_request`'s dry_run field as in `include/network_ipc.h`
--- @field cafile string | nil `CURLOPT_CAINFO` - path to Certificate Authority (CA) bundle
--- @field sslkey string | nil `CURLOPT_SSLKEY` - private key file for TLS and SSL client cert
--- @field sslcert string | nil `CURLOPT_SSLCERT` - SSL client certificate
--- @field ciphers string | nil `CURLOPT_SSL_CIPHER_LIST` - ciphers to use for TLS
--- @field proxy string | nil `CURLOPT_PROXY` - proxy to use
--- @field info string | nil `swupdate_request`'s info field as in `include/network_ipc.h`
--- @field auth_token string | nil String appended to Header
--- @field content_type string | nil `Content-Type:` and `Accept:` appended to Header
--- @field retry_sleep number | nil Time to wait prior to retry and resume a download
--- @field method suricatta.channel.method | nil Channel transfer method to use
--- @field retries number | nil Maximal download attempt count
--- @field low_speed_timeout number | nil `CURLOPT_LOW_SPEED_TIME` - low speed limit time period
--- @field connection_timeout number | nil `CURLOPT_CONNECTTIMEOUT` - timeout for the connect phase
--- @field format suricatta.channel.content | nil Content type passed over the channel
--- @field debug boolean | nil Set channel debug logging
--- @field usessl boolean | nil Enable SSL hash sum calculation
--- @field strictssl boolean | nil `CURLOPT_SSL_VERIFYHOST` + `CURLOPT_SSL_VERIFYPEER`
--- @field nocheckanswer boolean | nil Whether the reply is interpreted/logged and tried to be parsed
--- @field nofollow boolean | nil `CURLOPT_FOLLOWLOCATION` - follow HTTP 3xx redirects
--- @field max_download_speed string | nil `CURLOPT_MAX_RECV_SPEED_LARGE` - rate limit data download speed
--- @field headers_to_send table<string, string> | nil Header to send
options = {},
--- Open a new channel.
--
--- @param options suricatta.channel.options Channel default options overridable per operation
--- @return boolean # Whether operation was successful or not
--- @return suricatta.open_channel # Options of and operations on the opened channel
open = function(options) end,
}Quick spotlight on USE_PROXY_ENV: it's the Lua-alike of the proxy environment variable usage marker as in include/channel_curl.h, it's a string, and an empty proxy string means use proxy environment variables. So "" is not "no proxy", bestie — it's "yes proxy, from env". Do not get got. 👑
Also, content gives you NONE = 0, JSON = 1, RAW = 2, and method gives you GET = 0, POST = 1, PUT = 2, PATCH = 3, DELETE = 4, HEAD = 5. And every single one of those options fields is typed as | nil, so they're all optional. Live your life. 🫶
open(options) takes channel default options that you can override per operation, and hands back a boolean (whether the operation was successful or not) plus the suricatta.open_channel — the options of and operations on the opened channel.
🌸 operation channel (channel + options, glowed up)
Sometimes you need the channel plus per-operation overrides. That's this:
--- Channel and Options Table to use for an operation.
--
-- Channel to use for the download / installation operation as returned by `suricatta.channel.open()`
-- plus channel options overriding the defaults per operation (@see suricatta.channel.options)
-- and specific options to the download / installation operation, e.g., `drain_messages`.
--
--- @class suricatta.operation_channel
--- @field channel suricatta.open_channel Channel table as returned by `suricatta.channel.open()`
--- @field drain_messages boolean | nil Whether to flush all progress messages or only those while in-flight operation (default)
--- @field suricatta.channel.options | nil Channel options to override for this operationdrain_messages decides whether you flush all progress messages or only those while the operation is in flight — and in-flight-only is the default. So if you leave it alone, that's what you get. 🍓
And the HTTP response info coming back from a channel operation:
--- Channel HTTP response table.
--
-- HTTP response information in return of a channel operation.
--
--- @class suricatta.channel_operation_response
--- @field http_response_code number
--- @field received_headers table<string, string>Yes, it's a separate table from the result table — a number code and a table<string, string> of received headers. Two different things, both real.
💜 install (the one you've been waiting for)
Here's the moment. Install an update artifact from a remote server or a local file:
--- Install an update artifact from remote server or local file.
--
-- If the protocol specified in Table `install_channel`'s `url` field is `file://`,
-- a local update artifact file is installed. If it is, e.g., `https://`, the
-- update artifact is downloaded *and* installed.
-- Note that this file is to be deleted, if applicable, from the Lua realm.
--
--- @see suricatta.download
--- @param install_channel suricatta.operation_channel Channel to use for the download+installation operation
--- @return boolean # Whether operation was successful or not
--- @return suricatta.status # Suricatta return code
--- @return table<number, string> # Error messages, if any
--- @return suricatta.channel_operation_response # if `nocheckanswer` is not set, HTTP response information, if any
suricatta.install = function(install_channel) endSo the protocol in Table install_channel's url field is the whole plot twist here:
- if it's
file://— a local update artifact file gets installed - if it's, like,
https://— the update artifact gets downloaded and installed
Real talk though: that file is to be deleted, if applicable, from the Lua realm. That's on you, bestie. SWUpdate is not doing your chores. 💀
install_channel is a suricatta.operation_channel — the channel to use for the download+installation operation. And it returns up to four things, in this exact order: a boolean (whether operation was successful or not), a suricatta.status (the Suricatta return code), a table<number, string> of error messages if any, and — if nocheckanswer is not set — a suricatta.channel_operation_response with HTTP response information, if any.
And it also cross-references suricatta.download, so keep that in your back pocket. 🎀
suricatta.download()— bestie, this one downloads an update artifact from a remote server. That's it. It does NOT install it. It just grabs it. If you wanna install it later, you callsuricatta.install()and hand it an appropriateinstall_channelTable'surlfield. okay so:@see suricatta.install@param download_channel—suricatta.operation_channel— the Channel you use for the download operation@param localpath—string— the Path where you store the downloaded artifact@return boolean— whether the operation was successful or not@return suricatta.status— the Suricatta return code@return table<number, string>— error messages, if any@return suricatta.channel_operation_response— ifnocheckansweris not set, the HTTP response information, if any
suricatta.download = function(download_channel, localpath) endsuricatta.sleep()— Sleep for a number of seconds. It callsSLEEP(3)via the C realm (iykyk).@param seconds number— Number of seconds to sleep
suricatta.sleep = function(seconds) endsuricatta.get_tmpdir()— Get TMPDIR from SWUpdate, bestie.@seecore/util.c::get_tmpdir()@return string— TMPDIR path
suricatta.get_tmpdir = function() endsuricatta.version— SWUpdate version information, periodt. This is a class:@class suricatta.version@field [1] number— SWUpdate's version@field [2] number— SWUpdate's patch level@field [3] number— SWUpdate's sub level@field version number— SWUpdate's version@field patchlevel number— SWUpdate's patch level@field sublevel number— SWUpdate's sub level
suricatta.getversion()— Get the SWUpdate version.@return suricatta.version— Table withversionandpatchlevelfields
suricatta.getversion = function() endsuricatta.ipc— SWUpdate IPC types and definitions 💅@class suricatta.ipc
suricatta.ipc = {}suricatta.ipc.sourcetype—@enum suricatta.ipc.sourcetype. It's the Lua equivalent ofsourcetypeas ininclude/swupdate_status.h.
suricatta.ipc.sourcetype = {
SOURCE_UNKNOWN = 0,
SOURCE_WEBSERVER = 1,
SOURCE_SURICATTA = 2,
SOURCE_DOWNLOADER = 3,
SOURCE_LOCAL = 4,
SOURCE_CHUNKS_DOWNLOADER = 5
}suricatta.ipc.RECOVERY_STATUS—@enum suricatta.ipc.RECOVERY_STATUS. Lua equivalent ofRECOVERY_STATUSas ininclude/swupdate_status.h.
suricatta.ipc.RECOVERY_STATUS = {
IDLE = 0,
START = 1,
RUN = 2,
SUCCESS = 3,
FAILURE = 4,
DOWNLOAD = 5,
DONE = 6,
SUBPROCESS = 7,
PROGRESS = 8
}suricatta.ipc.progress_cause—@enum suricatta.ipc.progress_cause. Lua equivalent ofprogress_cause_tas ininclude/progress_ipc.h.
suricatta.ipc.progress_cause = {
CAUSE_NONE = 0,
CAUSE_REBOOT_MODE = 1,
}suricatta.ipc.progress_msg— Lua-alike ofprogress_msgas ininclude/progress_ipc.h. Fish:@class suricatta.ipc.progress_msg@field magic number— SWUpdate IPC magic number@field status suricatta.ipc.RECOVERY_STATUS— Update status@field dwl_percent number— Percent of downloaded data@field nsteps number— Total steps count@field cur_step number— Current step@field cur_percent number— Percent in current step@field cur_image string— Name of the current image to be installed (max: 256 chars)@field hnd_name string— Name of the running handler (max: 64 chars)@field source suricatta.ipc.sourcetype— The source that has triggered the update@field info string— Additional information about the installation (max: 2048 chars)@field jsoninfo table— Ifinfois JSON, according Lua Table
suricatta.ipc.ipc_commands— Lua enum of IPC commands as ininclude/network_ipc.h.
Heads up bestie: CMD_ENABLE is not passed through and hence not in ipc_commands, because it's handled directly in suricatta/suricatta.c.
@type {[string]: number}@class suricatta.ipc.ipc_commands@field ACTIVATION number—0@field CONFIG number—1@field GET_STATUS number—3
suricatta.ipc.ipc_message— Lua-alike ofipc_messageas ininclude/network_ipc.h.
Note: Some members are deliberately not passed through to the Lua realm such asipc_message.data.lensince that's transparently handled by the C-to-Lua bridge. Note: This is not a direct equivalent but rather a "sensible" selection as, e.g., thejsonfield is not present instruct ipc_message.
@class suricatta.ipc.ipc_message@field magic number— SWUpdate IPC magic number@field commands suricatta.ipc.ipc_commands— IPC commands@field cmd number— Command number, one ofipc_commands's values@field msg string— String data sent via IPC@field json table— Ifmsgis JSON, JSON as Lua Table
return suricatta🌈 CONFIG FOR HAWKBIT UNDER SSL/TLS USING PRIVATE CA / SUB CA 🌈
okay so this is a user-contributed recipe, no cap. It's based on hawkBit (0.2.0-SNAPSHOT) + swupdate (v2018.03). Take notes bestie 💖
🎀 Purpose
Use HTTPS on a hawkBit server to avoid server spoofing. Anonymous client connections are authorized.
🦋 Recipe
- On the PKI:
- Create a pkcs#12 (
.p12) file, rolling server key, server, private CA, sub CA certs into a single file. - Use a password on the server key you won't be ashamed of. (fr fr, pick a good one.)
- Also create a single
.pemfile for the private CA + sub-CA.
- Create a pkcs#12 (
- On the hawkBit host:
- hawkBit uses the Java KeyStore to access credentials, but a JKS is not designed apparently to hold CA certs, which is a problem for private CAs. The workaround is to make it gulp an entire pkcs#12 file. 💀
- It looks like a JKS like this cannot have a password different from the one protecting the
.p12. Keytool also seems to have a little tendency to destruct the.jksif you change your mind and want to change the password... Basically do everything you need with openssl and use only keytool for generating the.jksfile.
The following command imports a .p12 into a "pkcs12 Java keystore", keeping the same password:
keytool -importkeystore -srckeystore hb-pass.p12 -srcstoretype pkcs12 \
-destkeystore hb-pass.jks -deststoretype pkcs12 \
-alias 1 -deststorepass <password_of_p12>Then you need to adapt application.properties of the hawkBit server to make use of the keystore. There are extra requirements to make hawkBit send artifacts via HTTPS.
This is the relevant part of <hawkBit dir>/hawkbit-runtime/hawkbit-update-server/src/main/resources/application.properties:
# HTTPS mode working w/ swupdate
# See also https://docs.spring.io/spring-boot/docs/1.4.7.RELEASE/reference/html/howto-embedded-servlet-containers.html#howto-configure-ssl
# https://github.com/eclipse/hawkbit/issues/618
#
# Need to run as root to use port 443
server.hostname=hb.domain
server.port=8443
#
# Overriding some of hawkbit-artifactdl-defaults.properties is required
hawkbit.artifact.url.protocols.download-http.protocol=https
hawkbit.artifact.url.protocols.download-http.port=8443
#
# Upgrades http:8443 to https:8443
# Would redirect + upgrade http:80 to https:443
security.require-ssl=true
server.use-forward-headers=true
#
# Server cert+key w/ private CA + subCA
# See also https://stackoverflow.com/questions/906402/how-to-import-an-existing-x509-certificate-and-private-key-in-java-keystore-to-u
# http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html (2008, still relevant!?)
#
# File .jks is a .p12 imported via keytool. Only one password supported, set from openssl.
server.ssl.key-store=hb-pass.jks
server.ssl.key-password=password
server.ssl.key-store-password=password-yes_the_same_one
...- On the swupdate client host(s):
- The client needs the private CA certificate(s) to authenticate the server.
- There is a setting in swupdate to specify the path to a single CA cert, not a directory. Beyond that, libcurl looks into
/etc/ssl/certs. So we're using a compound "CA chain".pemfile to hold both private CA and sub-CA in our preferred location. ✨
This is the relevant part of /etc/swupdate/swupdate.conf:
...
suricatta :
{
tenant = "default";
id = "machineID";
url = "https://hb.domain:8443";
nocheckcert = false;
cafile = "/etc/swupdate/priv-cachain.pem"; /* CA + sub CA in one file */
/* sslkey = anon client: do not set; */
/* sslcert = anon client: do not set; */
...👑 SWUPDATE: API FOR EXTERNAL PROGRAMS 👑
🌸 Overview
SWUpdate contains an integrated web-server to allow remote updating. However, which protocols are involved during an update is project specific and differs significantly. Some projects can decide to use FTP to load an image from an external server, or using even a proprietary protocol. The integrated web-server uses this interface.
SWUpdate has a simple interface to let external programs communicate with the installer. Clients can start an upgrade and stream an image to the installer, querying then for the status and the final result. The API is at the moment very simple, but it can easily be extended in the future if new use cases will arise. 💅
💜 API Description
The communication runs via Unix Domain Socket (UDS). The socket path is determined by:
- the
CONFIG_SOCKET_CTRL_PATHcompile-time configuration, $RUNTIME_DIRECTORY/sockinstctrlif$RUNTIME_DIRECTORYis set,$TMPDIR/sockinstctrlif$TMPDIRis set, or/tmp/sockinstctrl,
in this order, and taken over from systemd or created by SWUpdate. An example using the direct socket approach is in tools/swupdate-ipc.c. The recommended approach though is to use the Client Library.
The exchanged packets are described in network_ipc.h
okay so the packet itself is literally this little guy right here:
typedef struct {
int magic;
int type;
msgdata data;
} ipc_message;and the fields are giving:
magic— a magic number, aka the simplest little proof that the packet is real 💫type— one ofREQ_INSTALL,ACK,NACK,GET_STATUS,POST_UPDATE,SWUPDATE_SUBPROCESS,SET_AES_KEYmsgdata— a buffer that the client uses to send the image, or that SWUpdate uses to report notifications and status back 🫶
The client sends a REQ_INSTALL packet and then just... waits for an answer. SWUpdate sends back ACK or NACK — NACK if, for example, an update is already in progress 😭.
After the ACK, the client streams the whole image. SWUpdate literally expects that every single byte after the ACK is part of the image that needs installing. It figures out the size of the image from the CPIO header. Any error and SWUpdate bails out of the update state, and after that every further packet gets ignored until a brand new REQ_INSTALL shows up. no cap 💀
[image]
It is recommended to use the client library to talk to SWUpdate, bestie. On the lower level with direct socket communication, it cannot be guaranteed that the structures will stay compatible in the future 😬. The client library got hit by this issue too, and it was changed so it accepts an opaque interface that survives API changes. Compatibility layers could get added on-demand later if the API changes.
Client Library ✨
Functions to start an update 🎀
There's a library that makes using the IPC wayyyyy easier, and it gives you a way to kick off an update asynchronously.
The library is literally one function and a few call-backs:
int swupdate_async_start(writedata wr_func, getstatus status_func,
terminated end_func, void *req, ssize_t size)
typedef int (*writedata)(char **buf, int *size);
typedef int (*getstatus)(ipc_message *msg);
typedef int (*terminated)(RECOVERY_STATUS status);swupdate_async_start spins up a new thread and starts the communication with SWUpdate, triggering a new update. The wr_func gets called to grab the image that needs installing — and it's your callback's job to hand over the buffer and the size of the chunk of data. main character responsibility fr 🫡
The getstatus call-back gets called after the stream has been downloaded, so you can check how the upgrade is going. You can totally omit it if all you want is the final result.
The terminated call-back gets called once SWUpdate has finished, and it hands you the result of the upgrade.
There's an example using this library in tools/swupdate-client.c 👑
The req structure gets casted to void so the API stays compatible, and a user should instantiate it as struct swupdate_request. It's holding fields that can control the update process:
struct swupdate_request {
unsigned int apiversion;
sourcetype source;
int dry_run;
size_t len;
char info[512];
char software_set[256];
char running_mode[256];
};A user should call swupdate_prepare_req() first:
void swupdate_prepare_req(struct swupdate_request *req);This fills the request structure with default values 💅. After that, the user can fill the other fields like:
sourcetype— one ofSOURCE_UNKNOWN,SOURCE_WEBSERVER,SOURCE_SURICATTA,SOURCE_DOWNLOADER,SOURCE_LOCALdry_run— one ofRUN_DEFAULT(set from command line),RUN_DRYRUN,RUN_INSTALL.info,len— a variable length data blob that can be forwarded to the progress interface. The installer in SWUpdate does not evaluate it. it's just vibes ✨software_setandrunning_mode— this lets you set the selection for the update, and it should be accepted by swupdate (the-q,--accepted-selectoption when swupdate is launched).
Functions to set AES keys 🔐
The key for decryption can be set with a command line parameter (see -K), but bestie you can also set it via IPC. That way, each update could have a completely different key 🌈
int swupdate_set_aes(char *key, char *ivt)The key is for AES-256. So the length for key and ivt is defined by the algorithm, and they're passed as an ASCII string — meaning the length must be 64 bytes for the key and 32 bytes for the IVT. don't get it wrong bestie, it's not giving room for error 😭
Functions to control SWUpdate 🎮
int ipc_send_cmd(ipc_message *msg);ipc_send_cmd sends a command to a SWUpdate subprocess (like suricatta). The function is synchronous — meaning it blocks until the subprocess has answered with ACK or NACK. It sets type to SWUPDATE_SUBPROCESS, and then the caller has to set the other fields in the message according to the destination. The msgdata field is a structure like:
struct {
sourcetype source; /* Who triggered the update */
int cmd; /* Optional encoded command */
int timeout; /* timeout in seconds if an answer is expected */
unsigned int len; /* Len of data valid in buf */
char buf[2048]; /*
* Buffer that each source can fill
* with additional information
*/
}The caller fills source with the subprocess that accepts the command. The values of cmd live in network_ipc.h 💖
Messages for suricatta 🦋
suricatta accepts messages in JSON format. The message has to be formatted inside the buf field of the message data.
Setting the polling time ⏰
{ "polling" : <value in seconds, range 0..X>}Setting it to 0 has the special meaning that the polling time gets retrieved from the Backend (if the server supports it, that is).
Enable / disable Suricatta daemon 🔛
{ "enable" : true }
{ "enable" : false }Set custom device attributes for Suricatta (for Hawkbit implementation) 🏷️
{ "identify" : [
{
"name" : "customizableAttributeOne",
"value" : "valueOne"
},
{
"name" : "customizableAttributeTwo",
"value" : "valueTwo"
}
]}New attributes can be added at runtime, and existing attributes can be modified the exact same way. The changes will be reflected on the server in the next poll iteration 🌸
Trigger a check on the server 📡
This one is iconic if the device is mostly offline and, whenever it comes online, it should check immediately whether an update exists and run it. Fr fr: after enabling the suricatta daemon, the update follows the usual states, and the daemon waits for a polling time before loading the new software. This command forces an update (if one is available) without changing the polling time.
{ "trigger" : true }Activate an already installed Software 🎉
After a software was installed, the new software boots, and if everything runs fine, an acknowledge should be sent to the hawkBit server. If this feature is used — for example to let the end user decide whether the new software is accepted — the parameters used by the installation should be stored during the update process.
{ "id" : <action id>,
"finished" : "success", "failure", "none",
"execution" : ["closed", "proceeding", canceled", "rejected", "resumed"]
"details" : [ ]
}Get hawkBit Server Status 👑
To provide the hawkBit server status to other processes, you can request it by sending an empty message with message type CMD_GET_STATUS.
The response is a JSON object containing the hawkBit server status <status>. <status> is a number representing the value of the channel_op_res_t enum from channel_op_res.h. Since the hawkBit server is polled, its status can only be updated when it has been polled — so the response also contains the time <time>, which is when the hawkBit server was polled the last time. It's provided as an ISO 8601 date and time string. (2021-10-14T13:42:37.000+00)
{ "server" : {
"status" : <status>
"time" : <time>
}
}API to the integrated Webserver 🌐
The integrated Webserver provides REST resources to push a SWU package and to get some info about the update process. This API is based on HTTP standards, and there are two kinds of interfaces:
- Install API to push a SWU and to restart the device after the update.
- A WebSocket interface to send the status of the update process.
Install API 📦
POST /uploadThis initiates an update: the initiator sends the request and starts to stream the SWU in the same way as described in API Description. The POST /upload request contains:
- the SWU file in its body
- the following header field
Content-Type: multipart/form-data; boundary...For example, to stream the SWU file and start the update procedure using curl:
curl -F file=@update_file.swu http://host:8080/uploadRestart API 🔄
POST /restartIf it's configured (see post update command), this request will restart the device.
WebSocket API 🧚
The integrated Webserver exposes a WebSocket API. The WebSocket protocol specification defines ws (WebSocket) and wss (WebSocket Secure) as two new uniform resource identifier (URI) schemes that are used for unencrypted and encrypted connections respectively, and both of them are supported by SWUpdate. A WebSocket provides full-duplex communication, but in SWUpdate it's used to send events to an external host after each change in the update process. The Webserver sends JSON formatted responses as results of internal events.
The response contains the field type, which defines which event is sent.
✨ Event Type ✨
ok so every event SWUpdate sends out has a type, and here's the whole lineup bestie:
status— event sent when SWUpdate's internal state changes 💫source— event to inform from which interface an update is received 📡info— event with custom message to be passed to an external process 💌message— event that contains the error message in case of error 😭step— event to inform about the running update 👟
💫 Status Change Event 💫
this one fires whenever the internal SWUpdate status change happens, like a little heartbeat update. these are the statuses it supports:
IDLE
START
RUN
SUCCESS
FAILURE
DONEexample, look at her:
{
"type": "status",
"status": "SUCCESS"
}📡 Source Event 📡
this event tells you from which interface a SWU got loaded. no gatekeeping:
{
"type": "source",
"source": "WEBSERVER"
}the source field can be one of these values:
UNKNOWN
WEBSERVER
SURICATTA
DOWNLOADER
LOCAL💌 Info Event 💌
this one forwards all internal logs that were sent with level=INFO. so all the little "hey just so you know" moments:
{
"type": "info",
"source": < text message >
}😭 Message Event 😭
this event contains the error message in case of failure. it's the "something went wrong bestie" notification, except it's structured.
Fields for message event
status—"message"level—"3"in case of error,"6"as infotext— message associated to the event
example (the sad one):
{
"type": "message",
"level": "3",
"text" : "[ERROR] : SWUPDATE failed [0] ERROR core/cpio_utils.c : ",
}👟 Step event 👟
this event tells you which step is currently running and what percentage of that step is already installed. it's giving progress bar energy fr fr.
Fields for step event
number— total number of steps N for this update 🔢step— running step in range[1..N]🪜name— filename of artifact to be installed 📄percent— percentage of the running step 📊
example:
{
"type": "step",
"number": "7",
"step": "2",
"name": "rootfs.ext4.gz",
"percent": "18"
}✨ GETTING INFORMATION ON RUNNING UPDATE ✨
ok so real talk: sometimes you don't just wanna know if the update succeeded, you wanna know how it's going while it's going. the operator deserves tea ☕. like, if the target has a display or a remote interface, you can forward which percentage of the update has been reached, so everyone can estimate how much longer this thing is gonna run. iconic behavior.
SWUpdate has an interface for exactly this (the "progress API"). an external process can register itself with SWUpdate, and then it will receive notifications whenever something in the update changed. and bestie, this is different from the IPC API — because the IPC API is mainly used to transfer the SWU image, and with that one you can only poll the interface to find out if the update is still running. no push notifications there, just you refreshing like it's 2011 💀.
🎀 API Description 🎀
so an external process registers itself to SWUpdate by doing a connect() request to the domain socket "/tmp/swupdateprog" — that's per the default configuration of SWUpdate. there's literally no information to send, SWUpdate just slides the new connection into the list of processes to be informed. and then? SWUpdate will send a frame back after any change in the update process with the following data (see include/progress_ipc.h):
struct progress_msg {
unsigned int magic; /* Magic Number */
unsigned int status; /* Update Status (Running, Failure) */
unsigned int dwl_percent; /* % downloaded data */
unsigned long long dwl_bytes; /* total of bytes to be downloaded */
unsigned int nsteps; /* No. total of steps */
unsigned int cur_step; /* Current step index */
unsigned int cur_percent; /* % in current step */
char cur_image[256]; /* Name of image to be installed */
char hnd_name[64]; /* Name of running handler */
sourcetype source; /* Interface that triggered the update */
unsigned int infolen; /* Len of data valid in info */
char info[2048]; /* additional information about install */
};the single fields have the following meaning, field by field, no skipping:
magicis not yet used, it could be added for simply verification of the frame. (yes she's a placeholder, she's manifesting her purpose ✨)statusis one of the values inswupdate_status.h(START,RUN,SUCCESS,FAILURE,DOWNLOAD,DONE).dwl_percentis the percentage of downloaded data whenstatus = DOWNLOAD.dwl_bytesis the total number of bytes that are to be downloaded.nstepsis the total number of installers (handlers) to be run.cur_stepis the index of the running handler.cur_stepis in the range1..nstepscur_percentis the percentage of work done inside the current handler. this is useful when updating a slow interface, such as a slow flash, and signals which is the percentage of image already copied into the destination. (slow flash, we've all been there 🐌)cur_imageis the name of the image in sw-description that is currently being installed.hnd_namereports the name of the running handler.sourceis the interface that triggered the update.infolenlength of data in the following info field.infoadditional information about installation.
and as an example for a progress client, tools/swupdate-progress.c prints the status on the console and drives "psplash" to draw a progress bar on a display. slay. 🫶
🌈 LANGUAGE BINDINGS 🌈
✨ Overview ✨
okay so bestie, here's the tea: SWUpdate does not care what language you speak to it in 💅. It's agnostic to a particular language it is operated from, and that's all thanks to SWUpdate's socket-based control <> and progress APIs <> for external programs. So as long as your language of choice has proper socket (library) support, you can absolutely drive SWUpdate with it. no cap.
But wait, there's more 🎀 — for convenience, a Lua language binding in terms of a shared library is provided, currently lua_swupdate.so.0.1.
🦋 Lua Language Binding 🦋
The Lua language binding is realized in terms of the lua_swupdate module that defines three bindings, namely:
- the control interface,
- the progress interface, and
- a convenience function yielding a table holding all local network interfaces including their IP addresses and submasks.
The lua_swupdate Lua module interface specification that details what functionality is made available by bindings/lua_swupdate.c is found in bindings/lua_swupdate.lua, reproduced for convenience in Section Lua Language Binding Interface Specification 👑. It serves as reference, for mocking purposes, and type checking thanks to the EmmyLua-inspired annotations. iconic fr fr.
lowkey important heads up 🙈: depending on the filesystem location of the Lua binding's shared library, Lua'spackage.cpathmay have to be adapted by setting the environment variableLUA_CPATH, modifyingpackage.cpathprior to arequire('lua_swupdate'), or , as last resort, usingpackage.loadlib()instead ofrequire('lua_swupdate'). (yes, that comma-before-space thing is literally how it's written, we're not touching it 💀)
💖 Control Interface 💖
The lua_swupdate module's control interface binding conveniently makes SWUpdate's socket-based control API <> available to pure Lua. slay.
The binding is captured in the swupdate_control object that is returned by a call to swupdate.control(). This object offers the three methods connect(), write(<chunkdata>), and close():
- The
connect()method initializes the connection to SWUpdate's control socket, sendsREQ_INSTALL, and waits forACKorNACK, returning the socket connection file descriptor, mostly for information purposes, or, in case of an error,nilplus an error message. - The artifact's data can then be sent to SWUpdate via the
write(<chunkdata>)method, returningtrue, or, in case of errors,nilplus an error message. - Finally, the
close()method closes the connection to SWUpdate's control socket after which it waits for SWUpdate to complete the update transaction and executes the post-install command, if given. periodt.
The following example snippet illustrates how to use the control interface binding:
local artifact = io.open("/some/path/to/artifact.swu", "rb" )
swupdate = require('lua_swupdate')
local ctrl = swupdate.control()
if not ctrl:connect() then
-- Deliberately neglecting error message.
io.stderr:write("Error connecting to SWUpdate control socket.\n")
return
end
while true do
local chunk = artifact:read(1024)
if not chunk then break end
if not ctrl:write(chunk) then
-- Deliberately neglecting error message.
io.stderr:write("Error writing to SWUpdate control socket.\n")
break
end
end
local res, msg = ctrl:close()
if not res then
io.stderr:write(string.format("Error finalizing update: %s\n", msg))
end🌸 Progress Interface 🌸
The lua_swupdate module's progress interface binding conveniently makes SWUpdate's socket-based progress API <> available to pure Lua. yes bestie, same energy, different socket ✨
The binding is captured in the swupdate_progress object that is returned by a call to swupdate.progress(). This object offers the three methods connect(), receive(), and close():
- The
connect()method connects to SWUpdate's progress socket, waiting until the connection has been established. Note that it is only really required to explicitly callconnect()to reestablish a broken connection as theswupdate_progressobject's instantiation already initiates the connection. - The
receive()method returns a table representation of thestruct progress_msgdescribed in the progress interface's API description <>. - The
close()method deliberately closes the connection to SWUpdate's progress socket.
🌈 IPv4 Interface 🌈
For convenience, the lua_swupdate module provides the ipv4() method returning a table holding the local network interfaces as the table's keys and their space-separated IP addresses plus subnet masks as respective values. It's giving "here's your network, all wrapped up with a bow 🎀".
👑 Lua Language Binding Interface Specification 👑
--[[
SWUpdate IPC Lua Module Interface.
Interface specification for the Lua IPC module.
See: bindings/lua_swupdate.c
Copyright (C) 2022, Siemens AG
Author: Christian Storm <christian.storm@siemens.com>
SPDX-License-Identifier: GPL-2.0-or-later
--]]
---@diagnostic disable: missing-return
---@diagnostic disable: unused-local
-- luacheck: no unused args
--- SWUpdate IPC Lua Module.
--- @class lua_swupdate
local lua_swupdate = {}
--- Get local IPv4 network interface(s) information.
--
-- The returned Table contains the network interface names
-- as keys and a space-separated IP address and subnet mask
-- as values, e.g., {['eth0']="192.168.1.1 255.255.255.0"}.
--
--- @return table<string, string>
lua_swupdate.ipv4 = function() end
--- @enum lua_swupdate.RECOVERY_STATUS
--- Lua equivalent of `RECOVERY_STATUS` as in `include/swupdate_status.h`.
lua_swupdate.RECOVERY_STATUS = {
IDLE = 0,
START = 1,
RUN = 2,
SUCCESS = 3,
FAILURE = 4,
DOWNLOAD = 5,
DONE = 6,
SUBPROCESS = 7,
PROGRESS = 8
}
--- @enum lua_swupdate.sourcetype
--- Lua equivalent of `sourcetype` as in `include/swupdate_status.h`.
lua_swupdate.sourcetype = {
SOURCE_UNKNOWN = 0,
SOURCE_WEBSERVER = 1,
SOURCE_SURICATTA = 2,
SOURCE_DOWNLOADER = 3,
SOURCE_LOCAL = 4,
SOURCE_CHUNKS_DOWNLOADER = 5
}
--- Lua equivalent of `struct progress_msg` as in `include/progress_ipc.h`.
--- @class lua_swupdate.progress_msg
--- @field apiversion number API Version for compatibility check
--- @field status lua_swupdate.RECOVERY_STATUS Update status, one of `lua_swupdate.RECOVERY_STATUS`'s values
--- @field download number Downloaded data percentage
--- @field source lua_swupdate.sourcetype Interface that triggered the update, one of `lua_swupdate.sourcetype`'s values
--- @field nsteps number Total number of steps
--- @field step number Current step
--- @field percent number Percentage done in current step
--- @field artifact string Name of image to be installed
--- @field handler string Name of running Handler
--- @field info string Additional information about installation
--- SWUpdate progress socket binding.
--
-- The returned Class Table contains methods to
-- interact with SWUpdate's progress socket.
--
lua_swupdate.progress = function()
return {
--- Connect to SWUpdate's progress socket.
--
--- @param self table This `lua_swupdate.progress` instance
--- @return number | nil # The connection handle or nil in case of error
--- @return nil | string # nil or an error message in case of error
connect = function(self) end,
--- Receive data from SWUpdate's progress socket.
--
--- @param self table This `lua_swupdate.progress` instance
--- @return table | lua_swupdate.progress_msg # This `lua_swupdate.progress` instance on error or the received progress message
--- @return nil # nil in case of error
receive = function(self) end,
--- Close connection to SWUpdate's progress socket.
--
--- @param self table This `lua_swupdate.progress` instance
--- @return table # This `lua_swupdate.progress` instance
close = function(self) end,
}
end
--- SWUpdate control socket binding.
--
-- The returned Class Table contains methods to
-- interact with SWUpdate's control socket.
--
lua_swupdate.control = function()
return {
--- Connect to SWUpdate's control socket.
--
--- @param self table This `lua_swupdate.control` instance
--- @return number | nil # Connection handle or nil in case of error
--- @return nil | string # nil or an error message in case of error
connect = function(self) end,
--- Write to connected SWUpdate control socket.
--
--- @param self table This `lua_swupdate.control` instance
--- @param data string Chunk data to write to SWUpdate's control socket
--- @return boolean | nil # true or nil in case of error
--- @return nil | string # nil or an error message in case of error
write = function(self, data) end,
--- Close connection to SWUpdate's control socket.
--
--- @param self table This `lua_swupdate.control` instance
--- @return boolean | nil # true or nil in case of error
--- @return nil | string # nil or an error message in case of error
close = function(self) end,
}
end
return lua_swupdate👑 BOOTLOADER INTERFACE 👑
🌈 Overview 🌈
So bestie, SWUpdate has bindings to a whole lineup of bootloaders, and their whole job is to store persistent state info across reboots. It's giving "we remember things even after you turn the board off". Currently, these are the bootloaders we support:
- A fake bootloader called "Environment in RAM",
- EFI Boot Guard (
https://github.com/siemens/efibootguard), - U-Boot (
https://www.denx.de/wiki/U-Boot), and - GRUB (
https://www.gnu.org/software/grub/).
Which bootloaders are actually in the (sub)set is a compile-time choice. And then at run-time, the compile-time set default bootloader interface implementation is the one that gets used — unless you overrule it to use a different bootloader interface implementation via the -B command line switch or a configuration file (via the bootloader setting in the globals section, see examples/configuration/swupdate.cfg). The way that you have options? Iconic.
Heads up, bestie 💖 — the run-time support for some bootloaders, currently U-Boot and EFI Boot Guard, relies on loading the respective bootloader's environment modification shared library at run-time. So even if support for a particular bootloader is compiled-in, the according shared library must be present and loadable on the target system at run-time for you to actually use this bootloader interface implementation. This is a feature, not a bug, fr fr: it lets, e.g., distributions ship a generic SWUpdate package and downstream integrators combine that generic package with the appropriate bootloader by just providing its environment modification shared library. Slay.
🧚 Bootloader Interface Description 🧚
The bootloader interface implementations live in bootloader/. Each bootloader has to implement the interface functions as defined in include/bootloader.h, more precisely:
char *env_get(const char *name);
int env_set(const char *name, const char *value);
int env_unset(const char *name);
int apply_list(const char *filename);And what do those lil guys do? They retrieve a key's value from the bootloader environment, set a key to a value in the bootloader environment, delete a key-value pair from the bootloader environment, and apply the key=value pairs found in a file. Periodt.
Then, each bootloader interface implementation has to register itself to SWUpdate at run-time by calling the register_bootloader(const char *name, bootloader *bl) function — that takes the bootloader's name and a pointer to struct bootloader as in include/bootloader.h, which is filled with pointers to the respective above mentioned interface functions. And plot twist 😭 — if the bootloader setup fails and hence it cannot be successfully registered, e.g., because the required shared library for environment modification cannot be loaded, NULL is to be returned as pointer to struct bootloader.
For example, let's say we have a bootloader named "trunk" and (static) interface functions implementations do_env_{get,set,unset}() as well as do_apply_list() in a bootloader/trunk.c file — the following snippet registers this bootloader to SWUpdate at run-time:
static bootloader trunk = {
.env_get = &do_env_get,
.env_set = &do_env_set,
.env_unset = &do_env_unset,
.apply_list = &do_apply_list
};
__attribute__((constructor))
static void trunk_probe(void)
{
(void)register_bootloader(BOOTLOADER_TRUNK, &trunk);
}with
#define BOOTLOADER_TRUNK "trunk"added to include/bootloader.h as a single central "trunk" bootloader name definition, which aids in maintaining the uniqueness of bootloader names. And this new "trunk" bootloader should also be added to the Suricatta Lua Module interface specification's bootloader Table suricatta.bootloader.bootloaders = { ... } in suricatta/suricatta.lua. Don't skip that step bestie, it's part of the assignment.
Attention: Take care to uniquely name the bootloader. 💀
See, e.g., bootloader/{uboot,ebg}.c for examples of a bootloader using a shared environment modification library, and bootloader/{grub,none}.c for a simpler bootloader support example. Both vibes are valid.
✨ Bootloader Build System Integration ✨
A bootloader support implementation needs to be registered to the kconfig build system. No sneaking in without an invite.
First, the bootloader support implementation — named "trunk" and implemented in bootloader/trunk.c for example — needs to be added to bootloader/Kconfig in the Bootloader Interfaces menu as follows:
...
menu "Bootloader"
menu "Bootloader Interfaces"
...
config BOOTLOADER_TRUNK
bool "TrUnK Bootloader"
help
Support for the TrUnK Bootloader
https://github.com/knurt/trunkThen, in order to enable the compile-time selection of the "trunk" bootloader as default, add a section to the Default Bootloader Interface choice submenu of the Bootloader menu as follows:
choice
prompt "Default Bootloader Interface"
help
Default bootloader interface to use if not explicitly
overridden via configuration or command-line option
at run-time.
...
config BOOTLOADER_DEFAULT_TRUNK
bool "TrUnK"
depends on BOOTLOADER_TRUNK
help
Use TrUnK as default bootloader interface.Finally, bootloader/Makefile needs to be adapted to build the "trunk" bootloader support code, given BOOTLOADER_TRUNK was enabled:
obj-$(CONFIG_BOOTLOADER_TRUNK) += trunk.oAnd if the "trunk" bootloader, for example, requires loading a shared environment modification library, then Makefile.flags needs to be adapted as well, e.g., as follows:
ifeq ($(CONFIG_BOOTLOADER_TUNK),y)
LDLIBS += dl
endif(yes bestie, that's spelled TUNK up there, verbatim, we don't fix the originals in this house 🙈 just copy it exactly.)
🌸 META-SWUPDATE: BUILDING WITH YOCTO 🌸
🦋 Overview 🦋
The Yocto-Project (http://www.yoctoproject.org) is a community project under the umbrella of the Linux Foundation that provides tools and template to create the own custom Linux-based software for embedded systems. Academia but make it embedded.
Add-on features can be added using layers. meta-swupdate (https://github.com/sbabic/meta-swupdate.git) is the layer to cross-compile the SWUpdate application and to generate the compound SWU images containing the release of the product. It contains the required changes for mtd-utils and for generating Lua. Using meta-SWUpdate is a straightforward process. As described in Yocto's documentation about layers, you should include it in your bblayers.conf file to use it.
Add meta-SWUpdate as usual to your bblayers.conf. You have also to add meta-oe to the list. Both, okay? Not optional.
In meta-SWUpdate there is a recipe to generate an initrd with a rescue system with SWUpdate. Use:
MACHINE=<your machine> bitbake swupdate-imageYou will find the result in your tmp/deploy/<your machine> directory. How to install and start an initrd is very target specific — please check in the documentation of your bootloader. We can't hold your hand through that one, it's giving "read your bootloader's docs".
🎀 What about libubootenv ? 🎀
This is a common issue when SWUpdate is built. SWUpdate depends on this library, that is generated from the U-Boot's sources. This library allows one to safe modify the U-Boot environment. It is not required if U-Boot is not used as bootloader. If SWUpdate cannot be linked, you are using an old version of U-Boot (you need at least 2016.05). If this is the case, you can add your own recipe for the package u-boot-fw-utils, adding the code for the library.
It is important that the package u-boot-fw-utils is built with the same sources of the bootloader and for the same machine. In fact, the target can have a default environment linked together with U-Boot's code, and it is not (yet) stored into a storage. SWUpdate should be aware of it, because it cannot read it: the default environment must be linked as well to SWUpdate's code. This is done inside the libubootenv.
Lowkey terrifying: if you build for a different machine, SWUpdate will destroy the environment when it tries to change it the first time. In fact, a wrong default environment is taken, and your board won't boot again. 😭 Not the kind of plot twist we want.
To avoid possible mismatch, a new library was developed to be hardware independent. A strict match with the bootloader is not required anymore. The meta-swupdate layer contains recipes to build the new library (libubootenv) and adjust SWUpdate to be linked against it. To use it as replacement for u-boot-fw-utils:
- set
PREFERRED_PROVIDER_u-boot-fw-utils = "libubootenv" - add to SWUpdate config:
CONFIG_UBOOT=yWith this library, you can simply pass the default environment as file (u-boot-initial-env). It is recommended for new project to switch to the new library to become independent from the bootloader. Do it bestie, future you will thank you 💜.
💅 The swupdate class 💅
meta-swupdate contains a class specific for SWUpdate. It helps to generate the SWU image starting from images built inside the Yocto. It requires that all components — that means the artifacts that are part of the SWU image — are present in the Yocto's deploy directory. This class should be inherited by recipes generating the SWU. The class defines new variables, all of them have the prefix SWUPDATE_ in the name. Cute naming convention, honestly.
SWUPDATE_IMAGES— this is a list of the artifacts to be packaged together. The list contains the name of images without any extension forMACHINEor filetype, that are added automatically. Example:
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"SWUPDATE_IMAGES_FSTYPES— extension of the artifact. Each artifact can have multiple extension according to theIMAGE_FSTYPESvariable. For example, an image can be generated as tarball and as UBIFS for target. Setting the variable for each artifact tells the class which file must be packed into the SWU image.
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"SWUPDATE_IMAGES_NOAPPEND_MACHINE— flag to use drop the machine name from the artifact file. Most images in deploy have the name of the Yocto's machine in the filename. The class adds automatically the name of theMACHINEto the file, but some artifacts can be deployed without it.
SWUPDATE_IMAGES_NOAPPEND_MACHINE[my-image] = "1"SWUPDATE_SIGNING— if set, the SWU is signed. There are 3 allowed values:RSA,CMS,CUSTOM. This value determines used signing mechanism.SWUPDATE_SIGN_TOOL— instead of usingopenssl, useSWUPDATE_SIGN_TOOLto sign the image. A typical use case is together with a hardware key. It is available ifSWUPDATE_SIGNINGis set toCUSTOM.SWUPDATE_PRIVATE_KEY— this is the file with the private key used to sign the image using RSA mechanism. Is available ifSWUPDATE_SIGNINGis set toRSA.SWUPDATE_PASSWORD_FILE— an optional file containing the password for the private key. It is available ifSWUPDATE_SIGNINGis set toRSA.SWUPDATE_CMS_KEY— this is the file with the private key used in signing process using CMS mechanism. It is available ifSWUPDATE_SIGNINGis set toCMS.SWUPDATE_CMS_CERT— this is the file with the certificate used in signing process using CMS method. It is available ifSWUPDATE_SIGNINGis set toCMS.SWUPDATE_AES_FILE— bestie this is literally just the file holding the AES password that encrypts your artifact. The class supports a whole new fstype for it (type:enc💅). You generate this file straight fromopensslwhen you make a fresh key, like so:
openssl enc -aes-256-cbc -k <PASSPHRASE> -P -md sha1 -nosalt > $SWUPDATE_AES_FILETo actually use it? Just drop IMAGE_FSTYPES += "enc" onto your artifact and you're done, no cap. And SWUpdate can decrypt compressed artifacts too, it's giving very flexible, e.g.:
IMAGE_FSTYPES += ".ext4.gz.enc"🌸 Automatic sha256 in sw-description 🌸
The swupdate class is the one doing the work here — it computes and inserts the sha256 hashes into your sw-description file for you. 🫶 The sha256 attribute is required if the image is signed, so every artifact needs:
sha256 = "$swupdate_get_sha256(artifact-file-name)"For example, to add sha256 to the standard Yocto core-image-full-cmdline:
sha256 = "$swupdate_get_sha256(core-image-full-cmdline-machine.ubifs)";⚠️ And btw, the file name has to match the one in the deploy directory. No sneaky renaming.
✨ BitBake variable expansion in sw-description ✨
Want a BitBake variable to show up inside the update file? Slap @@ both before and after the variable name. Obsessed with how easy that is. So to automatically set the version tag:
version = "@@DISTRO_VERSION@@";💖 Automatic versions in sw-description 💖
Set the version tag in the update file to $swupdate_get_pkgvar(<package-name>) and it gets automatically swapped for PV from BitBake's package-data-file for the package matching the <package-name> you gave it. Iconic. So like, to set the version tag to PV of package u-boot:
version = "$swupdate_get_pkgvar(u-boot)";And if you want to auto-insert a variable other than PV from BitBake's package-data-file (say PKGV), just tack the variable name onto the tag: $swupdate_get_pkgvar(<package-name>@<package-data-variable>). So for PKGV of package u-boot:
version = "$swupdate_get_pkgvar(u-boot@PKGV)";🦋 Using checksum for version 🦋
Okay so you can use the hash of an artifact as the version, which lets you use install-if-different — meaning versionless artifacts get skipped when the artifact in the update matches what's currently installed. Chef's kiss. 🎀
To do that, you have to use the sha256 hash file placeholder described above in Automatic sha256 in sw-description as the version. Every artifact needs:
version = "@artifact-file-name"And again — the file name must be the same as in the deploy directory. Periodt. 💅
🌈 Template for recipe using the class 🌈
DESCRIPTION = "Example recipe generating SWU image"
SECTION = ""
LICENSE = ""
# Add all local files to be added to the SWU
# sw-description must always be in the list.
# You can extend with scripts or whatever you need
SRC_URI = " \
file://sw-description \
"
# images to build before building swupdate image
IMAGE_DEPENDS = "core-image-full-cmdline virtual/kernel"
# images and files that will be included in the .swu image
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
# a deployable image can have multiple format, choose one
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
SWUPDATE_IMAGES_FSTYPES[uImage] = ".bin"
inherit swupdate⭐ Simplified version for just image ⭐
Lowkey so many cases have just one image inside the SWU — like when only the rootfs is getting updated. And the generic setup above meant you had to write and babysit an extra recipe. 😭 So they introduced a simplified version of the class that lets you build the SWU straight from the image recipe.
You just import the swupdate-image class. It already sets some variables for you. You still have to add a sw-description into a files directory — the class searches for it automatically — and you still need to set SWUPDATE_IMAGE_FSTYPES[your image] to the fstype that gets packed into the SWU. 🫶 And heads up: if that flag isn't set, an error is raised. No silent vibes, it will absolutely call you out.
So the simple way, your recipe looks like:
<your original recipe code>
SWUPDATE_IMAGES_FSTYPES[<name of your image>] = <fstype to be put into SWU>
inherit swupdate-image🔥 SWU image content's encryption 🔥
The swupdate class can encrypt the contents of the SWU image. Main character energy fr fr. To do it, first make sure SWUpdate is compiled with CONFIG_ENCRYPTED_IMAGES=y enabled. Then set SWUPDATE_AES_FILE to the full path of the key. Finally, for each content you want encrypted, add this to your SWU image recipe:
SWUPDATE_IMAGES_ENCRYPTED[content] = "1"Where content matches the files described in sw-description.
And if you want to encrypt sw-description itself, check that SWUpdate is being compiled with CONFIG_ENCRYPTED_SW_DESCRIPTION=y and then set this in your SWU image recipe:
SWUPDATE_ENCRYPT_SWDESC = "1"Oh and bestie — do not forget to add encrypted = true; on each of the contents that will require decryption in the sw-description file. 💀 Skipping that is how you get a mysterious sad little device.
Expanding on the previous example:
DESCRIPTION = "Example recipe generating SWU image"
SECTION = ""
LICENSE = ""
# Add all local files to be added to the SWU
# sw-description must always be in the list.
# You can extend with scripts or whatever you need
SRC_URI = " \
file://sw-description \
"
# images to build before building swupdate image
IMAGE_DEPENDS = "core-image-full-cmdline virtual/kernel"
# images and files that will be included in the .swu image
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
# a deployable image can have multiple format, choose one
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
SWUPDATE_IMAGES_FSTYPES[uImage] = ".bin"
SWUPDATE_IMAGES_ENCRYPTED[core-image-full-cmdline.ubifs] = "1"
SWUPDATE_IMAGES_ENCRYPTED[uImage] = "1"
SWUPDATE_ENCRYPT_SWDESC = "1"
inherit swupdate💜 What about grub ? 💜
To use swupdate with grub, swupdate has to be configured to use grub. Some of the important configurations are CONFIG_GRUBENV_PATH="/path/to/grubenv", where "/path/to/grubenv" is the path to the grub environment. Example: "/boot/EFI/BOOT/grubenv".
Now listen — the grubenv file should be created using the grub-editenv tool, because it is a 1024-byte file. 😭 So any modification with nano or vim will only corrupt it, and grub will just not be able to use it. Do not do that to yourself.
You can create a grubenv file using these commands, for instance:
GRUBENV="/path/to/grubenv"
grub-editenv $GRUBENV create
grub-editenv $GRUBENV set rootfs=2
grub-editenv $GRUBENV set kernel=2grub-editenv is a tool that is integrated into yocto. 🫶
Once the grubenv file exists, grub should be configured to use it. That configuration lives in the configuration file grub.cfg. Here's an example of grub.cfg that loads the environment file before booting:
# Take a kernel and a rootfs by default in case grubenv is corrupted
rootfs=1
kernel=1
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
default=boot
# set timeout to zero to boot without timeout
timeout=0
# load grubenv the environment file that contains the value of rootfs and kernel variables
load_env -f "/path/to/grubenv"
# detect which memory contains 5 partitions
for i in 1 2 3 4 5; do if [ -d (hd${i},gpt5)/ ]; then drive=${i};fi ; done
# detect which rootfs should we boot with
if [ ${rootfs} = "1" ]; then rootfs_part=4 ; elif [ ${rootfs} = "2" ]; then rootfs_part=5 ; fi
# detect which kernel should we boot with
if [ ${kernel} = "1" ]; then kernel_part="(hd${drive},gpt2)" ; elif [ ${kernel} = "2" ]; then kernel_part="(hd${drive},gpt3)" ; fi
# The menuentry that is used to boot (more can be added if it is wanted)
menuentry 'boot'{
linux ${kernel_part}/bzImage root=/dev/mmcblk1p${rootfs_part} rw rootwait quiet console=ttyS2,115200 console=tty0 panic=5
}The grub.cfg above is merely an example — you can modify it however you want, as long as it loads the environment variables, and it boots correctly using those environment variables. ✨
👑 SWUPDATE BEST PRACTICE 👑
This is meant as a general rule for integrating SWUpdate into a custom project.
SWUpdate is an update agent and it's thought of as a framework. Which means it's highly configurable, and SWUpdate should be configured to fit into your project — not the other way around. 💅 SWUpdate makes just a few requirements on the system and it has no fixed update schema. There's literally no restriction on how many partitions or which storage you're using. And in some more complex cases the update depends on a ton of conditions, and SWUpdate can run differently depending on the mode a device is started in. So think of SWUpdate not as a ready-to-use updater but as a framework, and hence you should first write a meaningful:
🎀 Update Concept 🎀
Take your time and write an update concept for your device first. It is not wasted time, bestie, I promise. You have to imagine the conditions when an update is not working, and try to write down the use cases where an update can fail and how the device can be restored. SWUpdate installs new software, but a successful update means the new software is started and runs flawlessly. The interface with the bootloader (or whatever starts the software) must be checked in detail. A successful update means:
- SWUpdate runs successfully,
- the device reboots,
- the bootloader can start the new software, and
- the new software runs, makes some consistency checks, and declares that the transaction (that is from old to new software) is terminated.
Which means some coordination between the bootloader and the update agent is necessary. In most cases this is done via persistent variables that are available to both SWUpdate and the bootloader. SWUpdate has two built-in variables:
recovery_status: this gets set when SWUpdate starts to write to the device, and it gets unset after the installation completes with success — or it's set tofailedif there's an error. A bootloader can use it to check whether an update was interrupted. This is a must in case a single copy of the software is on the device.ustate: this triggers a state machine. SWUpdate sets it to1(INSTALL) after an update. The bootloader can use it to check whether a new software must be tested. The bootloader should implement a counter mechanism to check how many times it tried to start a new software. When a threshold is reached, the bootloader should declare the new software as buggy, and proceed with a fallback in case the old software is available. 🦋
A fallback is always initiated by the bootloader, bestie — because the bootloader is the only one who actually knows whether the new software is running. It should toggle the copies and start the old software set. And to communicate this to user space and to SWUpdate, the bootloader sets the ustate variable to 3 (FAILED). SWUpdate uses that info when the result has to be forwarded to an external server (like a backend). ✨
There's a time window when a fallback can take place. Fr fr — after a reboot and some attempts, the update transaction is declared successful or failed, and later a new update can be executed. When a new update runs, the status of the stand-by copy is unknown, because it could be the result of an interrupted update. And running an incomplete software can lead to unpredictable results and must be strongly avoided. 😭 Like, do not do it.
A common pattern for a toggling in the bootloader is:
ustateis not set or set to0(no update pending). The bootloader runs the configured copy and won't ever toggle. In case of failure, a rescue system can be started if available.ustateis set to1(INSTALLED): the new software is under test. The bootloader initiates a fallback if the new software is not running and setsustateto3(FAILED). If the new software runs, it is the duty of user space (often SWUpdate or the init script for SWUpdate) to resetustateto0. Note that resetting the variable is project specific, and it could be set as last action after having sufficiently checked that the new software is running. This includes performing in the application a database migration, starting communicating with peers, whatever.
A possible diagram is shown in next picture — it is not supposed to work with any project, but it gives an idea how fallback is working together with the bootloaders. [image] 🦋
Check in advance which security topics are relevant for your project. This includes:
- signed images (SWU is verified before installing), and then which crypto mechanism is used (RSA keys, certificates, PKI)
- encryption for artifacts
- under which user and group SWUpdate and other components are allowed to run. Set user id and group id if not root in
swupdate.cfg. - if any version can be installed or if you forbid a downgrade, and then be sure to pass the range of versions you allow via
--M,-Rand--max-version. - hardware-software compatibility check and how your device knows which hardware revision is running.
🌸 Reduce dependencies to minimum 🌸
An update should be possible in any condition. Even if the system is degraded or in a bad shape, if an update can work, the device can be functional again without returning it back to the factory. That's the dream, no cap. 💅
SWUpdate is thought to be self contained: that means it does not make use of external tools. If your system is degraded and filesystems get corrupted, there are less chances to restore it if the update calls external tools. SWUpdate is started at boot time and there are good chances it succeeds even if your system has some (software) flaws.
Be careful to make an update depending on your application, or try to reduce the dependencies. In fact, the application is updated often and an introduction of new bugs can make the device no longer updatable. Take the dependencies under control, and if you have any, be sure that the update is still working. You can fix any bugs if the update works, but not anymore if the device cannot be updated. 😭 major vibe check.
🧚 Make a risk analysis 🧚
A more accurate analysis brings less surprises in the field. Think twice about what you want to update, which components should be updated, and the risks of updating a single point of failure.
Very often, this means the bootloader. Compare risks and benefits: it happens in many projects that having the possibility (with some risk) to update the bootloader is better that returning the devices back to service. A cost / benefits analysis should be part of the integration of the update agent. ✨
👑 SWUpdate builtin configuration 👑
SWUpdate has a compile time configuration. The default configuration delivered with meta-swupdate is not suitable for most projects. The easy way to check configuration in Yocto is to run:
bitbake -c menuconfig swupdateOutside Yocto, just run in SWUpdate's sources:
make menuconfigCheck security, bootloader, and which handlers should be installed. They depend strongly on your project. If you build with OE, add a swupdate_%.bbappend to one of your layers, and put the resulting configuration file as defconfig that can be fetched. Please review the following configuration:
- Security settings
- Interfaces required (where the software is coming from). Disable the interface you do not need.
- Handlers required for your project. Disable what you do not need, but consider if you could need some of them in future. As example, you can safely disable
ubivolif you do not use raw NAND, but you can letarchiveenabled if you plan to install artifacts from tarballs in future. - It is highly recommended to enable Lua to extend runtime behavior.
🎀 SWUpdate startup 🎀
An easy way to start SWUpdate is provided only with meta-swupdate and Yocto. A generic SystemV init script or a systemd unit for SWUpdate are executing a script swupdate.sh, that is delivered together with the SWUpdate binaries.
The script goes through /etc/swupdate/conf.d/ and sources all found files. The integrator can use a set of predefined variables to configure SWUpdate's command line parameters. Obsessed with this pattern tbh. 💖
SWUPDATE_WEBSERVER_ARGS: This string is passed if the webserver must be started. It consists of the webserver specific parameters. If this variable is set, the script will add-wto the list of parameters. Note: meta-swupdate contains a default configuration forSWUPDATE_WEBSERVER_ARGS, that uses/wwwas document root for the Website and default port8080.SWUPDATE_SURICATTA_ARGS: Suricatta (backend) specific parameters. There is no default.SWUPDATE_ARGS: Parameters not belonging to Webserver or Suricatta.
Note that swupdate.sh sources the files in sorted order, so it is possible to override the variables with a configuration file whose filename is loaded at the end. Preferred style is to use SystemV like names, for example 10-webserver, 11-suricatta, and so on.
🌈 Write sw-description 🌈
sw-description is the central file that describes a new software release and how a release must be installed. It should be a consequence of the update concept. There is not a single right way. ⭐
SWUpdate heavily uses 'selections' and links to extract just one part of the whole sw-description, that can be used for different situations and different ways to run the device. One use case for selections is to implement the dual-copy (often referred to as A/B) mode: one selection contains instructions for one copy, the other for the second copy. Which copy is the stand-by must be detected before running SWUpdate and passed via the -e <selection,mode> switch.
Other methods set up a link to the standby storage (like /dev/standby) during boot. Or the standby device can be detected at runtime with an embedded-script, as part of sw-description, with Lua code. Please note that for the last case, SWUpdate is extended with functions exported to the Lua context that simplify the detection. SWUpdate exports a getroot() function that returns type and value for the device used as rootfs. See SWUpdate documentation for a complete list of functions exported by SWUpdate that can be used in Lua. An embedded Lua script must just start with
require ('swupdate')to make use of them.
💫 Use OE variables as much as possible 💫
meta-swupdate replaces a special construct in sw-description with the values of build variables. The recognized construct in sw-description is delimited by @@, that is @@VARIABLE-NAME@@. The exception (for compatibility reasons) is the automatic generation of sha256. The syntax in that case is :
sha256 = "$swupdate_get_sha256(<name of artifact>)"You can again use variable substitution for artifact names. Example:
sha256 = "$swupdate_get_sha256(@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@)";Please note that each variable is double delimited (at the beginning and at the end) by @@. 🦋
💜 Deliver your scripts instead of relying on them being installed 💜
You have the freedom to call any tools during an update. However, take care if you are using some tools from the running rootfs / current software. This implies that the current software is running flawlessly, as well as the tools you are calling. And this may not always be the case. 🙈
🔥 Prefer Lua to shell scripts 🔥
Shell scripts are very popular, and they are often used even when they are not strictly required. They can raise security issues.
In fact, take as example a simple shell script. Goal of rootkits is often the shell, because taking control of the shell means to control the whole device. If the shell is compromised, the whole system is compromised. 💀
Running a shell script means that SWUpdate should call "fork" followed by an "exec". This means also that many resources are duplicated in the child process, and it could cause a further problem if system is getting rid of resources.
A better approach is to use Lua and to deliver the scripts inside the SWU. In fact, the Lua interpreter is linked to SWUpdate and runs in context of the SWUpdate process without forking a child process. Shell is not involved at all. Of course, Lua scripts should be written to be self-contained, too, and executing external tools should be done only if unavoidable. periodt. 💅
🍓 Use installed-directly when possible 🍓
SWUpdate can be enabled for zero-copy (or streaming mode), that is the incoming SWU is analyzed on the fly and it is installed by the associated handler without any temporary copy. If this is not set, SWUpdate creates a temporary copy in $TMPDIR before passing it to the handlers. Note that $TMPDIR generally points to a RAMDISK and storing files there reduces the amount of memory available for the application.
It makes sense to disable the flag in case the artifact is a single point of failure. A typical example could be the bootloader (not duplicated on the devices), and if the SWU is corrupted or the connection gets broken, the board is left in a bricked state. 😭 It makes sense then to download the whole artifact before installing.
⭐ Always enable sha256 verification ⭐
The SWU image is a CPIO archive with CRC (new ASCII format), but the check in CPIO is very weak. Do not trust it, but enable sha256 for each artifact. no cap, fr fr.
🌸 Always set the "type" attribute 🌸
SWUpdate sets some default handler if the type is not set. Do not use it, but set explicitly the type (that is, which handler should install the artifact) in sw-description.
Do not rely on install order
okay bestie, listen — SWUpdate does not require that artifacts are put into the CPIO in a specific order. ✨ It genuinely does not care. The one and only exception?? sw-description must be the first file in a SWU. That's it, that's the rule. 💅
And also like... try to avoid dependencies inside the SWU?? A dependency here means an artifact that can be installed only after another one was installed before it. It's giving house-of-cards energy and we do not love that. 🫠 If you really need it — for example if you want to install a file into a filesystem provided as image — then disable installed-directy for the file and enable it for the filesystem image instead. K? K. 🎀
Do not drop atomicity !
SWUpdate guarantees atomicity — as long as you don't do something that simply breaks it. 😭 The vibes are protected until you personally wreck them.
Example time: think about the bootloader's environment. In an sw-description there is a specific section where the environment can be set — adding, modifying, deleting variables, the whole menu. 🌸 And here's the slay part: SWUpdate does not change single variables. Instead it generates the resulting new environment for the supported bootloader and this is written in one shot, in a way that is power-cut safe (that's for U-Boot / EFIBootguard, not for GRUB — noted 👀).
You can of course change the environment in a postinstall script, like in the following way (for U-Boot):
fw_setenv var1 val1
fw_setenv var2 val2
fw_setenv var3 val3
fw_setenv var4 val4
fw_setenv var5 val5But if a power cut happens during two calls of fw_setenv?? The environment is left in an intermediate state and this can brick the device. 🔥 Not the cute kind of fire. So don't.
Plan to have a rescue system
Even if you have a double-copy setup, something can still go wrong, bestie. 🫶 So plan to have a rescue system (swupdate-image in meta-swupdate) and to install it on a separate storage than the main system, if it is possible. This is the moment where it saves you: when the main storage is corrupted, the device can be restored in the field without returning it back to the factory. 🏭➡️🚫
And pls pls pls — plan to update the rescue system as well. It is software, too, and its bugs should be fixed, too. No system is above bug fixes, not even the rescue one. 💅
✨ DELTA UPDATE WITH SWUPDATE ✨
Overview
update packages are getting BIG and it is not a vibe. 😭 Once upon a time the whole software was just a bunch of megabytes, but now?? It is not unusual that OS and applications on devices running Linux as OS reach a huge size of Gigabytes. Absolute units.
So like, several mechanisms can be used to reduce the size of downloaded data. The resulting images can be compressed! 🎀 But that is not enough when bandwidth is important and not cheap. Frugal queen behavior is required.
And here's the thing: it is very common that a device will be upgraded to a version that is similar to the running one but adds new features and solves some bugs. Specially in the case of just fixes, the new version is pretty much equal to the original one. So we need methods to download just the differences with the current software, without downloading a full image. 📦✨ And in case an update is performed from a known base, we talk about delta updates.
In the following chapter some well known algorithms are considered and verified to see if they can be integrated into SWUpdate. The criteria to find a suitable algorithm:
- license must be compatible with GPLv2
- good performance for smaller downloads, but not necessarily the best one.
- SWUpdate remains with the concept to deliver one package (SWU), the same independently from the source where the SWU is stored (USB, OTA, etc.)
- It must comply to SWUpdate's security requirements (signed images, privilege separation, etc.)
Specific ad-hoc delta updates mechanisms can be realized when the nature of the updated files is the same. And yeah, it is always possible with SWUpdate to install single files — but coherency and compatibility with the running software must be guaranteed by the integrator / manufacturer. 👑 That part is not covered here. The scope?? To get an efficient and content unaware delta mechanism, one that can upgrade in differential mode two arbitrary images, without any previous knowledge about what they contain. Periodt.
FOSS projects for delta encoding
There are several algorithms for delta encoding — that is, finding the difference between files, generally in binary format. 🌈 For SWUpdate we only consider algorithms available under a compatible FOSS license (GPLv2).
One of the goals in SWUpdate is that it should work independently of which is the format of the artifacts. Very specialized algorithms and libraries like Google's Courgette, used in Chromium, will give much better results — but it works on programs (ELF files) and takes advantages of the structure of compiled code. 🧠 And in case of an OTA update, not only software but any kind of artifact can be delivered, and this includes configuration data, databases, videos, docs, etc. So no assuming ELF, bestie. 💀
librsync — https://librsync.github.io/
librsync is an independent implementation for rsync and does not use the rsync protocol. It is well suited to generate offline differential update, and it is already integrated into SWUpdate. 🥳
But — however — lowkey a catch: librsync takes the whole artifact and generates a differential image that is applied on the whole image. It gives the best results in terms of reduced size when differences are very small... but the differential output tends to be very large as soon as the differences are meaningful. 😭 Differential images created for SWUpdate show that, as soon as the difference is larger, the resulting delta image can even become larger than the original one. Reverse glow-up, honestly.
SWUpdate supports librsync as delta encoder via the rdiff handler. 💅
xdelta — http://xdelta.org/
xdelta uses the VCDIFF algorithm to compute differences between binaries. It is often used to deliver smaller images for CD and DVD. 🎀
But the resulting images are created from an installed image that should be loaded entirely in main memory. 🫠 For this reason it does not scale well when the images are becoming larger, and it is unsuitable for embedded systems and SWUpdate. No cap.
casync — http://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html
casync is, according to his author, a tool for distributing images. It has several interesting aspects that can be helpful with OTA update. 👀 Files itself are grouped together in chunks and casync creates a "Chunk storage" where each chunk is stored on a separate file. The chunk storage is part of the delivery, and it must be stored on a server. casync checks if the chunk is already present on the target, and if not, downloads it.
If that sounds exactly like what you need... bestie, sit down, because there are some drawbacks if casync should be integrated in SWUpdate:
- because of the nature of casync, each chunk is a separate file. This causes a huge number of new connections, because each file is a separate
GETon the server. The overhead caused to re-instantiate a connection is high on small devices, where SSL connections are also increasing CPU load. There are downloads of hundreds or thousands of small files just to recreate the original metadata file. 😭 - casync has no authentication and verification and the index (
.caidxor.caibx) are not signed. This is known, but casync goals and scopes are outside the ones on embedded devices. 💀 - it is difficult to deliver a whole chunk storage. The common usage for OTA is to deliver artifacts, and they should be just a few. Thousands of files to be delivered to let casync compute the new image is not practical for companies: they have a new "firmware" or "software" and they need an easy way to deliver this file (the output from their build system) to the devices. In some cases, they are even not responsible for that, and the firmware is given to another authority that groups all packages from vendors and realizes a sort of OTA service. 🙈
- casync is quite a huge project — even if it was stated that it will be converted into a library, this never happened. This makes it difficult to interface to SWUpdate, and using it as external process is a no way in SWUpdate for security reason. It breaks privilege separation, and adds a lot of code that is difficult to maintain. 🔥
For all these reasons, even if the idea of a chunk storage is good for an OTA updater, casync is not a candidate for SWUpdate. An out-of-the-box solution cannot be found, and it is required to implement an own solution that better suits for SWUpdate. Periodt.
Zchunk — https://github.com/zchunk/zchunk — compression format
zchunk seems to combine the usage of a chunk storage without having to deliver it on a server. 🦋 zchunk is a FOSS project released under BSD by its author (https://www.jdieter.net/posts/2018/05/31/what-is-zchunk/). The goal of this project is something else: zchunk creates a new compression format that adds the ability to download the differences between new and old file. 💅 And that matches very well with SWUpdate.
A zchunk file contains a header that has metadata for all chunks, and according to the header, it is known which chunks must be downloaded and which ones can be reused. ✨ zchunk has utilities to download itself the missing chunks, but it could be just used to find which part of an artifact must be downloading, and SWUpdate can go on with its own way to do this.
One big advantage on this approach: metadata and compressed chunks are still bound into a single file, that can be built by the buildsystem and delivered as it is used to. 📦 The updater needs first the metadata, that is the header in the zchunk file, and processes it to detect which chunks need to be downloaded. Each chunk has its own hash, and the chunks already available on the device are verified against the hash to be sure they are not corrupted. 🫶
Zchunk supports multiple sha algorithms — to be compatible with SWUpdate, zchunk should be informed to generate sha256 hashes. Noted, bestie. ✅
Design Delta Update in SWUpdate
For all reasons stated before, zchunk is chosen as the format to deliver delta update in SWUpdate. 👑
An artifact can be generated in ZCK format and then the ZCK's header (as described in format https://github.com/zchunk/zchunk/blob/main/zchunk_format.txt) can be extracted and added to the SWU. In this way, a ZCK file is signed (and if requested, compressed and/or encrypted) as part of the SWU, and loading chunks from an external URL can be verified as well because the corresponding hashes are already verified as part of the header. 🌈✨
✨ Changes in ZCHUNK project ✨
Okay so Zchunk has an API that hides most of its internals and ships a whole set of tools for making and downloading files in ZCK format. Iconic. 💅 But lowkey there was a problem: Zchunk relies on hashes for the compressed (ZST) chunks, and support for uncompressed data was just... not there. And to glue SWUpdate and zchunk together, you have to be able to compare uncompressed data, because nobody wants a device to be forced to compress a huge pile of data just to do a comparison. Rude.
So here's the short list of what got changed in the Zchunk project:
- create hashes for uncompressed data and extend the format to support it. The header must be extended to include both size and hash of uncompressed data. 💖
- make the library embedded friendly, which means it reports errors on failure instead of just exiting, and finds a suitable way to integrate the log output for the caller.
- allow the use of
sha256(already foreseen in zchunk), since that's the only hash type used in SWUpdate. - add API so an external caller can decide for itself whether a chunk must be downloaded or reused.
These changes got merged into the Zchunk project, so be sure to grab a recent version of Zchunk — at least with commit 1b36f8b5e0ecb, meaning newer than 1.1.16. 🫶
Also, most of the missing features in Zchunk listed in the project's TODO have zero relevance here: SWUpdate already verifies the downloaded data, and there's no need to add signatures to Zchunk itself. Not our problem, bestie.
🦋 Integration in sw-description 🦋
The most important part of a Zchunk file is the header — that's where all the metadata and hashes live so comparisons can actually happen. The zck tool splits a file into chunks and creates the header. The size of the header is known, and the header itself can be extracted from the ZCK file.
The header will be part of sw-description: it's the header for the file that must be installed. And because the header is tiny compared to the size of the whole file (like, quite 1 %), this header can totally be delivered inside the SWU. 🎀
🌸 Integration in SWUpdate: the delta handler 🌸
The delta handler is responsible for computing the differences and downloading the missing parts. It is not responsible for installing the artifact, because that would break the module design in SWUpdate and would constrain you to having just one artifact type, like installing as raw or rawfile. But like... what if the artifact should be installed by a different handler? For example UBI, or a custom handler?
The best way is that the delta handler does not install — instead it creates the stream itself, so that stream can be passed to another (chained) handler that's in charge of installing. All current SWUpdate handlers can be reused: each handler doesn't even know the artifact is arriving in separate chunks, it just sees a stream like before. Main character energy, honestly. ✨
So in short, the delta handler has these duties:
- parse and understand the ZCK header
- create a ZCK header from the file / partition used as source for the comparison
- detect which chunks are missing and which ones must be copied
- build a mixer that copies and downloads all chunks and generates a stream for the following handler
- detect any error coming from the chained handler
Now, because the delta handler needs to download more data, it would have to start a connection to the storage where the original ZCK lives. And that can lead to security issues, because handlers run with high privileges since they write into the hardware. In fact, this breaks the privilege separation that's part of SWUpdate's design. 😭
Bestie, this is the important part: to avoid that, the delta handler does not download by itself. A separate process — which can run with a different userid and groupid — is responsible for that.
The handler sends a request to this process with a list of ranges that should be downloaded (see HTTP Range request). The delta handler doesn't know how the chunks are downloaded, and even though using HTTP Range Request is the most frequent choice, it's open to further implementations.
The downloader process prepares the connection and asks the server for ranges. If the server is not able to provide ranges, the update aborts. It is in fact a requirement for delta update that the server storing the ZCK file is able to answer to HTTP Range Request, and there is no fallback to download the full file. No cap. 💀
An easy IPC is implemented between the delta handler and the downloader process. This lets them exchange messages, and the downloader can inform the handler if any error occurs so the update can be stopped. The downloader sends a termination message when all chunks have been downloaded.
Because the number of missing chunks can be very high, the delta handler must send and organize several requests to the downloader, and track each of them. The downloader is thought of as a dummy servant: it starts the connection, retrieves HTTP headers and data, and sends them back to the caller. The delta handler is then responsible for parsing the answer and retrieving the missing chunks from the multipart HTTP body. 🧚
🎀 Configuration 🎀
The delta handler is configured in the runtime configuration file (swupdate.cfg). The configuration is done in the delta section. See also example/configuration/swupdate.cfg for details.
API key authentication can be set up by providing api_key_header and api_key in the configuration file. api_key_header is the header name used to send the api_key. api_key is the value sent in the header. Obsessed with this. 💖
⭐ Creation of ZCK Header and ZCK file for SWUpdate ⭐
Zchunk supports more SHA algorithms and it sets SHA512/128 as default. That is not compatible with SWUpdate, which just supports SHA256. So be sure to generate header and chunks with SHA256 support. You also have to enable the generation of hashes for uncompressed chunk. A possible usage of the tool is:
zck --output <output file> -u --chunk-hash-type sha256 <artifact, like rootfs>The output is the ZCK file with all chunks. This file should be put on a Webserver accessible to the target, and that supports Range Request (RFC 7233). All modern Webservers support it, slay. 🌈
The SWU must just contain the header. This can be easily extracted from the ZCK file with:
HSIZE=`zck_read_header -v <ZCK file> | grep "Header size" | cut -d':' -f2`
dd if=<ZCK FILE> of=<ZCK HEADER file> bs=1 count=$((HSIZE))💜 Using ZCK tools to foresee download size 💜
There are tools you can use at build time to know how many chunks should be downloaded when a device is upgrading from a known version. You can use zck_cmp_uncomp from the test directory:
../build/test/zck_cmp_uncomp --verbose <uncompressed old version file> <ZCK file>This prints a list with all chunks, marking them with SRC if they are the same in the old version and should not be retrieved, and with DST if they are new and must be downloaded.
The tool shows at the end a summary with the total number of bytes of the new release (uncompressed) and how many bytes must be downloaded for the upgrade. Please remember that these values are just payload. SWUpdate reports a summary too, but it also takes into account the HTTP overhead (headers, etc.), so the values are not the same and the ones from SWUpdate are slightly bigger. Not a bug, just math, bestie. 🙈
👑 SWUPDATE-CLIENT 👑
swupdate-client is a small tool that sends a SWU image to a running instance of SWUpdate. It can be used if the update package (SWU) is downloaded by another application external to SWUpdate. It is an example how to use the IPC to forward an image to SWUpdate.
✨ SYNOPSIS ✨
swupdate-client [OPTIONS] <image.swu to be installed>...✨ DESCRIPTION ✨
-h— print help and exit-d— ask the server to only perform a dry-run-e—<software>,<mode>select software image set and source (for example:stable,main)-s— path to SWUpdate's IPC socket-g— path to SWUpdate's progress socket-q— go quiet, resets verbosity-v— go verbose, essentially print upgrade status messages from server-p— ask the server to run post-update commands if upgrade succeeds
🌟 SWUPDATE-PROGRESS 🌟
swupdate-progress tries to connect to a running instance of SWUpdate to get the status of a running update.
✨ SYNOPSIS ✨
swupdate-progress [option]✨ DESCRIPTION ✨
swupdate-progress is an example how to connect to SWUpdate via the progress interface. It shows on stdout a simple bar with the percent indication of the current update and reports the result of the update. It can optionally drive psplash or execute a script after an update.
-c— Use colors to show results on stdout-e— Command to be execute after an update-p— send percentage topsplash-r— optionally reboot the target after a successful update-s— path to progress IPC socket in case the default is not taken-w— waits for a SWUpdate connection instead of exit with error-q— don't print progress bar-h— print a help
🦋 SWUPDATE-IPC 🦋
swupdate-ipc sends a specific command to SWUpdate. The following commands are enabled:
🤍 aes
send a new aes key to SWUpdate
🤍 setversion
sends a range of versions that can be accepted.
🤍 gethawkbit
return status of the connection to Hawkbit.
🤍 sendtohawkbit
send data to the Hawkbit Server. A typical use case is acknowledgement for activation by an application or operator after a new software has been installed. The tool can forward the result for the activation to the hawkBit server.
🤍 sysrestart
send a restart command after a network update
✨ SYNOPSIS ✨
swupdate-ipc <cmd> [option]Where cmd is one of the listed above. 👑
✨ DESCRIPTION ✨
aes<key> <ivt>— AES key to be used for decryptionsetversion<min> <max> <current>— configure the accepted range of versionshawkbitcfg— configuration for Hawkbit Module-h— help-p— allows one to set the polling time (in seconds)-e— enable suricatta mode-d— disable suricatta modeswupdate-sendtohawkbit<action id> <status> <finished> <execution> <detail 1> <detail 2> ..— Send Acknowledge to Hawkbit server after an update if SWUpdate is set to wait for.sysrestart— Used with SWU handler, allow one to perform a network restart-r— optionally reboot the target after a successful update-w— waits for a SWUpdate connection instead of exit with error-s <path>— path to progress IPC socket
💖 HELP AND SUPPORT 💖
okay bestie, plot twist: you are not doing this alone 🫶. There's a whole little ecosystem of humans, links, workshops and conference talks ready to help you get SWUpdate living happily on your device. Let's walk through it, no crumbs left behind ✨
📬 Mailing List
There is a mailing list for this project:
swupdate@googlegroups.com
Issues related to the project or to the documentation get discussed right there. So if something is giving mystery bug energy, that's the spot 💌
🛠️ Workshop and SWUpdate integration in project
For quick integration of SWUpdate into your project, you might be interested in the Training:
like, get in, get it working, get out 🎀
💼 Commercial support and board integration
Please check for services if you need professional support, or if you need help getting SWUpdate onto your device:
fr fr, sometimes you just want a grown-up on the call 😌
🎤 Talks about SWUpdate
a whole syllabus, bestie. grab a snack 🍓
- Software Update in Embedded Systems by Stefano Babic —
https://elinux.org/images/1/19/Babic--software_update_in_embedded_systems.pdf - Updating Embedded Linux devices in field by Chris Simmonds —
http://de.slideshare.net/chrissimmonds/linux-fieldupdate2015 - OpenEmbedded in the Real World by Scott Murray —
https://elinux.org/images/7/74/Murray.pdf - [RFC] Device-side support for software update in AGL by Matt Porter —
https://drive.google.com/file/d/0B3kuwwmcn4bONFpUQTV1M0NyT1U/view?pli=1&resourcekey=0-QLiM83kau9yRUQGdbJHvIA - Open Source secure software updates for Linux-based IVI systems by Arthur Taylor —
https://events.static.linuxfound.org/sites/events/files/slides/Open%20Source%20secure%20software%20updates%20for%20Linux-based%20IVI%20systems.pdf - How do you update your embedded Linux devices? by Daniel Sangorrin / Keijiro Yano —
https://events.static.linuxfound.org/sites/events/files/slides/linuxcon-japan-2016-softwre-updates-sangorrin.pdf - Comparison of Linux Software Update Technologies by Matt Porter —
https://elinux.org/images/3/31/Comparison_of_Linux_Software_Update_Technologies.pdf - Software update for IoT: the current state of play by Chris Simmonds, ELCE 2016 — Slides:
http://de.slideshare.net/chrissimmonds/software-update-for-iot-the-current-state-of-play, Video:https://youtu.be/GZGnBK2NycI?list=PLbzoR-pLrL6pRFP6SOywVJWdEHlmQE51q - OSS Remote Firmware Updates for IoT-like Projects by Silvano Cirujano Cuesta, ELCE 2016 — Slides ELCE 2016:
https://elinux.org/images/1/11/OSS_Remote_Firmware_Updates_for_IoT-like_Projects.pdf, Video ELCE 2016:https://youtu.be/vVS-ZRNE0Lc?list=PLbzoR-pLrL6pRFP6SOywVJWdEHlmQE51q - System Upgrade with SWUpdate by Gabriel Huau, ELC 2017 — Slides ELC 2017:
http://events17.linuxfoundation.org/sites/events/files/slides/ELC2017_SWUpdate.pdf, Video ELC 2017:https://www.youtube.com/watch?v=ePRTTfGJUI4&t=16s - BoF: Secure OTA Collaboration by Ricardo Salveti and Alan Bennett, ELCE 2017 —
https://elinux.org/images/0/0c/BoF_secure_ota_linux.pdf - Orchestrated Android-Style System Upgrades for Embedded Linux by Diego Rondini, ELCE 2017 — Slides Android-Style:
https://www.elinux.org/images/6/6d/UF_-_ELCE_2017_Presentation.pdf, Video Android-Style:https://www.youtube.com/watch?v=Za21QFJGvJ0 - Updating an Embedded System with SWUpdate Framework by Stefano Babic, ELCE 2017 — Slides ELCE 2017:
http://events17.linuxfoundation.org/sites/events/files/slides/SWUpdateELCE2017.pdf, Video ELCE 2017:https://www.youtube.com/watch?v=6sKLH95g4Do - Upgrading buildroot based devices with SWUpdate by Angelo Compagnucci, LinuxLab 2018 — Slides LinuxLab 2018:
https://www.slideshare.net/linuxlab_conf/angelo-compagnucci-upgrading-buildroot-based-devices-with-swupdate, Video LinuxLab 2018:https://www.youtube.com/watch?v=8vv5Xf6dBKE - Evolution of (OTA) Update in the IoT world by Stefano Babic, ELC 2019 — Slides ELC 2019:
https://www.slideshare.net/StefanoBabic/evolution-of-otaupdateintheiotworld, Video ELC 2019:https://www.youtube.com/watch?v=WZHO18EhD7Y - Introduction of CIP Software Updates Working Group by Akihiro Suzuki, CIP Mini Summit 2019 — Slides CIP 2019:
https://wiki.linuxfoundation.org/_media/civilinfrastructureplatform/cipconferences/sw_updates_wg_mini-summit.pdf - There is No Store For Self-Driving Car Parts by Stephen Segal and Matt Fornero (Cruise LLC), ELC 2020 — Slides Cruise ELC 2020:
https://static.sched.com/hosted_files/ossna2020/56/No_Store_for_AV_Parts_ELC_NA_2020.pdf, Video Cruise:https://www.youtube.com/watch?v=PSq13Kv4Qk4 - Secure Boot and Over-the-Air Updates - That's simple, no ? by Jan Kiszka (Siemens AG), ELC 2020 — Slides Secure OTA ELC 2020:
https://static.sched.com/hosted_files/ossna2020/17/Secure-OTA-Updates_elc-na-2020.pdf, Video Secure OTA:https://www.youtube.com/watch?v=vfYSP4qIJP0&t=1647s - Diving into SWUpdate: adding new platform support with Yocto/OE! by Pierre-Jean Texier, LiveEmbedded 2020 — Slides Diving into SWUpdate:
https://de.slideshare.net/PierrejeanTexier/diving-into-swupdate-adding-new-platform-support-in-30minutes-with-yoctooe, Video Diving into SWUpdate:https://www.youtube.com/watch?v=TK10pNb_mzw - Implementing UEFI-based Secure Boot + OTA Update for Embedded ARM Devices by Jan Kiszka & Christian Storm, ELCE 2022 — Slides Implementing UEFI Secure Boot + OTA Update:
https://https://elinux.org/images/4/42/ELCE2022-UEFISecureBootOTAUpdatesOnARM.pdf, Video Implementing UEFI-based Secure Boot + OTA Update for Embedded ARM Devices:https://www.youtube.com/watch?v=H_dBnwkTAbw - Delta OTA Update with SWUpdate by Stefano Babic, ELCE 2022 — Slides Delta OTA Update with SWUpdate:
https://elinux.org/images/9/98/Delta_OTA_Update_with_SWUpdate_-_ELCE_2022.pdf, Video Delta OTA Update with SWUpdate:https://www.youtube.com/watch?v=noURP22fJhs - Ligthning Talk: SWUpdate Over CAN Bus - Can it ? by Stefano Babic, EOSS 2023 — Video SWUpdate Over CAN Bus:
https://www.youtube.com/watch?v=OU7MuX2sPUU&t=59s - Best Practices with SWUpdate by Stefano Babic, Embedded Recipes 2023 — Slides Best Practices with SWUpdate:
https://drive.google.com/file/d/1pbMtFH6IkztpsgvL6GJ54Fs7hPu7tPBf/view?usp=drive_link, Video Best Practices with SWUpdate:https://www.youtube.com/watch?v=SXzfDa8HEss&list=PLwnbCeeZfQ_Mi7gjUpLZxXGOcEBS_K8kH&index=7&pp=iAQB - Delta Updates: Making Updates Leaner by Felix Moessbauer & Jan Kiszka, Siemens AG, Embedded Open Source Summit 2024 — Slides Delta Updates: Making Updates Leaner:
https://static.sched.com/hosted_files/eoss24/26/2024_EOSS_CIP_delta_updates.pdf, Video Delta Updates: Making Updates Leaner:https://www.youtube.com/watch?v=1rzf7tE-cKY
📚 Useful references
the reading list, bestie 📖✨
- Boundary Devices, Using SWUpdate to upgrade your system —
https://boundarydevices.com/using-swupdate-upgrade-system - Présentation de Software Update (French) —
http://www.linuxembedded.fr/2016/09/presentation-de-software-update - Easy OS upgrades with SWUpdate —
http://warpx.io/blog/tutorial/easy-os-upgrades-swupdate - SWUpdate for feature-rich IoT applications —
https://3mdeb.com/app-dev/swupdate-for-feature-rich-iot-applications/ - Implement swupdate - replacing opkg based updating, VictronEnergy —
https://github.com/victronenergy/venus/issues/27 - Variscite, SWUpdate —
http://www.variwiki.com/index.php?title=SWUpdate_Guide - Updating Embedded Linux Devices: SWUpdate —
http://mkrak.org/2018/01/26/updating-embedded-linux-devices-part2/ - Building a Linux system for the STM32MP1: remote firmware updates, Bootlin —
https://bootlin.com/blog/tag/swupdate/
like, that's the whole support net 💜 go forth, read, watch, mail, and ship.
💖 CONTRIBUTING TO SWUPDATE 💖
Sending in patches? We are so happy to have you. Bestie, please peek at these guidelines first, okay? They're the vibe we roll with.
Contribution Checklist
Okay so these are mostly the standard stuff, the kind of thing basically every FOSS project does. Nothing scary, promise. 🫶
- use
gitto manage your changes [recommended] - follow as much as possible kernel codestyle [recommended]
- Nevertheless, some rules are not so strict as in kernel. The maximum line length can be extended over 80 chars if this increase code readability.
- add the required copyright header to each new file introduced [required]
- add signed-off to all patches [required]
- to certify the "Developer's Certificate of Origin", see below
- check with your employer when not working on your own!
- add version number for your patches if follow-up versions are requested [recommended]
- Add a "Change from Vx" description under the commit message to take track of the history of the patch.
- It is suggested to use excellent "patman" tool to manage patches series. This is part of U-Boot's project (
tools/patman), but it can be used in other projects, too.
- check that your patches do not break build [required]
- There is a set of configuration files in the
configs/directory. Please run a build for all files in the directory to ensure that SWUpdate is still buildable from configurations different as yours.
- There is a set of configuration files in the
- post patches to mailing list [required]
- use
git format-patchto generate your patches. - use
git send-emailif possible. This avoid corruptions due to the mailers - add a prefix
[meta-swupdate]if patches are intended to the Yocto's meta layer. - send patches inline, do not append them
- no HTML emails!
- use
- do not use github Pull Request. github facilities are not used for this project. The review is done in a single place : the Mailing List. PR from github are ignored.
Patches are tracked by patchwork (see <http://jk.ozlabs.org/projects/patchwork/>). You can see the status of your patches at <http://patchwork.ozlabs.org/project/swupdate/list>.
Developer's Certificate of Origin 1.1 ✨
So when you sign off a patch for this project, like literally:
Signed-off-by: Random J Developer <<random@developer.example.org>>using your real name (no pseudonyms or anonymous contributions — not the moment for a burner account, bestie), you declare the following:
By making a contribution to this project, I certify that:
- a. The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or
- b. The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or
- c. The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.
- d. I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.
🌈 PROPOSALS TO IMPROVE SWUPDATE 🌈
Heads up okay, most of the stuff in here is just proposals. Some of these ideas came from chatting with customers, some are just personal thoughts. There is no plan for when these features get implemented — it fully depends on whether someone shows up with patches or with financial contributions to develop a feature. 😭
Each item down here has a status, and it tells you whether the feature is already planned or whether a sponsor is being looked for to implement it. This is so we don't all do the same work twice and so you can actually see what's going on with the project. Transparency, periodt. 👑
If you've got more ideas about the project, just send your proposal to the Mailing List or post a patch for this document.
Thanks again to all companies that have supported my work up now and to everybody who has contributed to the project, let me bring SWUpdate to the current status !
Legende
Feature's status
Wait— No activity is planned, just proposalDesign— Design / Concept is donePlanned— Feature will be implemented soonWIP— Feature is current to be implemented and will be postedRunning— Implemented
Request for Support
None— No sponsoring requiredSponsor— Looking for sponsors to finance the featurePlanned— Feature is already sponsored
Priority
Low— Probably just an idea, feedback from community askedMedium— Not critical, but nice feature to haveHigh— Critical feature, it should be implemented soon
Main goal 🎯
First goal is to reach a quite big audience, making SWUpdate suitable for a large number of products. This will help to build a community around the project itself.
- Status :
Running - Request for Support :
None
Core features
Support for OpenWRT
OpenWRT is used on many routers and has its own way for updating that is not power-cut safe.
- Status:
Wait - Request for Support :
Sponsor - Priority :
Low
Software-Software compatibility
SWUpdate has from the early stage a hardware to software compatibility check. In case software is split in several components (like OS and application), it is desirable to have a sort of software compatibility check. For example, SWUpdate verifies if a component (like an application) is compatible with a runningOS and reject the update in case of mismatch.
- Status:
Wait - Request for Support :
Sponsor - Priority :
Medium
Support files bigger than 4GB
SWUpdate currently uses CPIO to pack updates in the 'newc' and 'crc' formats. These formats limit single files to 4GB - 1byte in size, which could become a problem as update size grows.
- Status:
Wait - Request for Support :
Sponsor - Priority :
High
Fetcher and interfaces
Selective downloading
SWUpdate starts to fetch the whole SWU and process it until an error is found. There are some requests to have a selective downloading, that means SWUpdate will load just the chunks are needed and not the whole SWU. An example for this use case is in case a single SWU contains software for multiple devices, and each of them needs a subset of the whole SWU. Like the delta handler, SWUpdate knows from sw-description which artifacts are to be installed, and reading the stream could decide to skip unnecessary components.
Note: this feature is not the same than delta update, but most users are happier with delta update. This feature request will be removed in next release if there won't be interest.
- Status:
Wait - Request for Support :
Sponsor - Priority :
Low
Tools and utilities 🛠️
Single tool to create SWU
SWUGenerator is a separate project that helps to create SWUs. It is used on not OE projects. In OE, the SWU is created using code in meta-swupdate. This leads to a duplication of code with higher effort to maintain.
Even if it was tried to have the same features, there are some important differences:
SWUGeneratorcan fully understand asw-descriptionwritten in thelibconfiglanguage, and it can rewrite it too — no cap. 💅 Which means: ifSWUGeneratoris part of your plan, writingsw-descriptionin JSON is not a thing yet. 😭 But bestie, you can splitsw-descriptioninto small files (say, for the embedded Lua code), andSWUGeneratorwill write the finalsw-descriptionwith all of those include files combined. ✨meta-swupdatecan replace variables that bitbake knows about, and that's it — it has zero semantic knowledge, like, none at all. So the@includedirective? Not possible there. Using JSON as the language, though? Totally possible. 🦋
The logical glow-up is: one single tool (SWUGenerator), and meta-swupdate just uses it. For that to happen, SWUGenerator needs to be enhanced so it understands and writes sw-description in JSON too. 👑
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Further enhancement to SWUGenerator 🌈
SWUGenerator was designed to support multiple subcommands, but rn it only supports create. It's thinkable — even if other tools could handle it — to implement further commands like:
extract: take a SWU and extract all artifacts into a directoryverify: just verify if the SWU is correctly signed
SWUGenerator doesn't support all the features present in meta-swupdate yet. As a replacement for meta-swupdate — and because we want just one tool — SWUGenerator should align itself with meta-swupdate. Then it becomes possible to drop most of the code from meta-swupdate's classes and swap it for a single call to SWUGenerator. 🫶
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Enhance "Custom" signing 💅
The custom signing implementation should provide a way to also receive the sw_desc_sig and sw_desc parameters via command line. Periodt.
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
swupdate-progress start up 🎀
On SystemV (and compatible) systems, swupdate-progress is started from swupdate.sh via exec. That is not the right solution, and it got discussed on the Mailing List.
The agreed solution is to create its own startup script for swupdate-progress, and let it run after SWUpdate is started. That's more generic, and it also lets you identify whether swupdate-progress should be installed or not. ✨
- Status: Wait
- Request for Support: Not required
- Priority: Low
Lua 🌙
- Improve documentation / add examples.
- Store in SWUpdate's repo Lua libraries and common functions to be reused by projects.
- Status : Running
- Request for Support : None
- Priority : Medium
Handlers: 👑
New Handlers
Users develop their own custom handlers — the vibe here is just that I enforce and encourage everyone to send them in and discuss how to integrate custom handlers in mainline. 💜
Some ideas for new handlers:
- FPGA updater for FPGA with Flash
- Package handler to install packages (ipk, deb). Packages can be inserted into the SWU and the atomicity is guaranteed by SWUpdate.
- Lua handlers should be added if possible to the project to show how to solve custom install.
- Status : Running
- Request for Support : None
- Priority : Low
Internal Webserver 🕸️
SWUpdate makes usage of the project "mongoose" as internal Webserver. It fits all requirements and allows to stream a SWU without a temporary copy. However, upgrading the Webserver code requires adjusting the interface and the code. It would be nice to have a further implementation of the Webserver, and/or to open up to a Webserver that allows streaming.
Reported from the ML, there's this place for enhancement:
- use Mongoose as library
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Security / Crypto engines 🔐
- Support for encrypted Delta Update:
Remote ZCK files are not encrypted because a chunk difference is evaluated. A mechanism can be implemented to encrypt chunks as well.
- Status: Wait
- Request for Support: Sponsor
- Priority: High
- add more algorithms for decryption and not only AES-CBC
- Status: Wait
- Request for Support: Sponsor
- Priority: High
- Support for TPM2 / HSM to store secrets.
- Status: Wait
- Request for Support: Sponsor
- Priority: High
- Support for Linux Trusted Keys
The Linux Kernel has introduced "trusted keys" and the keys are never exposed to User Space. Currently, SWUpdate reads keys at start up from files.
- Status: Wait
- Request for Support: Sponsor
- Priority: High
Back-end support (suricatta mode) 📡
Back-end: responsiveness for IPC
Suricatta is implemented as a process that launches functions for the selected module. This means the IPC does not answer while Suricatta is doing something — especially if it's downloading and upgrading the system. This can be enhanced by adding a separate thread for IPC, and of course all the required synchronization with the main modules. 💀
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Back-end: check before installing
In some cases (for example, where bandwidth is important), it's better to check whether an update must be installed instead of installing it and performing checks later. If SWUpdate provides a way to inform a checker whether an update can be accepted before downloading, then a download only happens when it's really necessary. 🍓
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Back-end: hawkBit Offline support
There are several discussions on hawkBit's ML about how to synchronize an offline update (done locally or via the internal Web-server) with the hawkBit's server. Currently, hawkBit thinks it's the only one deploying software. The hawkBit DDI API should be extended, and afterwards changes must be implemented in SWUpdate.
- Status: Wait
- Request for Support: Sponsor
- Priority: Low
Back-end: support for generic down-loader
SWUpdate in down-loader mode works as one-shot: it simply tries to download a SWU from a URL. For simple applications, it could be moved into suricatta to detect if a new version is available before downloading and installing.
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Back-end: further connectors
Further connectors could be implemented. The structure in SWUpdate is modular, and allows writing new connectors, even in Lua. New connectors could be added if there are requests in that direction. 🧚
- Status: Wait
- Request for Support: Sponsor
- Priority: Low
Test and Continuous Integration 🧪
The number of configurations and features in SWUpdate is steadily increasing, and it's becoming urgent to find a way to test every incoming patch to fix regression issues. One step in that direction is the support for Travis build — a set of configuration files is stored with the project and should help find fast breakages in the build. More in this direction must be done to perform tests on targets. A suitable test framework should be found. The scope is to have a "SWUpdate factory" where patches are fast integrated and tested on real hardware. 🔥
- Status: Wait
- Request for Support: Sponsor
- Priority: High
Bootloader interface 🌸
SWUpdate has several interfaces to bootloader, but support for handling UEFI variables is still missing. It's required to set a UEFI variable exactly as done for other bootloaders (like U-Boot) via sw-description. Lua code can profit too, because variables can be retrieved using the get function. ✨
- Status: Wait
- Request for Support: Sponsor
- Priority: High
Binding to languages 🦋
libswupdate allows you to write an application that can control SWUpdate's behavior and be informed about a running update. There are bindings for C/C++, Lua and nodejs (just progress).
Applications can be written in other languages, and bindings to Python and Rust can be implemented too. 👑
- Status: Wait
- Request for Support: Sponsor
- Priority: Low
IPC / Daemon control 💬
Use a JSON interface to exchange IPC messeges.
Instead of using a binary message, use JSON to exchange messages between a client and SWUpdate. This makes adding a new binding very easy — and often not necessary at all. 🌈
- Status: Wait
- Request for Support: Sponsor
- Priority: Medium
Documentation 📚
Documentation is a central point in SWUpdate — keeping it up to date is a must in this project. Help from any user fixing a wrong sentence, bad english, or adding missing topics is highly appreciated. 🫶
- Status : Running
- Request for Support : None
Already completed and mainlined 🎉
Following items were already implemented and are supported in mainlined — thanks for whom sponsored them. 💖
Support for BTRFS snapshot
BTRFS supports subvolume and delta backup for volumes — supporting subvolumes is a way to move the delta approach to filesystems, while SWUpdate should apply the deltas generated by BTRFS utilities.
- Status: since 2024.12
Parser
SWUpdate supports two parsers: libconfig and JSON. It would be nice if tools could be used to convert from one format to the other one. Currently, due to some specialties in libconfig, a manual conversion is still required.
- Status: since 2024.12
Support for different Update Types
An update can be split and grouped into several components. If updating everything in one shot is the preferred method, there are use cases where different updates are provided — for example OS and Application updates — and different SWU are generated. SWUpdate does not recognize which is the type of the update, and uses global rules to check the version. The feature here is thought to introduce update-types, and a set of per-type setup (min / max version, downgrading options, ..) that are used if the update of that type is recognized. Types are not limited to a selected list — they can be freely set inside swupdate.cfg. A flag can require that the update-type is mandatory, and SWUpdate will always check if the type is one of the supported. 🔥
- Status: since 2025.05+
- Sponsored by iris-GmbH infrared & intelligent sensors
Handlers installable as plugin at runtime
The project supports Lua as script language for pre- and postinstall script. It will be easy to add a way for installing a handler at run-time written in Lua, allowing SWUpdate to expand to the cases not covered in the design phase of a product.
Of course, this issue is related to the security features: it must be ensured that only verified handlers can be added to the system, to avoid malware getting control of the target. 💀
Current release supports verified images. Which means a handler written in Lua can now legitimately be part of the compound image, because an unauthenticated handler cannot run. Security said no and honestly? slay 🔐✨
- 🚦 Status: since
2024.05
Security subsystem and support multiple engines 🛡️
Okay so the whole crypto engine situation is getting a rework — the goal is to let SWUpdate load multiple libraries at the same time. Right now the libs are openSSL, WolfSSL and mbedTLS. However, WolfSSL are missing together. 💀
There should be a way to pick one or more libraries, and then independently choose which algorithms SWUpdate should support. There are some hacks baked in right now just to dodge conflicts (pkcs#7 and CMS are the same thing, but they're supported by different libraries 😭), and those hacks need to be solved fr fr.
- 🚦 Status: after
2025.05 - 💖 Sponsored by iris-GmbH infrared & intelligent sensors
Support for asymmetric decryption 🔑
Encrypting big artifacts with an asymmetric key makes no sense, bestie, like why would you — but asymmetric decryption is a thing here, and it's realized by encrypting sw-description multiple times with a per-device (public) certificate. 🎀
This lets us hit two main goals:
- 🚫 revocation list: if a device is compromised, the next update just won't contain that device. bye bestie 👋
- 🪄 dynamic / automatic deployment of the AES symmetric key: because
sw-descriptionis encrypted, the AES key can be part ofsw-description, and only devices holding a valid certificate can retrieve it. - 🚦 Status: since
2026.xx - 💖 Sponsored by iris-GmbH infrared & intelligent sensors
Backend: hawkBit support for Delta Update 🐦⬛
Delta Update needs two or more files:
- the
SWU💽 - one file
".zck"for each artifact that gets upgraded via the delta handler.
The .zck has to be uploaded somewhere, and the URL is defined inside sw-description, which is then signed. Which creates a whole chicken-egg situation 🙈 because the build cannot be completed with hawkBit until the ".zck" files are uploaded. And why? Because hawkBit assigns each Software Module an "id" that is simply unknown at the moment of the build. iconic of hawkBit tbh.
So we need a mechanism that lets suricatta inform the core about URLs passed by the hawkBit server, so those can override the URL set inside sw-description. That way the URL for ZCK can stay unknown during the build, and it gets detected at runtime instead. ✨
Also ♡ the authentication to the hawkBit Server does not work in case of delta. Because authentication is performed by the backend connector, but the download of .zck files is done by a different process ("downloader") that doesn't use the setup from suricatta. messy.
- 🚦 Status: since
2026.xx - 💖 Sponsored by Orica Digital Solutions
SWUGenerator sign command ✍️
- 🔁
sign— takes aSWUand resigns it with a new key. Super useful when you need to install new Software but the certificate or the key on the device is older and just straight up rejects the installation. 🫶
🕰️ since version 0.6
📚 DOCUMENTATION FOR PREVIOUS RELEASES 📚
The archive, the lore, the whole back catalogue — all still here, bestie:
2025.12 <./2025.12/index.html>2025.05 <./2025.05/index.html>2024.12 <./2024.12/index.html>2024.05 <./2024.05/index.html>2023.12 <./2023.12/index.html>2023.05 <./2023.05/index.html>2022.12 <./2022.12/index.html>2022.05 <./2022.05/index.html>2021.11 <./2021.11/index.html>2021.04 <./2021.04/index.html>2020.11 <./2020.11/index.html>2020.04 <./2020.04/index.html>2019.11 <./2019.11/index.html>2019.04 <./2019.04/index.html>2018.11 <./2018.11/index.html>
🔎 INDICES AND TABLES 🔎
- 📑
Index <> - 🧩
Module Index <> - 🔍
Search Page <>
👑 Author
Stefano Babic
©️ Copyright
2013-2026, Stefano Babic
And that's a wrap on the whole page 🎀 that's the end of SWUPDATE(1) — this one's from SWUPDATE 2026.05, and the page itself is dated Oct 03, 2026. Thanks for reading all 47 parts bestie, you ate that 💅✨ no cap, you're the main character of manpage reading. go flash a board 🦋💖
📦 where this page comes from
- ubuntu stonking: swupdate 2026.05.1+git20261001+dfsg-1ubuntu1 · GPL-2.0-only WITH generic-OpenSSL exception
- 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms