CRIU(8)

📦 version 4.1, in opensuse-leap 16.0

💖 CRIU(8) — CRIU Manual 💖

✨ NAME ✨

criu — checkpoint/restore in userspace 💅

🌈 SYNOPSIS 🌈

criu command [option ...]

🦋 DESCRIPTION 🦋

Okay so criu is literally the tool for checkpointing and restoring running applications. It does this by saving their state as a collection of files (peep the dump command) and creating equivalent processes from those files (peep the restore command). The restore operation can be performed at a later time, on a different system, or both. It’s giving time travel but for processes. 🫶

🌸 OPTIONS 🌸

Most of the long flags can be prefixed with no- to negate the option (example: --display-stats and --no-display-stats). iykyk 😌

💖 Common options 💖

Common options are applicable to any command. periodt.

  • -v[v...], --verbosity — increase verbosity up from the default level. With the short option, multiple vs can be used, and each one increases verbosity by one. stack ’em ✨
  • -vnum, --verbosity=num — set verbosity level to num. The higher the level, the more output is produced.

The following levels are available:

  • -v0 — no output;
  • -v1 — only errors;
  • -v2 — the above plus warnings (this is the default level);
  • -v3 — the above plus information messages and timestamps;
  • -v4 — the above plus lots of debug.
  • --config file — pass a specific configuration file to criu.
  • --no-default-config — disable parsing of default configuration files.
  • --pidfile file — write root task, service or page-server pid into a file.
  • -o, --log-file file — write logging messages to a file.
  • --display-stats — during dump, as well as during restore, criu collects some statistics, like the time required to dump or restore the process, or the number of pages dumped or restored. This information is always saved to the stats-dump and stats-restore files, and can be shown using crit(1). The option --display-stats prints out this information on the console at the end of a dump or restore operation. numbers girlies rise 📊
  • -D, --images-dir path — use path as a base directory where to look for sets of image files.
  • --stream — dump/restore images using criu-image-streamer. See https://github.com/checkpoint-restore/criu-image-streamer for detailed usage.
  • --prev-images-dir path — use path as a parent directory where to look for sets of image files. This option makes sense in case of incremental dumps.
  • -W, --work-dir dir — use directory dir for putting logs, pidfiles and statistics. If not specified, path from -D option is taken.
  • --close fd — close file descriptor fd before performing any actions.
  • -L, --libdir path — path to plugins directory.
  • --enable-fs [fs[,fs...]] — specify a comma-separated list of filesystem names that should be auto-detected. The value all enables auto-detection for all filesystems.
Bestie, be careful: this option is not safe, use at your own risk. Auto-detecting a filesystem mount assumes that the mountpoint can be restored with mount(src, mountpoint, flags, options). When used, dump is expected to always succeed if a mountpoint is to be auto-detected, however restore may fail (or do something wrong) if the assumption for restore logic is incorrect. This option is not compatible with --external dev.
  • --action-script script — add an external action script to be executed at certain stages. The environment variable CRTOOLS_SCRIPT_ACTION is available to the script to find out which action is being executed, and its value can be one of the following:
    • pre-dump — run prior to beginning a dump
    • post-dump — run upon dump completion
    • pre-restore — run prior to beginning a restore
    • post-restore — run upon restore completion
    • pre-resume — run when all processes and resources are restored but tasks are stopped waiting for final kick to run. Must not fail.
    • post-resume — called at the very end, when everything is restored and processes were resumed
    • network-lock — run to lock network in a target network namespace
    • network-unlock — run to unlock network in a target network namespace
    • setup-namespaces — run once root task has just been created with required namespaces. Note it is an early stage of restore, when nothing is restored yet, except for namespaces themselves
    • post-setup-namespaces — called after the namespaces are configured
    • orphan-pts-master — called after master pty is opened and unlocked. This hook can be used only in the RPC mode, and the notification message contains a file descriptor for the master pty
    • query-ext-files — called after the process tree is stopped and network is locked. This hook is used only in the RPC mode. The notification reply contains file ids to be added to external file list (may be empty).
  • --unprivileged — this option tells criu to accept the limitations when running as non-root. Running as non-root requires criu at least to have CAP_SYS_ADMIN or CAP_CHECKPOINT_RESTORE. For details about running criu as non-root please consult the NON-ROOT section. respect the caps 👑
  • -V, --version — print program version and exit.
  • -h, --help — print some help and exit.

🦋 pre-dump 🦋

Performs the pre-dump procedure, during which criu creates a snapshot of memory changes since the previous pre-dump. Note that during this criu also creates the fsnotify cache which speeds up the restore procedure. pre-dump requires at least -t option (see dump below). In addition, page-server options may be specified. pre-game ritual ✨

  • --track-mem — turn on memory changes tracker in the kernel. If the option is not passed the memory tracker get turned on implicitly.
  • --pre-dump-mode=mode — there are two mode to operate pre-dump algorithm. The splice mode is parasite based, whereas the read mode is based on process_vm_readv syscall. The read mode incurs reduced frozen time and reduced memory pressure as compared to splice mode. Default is splice mode. choose your fighter ⚔️

🎀 dump 🎀

Performs a checkpoint procedure. main character operation 💅

  • -t, --tree pid — checkpoint the whole process tree starting from pid.
  • -R, --leave-running — leave tasks in running state after checkpoint, instead of killing. This option is pretty dangerous and should be used only if you understand what you are doing.
Okay but seriously: note if task is about to run after been checkpointed, it can modify TCP connections, delete files and do other dangerous actions. Therefore, criu can not guarantee that the next restore action will succeed. Most likely if this option is used, at least the file system snapshot must be made with the help of post-dump action script.

In other words, do not use it unless really needed. fr fr 💀

  • -s, --leave-stopped — leave tasks in stopped state after checkpoint, instead of killing.
  • --external type[id]:value — dump an instance of an external resource. The generic syntax is type of resource, followed by resource id (enclosed in literal square brackets), and optional value (prepended by a literal colon). The following resource types are currently supported: mnt, dev, file, tty, unix. Syntax depends on type. Note to restore external resources, either --external or --inherit-fd is used, depending on resource type. external era 🌍
  • --external mnt[mountpoint]:name — dump an external bind mount referenced by mountpoint, saving it to image under the identifier name.
  • --external mnt[]:flags — dump all external bind mounts, autodetecting those. Optional flags can contain m to also dump external master mounts, s to also dump external shared mounts (default behavior is to abort dumping if such mounts are found). If flags are not provided, colon is optional.
  • --external dev[major/minor]:name — allow to dump a mount namespace having a real block device mounted. A block device is identified by its major and minor numbers, and criu saves its information to image under the identifier name.
  • --external file[mnt_id:inode] — dump an external file, i.e. an opened file that is can not be resolved from the current mount namespace, which can not be dumped without using this option. The file is identified by mnt_id (a field obtained from /proc/pid/fdinfo/N) and inode (as returned by stat(2)). mystery file unlocked 🔍
  • --external tty[rdev:dev] — dump an external TTY, identified by st_rdev and st_dev fields returned by stat(2).
  • --external unix[id] — tell criu that one end of a pair of UNIX sockets (created by socketpair(2)) with the given id is OK to be disconnected.
  • --external net[inode]:name — mark a network namespace as external and do not include it in the checkpoint. The label name can be used with --inherit-fd during restore to specify a file descriptor to a preconfigured network namespace.
  • --external pid[inode]:name — mark a PID namespace as external. This can be later used to restore a process into an existing PID namespace. The label name can be used to assign another PID namespace during restore with the help of --inherit-fd.
  • --freeze-cgroup — use cgroup freezer to collect processes.
  • --manage-cgroups — collect cgroups into the image thus they gonna be restored then. Without this option, criu will not save cgroups configuration associated with a task. cgroups said see you later 🎀
  • --cgroup-props spec — specify controllers and their properties to be saved into the image file. criu predefines specifications for common controllers, but since the kernel can add new controllers and modify their properties, there should be a way to specify ones matched the kernel.

The spec argument describes the controller and properties specification in a simplified YAML form:

"c1":
 - "strategy": "merge"
 - "properties": ["a", "b"]
"c2":
 - "strategy": "replace"
 - "properties": ["c", "d"]

where c1 and c2 are controllers names, and a, b, c, d are their properties.

Note the format: double quotes, spaces and new lines are required. The strategy specifies what to do if a controller specified already exists as a built-in one: criu can either merge or replace such.

For example, the command line for the above example should look like this:

--cgroup-props "\"c1\":\n - \"strategy\": \"merge\"\n - \"properties\": [\"a\", \"b\"]\n \"c2\":\n - \"strategy\": \"replace\"\n - \"properties\": [\"c\", \"d\"]"
  • --cgroup-props-file file — literally the same as --cgroup-props, except the whole spec gets read from file instead. 🎀
  • --cgroup-dump-controller name — dumps a controller with the name name only, skipping everything else that was discovered automatically (usually via /proc). This is so useful when you need criu to skip some controllers. 💅
  • --cgroup-yard path — instead of trying to mount cgroups inside CRIU, you hand it a path to a directory that already has a cgroup yard in it. Iconic if you don't want to grant CAP_SYS_ADMIN to CRIU. For every cgroup mount there should be exactly one directory. If there's only one controller in that mount, the dir's name should be just the name of the controller. If there are multiple controllers comounted, the directory name should have them separated by a comma.

For example, if /proc/cgroups looks like this:

#subsys_name hierarchy num_cgroups enabled
cpu          1         1           1
devices      2         2           1
freezer      2         2           1

then you can create the cgroup yard by the following commands:

mkdir private_yard
cd private_yard
mkdir cpu
mount -t cgroup -o cpu none cpu
mkdir devices,freezer
mount -t cgroup -o devices,freezer none devices,freezer
  • --tcp-established — checkpoint established TCP connections. yes bestie, the ones already fully talking. 🔥
  • --tcp-close — don't dump the state of, or block, established tcp connections (and that includes a connection that was once established but is now closed). Useful when tcp connections aren't going to be restored anyway.
  • --skip-in-flight — skips in-flight TCP connections. If any TCP connections that aren't yet completely established are found, criu ignores these connections instead of erroring out. The TCP stack on the client side is expected to handle the re-connect gracefully. 🦋
  • --evasive-devices — use any path to a device file if the original one is inaccessible.
  • --page-server — send pages to a page server (peep the page-server command).
  • --force-irmap — force resolving names for inotify and fsnotify watches.
  • --auto-dedup — deduplicate "old" data in the pages images of a previous dump. This option implies incremental dump mode (see the pre-dump command).
  • -l, --file-locks — dump file locks. You have to make sure that all file lock users get taken into the dump, so it's only safe to use this for enclosed containers where locks aren't held by any processes outside the dumped process tree. ⚠️
  • --link-remap — allows linking unlinked files back, if possible (it modifies the filesystem during restore, fyi).
  • --timeout number — set a time limit in seconds for collecting tasks during the dump operation. The timeout is 10 seconds by default. ⏳
  • --ghost-limit size — set the maximum size of a deleted file that gets carried inside the image. By default up to 1M file is allowed. Using this option lets you keep big deleted files out of your images. The size argument may be postfixed with K, M or G, which stand for kilo-, mega-, and gigabytes, accordingly.
  • --ghost-fiemap — enable an optimization based on the fiemap ioctl that can reduce the number of system calls used when checkpointing highly sparse ghost files. This option is enabled by default, and it can be disabled with --no-ghost-fiemap. There's an automatic fallback to SEEK_HOLE/SEEK_DATA when fiemap isn't supported. ✨
  • -j, --shell-job — allows one to dump shell jobs. This implies the restored task will inherit session and process group ID from criu itself. It also allows migrating a single external tty connection, so you can migrate applications like top. If used with the dump command, it must be specified with restore as well.
  • --cpu-cap [cap[,cap...]] — specify CPU capabilities to write to an image file. The argument is a comma-separated list of:
    • none — ignore capabilities at all; the image won't be produced on dump, and neither is any check performed on restore;
    • fpu — check if the FPU module is compatible;
    • ins — check if the CPU supports all instructions required;
    • cpu — check if CPU capabilities are exactly matching;
    • all — for all of the above set.

By default the option is set to fpu and ins.

  • --cgroup-root [controller:]/newroot — change the root for the controller that will be dumped. By default criu simply dumps everything below where any of the tasks live. But if a container moves all of its tasks into a cgroup directory below the container engine's default directory for tasks, permissions won't be preserved on the upper directories that have no tasks in them, which may cause problems. 😭
  • --lazy-pages — do the dump procedure without writing memory pages into the image files, and get ready to service page requests over the network. When dump runs in this mode it presumes the lazy-pages daemon will connect to it and fetch memory pages to lazily inject them into the restored process address space. This option is intended for post-copy (lazy) migration and should be used in conjunction with restore with appropriate options. 🧚
  • --file-validation [mode] — set the method to be used to validate open files. Validation is done to ensure the version of the file being restored is the same version as when it was dumped. 👑

The mode may be one of the following:

  • filesize — explicitly use only the file size check all the time. This is the fastest and least intensive check.
  • buildid — validate ELF files with their build-ID. If the build-ID can't be obtained, the chksm-first method will be used. This is the default if mode is unspecified.
  • --network-lock [mode] — set the method to be used for network locking/unlocking. Locking is done to ensure that tcp packets are dropped between dump and restore. This avoids the kernel sending RST when a packet arrives destined for the dumped process. 🫶

The mode may be one of the following:

  • iptables — use iptables rules to drop the packets. This is the default if mode is not specified.
  • nftables — use nftables rules to drop the packets.
  • skip — don't lock the network. If --tcp-close is not used, the network must be locked externally to allow CRIU to dump TCP connections.

🌸 restore 🌸

Restores previously checkpointed processes. main character energy, honestly. 💖

  • --inherit-fd fd[N]:resource — inherit a file descriptor. This option lets criu use an already opened file descriptor N for restoring a file identified by resource. You can use it to restore an external resource dumped with the help of --external file, tty, pid and unix options.

The resource argument can be one of the following:

  • tty[rdev:dev]
  • pipe[inode]
  • socket[inode*]*
  • file[mnt_id:inode]
  • path/to/file

Note that the square brackets used in this option's arguments are literals and usually need to be escaped from the shell. 🙈

  • -d, --restore-detached — detach criu itself once restore is complete.
  • -s, --leave-stopped — leave tasks in stopped state after restore (rather than resuming their execution).
  • -S, --restore-sibling — restore root task as a sibling (makes sense only with --restore-detached).
  • --log-pid — write separate logging files per each pid.
  • -r, --root path — change the root filesystem to path (when run in a mount namespace). This option is required to restore a mount namespace. The directory path must be a mount point, and its parent must not be overmounted. ⭐
  • --external type[id]:value — restore an instance of an external resource. The generic syntax is: type of resource, followed by resource id (enclosed in literal square brackets), and an optional value (prepended by a literal colon). The following resource types are currently supported: mnt, dev, veth, macvlan. Syntax depends on type. Note: to restore external resources dealing with opened file descriptors (such as dumped with the help of --external file, tty, and unix options), the --inherit-fd option should be used. 🎀
  • --external mnt[name]:mountpoint — restore an external bind mount referenced in the image by name, bind-mounting it from the host mountpoint to a proper mount point.
  • --external mnt[] — restore all external bind mounts (dumped with the help of --external mnt[] auto-detection).
  • --external dev[name]:/dev/path — restore an external mount device, identified in the image by name, using the existing block device /dev/path.
  • --external veth[inner_dev]:outer_dev@bridge — set the outer VETH device name (corresponding to inner_dev being restored) to outer_dev. If the optional @bridge is specified, outer_dev is added to that bridge. If the option isn't used, outer_dev will be autogenerated by the kernel. 🌈
  • --external macvlan[inner_dev]:outer_dev — when restoring an image that has a MacVLAN device in it, you must use this option to specify which outer_dev (an existing network device in the CRIU namespace) the restored inner_dev should be bound to.
  • -J, --join-ns NS:{PID|NS_FILE}[,EXTRA_OPTS] — restore the process tree inside an existing namespace. The namespace can be specified in PID or NS_FILE path format (example: --join-ns net:12345 or --join-ns net:/foo/bar). Currently supported values for NS are: ipc, net, time, user, and uts. This option doesn't support joining a PID namespace, but that's possible using --external and --inheritfd. EXTRA_OPTS is optional and can be used to specify UID and GID for the user namespace (e.g., --join-ns user:PID,UID,GID). 🔥
  • --manage-cgroups [mode] — restore the cgroups configuration associated with a task from the image. Controllers are always restored in an optimistic way — if already present in the system, criu reuses it, otherwise it will be created. ✨

The mode may be one of the following:

  • none — don't restore cgroup properties, but require the cgroup to pre-exist at the moment of the restore procedure.
  • props — restore cgroup properties and require the cgroup to pre-exist.
  • soft — restore cgroup properties only if the cgroup was created by criu; otherwise don't restore properties. This is the default if mode is unspecified.
  • full — always restore all cgroups and their properties.
  • strict — restore all cgroups and their properties from scratch, requiring them to not be present in the system. Like, full reset energy 💅
    • ignore — don’t deal with cgroups at all and just pretend they don’t exist. Ghosting them, basically 🫥
  • --cgroup-yard path — instead of trying to mount cgroups in CRIU, give a path to a directory with an already-created cgroup yard. For more info, look in the dump section. It’s giving “bring your own cgroup setup” ✨
  • --cgroup-root [controller:]/newroot — change the root cgroup the controller will be installed into. No controller means that root is the default for all controllers not specified. Slay the hierarchy 🎀
  • --tcp-established — restore previously dumped established TCP connections. This implies that the network has been locked between dump and restore phases, so the other side of a connection simply notices a kind of lag. Not a glow-up, just a tiny pause 😭
  • --tcp-close — restore connected TCP sockets in closed state. Bye-bye connection, but make it official 💀
  • --veth-pair IN=OUT — correspondence between the outside and inside names of veth devices. Name matching, but make it network 🦋
  • -l, --file-locks — restore file locks from the image. Locks, restored, no cap 🔒
  • --lsm-profile type:name — specify an LSM profile to be used during restore. The type can be either apparmor or selinux. Pick your security flavor 💅
  • --lsm-mount-context context — specify a new mount context to be used during restore.

This option will only replace existing mount context information with the one specified with this option. Mounts without the context= option will not be changed.

If a mountpoint has been checkpointed with an option like

context="system_u:object_r:container_file_t:s0:c82,c137"

it is possible to change this option using

--lsm-mount-context "system_u:object_r:container_file_t:s0:c204,c495"

which will result in the mountpoint being restored with the new context=.

This option is useful if using selinux and the selinux labels need to be changed on restore, like if a container is restored into an existing Pod. Context switch, but make it secure 🌈

  • --auto-dedup — as soon as a page is restored it gets punched out from the image. One-and-done page behavior, lowkey iconic 🫶
  • -j, --shell-job — restore shell jobs; in other words, inherit session and process group ID from the criu itself. Family tree energy, but for processes 🌸
  • --cpu-cap [cap[,cap...]] — specify CPU capabilities to be present on the CPU the process is restoring. To inverse a capability, prefix it with ^. This option implies that --cpu-cap has been passed on dump as well, except in the fpu option case. The cap argument can be the following (or a set of comma-separated values):
    • all — require all capabilities. This is the default mode if --cpu-cap is passed without arguments. Most safe mode. All the things, all the safety 👑
    • cpu — require the CPU to have all capabilities in the image to match the runtime CPU. Bestie, your CPU better match the receipt 🧾
    • fpu — require the CPU to have a compatible FPU. For example, the process might be dumped with the xsave capability but attempted to restore without it present on the target CPU. In such case we refuse to proceed. This is the default mode if --cpu-cap is not present in the command line. Note this argument might be passed even if on the dump no --cpu-cap was specified, because FPU frames are always encoded into images. Math coprocessor vibes, but make it mandatory 🧮
    • ins — require CPU compatibility on instructions level. Instruction-level compatibility check, very demure, very mindful 🔍
    • none — ignore capabilities. Most dangerous mode. The behaviour is implementation dependent. Try to not use it until really required.

For example, this option can be used in case --cpu-cap=cpu was used during dump, and images are migrated to a less capable CPU and are to be restored. By default, criu shows an error that CPU capabilities are not adequate, but this can be suppressed by using --cpu-cap=none. Danger zone, bestie — proceed with caution 💀

  • --weak-sysctls — silently skip restoring sysctls that are not available. This allows restoring on an older kernel, or a kernel configured without some options. Not every sysctl is invited to the party, and that’s okay 🎀
  • --lazy-pages — restore the processes without filling out the entire memory contents. When this option is used, restore sets up the infrastructure required to fill memory pages either on demand when the process accesses them or in the background without stopping the restored process. This option requires a running lazy-pages daemon. Main character energy: pages show up exactly when they’re needed 💅
  • --file-validation [mode] — set the method to be used to validate open files. Validation is done to ensure that the version of the file being restored is the same version when it was dumped.

The mode may be one of the following:

  • filesize — to explicitly use only the file size check all the time. This is the fastest and least intensive check. Quick vibe check, size only 🍓
  • buildid — to validate ELF files with their build-ID. If the build-ID cannot be obtained, chksm-first method will be used. This is the default if mode is unspecified. The classic build-ID check, with a checksum fallback 🦋
  • --skip-file-rwx-check — skip checking file permissions (r/w/x for u/g/o) on restore. Permission checks? Not today, bestie 🙈

🕵️ check 🕵️

Checks whether the kernel supports the features needed by criu to dump and restore a process tree.

There are three categories of kernel support, as described below. criu check always checks Category 1 features unless --feature is specified, which only checks a specified feature.

🌸 Category 1 🌸

Absolutely required. These are features like support for /proc/PID/map_files, NETLINK_SOCK_DIAG socket monitoring, /proc/sys/kernel/ns_last_pid etc. Non-negotiable, must-have, no skips 💅

🌸 Category 2 🌸

Required only for specific cases. These are features like AIO remap, /dev/net/tun and others that are only required if a process being dumped or restored is using those. Situationship features — only relevant if you’re actually using them 🎀

🌸 Category 3 🌸

Experimental. These are features like task-diag that are used for experimental purposes (mostly during development). Lab-coat energy 🧪

If there are no errors or warnings, criu prints "Looks good." and its exit code is 0. Slay ✅

A missing Category 1 feature causes criu to print "Does not look good." and its exit code is non-zero. Not slay ❌

Missing Category 2 and 3 features cause criu to print "Looks good but ..." and its exit code is non-zero. Almost slay, but read the fine print 😭

Without any options, criu check checks Category 1 features. This behavior can be changed by using the following options:

  • --extra — check kernel support for Category 2 features.
  • --experimental — check kernel support for Category 3 features.
  • --all — check kernel support for Category 1, 2, and 3 features.
  • --feature name — check a specific feature. If name is list, a list of valid kernel feature names that can be checked will be printed. Very “show me the menu” of it all 📋

📡 page-server 📡

Launches criu in page server mode. Serving pages, serving looks 💅

  • --daemon — runs page server as a daemon (background process). Background bestie behavior 🌙
  • --status-fd — write \0 to the FD and close it once page-server is ready to handle requests. The status-fd allows to not daemonize a process and get its exit code at the end. It isn’t supposed to use --daemon and --status-fd together. Pick a lane, bestie 🚦
  • --address address — page server IP address or hostname. Where the page server lives 🏡
  • --port number — page server port number. The door number 🔢
  • --ps-socket fd — use provided file descriptor as socket for incoming connection. In this case --address and --port are ignored. Useful for intercepting page-server traffic e.g. to add encryption or authentication. Bring your own socket, we love a flexible setup 🔌
  • --lazy-pages — serve local memory dump to a remote lazy-pages daemon. In this mode the page-server reads local memory dump and allows the remote lazy-pages daemon to request memory pages in random order. Random access, but make it memory 🌀
  • --tls-cacert file — specifies the path to a trusted Certificate Authority (CA) certificate file to be used for verification of a client or server certificate. The file must be in PEM format. When this option is used only the specified CA is used for verification. Otherwise, the system’s trusted CAs and, if present, /etc/pki/CA/cacert.pem will be used. Trust, but verify — with the right CA 🛡️
  • --tls-cacrl file — specifies a path to a Certificate Revocation List (CRL) file which contains a list of revoked certificates that should no longer be trusted. The file must be in PEM format. When this option is not specified, the file, if present, /etc/pki/CA/cacrl.pem will be used. The naughty list for certs 📜
  • --tls-cert file — specifies a path to a file that contains a X.509 certificate to present to the remote entity. The file must be in PEM format. When this option is not specified, the default location (/etc/pki/criu/cert.pem) will be used. Show them your cert, bestie 🎀
  • --tls-key file — specifies a path to a file that contains the TLS private key. The file must be in PEM format. When this option is not specified, the default location (/etc/pki/criu/private/key.pem) will be used. Keep the key private, obviously 🔐
  • --tls — use TLS to secure remote connections. Encrypt the group chat 🔒

🧚 lazy-pages 🧚

Launches criu in lazy-pages daemon mode.

The lazy-pages daemon is responsible for managing user-level demand paging for the restored processes. It gets information required to fill the process memory pages from the restore and from the checkpoint directory. When a restored process accesses a certain memory page for the first time, the lazy-pages daemon injects its contents into the process address space. The memory pages that are not yet requested by the restored processes are injected in the background. Fairy godmother, but for memory pages 🧚✨

💀 exec 💀

Executes a system call inside a destination task’s context. This functionality is deprecated; please use Compel instead. We don’t do this anymore, bestie — it’s giving legacy 💀

🔌 service 🔌

Launches criu in RPC daemon mode, where criu is listening for RPC commands over a socket to perform. This is convenient for a case where the daemon itself is running in a privileged (superuser) mode but clients are not. Fancy daemon with non-privileged friends 🫶

🗑️ dedup 🗑️

Starts pagemap data deduplication procedure, where criu scans over all pagemap files and tries to minimize the number of pagemap entries by obtaining the references from a parent pagemap image. Duplicate entries? Not on our watch 🧹

🍓 cpuinfo dump 🍓

Fetches current CPU features and writes them into an image file. CPU tea, spilled into a file ☕

🍓 cpuinfo check 🍓

Fetches current CPU features (i.e. CPU the criu is running on) and tests if they are compatible with the ones present in an image file. Compatibility check, but make it CPU 🧠✨

🎀 CONFIGURATION FILES 🎀

okay so CRIU is obsessed with not making you type every single option on the command line every time — that's literally what config files are for, bestie. 💅 They're highkey useful when you keep using the same options over and over.

You can pass a specific config file with --config file. If you don't pass one, the default configuration files /etc/criu/default.conf and $HOME/.criu/default.conf are parsed (if they're present on the system). And if the environment variable CRIU_CONFIG_FILE is set, it will also be parsed. ✨

The options passed to CRIU via CLI, RPC or configuration file are evaluated in this exact order:

  • apply_config(/etc/criu/default.conf)
  • apply_config($HOME/.criu/default.conf)
  • apply_config(CRIU_CONFIG_FILE)
  • apply_config(--config file)
  • apply_config(CLI) or apply_config(RPC)
  • apply_config(RPC configuration file) (only for RPC mode)

Default configuration file parsing can be deactivated with --no-default-config if needed. Parsed configuration files are merged with command line options, which allows overriding boolean options. 🫶

📝 Configuration file syntax

Comments are supported using the # sign — the rest of the line is ignored. Options are the same as command line options without the -- prefix, and you use one option per line (with the corresponding argument if applicable, divided by whitespaces). If needed, the argument can be provided in double quotes (this should be needed only if the argument contains whitespaces). In case this type of argument contains a literal double quote as well, it can be escaped using the \ sign. Usage of commands is disallowed, and all other escape sequences are interpreted literally. 🔥

Example of a configuration file to illustrate the syntax:

$ cat ~/.criu/default.conf
tcp-established
work-dir "/home/USERNAME/criu/my \"work\" directory"
#this is a comment
no-restore-sibling   # this is another comment

🛰️ Configuration files in RPC mode

Not only does CRIU evaluate configuration files in CLI mode, it also evaluates them in RPC mode. Just like in CLI mode, the configuration file values are evaluated first. That means any option set via RPC will overwrite the configuration file setting. So the user can change CRIU's default behavior, but it is not possible to change settings that are explicitly set by the RPC client. 🙈

The RPC client can, however, specify an additional configuration file that will be evaluated after the RPC options (see the option evaluation order above). The RPC client can specify this additional configuration file via req.opts.config_file = /path/to/file. The values from this configuration file will overwrite all other configuration file settings or RPC options.

okay careful, bestie: this can lead to undesired behavior of CRIU and should only be used carefully. 💀

🌈 NON-ROOT 🌈

CRIU can be used as non-root with either the CAP_SYS_ADMIN capability or with the CAP_CHECKPOINT_RESTORE capability introduced in Linux kernel 5.9. CAP_CHECKPOINT_RESTORE is the minimum that is required. 👑

CRIU also needs either CAP_SYS_PTRACE or a value of 0 in /proc/sys/kernel/yama/ptrace_scope (see ptrace(2)) to be able to interrupt the process for dumping. 🫶

Running CRIU as non-root has many limitations, and depending on the process to checkpoint and restore, it may not be possible. 😭

In addition to CAP_CHECKPOINT_RESTORE, it is possible to give CRIU additional capabilities to enable additional features in non-root mode. Currently CRIU can benefit from the following additional capabilities:

  • CAP_NET_ADMIN
  • CAP_SYS_CHROOT
  • CAP_SETUID
  • CAP_SYS_RESOURCE

Note that for some operations, having a capability in a namespace other than the init namespace (i.e. the default/root namespace) is not sufficient. For example, in order to read symlinks in proc/[pid]/map_files, CRIU requires CAP_CHECKPOINT_RESTORE in the init namespace; having CAP_CHECKPOINT_RESTORE while running in another user namespace (e.g. in a container) does not allow CRIU to read symlinks in /proc/[pid]/map_files. 🌸

Without access to /proc/[pid]/map_files, checkpointing/restoring processes that have mapped deleted files may not be possible. 💀

Independent of the capabilities, it is always necessary to use --unprivileged to accept CRIU's limitation in non-root mode. ✨

✨ EXAMPLES ✨

To checkpoint a program with a pid of 1234 and write all image files into the directory checkpoint:

criu dump -D checkpoint -t 1234

To restore this program detaching CRIU itself:

criu restore -d -D checkpoint

👑 AUTHOR 👑

The CRIU team.

Copyright (C) 2011-2016, Parallels Holdings, Inc.

okay that's the whole page, bestie 🫶 source/version: criu; date: 2025-05-26; manual: CRIU(8) — iconic, no cap 💅✨

📦 where this page comes from