vzctl(8)

📦 version 4.9.4, in ubuntu jammy

🦋 vzctl(8) — Containers 🦋

okay bestie, sit down, hydrate, because this is the vzctl(8) manual page from the Containers section and it is long. But it's also iconic, and by the end of it you're gonna be running OpenVZ containers like it's nothing. no cap. 💅

✨ NAME ✨

vzctl — perform various operations on an OpenVZ container 🫶

🎀 SYNOPSIS 🎀

vzctl [flags] create CTID --parameter value [...]
vzctl [flags] start CTID [--wait] [--force] [--skip-fsck] [--skip-remount]
vzctl [flags] stop CTID [--fast] [--skip-umount]
vzctl [flags] restart CTID [--wait] [--force] [--fast] [--skip-fsck] [--skip-remount]
vzctl [flags] suspend | resume CTID [--dumpfile name]
vzctl [flags] snapshot CTID [--id uuid] [--name name] [--description desc] [--skip-suspend] [--skip-config]
vzctl [flags] snapshot-switch CTID [--skip-resume|--must-resume] [--skip-config] --id uuid
vzctl [flags] snapshot-delete CTID --id uuid
vzctl [flags] snapshot-mount CTID --id uuid --target dir
vzctl [flags] snapshot-umount CTID --id uuid
vzctl [flags] snapshot-list CTID [-H] [-o field[,field...] [--id uuid]
vzctl [flags] set CTID --parameter value [...]  [--save] [--force] [--setmode restart|ignore]
vzctl [flags] set CTID --reset_ub
vzctl [flags] destroy | delete | mount | umount | status | quotaon | quotaoff | quotainit CTID
vzctl [flags] console CTID [ttynum]
vzctl [flags] convert CTID [--layout ploop[:{expanded|plain|raw}]]
vzctl [flags] compact CTID
vzctl [flags] exec | exec2 CTID command [arg ...]
vzctl [flags] enter CTID [--exec command [arg ...]]
vzctl [flags] runscript CTID script
vzctl --help | --version

💖 DESCRIPTION 💖

vzctl is a utility that lives on the host system (which the cool kids also call the Hardware Node, or HN), and it does direct manipulations with containers (CTs). like, it's the one holding the reins. 🐎✨

Containers can be referred to by either their numeric CTID or by name (peep the --name option for that one). 🎀

heads up bestie: CT ID <= 100 are reserved for OpenVZ internal purposes. don't touch. and a numeric ID should not be more than 2147483644. that's the ceiling, fr fr. 🔥

🌈 OPTIONS 🌈

🚩 Flags

These flags come before a command, and you can use them with any command. They affect logging to console (terminal) only, and they do not affect logging to a log file. so the log file stays unbothered, periodt. 💅

  • --quiet — Disables output. But ngl, scripts run by vzctl are still able to produce some output. so "quiet" is doing a little bit of a vibe, not a full silence. 😭
  • --verbose — Increments the logging level up from the default. You can use it multiple times (stack it like it's limited edition 💅). The default value is set to the value of the VERBOSE parameter in the global configuration file vz.conf(5), or to 0 if it's not set by the VERBOSE parameter.

🛠️ Setting container parameters

set CTID [--onboot yes|no] [--bootorder number] [--root path]
    [--private path] [--mount_opts options] [--userpasswd user:pass]
    [--disabled yes|no] [--name name] [--description string]
    [--ostemplate string] [--stop-timeout seconds] [--ipadd addr]
    [--ipdel addr|all] [--hostname name] [--nameserver addr]
    [--searchdomain name] [--netif_add dev[,params...]]
    [--netif_del dev|all] [--ifname dev [--mac hwaddr]
    [--host_ifname dev] [--host_mac hwaddr] [--bridge name]
    [--mac_filter on|off]] [--numproc items] [--numtcpsock items]
    [--numothersock items] [--vmguarpages pages] [--kmemsize bytes]
    [--tcpsndbuf bytes] [--tcprcvbuf bytes] [--othersockbuf bytes]
    [--dgramrcvbuf bytes] [--oomguarpages pages] [--lockedpages pages]
    [--privvmpages pages] [--shmpages pages] [--numfile items]
    [--numflock items] [--numpty items] [--numsiginfo items]
    [--dcachesize bytes] [--numiptent num] [--physpages pages]
    [--swappages pages] [--ram bytes] [--swap bytes]
    [--vm_overcommit float] [--cpuunits num] [--cpulimit num]
    [--cpus num] [--cpumask cpus|auto|all] [--nodemask nodes|all]
    [--meminfo none|mode:value] [--iptables name[,...]]
    [--netfilter disabled|stateless|stateful|full] [--netdev_add ifname]
    [--netdev_del ifname] [--diskquota yes|no] [--diskspace num]
    [--diskinodes num] [--quotatime seconds] [--quotaugidlimit num]
    [--capability capname:on|off[,...]]  [--devnodes param]
    [--devices param] [--pci_add dev] [--pci_del dev]
    [--features name:on|off[,...]]  [--applyconfig name]
    [--applyconfig_map group] [--ioprio num] [--iolimit mbps]
    [--iopslimit iops] [--save] [--force] [--reset_ub]
    [--setmode restart|ignore]

This command sets various container parameters. If the container is currently running, vzctl applies these parameters to the container. The following options can be used with the set command. 🫶

🚩 Flags

  • --save — If this flag is given, parameters are saved in the container configuration file ctid.conf(5). 💾
  • --force — If this flag is given together with --save, parameters are saved even if the current kernel doesn't support OpenVZ. ⚠️ Note this flag does not make sense without --save, so --save is required. like, it needs its buddy.
  • --reset_ub — If this flag is given, vzctl applies all User Beancounter parameters from the configuration file to a running container. Super helpful in case the configuration file was modified manually. Please note this flag is exclusive, i.e. it can not be combined with any other options or flags. it's a solo act, main character energy only. 👑
  • --setmode restart | ignore — A few parameters can only be applied by restarting the container. By default, vzctl prints a warning if such parameters are supplied and a container is running. Use --setmode restart together with the --save flag to restart a container in such a case, or --setmode ignore to suppress the warning. 🙈

🌸 Miscellaneous

  • --onboot yes | no — Sets whether the container will be started during system boot. The container will be started on boot by the vz initscript if either this parameter is set to yes, or the container was running just before the last reboot, and this parameter is not set to no. Default value is unset, meaning the container will be started if it was running before the last reboot. ⭐
  • --bootorder number — Sets the boot order priority for this CT. The higher the number is, the earlier in the boot process this container starts. By default this parameter is unset, which is considered to be the lowest priority, so containers with unset bootorder will start last. 🐢
  • --root path — Sets the path to the root directory (VE_ROOT) for this container. This is essentially a mount point for the container's root directory. The argument can contain the literal string $VEID, which will be substituted with the numeric CT ID. ✨
  • --private path — Sets the path to the private directory (VE_PRIVATE) for this container. This is a directory in which all the container's files are stored. The argument can contain the literal string $VEID, which will be substituted with the numeric CT ID. 📁
  • --mount_opts option[,option...] — Sets additional mount options for the container file system. Only applicable for ploop layout, ignored otherwise. 🙈
  • --userpasswd user:password — Sets the password for the given user in a container, creating the user if it does not exist. Note that this option is not saved in the configuration file at all (so the --save flag is useless here), it is applied directly to the container, by running distribution-specific programs inside the container. It is not recommended to combine this option with any other options. 💀

In case the container was not running, it is automatically started, then all the appropriate changes are applied, then it is stopped. so it does a little pop-in, does its thing, and dips. 🎀

Note that the container should be created before using this option.

  • --disabled yes | no — Disable container start. To force the start of a disabled container, use vzctl start --force. 🔥
  • --name name — Add a name for a container. The name can later be used in subsequent calls to vzctl in place of CTID. Note this option can not be used without --save.
  • --description string — Add a textual description for a container. cute little label for your baby. 🎀
  • --ostemplate string — Sets a new value of the OSTEMPLATE parameter in the container configuration file ctid.conf(5). Requires the --save flag. Useful after a change/upgrade of a distribution running inside the container, as vzctl uses the value of OSTEMPLATE to run distribution-specific scripts. 🦋
  • --stop-timeout seconds — Sets a time to wait for the container to stop on vzctl stop before forcibly killing it, in seconds. Note this option can not be used without the --save flag. ⏳

A special value of 0 means to use the compiled-in default.

🌐 Networking

  • --ipadd addr — Adds an IP address addr to a given container. The address can optionally have a netmask specified in the CIDR notation (e.g. 10.1.2.3/25). Note that this option is incremental, so addr are added to the already existing ones. ➕
  • --ipdel addr | all — Removes IP address addr from a container. If you want to remove all the addresses, use --ipdel all. 🗑️
  • --hostname name — Sets the container hostname. vzctl writes it to the appropriate file inside the container (distribution-dependent). 💖
  • --nameserver addr — Sets the DNS server IP address for a container. If you want to set several nameservers, you should do it at once, so use the --nameserver option multiple times in one call to vzctl, because all the name server values set in previous calls to vzctl are overwritten. 😤

A special value of inherit can be used to auto-propagate nameserver value(s) from the host system's /etc/resolv.conf file. 🧚

  • --searchdomain name — Sets DNS search domains for a container. If you want to set several search domains, you should do it at once, so use the --searchdomain option multiple times in one call to vzctl, because all the search domain values set in previous calls to vzctl are overwritten. 😤

A special value of inherit can be used to auto-propagate search domain value(s) from the host system's /etc/resolv.conf file. 🧚

  • --netif_add ifname[,mac,host_ifname,host_mac,bridge] — Adds a virtual Ethernet device (veth) to a given container. Here ifname is the Ethernet device name in the container, mac is its MAC address, host_ifname is the Ethernet device name on the host, and host_mac is its MAC address. MAC addresses should be in the format like XX:XX:XX:XX:XX:XX. bridge is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. All parameters except ifname are optional and are automatically generated if not specified. 🌈
  • --netif_del dev_name | all — Removes a virtual Ethernet device from a container. If you want to remove all devices, use all. 💅

🦋 veth interface configuration

The following options can be used to reconfigure the already-created virtual Ethernet interface. To select the interface to configure, use the --ifname name option. 🎯

  • --mac XX:XX:XX:XX:XX:XX — MAC address of the interface inside a container.
  • --host_ifname name — interface name for the virtual interface in the host system.
  • --host_mac XX:XX:XX:XX:XX:XX — the MAC address of the interface in the host system, bestie. 💅

If you want an independent communication with the Container through the bridge, you should specify a multicast MAC address here (FE:FF:FF:FF:FF:FF). ✨

  • --bridge name — bridge name. Custom network start scripts can use this value to automatically add the interface to a bridge. 🌉
  • --mac_filter on | off — enables/disables MAC address filtering for the Container veth device and the possibility of configuring the MAC address of this device from inside the Container. If the filtering is turned on:
    • the veth device accepts only those packets that have a MAC address in their headers corresponding to that of this device (excluding all broadcast and multicast packets);
    • it is impossible to modify the veth MAC address from inside the Container.

By default, this functionality is enabled for all veth devices existing inside the Container. 💖

✨ VSwap limits ✨

The following options sets memory and swap limits for VSwap-enabled kernels (kernel version 042stab042 or greater). No cap. 🧚

Argument is in bytes, unless otherwise specified by an optional suffix. Available suffixes are:

  • T, t — terabytes;
  • G, g — gigabytes;
  • M, m — megabytes;
  • K, k — kilobytes;
  • P, p — memory pages (arch-specific, usually 4KB);
  • B, b — bytes (this is the default).
  • --ram bytes — sets physical memory (RAM) available to a container. Actually, the option is a shortcut for setting --physpages limit (the barrier is set to 0). 🧠
  • --swap bytes — set swap space available to a container. Actually, the option is a shortcut for setting --swappages limit (the barrier is set to 0). 💾
  • --vm_overcommit float — set VM overcommitment value to float. If set, it is used to calculate privmmpages parameter in case it is not set explicitly (see below). Default value is 0, meaning unlimited privvmpages. 🌈

vzctl checks if running kernel is VSwap capable, and refuses to use these parameters otherwise. This behavior can be overriden by using --force flag before parameters. 👀

Bestie, if your kernel isn't VSwap capable, vzctl will straight up refuse. Use --force if you know what you're doing. 💀

In VSwap mode, all beancounters other than RAM and swap become optional. Note though that if some optional beancounters are not set, they are calculated and set by vzctl implicitly, using the following formulae:

  • lockedpages.barrier = oomguarpages.barrier = ram
  • lockedpages.limit = oomguarpages.limit = unlimited
  • vmguarpages.barrier = vmguarpages.limit = ram + swap
  • privvmpages.barrier = privvmpages.limit = (ram + swap) * vm_overcommit

(if vm_overcommit is 0 or not set, privvmpages is set to "unlimited")

Here is an example of setting container 777 to have 512 megabytes of RAM and 1 gigabyte of swap:

vzctl set 777 --ram 512M --swap 1G --save

✨ User Beancounter limits ✨

The following options sets barrier and limit for various user beancounters. 🎀

Note that for VSwap-enabled kernels (version 042stab042 or greater) these limits are optional, you must only set --ram and --swap (see above). For older kernels, these limits are obligatory. 💀

Each option requires one or two arguments. In case of one argument, vzctl sets barrier and limit to the same value. In case of two colon-separated arguments, the first is a barrier, and the second is a limit. Each argument is either a number, a number with a suffix, or a special value unlimited. ✨

Arguments are in items, pages or bytes. Note that page size is architecture-specific, it is 4096 bytes on x86 and x86_64 platforms. 📏

You can also specify different suffixes for User Beancounter parameters (except for those which names start with num). For example, vzctl set CTID --privvmpages 5M:6M should set privvmpages' barrier to 5 megabytes and its limit to 6 megabytes. 🌸

Available suffixes are:

  • T, t — terabytes;
  • G, g — gigabytes;
  • M, m — megabytes;
  • K, k — kilobytes;
  • P, p — memory pages (arch-specific, usually 4KB);
  • B, b — bytes.

You can also specify the literal word unlimited in place of a number. In that case the corresponding value will be set to LONG_MAX, i. e. the maximum possible value. 👑

  • --numproc items[:items] — maximum number of processes and kernel-level threads. Setting the barrier and the limit to different values does not make practical sense. 🧵
  • --numtcpsock items[:items] — maximum number of TCP sockets. This parameter limits the number of TCP connections and, thus, the number of clients the server application can handle in parallel. Setting the barrier and the limit to different values does not make practical sense. 🔌
  • --numothersock items[:items] — maximum number of non-TCP sockets (local sockets, UDP and other types of sockets). Setting the barrier and the limit to different values does not make practical sense. 📡
  • --vmguarpages pages[:pages] — memory allocation guarantee. This parameter controls how much memory is available to a container. The barrier is the amount of memory that container's applications are guaranteed to be able to allocate. The meaning of the limit is currently unspecified; it should be set to unlimited. 🛡️
  • --kmemsize bytes[:bytes] — maximum amount of kernel memory used. This parameter is related to --numproc. Each process consumes certain amount of kernel memory - 16 KB at least, 30-50 KB typically. Very large processes may consume a bit more. It is important to have a certain safety gap between the barrier and the limit of this parameter: equal barrier and limit may lead to the situation where the kernel will need to kill container's applications to keep the kmemsize usage under the limit. 💀
Safety gap, bestie! Don't skip it. ✋
  • --tcpsndbuf bytes[:bytes] — maximum size of TCP send buffers. Barrier should be not less than 64 KB, and difference between barrier and limit should be equal to or more than value of numtcpsock multiplied by 2.5 KB. 📤
  • --tcprcvbuf bytes[:bytes] — maximum size of TCP receive buffers. Barrier should be not less than 64 KB, and difference between barrier and limit should be equal to or more than value of numtcpsock multiplied by 2.5 KB. 📥
  • --othersockbuf bytes[:bytes] — maximum size of other (non-TCP) socket send buffers. If container's processes needs to send very large datagrams, the barrier should be set accordingly. Increased limit is necessary for high performance of communications through local (UNIX-domain) sockets. 📨
  • --dgramrcvbuf bytes[:bytes] — maximum size of other (non-TCP) socket receive buffers. If container's processes needs to receive very large datagrams, the barrier should be set accordingly. The difference between the barrier and the limit is not needed. 📩
  • --oomguarpages pages[:pages] — guarantees against OOM kill. Under this beancounter the kernel accounts the total amount of memory and swap space used by the container's processes. The barrier of this parameter is the out-of-memory guarantee. If the oomguarpages usage is below the barrier, processes of this container are guaranteed not to be killed in out-of-memory situations. The meaning of limit is currently unspecified; it should be set to unlimited. 🚫
  • --lockedpages pages[:pages] — maximum number of pages acquired by mlock(2). 🔒
  • --privvmpages pages[:pages] — allows controlling the amount of memory allocated by the applications. For shared (mapped as MAP_SHARED) pages, each container really using a memory page is charged for the fraction of the page (depending on the number of others using it). For "potentially private" pages (mapped as MAP_PRIVATE), container is charged either for a fraction of the size or for the full size if the allocated address space. In the latter case, the physical pages associated with the allocated address space may be in memory, in swap or not physically allocated yet. 🧠

The barrier and the limit of this parameter control the upper boundary of the total size of allocated memory. Note that this upper boundary does not guarantee that container will be able to allocate that much memory. The primary mechanism to control memory allocation is the --vmguarpages guarantee. 💡

  • --shmpages pages[:pages] — maximum IPC SHM segment size. Setting the barrier and the limit to different values does not make practical sense. 🧩
  • --numfile items[:items] — maximum number of open files. In most cases the barrier and the limit should be set to the same value. Setting the barrier to 0 effectively disables pre-charging optimization for this beancounter in the kernel, which leads to the held value being precise but could slightly degrade file open performance. 📂
  • --numflock items[:items] — maximum number of file locks. Safety gap should be between barrier and limit. 🔐
  • --numpty items[:items] — number of pseudo-terminals (PTY). Note that in OpenVZ each container can have not more than 255 PTYs. Setting the barrier and the limit to different values does not make practical sense. 🖥️
  • --numsiginfo items[:items] — number of siginfo structures. Setting the barrier and the limit to different values does not make practical sense. 📶
  • --dcachesize bytes[:bytes] — maximum size of filesystem-related caches, such as directory entry and inode caches. Exists as a separate parameter to impose a limit causing file operations to sense memory shortage and return an errno to applications, protecting from memory shortages during critical operations that should not fail. Safety gap should be between barrier and limit. 🗂️
  • --numiptent num[:num] — number of iptables (netfilter) entries. Setting the barrier and the limit to different values does not make practical sense. 🔥
  • --physpages pages[:pages] — on VSwap-enabled kernels, this limits the amount of physical memory (RAM) available to a container. The barrier should be set to 0, and the limit to a total size of RAM that can be used used by a container. 🧠

For older kernels, this is an accounting-only parameter, showing the usage of RAM by this container. Barrier should be set to 0, and limit should be set to unlimited. 📜

  • --swappages pages[:pages] — okay so swap, bestie. On VSwap-enabled kernels (042stab042 or greater), this caps how much swap space a container can actually touch. You set the barrier to 0, and the limit to the total swap a container is allowed to use. 🫧

On the old-school pre-VSwap kernels it's a whole different story: the limit is just used to report a total amount of swap space available inside the container, and the barrier gets completely ignored. Default is unlimited, which means total swap gets reported as 0. 💀

🎀 CPU fair scheduler parameters 🎀

These ones run the CPU usage of a container, no cap.

  • --cpuunits num — the CPU weight for a container. Argument is a positive non-zero number, passed to and used in the kernel fair scheduler. Bigger number = more CPU time this container gets. Maximum is 500000, minimum is 8. The number is relative to the weights of all the other running containers, so it's giving "everybody's fighting for the same slice" energy. If you don't specify cpuunits, the default 1000 is used. ✨

You can also set CPU weight for CT0 (the host system itself, yes really) with vzctl set 0 --cpuunits num. Usually the OpenVZ initscript (/etc/init.d/vz) handles this for you, so don't stress.

  • --cpulimit num[%] — the limit of CPU usage for the container, in per cent. Heads up: if the computer has 2 CPUs, it has a total of 200% CPU time, so the percentages stack up. Default CPU limit is 0 (no CPU limit at all, free rein 💅).
  • --cpus num — sets the number of CPUs available in the container.
  • --cpumask cpus | auto | all — sets the list of allowed CPUs for the container. Input format is a comma-separated list of decimal numbers and/or ranges. Consecutively set bits show up as two hyphen-separated decimal numbers: the smallest and largest bit numbers set in the range. For example, if you want the container to execute on CPUs 0, 1, 2, 7, you pass 0-2,7. Default value is all (container can execute on any CPU, main character energy). If you use it with the --nodemask option, the value auto assigns all CPUs from the specified NUMA node to a container. 🧚
  • --nodemask nodes | all — sets the list of allowed NUMA nodes for the container. Input format is the same as for --cpumask. Note that --nodemask must be used together with the --cpumask option, they're a package deal. 🫶

🌸 Memory output parameters 🌸

For VSwap-enabled kernels (042stab042 or greater), this parameter is ignored, periodt. For older kernels, it controls the output of /proc/meminfo inside a container.

  • --meminfo none — no /proc/meminfo virtualization (the same as on the host system).
  • --meminfo mode:value — configure the total memory output in a container. The reported free memory gets evaluated according to the mode you set. The reported swap gets evaluated according to the settings of the --swappages parameter. 🌈

You can use these modes for mode:

  • pages:value — sets total memory in pages;
  • privvmpages:value — sets total memory as privvmpages * value.

Default is privvmpages:1.

💜 Netfilter (iptables) control parameters 💜

  • --netfilter disabled|stateless|stateful|full — restricts access to the netfilter/iptables modules for a container. This option replaces the obsoleted --iptables.
Heads up bestie: changing this parameter requires a container restart, so consider using the --setmode option. 🔥

The following arguments can be used:

  • disabled — no modules are allowed
  • stateless — all modules except NAT and conntracks are allowed (i.e. filter and mangle); this is the default
  • stateful — all modules except NAT are allowed
  • full — all modules are allowed
  • --iptables name[,...] — ngl, this option is obsoleted, --netfilter should be used instead.

Allows you to use the functionality of the name iptables module inside the container. Multiple comma-separated names can be specified.

The default list of enabled iptables modules is defined by the IPTABLES variable in vz.conf(5).

You can use the following values for name: iptable_filter, iptable_mangle, ipt_limit, ipt_multiport, ipt_tos, ipt_TOS, ipt_REJECT, ipt_TCPMSS, ipt_tcpmss, ipt_ttl, ipt_LOG, ipt_length, ip_conntrack, ip_conntrack_ftp, ip_conntrack_irc, ipt_conntrack, ipt_state, ipt_helper, iptable_nat, ip_nat_ftp, ip_nat_irc, ipt_REDIRECT, xt_mac, ipt_recent, ipt_owner.

🦋 Network devices control parameters 🦋

  • --netdev_add name — move network device from the host system to a specified container.
  • --netdev_del name — delete network device from a specified container.

🍓 Disk quota parameters 🍓

  • --diskquota yes | no — lets you enable or disable disk quota for a container. By default, a global value (DISK_QUOTA) from vz.conf(5) is used.

Note that this parameter is ignored for ploop layout.

  • --diskspace num[:num] — for simfs layout, sets soft and hard disk quota limits. First parameter is the soft limit, second is the hard limit.

For ploop layout, this initiates the procedure of resizing the ploop image file to the new size. Since there's no soft/hard limit concept in ploop, the second num, if specified, is ignored. 🙈

By default, ploop resize is done online, i.e. on a mounted ploop. This is the preferred way of doing resize. Although — rare case incoming — if a container was using lots of disk space and should now be resized to a much smaller size, an offline resize might be more appropriate. In that case, make sure the container is stopped and unmounted and use the additional --offline-resize option.

Note that ploop resize is NOT performed on container start, so for consistency --diskspace must be used together with the --save flag. Don't forget it, fr fr. 💀

Suffixes G, M, K can also be specified (see the Resource limits section for more info on suffixes). If no suffix is specified, the value is in kilobytes.

  • --diskinodes num[:num] — sets soft and hard disk quota limits, in i-nodes. First parameter is the soft limit, second is the hard limit.

Note that this parameter is ignored for ploop layout.

  • --quotatime seconds — sets the quota grace period. The container is permitted to exceed its soft limits for the grace period, but once it has expired, the soft limit is enforced as a hard limit. So it's giving "you have a little wiggle room and then we get strict". ✨

Note that this parameter is ignored for ploop layout.

  • --quotaugidlimit num — enables or disables in-container per-user and per-group disk quotas. If the value is set to 0 or not set, disk quotas inside the container are disabled and not accounted.

For simfs layout containers, a non-zero value sets the maximum number of user/group IDs for which disk quota is accounted.

For ploop layout containers, any non-zero value enables disk quota inside the container; the number of user/group IDs used by disk quota is not limited by OpenVZ.

Note that enabling or disabling in-container disk quotas requires a container restart, so consider using the --setmode option.

👑 Capability option 👑

  • --capability capname:on|off[,...] — sets a capability for a container. Multiple comma-separated capabilities can be specified.
Note that setting a capability while the container is running does not take immediate effect; restart the container in order for the changes to take effect (consider using the --setmode option).

A container has the default set of capabilities, so any operation on capabilities is a "logical AND" with the default capability mask.

You can use the following values for capname: chown, dac_override, dac_read_search, fowner, fsetid, kill, setgid, setuid, setpcap, linux_immutable, net_bind_service, net_broadcast, net_admin, net_raw, ipc_lock, ipc_owner, sys_module, sys_rawio, sys_chroot, sys_ptrace, sys_pacct, sys_admin, sys_boot, sys_nice, sys_resource, sys_time, sys_tty_config, mknod, lease, setveid, ve_admin. For a detailed description, see capabilities(7).

WARNING: setting some of those capabilities may have far reaching security implications, so do not do it unless you know what you are doing. Also note that setting setpcap:on for a container will most probably lead to inability to start it. 😭 Please be careful, bestie.

🎀 Device access management 🎀

  • --devnodes device:[r][w][q]|none — give the container access (r — read, w — write, q — disk quota management, none — no access) to a device designated by the special file /dev/device. The device file is created in a container by vzctl. Example:
vzctl set 777 --devnodes sdb:rwq
  • --devices b|c:major:minor|all:[r][w][q]|none — give the container access to a block or character device designated by its major and minor numbers. The device file has to be created manually.

💜 PCI device management 💜

  • --pci_add [domain:]bus:slot.func — give the container access to a specified PCI device. All numbers are hexadecimal (as printed by lspci(8) in the first column). 🌈
  • --pci_del [domain:]bus:slot.func — delete a PCI device from the container.

Note that the vps-pci configuration script is executed by vzctl when configuring PCI devices. The script is usually located at /usr/lib/vzctl/vzctl/scripts/.

🌸 Features management 🌸

  • --features name:on|off[,...] — enable or disable a specific container feature. Known features are: sysfs, nfs, sit, ipip, ppp, ipgre, bridge, nfsd. A few features can be specified at once, comma-separated. 🧚

⭐ Apply config ⭐

  • --applyconfig name — read container parameters from the container sample configuration file /etc/vz/conf/ve-name.conf-sample, and apply them; if the --save option is specified, save to the container config file. The following parameters are not changed: HOSTNAME, IP_ADDRESS, OSTEMPLATE, VE_ROOT, and VE_PRIVATE.
  • --applyconfig_map group — apply container config parameters selected by group. Right now the only possible value for group is name: to restore the container name based on the NAME variable in the container configuration file.

🎀 I/O scheduling 🎀

  • --ioprio priority — assigns disk I/O priority to a container. Priority range is 0-7. The greater the priority is, the more time for I/O activity the container has. By default each container has a priority of 4. 💅
  • --iolimit limit[B|K|M|G] — bestie, this is your disk I/O bandwidth cap for the container. The value is either a number with an optional suffix, or the literal string unlimited. A value of 0 means "unlimited" too. Out of the box a container has no I/O limit at all. The max you're allowed is 2 gigabytes per second — anything over that gets truncated. 💅

If you don't give a suffix, it's assumed to be megabytes per second. The suffixes you can use are:

  • b, B — bytes per second
  • k, K — kilobytes per second
  • m, M — megabytes per second (this is the default)
  • g, G — gigabytes per second
  • --iopslimit iops — this one caps the IOPS, aka input/output operations per second, for a container. Value is a number or the literal string unlimited. 0 means "unlimited". By default a container has no IOPS limit. ✨

Suspending and resuming

Checkpointing is a feature of the OpenVZ kernel that lets you save a complete in-kernel state of a running container, and restore it later. It's giving "save point in a video game" honestly. 🎀

suspend|chkpnt CTID [--dumpfile name]

Suspends a container to a dump file. If you don't set the --dumpfile option, the default dump file name /var/lib/vz/dump/Dump.CTID is used.

resume|restore CTID [--dumpfile name]

Restores a container from the dump file that the suspend command created. 🦋

Snapshotting

Snapshotting is a feature built on checkpointing and ploop snapshots. It lets you save a complete state of the container file system. Plus, if the container is running, it grabs its in-memory state too (just like checkpointing). Heads up though: the snapshot functionality only works for containers on a ploop device. 👑

snapshot CTID [--id uuid] [--name name] [--description desc] [--skip-suspend] [--skip-config]

Creates a container snapshot, i.e. saves the current container state — including its file system state, running processes state, and configuration file. It's the whole vibe, preserved. 🌸

If the container is running, and --skip-suspend is not specified, the container gets checkpointed and then restored, and the CT memory dump becomes part of the snapshot.

Unless --skip-config is given, the container configuration file is saved to the snapshot too.

If uuid isn't specified, it's auto-generated. --name and --description let you set the snapshot name and description, respectively. The name is what snapshot-list displays.

snapshot-switch CTID [--skip-resume | --must-resume] [--skip-config] --id uuid

Switches the container to a snapshot identified by uuid, restoring its file system state, configuration (if available) and its running state (if available).

⚠️ Note that the current state of the container — including its file system state and its configuration file — is lost! No undo, bestie, this is not a drafts folder. 💀

Option --skip-resume is used to ignore a CT memory dump file in a snapshot; as a result the container will end up in a stopped state (same as if the snapshot was taken with --skip-suspend).

If --must-resume is set, the absence of a memory dump is treated as an error, and the inability to restore from the memory dump is treated as an error rather than a warning.

The --skip-config option is used to ignore the CT configuration file in a snapshot, i.e. the current configuration file will be left as is.

snapshot-delete CTID --id uuid

Removes a specified snapshot. bye bye 🫶

snapshot-mount CTID --id uuid --target directory

Mounts a snapshot specified by uuid to a directory. Note this mount is read-only.

snapshot-umount CTID --id uuid

Unmounts a specified snapshot.

snapshot-list CTID [-H] [-o field[,field...] [--id uuid]

Lists the container's snapshots.

You can suppress the header using the -H option.

You can use the -o option to display only the specified field(s). The list of available fields can be obtained using the -L option.

Performing container actions

create CTID [--ostemplate name] [--config name]
       [--layout simfs|ploop[:{expanded|plain|raw}]]
       [--diskspace kbytes] [--diskinodes num] [--private path]
       [--root path] [--ipadd addr] [--hostname name] [--name name]
       [--local_uid uid] [--local_gid gid]

Creates a new container area. This is a once-and-done operation — you do it before the first start of the container. ✨

By default, an OS template denoted by the DEF_OSTEMPLATE parameter of vz.conf(5) is used to create a container. You can overwrite that with the --ostemplate option.

By default, a new container configuration file is created from a sample configuration denoted by the value of the CONFIGFILE parameter of vz.conf(5). If the container configuration file already exists, it will not be modified.

The value of CONFIGFILE can be overwritten by using the --config name option. This option can not be used if the container configuration file already exists.

A new container can either be created using the simfs filesystem or on a ploop device. The default is set by the value of the VE_LAYOUT parameter of vz.conf(5) and can be overwritten by the --layout option. If ploop is used, you can additionally specify the ploop disk image format after a colon. The possible ploop formats are expanded, plain and raw. Default is expanded. Using a value other than expanded is not recommended and is currently not supported. 😭

You can use the --diskspace and --diskinodes options to specify the container file system size. Note that for ploop layout, you will not be able to change the inodes value later.

If DISKSPACE is not specified either in the sample configuration file used for creation or in the global configuration file vz.conf(5), the --diskspace parameter is required for ploop layout.

Suffixes G, M, K can also be specified (see the Resource limits section for more info on suffixes).

You can use the --root path option to set the path to the mount point for the container root directory (default is VE_ROOT specified in the vz.conf(5) file). The argument can contain the literal string $VEID, which will be substituted with the numeric CT ID.

You can use the --private path option to set the path to the directory in which all the files and directories specific to this very container are stored (default is VE_PRIVATE specified in the vz.conf(5) file). The argument can contain the literal string $VEID, which will be substituted with the numeric CT ID.

You can use the --ipadd addr option to assign an IP address to a container. Note that this option can be used multiple times. 🎀

You can use the --hostname name option to set a host name for a container.

When running with an upstream Linux Kernel that supports user namespaces (>= 3.8), the parameters --local_uid and --local_gid can be used to select which uid and gid respectively will be used as a base user in the host system. Note that user namespaces provide a 1:1 mapping between container users and host users. If these options aren't specified, the values LOCAL_UID and LOCAL_GID from the global configuration file vz.conf(5) are used. An explicit --local_uid value of 0 will disable user namespace support, and run the container as a privileged user. In that case, --local_gid is ignored.

⚠️ Warning: use --local_uid and --local_gid with care, especially when migrating containers. In all situations, the container's files in the filesystem need to be correctly owned by the host-side users. No cap, be careful. 💅

destroy | delete CTID

Removes a container private area by deleting all files, directories and the configuration file of this container. poof, gone 🦋

start CTID [--wait] [--force] [--skip-fsck] [--skip-remount]

Mounts (if necessary) and starts a container. Unless the --wait option is specified, vzctl will return immediately; otherwise an attempt to wait till the default runlevel is reached will be made by vzctl.

Specify --force if you want to start a container which is disabled (see --disabled).

Specify --skip-fsck to skip fsck for a ploop-based container filesystem (this option is used by the vz initscript).

By default, if a container to be started happens to be already mounted, it is unmounted and mounted again. This behavior can be turned off by using the --skip-remount flag.

Note that this command can lead to execution of premount, mount and start action scripts (see ACTION SCRIPTS below).

stop CTID [--fast] [--skip-umount]

Stops a container and unmounts it (unless --skip-umount is given). Normally, halt(8) is executed inside a container; the --fast option makes vzctl use the reboot(2) syscall instead, which is faster but can lead to an unclean container shutdown. 🙈

Note that vzctl stop is not asynchronous — in other words, vzctl waits for the container's init to exit (unless --fast is given), which can take up to a few minutes. The default wait timeout is 120 seconds; it can be changed globally, by setting STOP_TIMEOUT in vz.conf(5), or per container (STOP_TIMEOUT in ctid.conf(5), see --stop-timeout).

Note that this command can lead to execution of stop, umount and postumount action scripts (see ACTION SCRIPTS below).

restart CTID [--wait] [--force] [--fast] [--skip-fsck]

Restarts a container, i.e. stops it if it's running, and starts it again. Accepts all the start and stop options.

Note that this command can lead to execution of some action scripts (see ACTION SCRIPTS below).

status CTID

Shows a container status. This is a line with five or six words, separated by spaces. She's giving telegram, honestly. 💅

  • First word is literally CTID.
  • Second word is the numeric CT ID.
  • Third word shows whether this container exists or not; it can be either exist or deleted.
  • Fourth word shows the status of the container filesystem; it can be either mounted or unmounted.
  • Fifth word shows if the container is running; it can be either running or down.
  • Sixth word, if it exists, is suspended. It appears if a dump file exists for a stopped container (see suspend).

This command can also be usable from scripts. 🔥

mount CTID

Mounts the container private area. Note that this command can lead to execution of premount and mount action scripts (see ACTION SCRIPTS below).

umount CTID

Unmounts the container private area. Note that this command can lead to execution of umount and postumount action scripts (see ACTION SCRIPTS below).

Note that stop does umount automatically. 🌈

  • convert CTID [--layout ploop[:{expanded|plain|raw}]] — Convert the CT private area so it lives on a ploop device (available in kernel version 042stab052.8 and greater). Do this while the container is stopped, and also set the disk space quota, bestie. ✨
  • compact CTID — Compact the container image. This only makes sense for ploop layout. 🧼
  • quotaon CTID — Turn disk quota on. Note that mount and start do that automatically anyway. 👑
  • quotaoff CTID — Turn disk quota off. Note that umount and stop do that automatically anyway. 💤
  • quotainit CTID — Initialize disk quota (i.e. run vzquota init) with the parameters taken from the CT configuration file ctid.conf(5). 🛠️
  • exec CTID command — Executes command in a container. Environment variables are not set inside the container. Signal handlers may differ from default settings. If command is -, commands are read from stdin. 📥
  • exec2 CTID command — Same as exec, but the return code is that of command. 🎯
  • runscript CTID script — Run the specified shell script in the container. The argument script is a file on the host system; its contents are read by vzctl and executed in the context of the container. For a running container, the command jumps into the container and executes the script. For a stopped container, it enters the container, mounts the container's root filesystem, executes the script, and unmounts CT root. In that latter case, the container is not really started, no file systems other than root (such as /proc) are mounted, no startup scripts are executed, etc. So the environment in which the script runs is far from normal and only usable for very basic operations. ⚠️
  • enter CTID [--exec command [arg ...]] — Enters into a container (giving you a container's root shell). This option is a back-door for host root only. The proper way to get a CT root shell is to use ssh(1). 🚪

Option --exec is used to run command with arguments after entering the container. This is useful if the command to be run requires a terminal (so vzctl exec can not be used) and for some reason you can not use ssh(1). 🖥️

You need to log out manually from the shell to finish the session (even if you specified --exec). 👋

  • console CTID [ttynum] — Attach to a container console. The optional ttynum argument is the tty number (such as 4 for tty4); the default is 1, which is used for the container's /dev/console. 🖤

Note the consoles are persistent, meaning that:

  • it can be attached to even if the container is not running;
  • there is no automatic detachment upon the container stop;
  • detaching from the console leaves anything running in this console as is.

The following escape sequences are recognized by vzctl console. Note that these sequences are only recognized at the beginning of a line. ⌨️

  • Esc then . to detach from the console. 🚪
  • Esc then ! to kill anything running on the console (SAK). This is helpful when one expects a login prompt but there isn't one. 💀

✨ Other options ✨

  • --help — Prints a help message with a brief list of possible options. 🆘
  • --version — Prints vzctl version. 📦

🎬 ACTION SCRIPTS 🎬

vzctl has the ability to execute user-defined scripts when a specific vzctl command is run for a container. The following vzctl commands can trigger execution of action scripts: start, stop, restart, mount, and umount. 🎭

Action scripts live in the /etc/vz/conf/ directory. There are global and per-CT scripts. Global scripts have a literal prefix of vps. and are executed for all containers. Per-CT scripts have a CTID. numeric prefix and are executed for the given container only. 🗂️

Please note: scripts are executed in a host system (CT0) context, with the exception of .start and .stop scripts, which are executed in a container context. 🧠

The following action scripts are currently defined:

  • vps.premount, CTID.premount — Global and per-CT mount scripts which are executed for a container before it is mounted. Scripts are executed in the host system context, while a CT is not yet mounted or running. The global script, if it exists, is executed first. 🏗️
  • vps.mount, CTID.mount — Global and per-CT mount scripts which are executed for a container right after it is mounted. Otherwise they are the same as .premount scripts. 🪄
  • CTID.start — Right after vzctl has started a container, it executes this script in a container context. 🚀
  • CTID.stop — Right before vzctl has stopped a container, it executes this script in a container context. 🛑
  • vps.umount, CTID.umount — Global and per-CT umount scripts which are executed for a container before it is unmounted. Scripts are executed in the host system context, while a CT is mounted. The global script, if it exists, is executed first. 🧹
  • vps.postumount, CTID.postumount — Global and per-CT umount scripts which are executed for a container right after it is unmounted. Otherwise they are the same as .umount scripts. ✨

The environment passed to all the *mount scripts is the standard environment of the parent (i.e. vzctl) with two additional variables: $VEID and $VE_CONFFILE. The first one holds the ID of the container, and the second one holds the full path to the container configuration file. If the script needs to get other CT configuration parameters, such as $VE_ROOT, it needs to get those from global and per-CT configuration files. 🌱

Here is an example of a mount script, which makes the host system's /mnt/disk available to container(s). Script name can either be /etc/vz/conf/vps.mount or /etc/vz/conf/CTID.mount. 📜

# If one of these files does not exist then something
# is really broken
[ -f /etc/vz/vz.conf ] || exit 1
[ -f $VE_CONFFILE ] || exit 1
# Source both files. Note the order is important.
. /etc/vz/vz.conf
. $VE_CONFFILE
SRC=/mnt/disk
DST=/mnt/disk
mount -n -t simfs $SRC ${VE_ROOT}${DST} -o $SRC

🚦 EXIT STATUS 🚦

Returns 0 upon success, or an appropriate error code in case of an error:

  • 1 — Failed to set a UBC parameter
  • 2 — Failed to set a fair scheduler parameter
  • 3 — Generic system error
  • 5 — The running kernel is not an OpenVZ kernel (or some OpenVZ modules are not loaded)
  • 6 — Not enough system resources
  • 7 — ENV_CREATE ioctl failed
  • 8 — Command executed by vzctl exec returned non-zero exit code
  • 9 — Container is locked by another vzctl invocation
  • 10 — Global OpenVZ configuration file vz.conf(5) not found
  • 11 — A vzctl helper script file not found
  • 12 — Permission denied
  • 13 — Capability setting failed
  • 14 — Container configuration file ctid.conf(5) not found
  • 15 — Timeout on vzctl exec
  • 16 — Error during vzctl suspend
  • 17 — Error during vzctl resume
  • 18 — Error from setluid() syscall
  • 20 — Invalid command line parameter
  • 21 — Invalid value for command line parameter
  • 22 — Container root directory (VE_ROOT) not set
  • 23 — Container private directory (VE_PRIVATE) not set
  • 24 — Container template directory (TEMPLATE) not set
  • 28 — Not all required UBC parameters are set, unable to start container
  • 29 — OS template is not specified, unable to create container
  • 31 — Container not running
  • 32 — Container already running
  • 33 — Unable to stop container
  • 34 — Unable to add IP address to container
  • 40 — Container not mounted
  • 41 — Container already mounted
  • 43 — Container private area not found
  • 44 — Container private area already exists
  • 46 — Not enough disk space
  • 47 — Bad/broken container (/sbin/init or /bin/sh not found)
  • 48 — Unable to create a new container private area
  • 49 — Unable to create a new container root area
  • 50 — Unable to mount container
  • 51 — Unable to unmount container
  • 52 — Unable to delete a container
  • 53 — Container private area not exist
  • 60 — vzquota on failed
  • 61 — vzquota init failed
  • 62 — vzquota setlimit failed
  • 63 — Parameter DISKSPACE not set
  • 64 — Parameter DISKINODES not set
  • 65 — Error setting in-container disk quotas
  • 66 — vzquota off failed
  • 67 — ugid quota not initialized
  • 71 — Incorrect IP address format
  • 74 — Error changing password
  • 78 — IP address already in use
  • 79 — Container action script returned an error
  • 82 — Config file copying error
  • 86 — Error setting devices (--devices or --devnodes)
  • 89 — IP address not available
  • 91 — OS template not found
  • 99 — Ploop is not supported by either the running kernel or vzctl.
  • 100 — Unable to find container IP address
  • 104 — VE_NETDEV ioctl error
  • 105 — Container start disabled
  • 106 — Unable to set iptables on a running container
  • 107 — Distribution-specific configuration file not found
  • 109 — Unable to apply a config
  • 129 — Unable to set meminfo parameter
  • 130 — Error setting veth interface
  • 131 — Error setting container name
  • 133 — Waiting for container start failed
  • 139 — Error saving container configuration file
  • 148 — Error setting container IO parameters (ioprio)
  • 150 — Ploop image file not found
  • 151 — Error creating ploop image
  • 152 — Error mounting ploop image
  • 153 — Error unmounting ploop image
  • 154 — Error resizing ploop image
  • 155 — Error converting container to ploop layout
  • 156 — Error creating ploop snapshot
  • 157 — Error merging ploop snapshot
  • 158 — Error deleting ploop snapshot
  • 159 — Error switching ploop snapshot
  • 166 — Error compacting ploop image
  • 167 — Error listing ploop snapsots

🌸 EXAMPLES 🌸

okay bestie, real talk, this is the part everyone actually scrolls to, so let's go. ✨

Wanna create AND start a "basic" container with ID <code>1000</code> using the <code>centos-5</code> OS template and IP <code>192.168.10.200</code>? Boom, three lines, you're a sysadmin:

vzctl create 1000 --ostemplate centos-5 --config basic
vzctl set 1000 --ipadd 192.168.10.200 --save
vzctl start 1000

Setting the number of processes barrier/limit to <code>80/100</code>, plus PTY barrier/limit to <code>16/20</code> PTYs? Say less:

vzctl set 1000 --numproc 80:100 -t 16:20 --save

Running <code>ls -la</code> inside this container? Exec it, periodt:

vzctl exec 1000 /bin/ls -la

Running a whole pipe, like <code>ls -l / | sort</code>, inside this container? Just quote it, bestie:

vzctl exec 1000 'ls -l / | sort'

Entering this container and running <code>apt-get install vim</code>?

vzctl enter 1000 --exec apt-get install vim
⚠️ heads up, no cap: in the example above you WILL need to log out from the container's shell after apt-get finishes. it's not gonna do it for you. 🙈

okay so now the glow-up version. Wanna enter this container, run <code>apt-get install vim</code>, and log out after a successful install — but stay inside the container if the installation failed? Use &&:

vzctl enter 1000 --exec "apt-get install vim && logout"

Wanna enter this container, run <code>apt-get install vim</code>, and log out no matter what the exit code of the install was? Use ;:

vzctl enter 1000 --exec "apt-get install vim ; logout"
💅 important: you need to quote the command if you use && or ;. no quoting, no vibes.

Stopping this container?

vzctl stop 1000

Permanently removing this container? (gone gone, not coming back 😭)

vzctl destroy 1000

📁 FILES 📁

here's where everything lives, bestie. memorize it, iykyk:

  • /etc/vz/vz.conf
  • /etc/vz/conf/CTID.conf
  • /etc/vz/conf/vps.{premount,mount,umount,postumount}
  • /etc/vz/conf/CTID.{premount,mount,start,stop,umount,postumount}
  • /proc/vz/veinfo
  • /proc/vz/vzquota
  • /proc/user_beancounters
  • /proc/bc/*
  • /proc/fairsched

👀 SEE ALSO 👀

go read your sisters, they're all iconic:

⚖️ LICENSE ⚖️

Copyright (C) 2000-2013, Parallels, Inc. Licensed under GNU GPL.

and that's the whole page, bestie!! 🫶 this one came from OpenVZ, dated 3 Jul 2014, and it's the manual for vzctl(8). go forth and containerize responsibly. slay. 💖✨🎀

📦 where this page comes from