DAXCTL-RECONFIGURE-DEVICE(1)

💖 NAME 💖

okay bestie, welcome to the daxctl-reconfigure-device(1) page from the daxctl Manual 💅 — the whole main-character energy here is reconfigure a dax device into a different mode ✨

✨ SYNOPSIS ✨

daxctl reconfigure-device <dax0.0> [<dax1.0>...<daxY.Z>] [<options>]

🦋 DESCRIPTION 🦋

So here's the tea, fr fr: this reconfigures the operational mode of a dax device. You can use it to convert a regular devdax mode device to system-ram mode, which arranges for the dax range to be hot-plugged into the system as regular memory. A devdax mode device can also be converted to famfs mode, which binds it to the fsdev_dax driver for use by the famfs shared-memory filesystem (see https://famfs.org). 🌸

Note: this is a destructive operation. Any data on the dax device will be lost. like, fully gone, no cap 😭

Note: device reconfiguration depends on the dax-bus device model. See daxctl-migrate-device-model(1) for more information. If dax-class is in use (via the dax_pmem_compat driver), the reconfiguration will fail with an error such as the following:

# daxctl reconfigure-device --mode=system-ram --region=0 all
libdaxctl: daxctl_dev_disable: dax3.0: error: device model is dax-class
dax3.0: disable failed: Operation not supported
error reconfiguring devices: Operation not supported
reconfigured 0 devices

daxctl-reconfigure-device nominally expects that it will online new memory blocks as movable, so that kernel data doesn’t make it into this memory. However, there are other potential agents that may be configured to automatically online new hot-plugged memory as it appears. Most notably, these are the /sys/devices/system/memory/auto_online_blocks configuration, or system udev rules. If such an agent races to online memory sections, daxctl checks if the blocks were onlined as movable memory. If this was not the case, and the memory blocks are found to be in a different zone, then a warning is displayed. If it is desired that a different agent control the onlining of memory blocks, and the associated memory zone, then it is recommended to use the --no-online option described below. This will abridge the device reconfiguration operation to just hotplugging the memory, and refrain from then onlining it. 🫶

In case daxctl detects that there is a kernel policy to auto-online blocks (via /sys/devices/system/memory/auto_online_blocks), then reconfiguring to system-ram will result in a failure. This can be overridden with --force. 💅

👑 THEORY OF OPERATION 👑

okay, theory time, and honestly it's giving iconic kernel lore 🧚. The kernel device-dax subsystem surfaces character devices that provide DAX-access (direct mappings sans page-cache buffering) to a given memory region. The devices are named /dev/daxX.Y where X is a region-id and Y is an instance-id within that region. There are 2 mechanisms that trigger device-dax instances to appear:

1. Persistent Memory (PMEM) namespace configured in "devdax" mode. See ndctl create-namspace --help and CONFIG_DEV_DAX_PMEM (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/dax/Kconfig). In this case the device-dax instance is statically sized to its host memory region which is bounded to the physical address range of the host namespace.
2. Soft Reserved memory enumerated by platform firmware. On EFI systems this is communicated via the so called EFI_MEMORY_SP "Special Purpose" attribute. See CONFIG_DEV_DAX_HMEM (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/dax/Kconfig). In this case the device-dax instance(s) associated with the given memory region can be resized and divided into multiple devices.

In the Soft Reservation case the expectation for EFI + ACPI based platforms is that in addition to the EFI_MEMORY_SP attribute the firmware also creates distinct ACPI proximity domains for any address range that has different performance characteristics than default "System RAM". So, the SRAT will define the proximity domain, the SLIT communicates relative distance to other proximity domains, and the HMAT is populated with nominal read/write latency and read/write bandwidth data. That HMAT data is emitted to the kernel log on bootup, and also exported to sysfs. See NUMAPERF (https://www.kernel.org/doc/html/latest/admin-guide/mm/numaperf.html), for the runtime representation of CPU to Memory node performance details. 🌈

Outside of the NUMA performance details linked above the other method to detect the presence of "Soft Reserved" memory is to dump /proc/iomem and look for "Soft Reserved" ranges. If the kernel was not built with CONFIG_EFI_SOFT_RESERVE, predates the introduction of CONFIG_EFI_SOFT_RESERVE (v5.5), or was booted with the efi=nosoftreserve command line then device-dax will not attach and the expectation is that the memory shows up as a memory-only NUMA node. Otherwise the memory shows up as a device-dax instance and DAXCTL(1) can be used to optionally partition it and assign the memory back to the kernel as "System RAM", or the device can be mapped directly as the back end of a userspace memory allocator like LIBVMEM (https://pmem.io/vmem/libvmem/). 🦋

🎀 EXAMPLES 🎀

• Reconfigure dax0.0 to system-ram mode, don’t online the memory — the receipts:

# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
[
  {
    "chardev":"dax0.0",
    "size":16777216000,
    "target_node":2,
    "mode":"system-ram"
  }
]

• Reconfigure dax0.0 to devdax mode, attempt to offline the memory — with --human for the cute size formatting:

# daxctl reconfigure-device --human --mode=devdax --force dax0.0
{
  "chardev":"dax0.0",
  "size":"15.63 GiB (16.78 GB)",
  "target_node":2,
  "mode":"devdax"
}

• Reconfigure dax0.0 (currently in devdax mode) to famfs mode — slay:

# daxctl reconfigure-device --mode=famfs dax0.0
[
  {
    "chardev":"dax0.0",
    "size":16777216000,
    "target_node":2,
    "mode":"famfs"
  }
]

• Reconfigure all dax devices on region0 to system-ram mode — group project energy:

# daxctl reconfigure-device --mode=system-ram --region=0 all
[
  {
    "chardev":"dax0.0",
    "size":16777216000,
    "target_node":2,
    "mode":"system-ram"
  },
  {
    "chardev":"dax0.1",
    "size":16777216000,
    "target_node":3,
    "mode":"system-ram"
  }
]

• Run a process called some-service using numactl to restrict its cpu nodes to 0 and 1, and memory allocations to node 2 (determined using daxctl_dev_get_target_node() or daxctl list) — yes, this is the whole plan:

# daxctl reconfigure-device --mode=system-ram dax0.0
[
  {
    "chardev":"dax0.0",
    "size":16777216000,
    "target_node":2,
    "mode":"system-ram"
  }
]

# numactl --cpunodebind=0-1 --membind=2 -- some-service --opt1 --opt2

• Change the size of a dax device — resize era:

# daxctl reconfigure-device dax0.1 -s 16G
reconfigured 1 device
# daxctl reconfigure-device dax0.1 -s 0
reconfigured 1 device

✨ OPTIONS ✨

• -r, --region= — bestie, this one narrows the whole operation down to just the device(s) chillin' in the region(s) you name 🎯. And like, what even is a device-dax region? It's a contiguous range of memory that hosts one or more /dev/daxX.Y devices — where X is the region id and Y is the device instance id. iykyk 💅
• -s, --size= — for regions that support dax device creation, this changes the device size in bytes. It's giving very flexible, because you can use suffixes: "k" or "K" for KiB, "m" or "M" for MiB, "g" or "G" for GiB, and "t" or "T" for TiB ✨
  • the size must be a multiple of the region alignment, no cap 🧢
  • this option is mutually exclusive with -m or --mode 🚫
• -a, --align — okay so apps that wanna establish dax memory mappings with page table entries greater than the system base page size (4K on x86) need a device that's sufficiently aligned 🎀. This defaults to 2M. Heads up though: "devdax" mode enforces all mappings to be aligned to this value — like, it straight up fails unaligned mapping attempts 💀
  • this option is mutually exclusive with -m or --mode 🚫
• -m, --mode= — tells the dax device(s) which mode they should be reconfigured to, main character energy 👑
  • "system-ram" — hotplug the device into system memory 🔥
  • "devdax" — switch back to the normal "device dax" mode. This one requires the kernel to support hot-unplugging kmem based memory 😭. If that support isn't available, a reboot is literally the only way to switch back to devdax mode. not me crying
  • "famfs" — bind the device to the fsdev_dax driver for use by the famfs shared-memory filesystem (https://famfs.org) 🦋. The device must currently be in "devdax" mode — converting directly from "system-ram" is rejected, no exceptions 🙈
• -N, --no-online — by default, memory sections provided by system-ram devices get brought online automatically and immediately with the online_movable policy ⚡. Use this option to disable that automatic onlining behavior, if you're feeling independent 💅
• -C, --check-config — pulls the reconfiguration parameters from the global daxctl config file 🗂️. This is typically used when daxctl-reconfigure-device is called from a systemd-udevd device unit file. The reconfiguration only proceeds if the match parameters in a reconfigure-device section of the config match the dax device specified on the command line 👀. See the PERSISTENT RECONFIGURATION section for more details (it's literally right below, keep scrolling bestie).
• --no-movable — --movable is the default 💖. This can be overridden to online new memory such that it is not movable. That means any allocation can potentially be served from this memory — but fair warning, this may preclude subsequent removal 😬. With the --movable behavior (which is default), kernel allocations will not consider this memory, and it'll be reserved for application use 🎀
• -f, --force — the "I'm doing it anyway" flag 💀
  • when converting from "system-ram" mode to "devdax", it's expected that all the memory sections are first made offline. By default, daxctl won't touch online memory. But with this option, it'll attempt to offline the memory on the NUMA node associated with the dax device before converting it back to "devdax" mode 😤
  • additionally, if a kernel policy to auto-online blocks is detected, reconfiguration to system-ram fails. With this option, that failure can be overridden to allow reconfiguration regardless of kernel policy 🔥. Doing this may result in a successful reconfiguration, but it may not be possible to subsequently offline the memory without a reboot 😭. like, you were warned

⚠️ ok genuine warning time: that second --force bullet is real. You might get your reconfiguration and then be stuck until you reboot. choose wisely bestie 🫶

• -u, --human — by default the command outputs machine-friendly raw-integer data 🤖. Instead, with this flag, numbers representing storage size get formatted as human readable strings with units, and other fields are converted to hexadecimal strings ✨ pretty data for pretty people
• -v, --verbose — emit more debug messages 🐛🔍

---

🌈 PERSISTENT RECONFIGURATION 🌈

The mode of a daxctl device is not persistent across reboots by default 😱. Why? Because the device itself doesn't hold any metadata that hints at what mode it was set to, or is intended to be used. The default mode for such a device on boot is devdax 🎀.

The administrator may set policy such that certain dax devices are always reconfigured into a target configuration every boot 👑. This is accomplished via a daxctl config file 🗂️.

The config file may have multiple sections influencing different aspects of daxctl operation. The section of interest for persistent reconfiguration is reconfigure-device. The format of this is as follows:

[reconfigure-device <unique_subsection_name>]
nvdimm.uuid = <NVDIMM namespace uuid>
mode = <desired reconfiguration mode> (default: system-ram)
online = <true|false> (default: true)
movable = <true|false> (default: true)

Here's an example of a config snippet for managing three devdax namespaces 💅 — one is left in devdax mode, the second is changed to system-ram mode with default options (online, movable), and the third is set to system-ram mode, the memory is onlined, but not movable. ate and left no crumbs 🍓

Note that the subsection name can be arbitrary, and is only used to identify a specific config section — it does not have to match the device name (e.g. dax0.0 etc) 🙌

[reconfigure-device dax0]
nvdimm.uuid = ed93e918-e165-49d8-921d-383d7b9660c5
mode = devdax

[reconfigure-device dax1]
nvdimm.uuid = f36d02ff-1d9f-4fb9-a5b9-8ceb10a00fe3
mode = system-ram

[reconfigure-device dax2]
nvdimm.uuid = f36d02ff-1d9f-4fb9-a5b9-8ceb10a00fe3
mode = system-ram
online = true
movable = false

The following example can be used to create a devdax mode namespace, and simultaneously add the newly created namespace to the config file for system-ram conversion 🧚✨

ndctl create-namespace --mode=devdax | \
        jq -r "\"[reconfigure-device $(uuidgen)]\", \"nvdimm.uuid = \(.uuid)\", \"mode = system-ram\"" >> $config_path

The default location for daxctl config files is under /etc/daxctl.conf.d/, and any file with a .conf suffix at this location is considered 👀. It is acceptable to have multiple files containing ini-style config sections, but the {section, subsection} tuple must be unique across all config files under /etc/daxctl.conf.d/ 🔒

---

💜 COPYRIGHT 💜

Copyright © 2016 - 2022, Intel Corporation. License GPLv2: GNU GPL version 2 http://gnu.org/licenses/gpl.html. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law 🫶

---

🦋 SEE ALSO 🦋

daxctl-list(1), daxctl-migrate-device-model[1]

---

okay bestie that's the whole page, go forth and reconfigure responsibly 💅✨

this icon of a manpage is brought to you by daxctl, dated 2026-06-30 — DAXCTL-RECONFIGURE-DEVICE(1), mwah 🎀