MULTIPATH.CONF(5)
โจ MULTIPATH.CONF(5) โจ
okay bestie, so this one is from the File Formats Manual and it's literally the glow-up guide for your multipath config, no cap ๐
๐ NAME ๐
/etc/multipath.conf, /etc/multipath/conf.d/*.conf โ the configuration file for the multipath daemon. yes, that's the whole name section, she's lowkey but she runs the whole show ๐
๐ธ DESCRIPTION ๐ธ
/etc/multipath.conf is the configuration file for the multipath daemon. Its whole job? Overwriting the built-in configuration table of multipathd. Main character energy fr fr ๐
Any line whose first non-white-space character is a '#' is a comment line. Empty lines? Ignored, they don't get an invite to the party ๐
The configuration multipathd is currently using can be displayed with the multipath -t or multipathd show config command. use it, it's giving "let me check my outfit before I leave" โจ
You can also add extra configuration in drop-in files under /etc/multipath/conf.d/. Files ending in .conf in this directory are read in alphabetical order, after reading /etc/multipath.conf. They use the same syntax as /etc/multipath.conf itself, and support all sections and keywords. And if a keyword occurs in the same section in multiple files? The last occurrence takes precedence over all others. last one wins, periodt ๐
๐ฆ SYNTAX ๐ฆ
The configuration file contains entries of the form:
<section> {
<attribute> <value>
...
<subsection> {
<attribute> <value>
...
}
}
Each section contains one or more attributes or subsections. The recognized keywords for attributes or subsections depend on the section in which they occur. iykyk ๐ง
<attribute> and <value> must be on a single line. <attribute> is one of the keywords listed in this man page. <value> is either a simple word (containing no whitespace and none of the characters '"', '#', and '!') or one string enclosed in double quotes ("...").
Outside a quoted string, text starting with '#' and '!' is regarded as a comment and ignored until the end of the line. Inside a quoted string, '#' and '!' are normal characters, and whitespace is preserved ๐ซถ
To represent a double quote character inside a double quoted string, use two consecutive double quotes ('""'). Thus '2.5" SSD' can be written as "2.5"" SSD". yes bestie, the double-double-quote is real and it slays ๐
Opening braces ('{') must follow the (sub)section name on the same line. Closing braces ('}') that mark the end of a (sub)section must be the only non-whitespace character on the line. Whitespace is ignored except inside double quotes โ so the indentation shown in the example above is helpful for human readers but not mandatory. it's giving aesthetic, not obligation ๐
Note on regular expressions: the/etc/multipath.confsyntax allows many attribute values to be specified as POSIX Extended Regular Expressions (seeregex(7)). These regular expressions are case-sensitive and not anchored, thus the expression"bar"matches"barbie","rhabarber", and"wunderbar", but not"Barbie". To avoid unwanted substring matches, standard regular expression syntax using the special characters"^"and"$"can be used. so if you wanna be picky, anchor her โ
The following section keywords are recognized:
defaultsโ This section defines default values for attributes which are used whenever no values are given in the appropriate device or multipath sections. the fallback bestie ๐blacklistโ This section defines which devices should be excluded from the multipath topology discovery. not invited, blocked, unfollowed ๐blacklist_exceptionsโ This section defines which devices should be included in the multipath topology discovery, despite being listed in the blacklist section. the "wait no, I actually like them" list ๐multipathsโ This section defines the multipath topologies. They are indexed by a World Wide Identifier(WWID). For details on the WWID generation see section WWID generation below. Attributes set in this section take precedence over all others. top of the hierarchy, no notes ๐devicesโ This section defines the device-specific settings. Devices are identified by vendor, product, and revision. very "what's your sign" but for hardware โญoverridesโ This section defines values for attributes that should override the device-specific settings for all devices. the "actually, forget all that" section ๐ฅ
๐ defaults section ๐
The defaults section recognizes the following keywords:
verbosityโ Default verbosity. Higher values increase the verbosity level. Valid levels are between0and6.
The default is: 2
polling_intervalโ Interval between two path checks in seconds. For properly functioning paths, the interval between checks will gradually increase tomax_polling_interval. This value will be overridden by theWatchdogSecsetting in themultipathd.servicedefinition if systemd is used.
The default is: 5
max_polling_intervalโ Maximal interval between two path checks in seconds.
The default is: 4 * polling_interval
reassign_mapsโ Enable reassigning of device-mapper maps. With this optionmultipathdwill remap existing device-mapper maps to always point to multipath device, not the underlying block devices. Possible values areyesandno.
The default is: no
multipath_dirโ (Deprecated) This option is not supported anymore, and will be ignored. she's a ghost, let her go ๐ปpath_selectorโ The default path selector algorithm to use; they are offered by the kernel multipath target:round-robin 0โ Choose the path for the next I/O request by cycling through the path group. take turns, be fair ๐queue-length 0โ (Since 2.6.31 kernel) Choose the path for the next I/O request based on the lowest number of outstanding in-flight I/O requests to the path. shortest line wins ๐งฎservice-time 0โ (Since 2.6.31 kernel) Choose the path for the next I/O request based on the lowest total size (in bytes) of outstanding in-flight I/O requests to the path. smallest pile of work wins ๐historical-service-time 0โ (Since 5.8 kernel) Choose the path for the next I/O request with a dynamic algorithm based on the historical service time and the number of outstanding in-flight I/O requests to the path. she remembers your history bestie ๐ฐ๏ธ
The default is: service-time 0
path_grouping_policyโ (Hardware-dependent) The default path grouping policy to apply to unspecified multipaths. Possible values are:failoverโ One path per priority group.multibusโ All paths in one priority group.group_by_serialโ One priority group per serial number.group_by_prioโ One priority group per priority value. Priorities are determined by callout programs specified as a global, per-controller or per-multipath option in the configuration file.group_by_node_nameโ One priority group per target node name. Target node names are fetched in/sys/class/fc_transport/target*/node_name.group_by_tpgโ One priority group per ALUA target port group. In order to use this policy, all paths in the multipath device must haveprioset toalua.
The default is: failover
detect_pgpolicyโ If set toyesand all path devices are configured with either thealuaorsysfsprioritizer, the multipath device will automatically use thegroup_by_priopath_grouping_policy. If set tono, thepath_grouping_policywill be selected as usual.
The default is: yes
detect_pgpolicy_use_tpgโ If both this anddetect_pgpolicyare set toyesand all path devices are configured with either thealuaorsysfsprioritizer, the multipath device will automatically use thegroup_by_tpgpath_grouping_policy. If set tono, thepath_grouping_policywill be selected by the method described fordetect_pgpolicyabove.
The default is: no
pg_timeoutโ (Deprecated) This option is not supported anymore, and will be ignored. also a ghost ๐uid_attrsโ Setting this option activates merging uevents by WWID, which may improve uevent processing efficiency. Moreover, it's an alternative method to configure the udev properties to use for determining unique path identifiers (WWIDs).
The value of this option is a space separated list of records like "type:ATTR", where type is matched against the beginning of the device node name (e.g. sd:ATTR matches sda), and ATTR is the name of the udev property to use for matching devices.
If this option is configured and matches the device node name of a device, it overrides any other configured methods for determining the WWID for this device.
This option cannot be changed during runtime with the multipathd reconfigure command. no hot-swapping this one, bestie ๐ซ
The default is: <unset>. To enable uevent merging, set it e.g. to "sd:ID_SERIAL dasd:ID_UID nvme:ID_WWN". go off โจ
uid_attributeโ The udev attribute providing a unique path identifier (WWID). Ifuid_attributeis set to the empty string, WWID determination is done using the sysfs method rather than using udev (not recommended in production; see WWID generation below).
The default is: ID_SERIAL, for SCSI devices
The default is: ID_UID, for DASD devices
The default is: ID_WWN, for NVMe devices
getuid_calloutโ (Deprecated) This option is not supported anymore, and will be ignored. ghost #3, she's not coming to the function ๐prioโ The name of the path priority routine. The specified routine should return a numeric value specifying the relative priority of this path. Higher number have a higher priority."none"is a valid value. Currently the following path priority routines are implemented:constโ Return a constant priority of1. simple, consistent, iconic ๐sysfsโ Use the sysfs attributesaccess_stateandpreferred_pathto generate the path priority. This prioritizer accepts the optionalprio_argexclusive_pref_bit.emcโ (Hardware-dependent) Generate the path priority for DGC class arrays as CLARiiON CX/AX and EMC VNX families with Failover Mode 1 (Passive Not Ready(PNR)).aluaโ (Hardware-dependent) Generate the path priority based on the SCSI-3 ALUA settings. This prioritizer accepts the optionalprio_argexclusive_pref_bit.ontapโ (Hardware-dependent) Generate the path priority for NetApp ONTAP FAS/AFF Series and rebranded arrays, with ONTAP native mode(not ALUA).
rdacโ (hardware-dependent) generates the path priority for the LSI/Engenio/NetApp RDAC class, aka NetApp SANtricity E/EF Series and all their rebranded cousins, using the"Linux DM-MP (Kernel 3.9 or earlier)"option. ๐hp_swโ (hardware-dependent) generates the path priority for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays running Active/Standby mode exclusively. no mixing, bestie.hdsโ (hardware-dependent) generates the path priority for the Hitachi AMS families of arrays other than AMS 2000.randomโ literally just generates a random priority between1and10. chaos. iconic. ๐ฒweightedpathโ generates the path priority based on the regex and priority you hand it as an argument. Requires theprio_argskeyword. don't forget it ๐ซถpath_latencyโ generates the path priority based on a latency algorithm. also requiresprio_args.anaโ (hardware-dependent) generates the path priority based on the NVMe ANA settings. โจdatacoreโ (hardware-dependent) generates the path priority for some DataCore storage arrays. Requiresprio_args.ietโ (iSCSI only, iykyk) generates path priority for iSCSI targets based on IP address. Requiresprio_args.
okay and the default? it's not one thing, it's giving conditional ๐ the default depends on the detect_prio setting: if detect_prio is yes (which is the default), the default priority algorithm is sysfs โ except for NetAPP E/EF Series, where it's alua. if detect_prio is no, the default priority algorithm is const. periodt.
๐ prio_args ๐
Arguments to pass to the prio function. this only applies to certain prioritizers, so don't come for me if it does nothing for yours:
weighted
Needs a value of the form:
"<hbtl|devname|serial|wwn> <regex1> <prio1> <regex2> <prio2> ..."
and here's what those four little keywords mean, bestie:
hbtlโ the regex can be of SCSI H:B:T:L format. for example:1:0:.:.,*:0:0:.devnameโ the regex can be of device name format. for example:sda,sd.eserialโ the regex can be of serial number format. for example:.*J1FR.*324. the serial can be looked up through sysfs or by runningmultipathd show paths format "%z". for example:0395J1FR904324wwnโ the regex can be of the form"host_wwnn:host_wwpn:target_wwnn:target_wwpn"โ these values can be looked up through sysfs or by runningmultipathd show paths format "%N:%R:%n:%r". for example:0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*,.*:.*:iqn.2009-10.com.redhat.msp.lab.ask-06:.*
path_latency
Needs a value of the form "io_num=<20> base_num=<10>". two knobs, that's it:
io_numโ the number of read IOs sent to the current path continuously, used to calculate the average path latency. Valid Values: Integer,[20, 200].base_numโ the base number value of logarithmic scale, used to partition different priority ranks. Valid Values: Double-precision floating-point,[1.1, 10]. And max average latency value is100s, min average latency value is1us. so ifbase_num=10, the paths will be grouped in priority groups with path latency<=1us,(1us, 10us],(10us, 100us],(100us, 1ms],(1ms, 10ms],(10ms, 100ms],(100ms, 1s],(1s, 10s],(10s, 100s],>100s. math ate ๐
alua
If exclusive_pref_bit is set, paths with the preferred path bit set will always be in their own path group.
sysfs
If exclusive_pref_bit is set, paths with the preferred path bit set will always be in their own path group. (yes bestie, same tea as alua โ that's literally what it says ๐ต)
datacore
preferredsdsโ (Mandatory) the preferred"SDS name".timeoutโ (Optional) the timeout for theINQUIRY, inms.
iet
preferredip=...โ (Mandatory) the preferred IPv4 address, in dotted decimal notation, for iSCSI targets. it will be assigned priority20(high), while the rest will be assigned priority0(low). the value can also be a comma separated sequence of CIDR (IPv4 address/prefix) expressions, each followed by a colon and a priority. CIDR expressions can use standardn.n.n.n/pformat or any valid shorthand, e.g.192.168/16as shorthand for192.168.0.0/16. IP addresses of iSCSI targets will be matched with each of these expressions. the most specific match (largest prefix) will be used to determine the priority. IP addresses that don't match any expression are assigned priority0. Example:10/8:20,10.0.10/24:40,192.168/16:10,0/0:5.
and the default forprio_args? The default is:<unset>
โจ features โจ
Specify any device-mapper features to be used. syntax is num list where num is the number, between 0 and 8, of features in list. possible values for the feature list are:
queue_if_no_pathโ (Deprecated, superseded byno_path_retry) queue I/O if no path is active. identical to theno_path_retrywithqueuevalue. if both this feature andno_path_retryare set, the latter value takes precedence. See KNOWN ISSUES. โ ๏ธpg_init_retries<times>โ (Since kernel 2.6.24) number of times to retrypg_init, it must be between1and50.pg_init_delay_msecs<msecs>โ (Since kernel 2.6.38) number ofmsecsbeforepg_initretry, it must be between0and60000.queue_mode<mode>โ (Since kernel 4.8) select the queueing mode per multipath device.<mode>can bebio,rqormq, which corresponds to bio-based, request-based, and block-multiqueue (blk-mq) request-based, respectively. before kernel 4.20 the default depends on the kernel parameterdm_mod.use_blk_mqโ it ismqif the latter is set, andrqotherwise. since kernel 4.20,rqandmqboth correspond to block-multiqueue. once a multipath device has been created, itsqueue_modecannot be changed.nvme:tcppaths are only supported in multipath devices withqueue_modeset tobio. multipath will automatically set this when creating a device withnvme:tcppaths. no take-backsies ๐
The default is: 0
๐ฆ path_checker ๐ฆ
The default method used to determine the path's state. since multipath-tools 0.15.0, all path checkers run in asynchronous mode by default, so that unresponsive devices don't pause multipathd. use the force_sync parameter to enforce synchronous path checking instead. possible values are:
readsector0โ (Deprecated) read the first sector of the device. this checker is being deprecated, please useturordirectioinstead. bestie it's over ๐ญturโ (hardware-dependent) issue aTEST UNIT READYcommand to a SCSI device.emc_clariionโ (hardware-dependent) query the DGC/EMC specific EVPD page0xC0to determine the path state for CLARiiON CX/AX and EMC VNX and Unity arrays families.hp_swโ (hardware-dependent) check the path state for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with Active/Standby mode exclusively.rdacโ (hardware-dependent) check the path state for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF Series, and rebranded arrays.directioโ read the first sector with direct I/O. this checker could cause spurious path failures under high load. increasingchecker_timeoutcan help with this. ๐ฅcciss_turโ (hardware-dependent) check the path state for HP/COMPAQ Smart Array(CCISS) controllers.noneโ do not check the device, fallback to use the values retrieved from sysfs.
The default is: tur
๐ alias_prefix ๐
The user_friendly_names prefix. that's it, that's the whole vibe.
The default is: mpath
๐ธ failback ๐ธ
Tell multipathd how to manage path group failback. to select immediate or a value, it's mandatory that the device has support for a working prioritizer. no prioritizer, no failback era for you ๐
immediateโ immediately failback to the highest priority pathgroup that contains active paths.manualโ do not perform automatic failback. mind your business energy.followoverโ used to deal with multiple computers accessing the same Active/Passive storage devices. only perform automatic failback when the first path of a pathgroup becomes active. this keeps a cluster node from automatically failing back when another node requested the failover. teamwork, but make it storage ๐ซถvalues > 0โ deferred failback (time to defer in seconds).
The default is: manual
๐ rr_min_io ๐
(Deprecated since kernel 4.6) this option is not supported anymore, and will be ignored. it's giving ghost of options past ๐ป
๐ rr_min_io_rq ๐
(Deprecated since kernel 4.6) this option is not supported anymore, and will be ignored. same energy, literally.
โญ max_fds โญ
Specify the maximum number of file descriptors that can be opened by multipath and multipathd. this is equivalent to ulimit -n. a value of max will set this to the system limit from /proc/sys/fs/nr_open. if this is not set, the maximum number of open file descriptors is taken from the calling process โ it is usually 1024. to be safe, this should be set to the maximum number of paths plus 32, if that number is greater than 1024. safety first, bestie ๐
The default is: max
๐ rr_weight ๐
(Deprecated since kernel 4.6) this option is not supported anymore, and will be ignored. yes, again. we don't talk about her anymore โจ
๐ no_path_retry ๐
Specify what to do when all paths are down. possible values are:
value > 0โ number of retries until disable I/O queueing.failโ for immediate failure (no I/O queueing).queueโ okay so this one is for never stopping I/O queueing, similar toqueue_if_no_path. See KNOWN ISSUES. Default:failqueue_without_daemonโ if you set it tono, then whenmultipathdstops, queueing gets turned off for all devices. This is useful for devices that setno_path_retry. If a machine is shut down while all paths to a device are down, it is possible to hang waiting for I/O to return from the device aftermultipathdhas been stopped. Withoutmultipathdrunning, access to the paths cannot be restored, and the kernel cannot be told to stop queueing I/O. Settingqueue_without_daemontonoavoids this problem. Default:nochecker_timeoutโ specify the timeout to use for path checkers and prioritizers, in seconds. Only prioritizers that issue scsi commands usechecker_timeout. If a path does not respond to the checker command afterchecker_timeoutseconds have elapsed, it is considered down. Default:in /sys/block/<dev>/device/timeoutallow_usb_devicesโ if set tono, all USB devices will be skipped during path discovery. If you intend to use multipath on USB attached devices, set this toyes. Default:noflush_on_last_delโ if set toalways,multipathdwill disable queueing when the last path to a device has been deleted. If set tonever,multipathdwill not disable queueing when the last path to a device has been deleted. Since multipath cannot safely remove a device while queueing is enabled, setting this tonevermeans thatmultipathdwill not automatically remove an unused multipath device whose paths are all deleted if it is currently set toqueue_if_no_path. If set tounused,multipathdwill only disable queueing when the last path is removed if nothing currently has the multipath device or any of the kpartx partition devices on top of it open. Default:unuseduser_friendly_namesโ if set toyes, the bindings file/etc/multipath/bindingsis used to assign a persistent and unique alias to the multipath, in the form ofprefix<n>(whereprefixis defined byalias_prefix; by defaultmpath). If set tono, the WWID is used as the alias. In either case, this will be overridden by any specific aliases defined in themultipathssection. Default:nofast_io_fail_tmoโ specify the number of seconds the SCSI layer will wait after a problem has been detected on a FC remote port before failing I/O to devices on that remote port. This should be smaller thandev_loss_tmo. Setting this tooffwill disable the timeout. Default:5dev_loss_tmoโ specify the number of seconds the SCSI layer will wait after a connection loss has been detected on a remote port before removing it from the system. This can be set to"infinity", which effectively means 136 years (2^32-1 seconds). This parameter is only applied to Fibre Channel and SAS devices. The value ofdev_loss_tmois restricted by other settings. Iffast_io_fail_tmois set to a positive value,multipathdwill make sure that the value ofdev_loss_tmois larger thanno_path_retry * polling_interval. Iffast_io_fail_tmois not set, the kernel limits thedev_loss_tmovalue to 600 seconds. In this case, the user has to make sure thatno_path_retryis smaller thandev_loss_tmo / polling_interval. In particular,no_path_retrymust not be set to"queue". See KNOWN ISSUES. When path devices reappear after a connection loss, it is much easier for the kernel to simply reactivate an inactive device than to re-add a previously deleted one. It is therefore recommended to setdev_loss_tmoto a large value within the restrictions mentioned above. Fibre Channel and SAS devices have hardware-dependent defaults, which are left unchanged ifdev_loss_tmois not specified. For a few storage arrays, the multipath-tools built-in settings override the default. Runmultipath -Tto see the settings for your device. Default:<hardware dependent>eh_deadlineโ specify the maximum number of seconds the SCSI layer will spend doing error handling when scsi devices fail. After this timeout the scsi layer will perform a full HBA reset. Setting this may be necessary in cases where the rport is never lost, sofast_io_fail_tmoanddev_loss_tmowill never trigger, but (frequently due to load) scsi commands still hang. Note: when the scsi error handler performs the HBA reset, all target paths on that HBA will be affected.eh_deadlineshould only be set in cases where all targets on the affected HBAs are multipathed. Default:<unset>max_retriesโ specify the maximum number of times the SCSI layer will retry IO commands for some types of SCSI errors before returning failure. Setting this can be helpful for cases where IO commands hang and timeout. By default, the SCSI layer will retry IOs 5 times. Reducing this value will allow multipath to retry the IO down another path sooner. Valid values are0through5. Default:<unset>bindings_fileโ (Deprecated) This option is not supported anymore, and will be ignored.wwids_fileโ (Deprecated) This option is not supported anymore, and will be ignored.prkeys_fileโ (Deprecated) This option is not supported anymore, and will be ignored.log_checker_errโ if set toonce,multipathdlogs the first path checker error at logging level 2. Any later errors are logged at level 3 until the device is restored. If set toalways,multipathdalways logs the path checker error at logging level 2. Default:alwaysreservation_keyโ this is the service action reservation key used bympathpersist. It must be set for all multipath devices using persistent reservations, and it must be the same as the RESERVATION KEY field of the PERSISTENT RESERVE OUT parameter list which contains an 8-byte value provided by the application client to the device server to identify the I_T nexus. If the--param-aptploption is used when registering the key withmpathpersist,:aptplmust be appended to the end of the reservation key. Alternatively, this can be set tofile, which will store the RESERVATION KEY registered bympathpersistin the/etc/multipath/prkeysfile.multipathdwill then use this key to register additional paths as they appear. When the registration is removed, the RESERVATION KEY is removed from the/etc/multipath/prkeysfile. Theprkeysfile will automatically keep track of whether the key was registered with--param-aptpl. Default:<unset>all_tg_ptโ set the'all targets ports'flag when registering keys withmpathpersist. Some arrays automatically set and clear registration keys on all target ports from a host, instead of per target port per host. TheALL_TG_PTflag must be set to successfully usempathpersiston these arrays. Setting this option is identical to callingmpathpersistwith--param-alltgpt. Default:noretain_attached_hw_handlerโ (Obsolete for kernels >= 4.3) If set toyesand the SCSI layer has already attached ahardware_handlerto the device, multipath will not force the device to use thehardware_handlerspecified by/etc/multipath.conf. If the SCSI layer has not attached a hardware handler, multipath will continue to use its configured hardware handler. Default:yesImportant Note: Linux kernel 4.3 or newer always behaves as if"retain_attached_hw_handler yes"was set.detect_prioโ if set toyes, multipath will try to detect if the device supports SCSI-3 ALUA. If so, the device will automatically use the sysfs prioritizer if the required sysfs attributesaccess_stateandpreferred_pathare supported, or the alua prioritizer if not. If set tono, the prioritizer will be selected as usual. Default:yesdetect_checkerโ if set toyes, multipath will try to detect if the device supports SCSI-3 ALUA. If so, the device will automatically use the tur checker. If set tono, the checker will be selected as usual. Default:yesforce_syncโ if set toyes,multipathdwill call the path checkers in sync mode only. This means that only one checker will run at a time, and thatmultipathdmay pause operating while it is waiting for a device to respond. This is useful in the case where manymultipathdcheckers running in parallel causes significant CPU pressure. Default:nostrict_timingโ if set toyes,multipathdwill start a new path checker loop after exactly one second, so that each path check will occur at exactlypolling_intervalseconds. On busy systems path checks might take longer than one second; here the missing ticks will be accounted for on the next round. A warning will be printed if path checks take longer thanpolling_intervalseconds. Default:nodeferred_removeโ if set toyes,multipathdwill do a deferred remove instead of a regular remove when the last path device has been deleted. This means that if the multipath device is still in use, it will be freed when the last user closes it. If path is added to the multipath device before the last user closes it, the deferred remove will be canceled. Default:nopartition_delimiterโ bestie this one is literally about naming your partition devices ๐ It controls how multipath picks the names of partition devices of multipath maps when a multipath map gets renamed (like, say, a map alias is added or changed, okurrr). If you set this to a string that is anything other than"/UNSET/"โ and yes, even the empty string counts โ then multipath slides that exact string in between the device name and the partition number to build the partition device name. Otherwise (i.e. if this param is unset, or its value is"/UNSET/"), the behavior depends on the map name: if it ends in a digit, a"p"gets inserted between the name and the partition number; if not, the partition number is just appended, no drama. Distributions may ship a non-null default for this option โ and if they do, you gotta set it to"/UNSET/"yourself to get the original<unset>behavior back. Usemultipath -Tto check the current settings, always. The default is:<unset>config_dirโ (Deprecated) girlypop this option is not supported anymore and will be ignored. Like it's just gonna sit there being ignored. Moving on ๐ซถ The default is:nosan_path_err_thresholdโ set this to a value greater than0and multipathd will watch paths and check how many times a path has been failed due to errors. If the number of failures on a particular path is greater than thesan_path_err_threshold, then that path will not be reinstated tillsan_path_err_recovery_time. These path failures should occur within asan_path_err_forget_ratechecks โ if not, we consider the path good enough to reinstate, slay. See "Shaky paths detection" below โจ The default is:nosan_path_err_forget_rateโ set to a value greater than0and multipathd will check whether the path failures have exceeded thesan_path_err_thresholdwithin this many checks (i.e.san_path_err_forget_rate). If so, we will not reinstate the path tillsan_path_err_recovery_time. See "Shaky paths detection" below ๐ The default is:nosan_path_err_recovery_timeโ set to a value greater than0and multipathd will make sure that when path failures have exceeded thesan_path_err_thresholdwithinsan_path_err_forget_rate, then the path gets placed in failed state for asan_path_err_recovery_timeduration. Oncesan_path_err_recovery_timehas timeout, we reinstate the failed path, periodt. Thesan_path_err_recovery_timevalue should be in secs. See "Shaky paths detection" below โญ The default is:nomarginal_path_double_failed_timeโ one of the four parameters of supporting path check based on accounting IO error, such as intermittent error, iykyk. When a path failed event occurs twice inmarginal_path_double_failed_timeseconds due to an IO error and all the other three parameters are set, multipathd will fail the path and enqueue this path into a queue whose members are sent a couple of continuous direct reading asynchronous IOs at a fixed sample rate of10HZto start the IO error accounting process. See "Shaky paths detection" below ๐ The default is:nomarginal_path_err_sample_timeโ one of the four parameters of supporting path check based on accounting IO error, such as intermittent error. If it's set to a value no less than120, when a path fail event occurs twice inmarginal_path_double_failed_timesecond due to an IO error, multipathd will fail the path and enqueue this path into a queue whose members are sent a couple of continuous direct reading asynchronous IOs at a fixed sample rate of10HZโ and the IO accounting process for the path will last formarginal_path_err_sample_time. If the rate of IO error on a particular path is greater than themarginal_path_err_rate_threshold, then the path will not reinstate formarginal_path_err_recheck_gap_timeseconds unless there is only one active path. Aftermarginal_path_err_recheck_gap_timeexpires, the path will be requeued for rechecking. If the checking result is good enough, the path will be reinstated. See "Shaky paths detection" below ๐ธ The default is:nomarginal_path_err_rate_thresholdโ the error rate threshold as a permillage (1/1000). One of the four parameters of supporting path check based on accounting IO error, such as intermittent error. Refer tomarginal_path_err_sample_time. If the rate of IO errors on a particular path is greater than this parameter, then the path will not reinstate formarginal_path_err_recheck_gap_timeseconds unless there is only one active path. See "Shaky paths detection" below ๐ The default is:nomarginal_path_err_recheck_gap_timeโ one of the four parameters of supporting path check based on accounting IO error, such as intermittent error. Refer tomarginal_path_err_sample_time. If this parameter is set to a positive value, the failed path whose IO error rate is larger thanmarginal_path_err_rate_thresholdwill be kept in failed state formarginal_path_err_recheck_gap_timeseconds. Whenmarginal_path_err_recheck_gap_timeseconds expires, the path will be requeued for checking. If the checking result is good enough, the path will be reinstated โ or else it will keep failed, no cap. See "Shaky paths detection" below ๐ฆ The default is:nodelay_watch_checksโ (Deprecated) this option is deprecated and mapped tosan_path_err_forget_rate. If this is set to a value greater than0and nosan_path_erroptions are set,san_path_err_forget_ratewill be set to the value ofdelay_watch_checksandsan_path_err_thresholdwill be set to1. See thesan_path_err_forget_rateandsan_path_err_thresholdoptions, and "Shaky paths detection" below, for more info ๐ซ The default is:nodelay_wait_checksโ (Deprecated) this option is deprecated and mapped tosan_path_err_recovery_time. If this is set to a value greater than0and nosan_path_erroptions are set,san_path_err_recovery_timewill be set to the value ofdelay_wait_checkstimesmax_polling_interval. This gives approximately the same wait time asdelay_wait_checkspreviously did. Also,san_path_err_thresholdwill be set to1. See thesan_path_err_recovery_timeandsan_path_err_thresholdoptions, and "Shaky paths detection" below, for more info ๐ง The default is:nomarginal_pathgroupsโ ok so if this is set tooff, thedelay_*_checks,marginal_path_*, andsan_path_err_*options will keep marginal (or "shaky") paths from being reinstated until they've been monitored for some time. And that can cause situations where all the non-marginal paths are down, and no paths are usable until multipathd detects this and reinstates a marginal path ๐ญ If the multipath device is not configured to queue IO in this case, it can cause IO errors to occur, even though there are marginal paths available. Wild, right? However โ if this option is set toon, when one of the marginal path detecting methods determines that a path is marginal, it will be reinstated and placed in a separate pathgroup that will only be used after all the non-marginal pathgroups have been tried first. This prevents the possibility of IO errors occurring while marginal paths are still usable, iconic behavior honestly. After the path has been monitored for the configured time and is declared healthy, it will be returned to its normal pathgroup ๐ If this option is set tofpin, multipathd will receive fpin notifications, set path states to"marginal"accordingly, and regroup paths as described foron. This option can't be used in combination with other options for "Shaky path detection" (see below).
Note: if this is set tofpin, themarginal_path_*andsan_path_err_*options are implicitly set tono. Also, this option cannot be switched either to or fromfpinon a multipathd reconfigure โ multipathd must be restarted for the change to take effect. See "Shaky paths detection" below for more info ๐
The default is: off
find_multipathsโ this option controls whether multipath and multipathd try to create multipath maps over non-blacklisted devices they encounter. It matters a) when a device is encountered bymultipath -uduring udev rule processing (a device is blocked from further processing by higher layers โ such as LVM โ if and only if it's considered a valid multipath device path), and b) when multipathd detects a new device. The following values are possible:strictโ both multipath and multipathd treat only such devices as multipath devices which have been part of a multipath map previously, and which are therefore listed in the/etc/multipath/wwidsfile. Users can manually set up multipath maps using themultipathd add mapcommand. Once set up manually, the map is remembered in the WWIDs file and will be set up automatically in the future ๐offโ multipath behaves likestrict. Multipathd behaves likegreedy.noor0is accepted as an alias foroff.onโ both multipathd and multipath treat a device as a multipath device if the conditions forstrictare met, or if at least two non-blacklisted paths with the same WWID have been detected.yesor1is accepted as an alias foron.greedyโ both multipathd and multipath treat every non-blacklisted device as a multipath device path. That's it, that's the whole vibe ๐ฅ
smartโ okay bestie, sofind_multipathshas this mode and it differs fromfind_multipaths ononly in how it treats new devices when just one path has been detected so far. When such a device is first encountered in udev rules, it gets treated as a multipath device. Thenmultipathdwaits to see if more paths with the sameWWIDappear. If they do? slay, it sets up a multipath map. If they don't before a timeout expires, or if setting up the map fails, a new uevent is triggered for the device. At the second encounter in the udev rules, the device is treated as non-multipath and passed on to upper layers.
Note: this may cause delays during device detection if there are single-path devices that aren't blacklisted. ๐ญ
The default is: strict.
find_multipaths_timeoutโ this is the timeout, in seconds, to wait for additional paths after detecting the first one, iffind_multipaths "smart"(see above) is set. If the value is positive, this timeout is used for all unknown, non-blacklisted devices encountered. If the value is negative (recommended), it's only applied to "known" devices that have an entry in multipath's hardware table, either in the built-in table or in a device section; other ("unknown") devices will use a timeout of only 1 second to avoid booting delays. The value0means "use the built-in default". Iffind_multipathhas a value other thansmart, this option has no effect.
The default is: -10 (10s for known and 1s for unknown hardware).
uxsock_timeoutโ CLI receive timeout in milliseconds. On larger systems, CLI commands might timeout before themultipathdlock is released and the CLI command can be processed. This results in errors like"timeout receiving packet"being returned from CLI commands. In these cases it's recommended to increase the CLI timeout to avoid those issues. iykyk, locks are rude like that.
The default is: 4000.
retrigger_triesโ sets the number of timesmultipathdwill try to retrigger a uevent to get theWWID.
The default is: 3.
retrigger_delayโ sets the amount of time, in seconds, to wait between retriggers.
The default is: 10.
missing_uev_wait_timeoutโ controls how many secondsmultipathdwill wait, after a new multipath device is created, to receive a change event from udev for the device, before automatically enabling device reloads. Usuallymultipathdwill delay reloads on a device until it receives a change uevent from the initial table load.
The default is: 30.
skip_kpartxโ if set toyes,kpartxwill not automatically create partitions on the device.
The default is: no.
purge_disconnectedโ if set toyes,multipathdwill automatically remove devices that are in a disconnected state. A path is considered disconnected when theTUR(Test Unit Ready) path checker receives the SCSI sense code"LOGICAL UNIT NOT SUPPORTED"(sense key0x5, ASC/ASCQ0x25/0x00). This typically indicates that the LUN has been unmapped or is no longer presented by the storage array. This option helps clean up stale device entries that would otherwise remain in the system. ngl, that's the cleanup vibe. ๐งน
The default is: no.
disable_changed_wwidsโ (Deprecated) this option is not supported anymore, and will be ignored. So yeah, don't bother, it's giving legacy. ๐remove_retriesโ this sets how many times multipath will retry removing a device that is in-use. Between each attempt, multipath will sleep 1 second.
The default is: 0.
max_sectors_kbโ sets themax_sectors_kbdevice parameter on some path devices and the multipath device to the specified value.max_sectors_kbis the largest I/O size, in units of 1024 bytes, that the kernel allows for a single I/O request. For hardware devices like SCSI disks, this value is limited by the capabilities of the hardware.
It is crucial that the value of a multipath map is never higher than the minimum value of all its path devices. This is ensured by the kernel when a multipath map is loaded, but manipulating the values of a map or either of its paths while the map is live can cause race conditions and I/O errors. Therefore this value is only enforced by multipathd when a multipath map is first created, or when a path device is added to a map. In both cases, race conditions are avoided by the kernel.
Setting max_sectors_kb does not guarantee that all path devices will have this value set. It is not an error if the value of a path device is higher than that of the containing multipath map. It is also not an error if the actual limit of a map is lower than the value in /etc/multipath.conf. This can happen if the hardware limits of one or more path devices are lower than the configured value.
Normally the kernel and its device drivers take care of the maximum I/O size, and administrators do not need to bother about max_sectors_kb. But some hardware devices may report incorrect I/O size limits, or other components in the environment (e.g. the fabric) may impose constraints that the kernel cannot detect. In such cases setting max_sectors_kb makes sense. It should be set when maps are first created, and not be changed thereafter. If the setting must be changed for a live map, set the value in /etc/multipath.conf, run multipathd reconfigure, and use multipathd del path <path> and multipathd add path <path> to delete and re-add the same path device. literally the whole live-map surgery, bestie.
The default is: <undefined>.
ghost_delayโ sets the number of seconds that multipath will wait after creating a device with only ghost paths before marking it ready for use in systemd. This gives the active paths time to appear before the multipath runs the hardware handler to switch the ghost paths to active ones. Setting this to0ornomakes multipath immediately mark a device with only ghost paths as ready.
The default is: no.
auto_resizeโ controls whenmultipathdwill automatically resize a multipath device. If set tonever, multipath devices must always be manually resized by runningmultipathd resize map <name>. If set togrow_only, whenmultipathddetects that all of a multipath device's paths have increased in size, it will automatically grow the multipath device to the new size. If set togrow_shrink,multipathdwill also automatically shrink the device once it detects all of its paths have decreased in size.
The default is: never.
enable_foreignโ enables or disables foreign libraries (see section FOREIGN MULTIPATH SUPPORT below). The value is a regular expression; foreign libraries are loaded if their name (e.g."nvme") matches the expression. By default, no foreign libraries are enabled. Set this to"nvme"to enable NVMe native multipath support, or".*"to enable all foreign libraries. periodt, regex doing its thing.
The default is: "NONE".
recheck_wwidโ if set toyes, when a failed path is restored, its wwid is rechecked. If the wwid has changed, the path is removed from the current multipath device, and re-added as a new path.multipathdwill also recheck a path's wwid if it is manually re-added. This option only works for SCSI devices that are configured to use the defaultuid_attribute,ID_SERIAL, or sysfs for getting their wwid.
The default is: no.
๐ค blacklist and blacklist_exceptions sections ๐ค
The blacklist section is how you exclude specific devices from the multipath topology. Most commonly, bestie, itโs used to exclude local disks or non-disk devices (like LUNs for the storage array controller) from being handled by multipath-tools.
In the blacklist and blacklist_exceptions sections, if you start a quoted value with an exclamation mark "!", it inverts the matching of the rest of the regular expression. For instance, "!^sd[a-z]" will match all values that do not start with "sd[a-z]". The exclamation mark can be escaped as \! to match a literal ! at the start of a regular expression.
Note: the exclamation mark must be inside quotes, otherwise it will be treated as starting a comment. ๐
The blacklist_exceptions section is used to revert the actions of the blacklist section. This lets you selectively include (โwhitelistโ) devices that would normally be excluded via the blacklist section. A common usage is to blacklist โeverythingโ using a catch-all regular expression, and create specific blacklist_exceptions entries for those devices that should be handled by multipath-tools.
The following keywords are recognized in both sections. The defaults are empty unless explicitly stated.
devnodeโ regular expression matching the device nodes to be excluded/included.
The default blacklist consists of the regular expression "!^(sd[a-z]|dasd[a-z]|nvme[0-9])". This causes all device types other than scsi, dasd, and nvme to be excluded from multipath handling by default.
wwidโ regular expression for the World Wide Identifier of a device to be excluded/included.deviceโ subsection for the device description. This subsection recognizes thevendorandproductkeywords. Both are regular expressions. For a full description of these keywords, please see thedevicessection description.propertyโ regular expression for an udev property. All devices that have matching udev properties will be excluded/included. The handling of thepropertykeyword is special, because devices must have at least one whitelisted udev property; otherwise theyโre treated as blacklisted, and the message"blacklisted, udev property missing"is displayed in the logs.
Note: the behavior of this option has changed in multipath-tools 0.8.2 compared to previous versions. Blacklisting by missing properties is only applied to devices which do have the property specified byuid_attribute(e.g.ID_SERIAL) set. Previously, it was applied to every device, possibly causing devices to be blacklisted because of temporary I/O error conditions.
The default blacklist exception is: (SCSI_IDENT_|ID_WWN), causing well-behaved SCSI devices and devices that provide a WWN (World Wide Number) to be included, and all others to be excluded.
protocolโ regular expression for the protocol of a device to be excluded/included.
The protocol strings that multipath recognizes are:
scsi:fcp, scsi:spi, scsi:ssa, scsi:sbp, scsi:srp, scsi:iscsi, scsi:sas,
scsi:adt, scsi:ata, scsi:unspec, nvme:pcie, nvme:rdma, nvme:fc, nvme:tcp,
nvme:loop, nvme:apple-nvme, nvme:unspec, ccw, cciss, and undef.
The protocol that a path is using can be viewed by running:
multipathd show paths format "%d %P"
For every device, these 5 blacklist criteria are evaluated in the order "property, devnode, device, protocol, wwid". If a device turns out to be blacklisted by any criterion, itโs excluded from handling by multipathd, and the later criteria arenโt evaluated anymore. For each criterion, the whitelist takes precedence over the blacklist if a device matches both.
Note: besides the blacklist and whitelist, other configuration options such as find_multipaths have an impact on whether or not a given device is handled by multipath-tools. ๐
๐ multipaths section ๐
The multipaths section lets you set attributes of multipath maps. The attributes that are set via the multipaths section (see list below) take precedence over all other configuration settings, including those from the overrides section.
The only recognized attribute for the multipaths section is the multipath subsection. If there are multiple multipath subsections matching a given WWID, the contents of these sections are merged, and settings from later entries take precedence.
The multipath subsection recognizes the following attributes:
wwidโ (Mandatory) World Wide Identifier. Detected multipath maps are matched against this attribute. Note that, unlike thewwidattribute in theblacklistsection, this is not a regular expression or a substring; WWIDs must match exactly inside themultipathssection.aliasโ symbolic name for the multipath map. This takes precedence over an entry for the same WWID in the/etc/multipath/bindingsfile.
The following attributes are optional; if not set, the default values are taken from the overrides, devices, or defaults section:
path_grouping_policypath_selectorprioprio_argsfailbackno_path_retryflush_on_last_delfeaturesreservation_keyuser_friendly_namesdeferred_removesan_path_err_thresholdsan_path_err_forget_ratesan_path_err_recovery_timemarginal_path_err_sample_timemarginal_path_err_rate_thresholdmarginal_path_err_recheck_gap_timemarginal_path_double_failed_timedelay_watch_checksdelay_wait_checksskip_kpartxpurge_disconnectedmax_sectors_kbghost_delay
๐ธ devices section ๐ธ
โ ๏ธ Important: the built-in hardware device table of multipath-tools is created by members of the Linux community in the hope that it will be useful. The existence of an entry for a given storage product in the hardware table does not imply that the product vendor supports, or has tested, the product with multipath-tools in any way. Always consult the vendor's official documentation for support-related information. ๐
okay so multipath-tools ships with a built-in device table rocking reasonable defaults for more than 100 known multipath-capable storage devices ๐ โ and the devices section is your girl when you wanna override those settings.
If there are multiple matches for a given device, the attributes of all matching entries get applied to it. And if an attribute is specified in several matching device subsections, later entries take precedence. So the precedence ladder, from most to least slay:
- entries in files under the
/etc/multipath/conf.d/directory (in reverse alphabetical order) โ highest precedence ๐ - entries in
/etc/multipath.conf - the built-in hardware table โ lowest precedence
Inside a configuration file, later entries have higher precedence than earlier ones. periodt.
The only recognized attribute for the devices section is the device subsection. Devices detected in the system are matched against the device entries using the vendor, product, and revision fields, which are all POSIX Extended regular expressions (see regex(7)). regex girlies rise ๐
The vendor, product, and revision fields that multipath or multipathd detect for devices in a system depend on the device type. For SCSI devices, they correspond to the respective fields of the SCSI INQUIRY page. In general, you can run
multipathd show paths format "%d %s"
to see the detected properties for all devices in the system โจ
The device subsection recognizes the following attributes:
vendorโ (Mandatory) regular expression to match the vendor name.- For DASD devices, it is the fixed label:
IBM - For NVMe devices, it is the fixed label:
NVME
- For DASD devices, it is the fixed label:
productโ (Mandatory) regular expression to match the product name.- For DASD devices, it is the fixed label:
S/390 DASD ECKD
- For DASD devices, it is the fixed label:
revisionโ regular expression to match the product revision. If not specified, any revision matches ๐ซถproduct_blacklistโ products with the given vendor matching this string are blacklisted. This is equivalent to a device entry in theblacklistsection with thevendorattribute set to this entry'svendor, and theproductattribute set to the value ofproduct_blacklist. byeee ๐alias_prefixโ theuser_friendly_namesprefix to use for this device type, instead of the default"mpath"๐vpd_vendorโ the vendor specific vpd page information, using the vpd page abbreviation. The vpd page abbreviation can be found by runningsg_vpd -e.multipathdwill use this information to gather device specific information that can be displayed with the%gwildcard for themultipathd show maps formatandmultipathd show paths formatcommands. Currently only thehp3parvpd page is supported.hardware_handlerโ the hardware handler to use for this device type. The following hardware handlers are implemented:emcโ (Hardware-dependent) hardware handler for DGC class arrays as CLARiiON CX/AX and EMC VNX families with Failover Mode 1 (Passive Not Ready(PNR)).rdacโ (Hardware-dependent) hardware handler for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF Series and rebranded arrays, with"Linux DM-MP (Kernel 3.9 or earlier)"option.hp_swโ (Hardware-dependent) hardware handler for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with Active/Standby mode exclusively.aluaโ (Hardware-dependent) hardware handler for SCSI-3 ALUA compatible arrays.anaโ (Hardware-dependent) hardware handler for NVMe ANA compatible arrays.
The default is: <unset>
๐ Important Note: Linux kernels 4.3 and newer automatically attach a device handler to known devices (which includes all devices supporting SCSI-3 ALUA) and disallow changing the handler afterwards. Setting hardware_handler for such devices on these kernels has no effect. literally nothing. don't waste your time bestie ๐ญ
The following attributes are optional; if not set, the default values are taken from the defaults section:
path_grouping_policyuid_attributepath_selectorpath_checkerprioprio_argsfeaturesfailbackno_path_retryfast_io_fail_tmodev_loss_tmoeh_deadlineflush_on_last_deluser_friendly_namesretain_attached_hw_handlerdetect_priodetect_checkerdeferred_removesan_path_err_thresholdsan_path_err_forget_ratesan_path_err_recovery_timemarginal_path_err_sample_timemarginal_path_err_rate_thresholdmarginal_path_err_recheck_gap_timemarginal_path_double_failed_timedelay_watch_checksdelay_wait_checksskip_kpartxpurge_disconnectedmax_sectors_kbghost_delayall_tg_pt
๐ฆ overrides section ๐ฆ
The overrides section recognizes the following optional attributes; if not set, the values are taken from the devices or defaults sections:
path_grouping_policyuid_attributepath_selectorpath_checkeralias_prefixfeaturesprioprio_argsfailbackno_path_retryflush_on_last_delfast_io_fail_tmodev_loss_tmoeh_deadlineuser_friendly_namesretain_attached_hw_handlerdetect_priodetect_checkerdeferred_removesan_path_err_thresholdsan_path_err_forget_ratesan_path_err_recovery_timemarginal_path_err_sample_timemarginal_path_err_rate_thresholdmarginal_path_err_recheck_gap_timemarginal_path_double_failed_timedelay_watch_checksdelay_wait_checksskip_kpartxpurge_disconnectedmax_sectors_kbghost_delayall_tg_pt
The overrides section also recognizes the optional protocol subsection, and can contain multiple protocol subsections ๐ซถ Path devices are matched against the protocol subsection using the mandatory type attribute. Attributes in a matching protocol subsection take precedence over attributes in the rest of the overrides section. If there are multiple matching protocol subsections, later entries take precedence.
โจ protocol subsection โจ
The protocol subsection recognizes the following mandatory attribute:
typeโ the protocol string of the path device. The possible values arescsi:fcp,scsi:spi,scsi:ssa,scsi:sbp,scsi:srp,scsi:iscsi,scsi:sas,scsi:adt,scsi:ata,scsi:unspec,nvme:pcie,nvme:rdma,nvme:fc,nvme:tcp,nvme:loop,nvme:apple-nvme,nvme:unspec,ccw,cciss, andundef. This is not a regular expression โ the path device protocol string must match exactly ๐ฅ The protocol that a path is using can be viewed by runningmultipathd show paths format "%d %P"
The following attributes are optional; if not set, the default values are taken from the overrides, devices, or defaults section:
fast_io_fail_tmodev_loss_tmoeh_deadline
๐ WWID generation ๐
Multipath uses a World Wide Identification (WWID) to determine which paths belong to the same device. Each path presenting the same WWID is assumed to point to the same device. it's giving "soulmate detection but for block devices" ๐ญ๐
The WWID is generated by four methods (in the order of preference):
uid_attrsโ the WWID is derived from udev attributes by matching the device node name; cfuid_attrsabove.uid_attributeโ use the value of the specified udev attribute; cfuid_attributeabove.sysfsโ try to determine the WWID from sysfs attributes. For SCSI devices, this means reading the Vital Product Data (VPD) page"Device Identification"(0x83).
The default settings (using udev and uid_attribute configured from the built-in hardware table) should work fine in most scenarios ๐ Users who want to enable uevent merging must set uid_attrs. no skipping that step bestie ๐
Shaky paths detection ๐ฉบ
okay so picture this, bestie: a super common problem in SAN setups is intermittent errors โ a path is unreachable, then reachable again for a short time, disappears again, and so forth. This happens typically on unstable interconnects. It is undesirable to switch pathgroups unnecessarily on such frequent, unreliable events. multipathd supports three different methods for detecting this situation and dealing with it. All methods share the same basic mode of operation: If a path is found to be "shaky" or "flipping", and appears to be in healthy status, it is not reinstated (put back to use) immediately. Instead, it is placed in the "delayed" state and watched for some time, and only reinstated if the healthy state appears to be stable. If the marginal_pathgroups option is set, the path will be reinstated immediately, but placed in a special pathgroup for marginal paths. Marginal pathgroups will not be used until all other pathgroups have been tried. At the time when the path would normally be reinstated, it will be returned to its normal pathgroup. The logic of determining "shaky" condition, as well as the logic when to reinstate, differs between the three methods.
delay_checks failure tracking (Deprecated) ๐ฌ
(Deprecated) This method is deprecated and mapped to the san_path_err method. See the delay_watch_checks and delay_wait_checks options above for more information.
marginal_path failure tracking ๐
If a second failure event (good->bad transition) occurs within marginal_path_double_failed_time seconds after a failure, high-frequency monitoring is started for the affected path: I/O is sent at a rate of 10 per second. This is done for marginal_path_err_sample_time seconds. During this period, the path is not reinstated. If the rate of errors remains below marginal_path_err_rate_threshold during the monitoring period, the path is reinstated. Otherwise, it is kept in failed state for marginal_path_err_recheck_gap_time, and after that, it is monitored again. For this method, time intervals are measured in seconds.
san_path_err failure tracking ๐งฎ
multipathd counts path failures for each path. Once the number of failures exceeds the value given by san_path_err_threshold, the path is not reinstated for san_path_err_recovery_time seconds. While counting failures, multipathd "forgets" one past failure every san_path_err_forget_rate ticks; thus if errors don't occur more often than once in the forget rate interval, the failure count doesn't increase and the threshold is never reached. Ticks are the time between path checks by multipathd, which is variable and controlled by the polling_interval and max_polling_interval parameters.
This algorithm is superseded by the marginal_path failure tracking, but remains supported for backward compatibility.
FPIN failure tracking ๐ก
Fibre channel fabrics can notify hosts about fabric-level issues such as integrity failures or congestion with so-called Fabric Performance Impact Notifications (FPINs). On receiving the fpin notifications through ELS, multipathd will move the affected path and port states to marginal.
See the documentation of the individual options above for details.
โ ๏ธ Warning, bestie: It is strongly discouraged to use more than one of these methods for any given multipath map, because the two concurrent methods may interact in unpredictable ways. If themarginal_pathmethod is active, thesan_path_errparameters are implicitly set to0.
FOREIGN MULTIPATH SUPPORT ๐โจ
multipath and multipathd can load "foreign" libraries to add support for other multipathing technologies besides the Linux device mapper. Currently this support is limited to printing detected information about multipath setup. In topology output, the names of foreign maps are prefixed by the foreign library name in square brackets, as in this example:
# multipath -ll
uuid.fedcba98-3579-4567-8765-123456789abc [nvme]:nvme4n9 NVME,Some NVMe controller,FFFFFFFF
size=167772160 features='n/a' hwhandler='ANA' wp=rw
|-+- policy='n/a' prio=50 status=optimized
| `- 4:38:1 nvme4c38n1 0:0 n/a optimized live
`-+- policy='n/a' prio=50 status=optimized
`- 4:39:1 nvme4c39n1 0:0 n/a optimized live
The nvme foreign library provides support for NVMe native multipathing in the kernel. It is part of the standard multipath package.
KNOWN ISSUES ๐๐ญ
The usage of queue_if_no_path option can lead to D state processes being hung and not killable in situations where all the paths to the LUN go offline. It is advisable to use the no_path_retry option instead.
The use of queue_if_no_path or no_path_retry might lead to a deadlock if the dev_loss_tmo setting results in a device being removed while I/O is still queued. The multipath daemon will update the dev_loss_tmo setting accordingly to avoid this deadlock. Hence if both values are specified the order of precedence is no_path_retry, queue_if_no_path, dev_loss_tmo.
FILES ๐๐
Default paths for configuration files and directories:
/etc/multipath.confโ Main configuration file./etc/multipath/conf.d/โ Directory for configuration drop-in files./etc/multipath/bindingsโ Bindings file, used when theuser_friendly_namesoption is set toyesto assign a persistent and unique alias to each multipath device./etc/multipath/prkeysโ Persistent reservation keys file, used bymultipathdto keep track of the reservation key used for a specific WWID whenreservation_keyis set tofile./etc/multipath/wwidsโ WWIDs file, used bymultipathto keep track of the WWIDs of LUNs for which multipath devices have been created in the past./run/multipathd.pidโ PID file.
SEE ALSO ๐๐
AUTHORS ๐ซถ๐
multipath-tools was developed by Christophe Varoqui, <christophe.varoqui@opensvc.com> and others.
okay thatโs the whole multipath.conf page, bestie ๐ โ from the Linux manual set, dated 2026-06-13, and itโs MULTIPATH.CONF(5), periodt. slay โจ
๐ฆ where this page comes from
- arch rolling: multipath-tools 0.15.1-1 ยท GPL-2.0-or-later AND GPL-3.0-or-later AND LGPL-2.0-or-later AND LGPL-2.1-or-later
- ๐ girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms