md_monitor(8)

🌸 md_monitor(8) — System Manager's Manual 🌸

okay bestie, so this is the page for md_monitor, the MD device monitor, and no cap it's kind of a big deal. it lives in the System Manager's Manual and it's here to keep an eye on your MD arrays so you don't have to 👀

---

✨ NAME ✨

md_monitor — MD device monitor 🧚

🦋 SYNOPSIS 🦋

num|--expires=num] [-P num|--process-limit=num] [-O num|--open-file-
limit=num] [-m|--fail-mirror] [-o|--fail-disk] [-r num|--retries=num]
[-p prio|--log-priority=prio] [-v|--verbose] [-y|--check-in-sync] [-c
cmd|--command=cmd] [-V|--version] [-h|--help]

like, that's a lot of flags, but we're gonna go through every single one, periodt 💅

💖 DESCRIPTION 💖

md_monitor is literally the girl who watches the component devices of each MD array for I/O issues. every time the status changes, she updates the monitored MD arrays — setting devices to 'faulty' or re-integrating working devices. she's not dramatic, she's just thorough 🫶

🎀 OPTIONS 🎀

md_monitor recognizes the following command-line options:

• -c cmd, --command=cmd — send command cmd to the daemon 📨
• -d, --daemonize — start md_monitor in the background 🌙
• -e num, --expires=num — set failfast_expires to num
• -f file, --logfile=file — write logging information into file instead of stdout 📝
• -h, --help — display md_monitor usage information
• -m, --fail-mirror — fail and reset the entire mirror half when one device failed. this is the default ✨
• -O num, --open-file-limit=num — set the maximum number of open files (RLIMIT_NOFILE, see getrlimit(2)) to num. default is 4096
• -o, --fail-disk — only fail the affected disk when one device failed. this is the opposite of --fail-mirror 🙈
• -P num, --process-limit=num — set the maximum number of processes (RLIMIT_NPROC, see getrlimit(2)) to num
• -p prio, --log-priority=prio — set logging priority to prio
• -r num, --retries=num — set failfast_retries to num
• -s, --syslog — write logging information to syslog
• -t secs, --check-timeout=secs — run path checker every secs seconds. default is 1 ⏱️
• -v, --verbose — increase logging priority
• -y, --check-in-sync — run path checkers for 'in_sync' devices. without this option, path checkers will be stopped whenever a device is detected to be 'in_sync'. they will be re-started once a device has been marked as 'faulty' or 'timeout' 🔄
• -V, --version — display md_monitor version information
• -h... wait, no, we already did that one bestie, keep scrolling 😌

👑 MD_MONITOR COMMAND MODE 👑

when you specify --command, the md_monitor program connects to an already running md_monitor program and sends a pre-defined command. the command has the following syntax:

cmd:md@dev

these are the recognised values for cmd. unless specified otherwise, md needs to be the device node of an existing MD array.

• Shutdown — shut down md_monitor; the md argument should be /dev/console
• RebuildStarted — rebuild has started on array md
• RebuildFinished — rebuild has finished on array md
• DeviceDisappeared — array md has been stopped; md_monitor will stop monitoring the component devices for array md
• NewArray — array md has been started. this event is ignored by md_monitor; new arrays will be detected via uevents
• Fail — MD detected a failure on the component device dev of array md. md_monitor will re-check the device every failfast_expires seconds
• FailSpare — MD detected a failure on the spare device dev of array md. md_monitor will re-check the device every failfast_expires seconds
• Remove — the component device dev has been removed from the MD array md. md_monitor will stop monitoring this device
• SpareActive — MD has integrated the device dev into array md. md_monitor will re-start monitoring of this device every failfast_expires seconds. the check interval will be increased for each successful check up to a maximum of failfast_expires * failfast_retries seconds 📈
• ArrayStatus — return the current internal status of the monitored devices
• MirrorStatus — return the status of the MD component devices in abbreviated form. each character represents the status of the MD component device at that position. for the possible states see the next paragraph 👀
• MonitorStatus — return the current I/O status of the monitored devices in abbreviated form. each character represents the I/O status of the monitored device in abbreviated form

🌈 DEVICE STATUS DISPLAY 🌈

md_monitor will be displaying state information about the monitored devices when the CLI command MirrorStatus or MonitorStatus is sent. each character of the returned string represents the state of the device at that location.

the possible states for MirrorStatus are:

• . — Unknown
• A — In_Sync
• W — Faulty
• T — Timeout
• S — Spare
• - — Removed
• R — Recovery pending
• P — Removal pending
• B — Blocked

R and P are intermediate states, which are set by md_monitor whenever a command has been sent to mdadm, but no notification has been received yet. B is set when MD attempts to fail the second half of the mirror when the first half is already failed. MD will hold off I/O to the entire mirror until the second half is useable again 😭

the possible states for MonitorStatus are:

• . — Unknown
• X — MD will be stopped
• A — I/O ok
• W — I/O failed
• P — I/O pending
• T — I/O timeout — Removed S Spare R Recovery

P and T describe the same condition, ie I/O has been stalled. the state will switch from P to T when the timeout as set by failfast_expires * failfast_retries seconds has expired. -, S, and R, are steps MD takes to recover a device; first the device will be removed, then it will be re-added as a 'spare' device, and then recovery will be starting for re-adding the spare device into the MD array 🧚

🔥 THEORY OF OPERATION 🔥

md_monitor sets up a path checker thread for each MD component device. this path checker will issue every check-time seconds an asynchronous I/O request to the device. it will then wait up to failfast_expires * failfast_retries seconds for this I/O to complete. if no response has been received during that time, the monitor status for this path is set to I/O timeout. if the I/O completed, the monitor status for this path will be set to I/O ok or I/O failed, depending on whether the I/O completed without error or not. if the path checker has been interrupted during waiting, the monitor status for this path will be set to I/O pending. after the monitor status has been updated, the path checker thread will update the MD status for this device and invoke an action, depending on these two states. if check-in-sync has been specified, the path checker continues to run even for in_sync paths. otherwise the path checker is stopped when a path is marked as in_sync. path checkers will be restarted whenever a device is marked as faulty or timeout. she's literally the main character of your storage stack, fr fr 💅

🫶 MDADM INTEGRATION 🫶

md_monitor listens to udev events for any device changes. it is designed to integrate into MD via the --monitor functionality of mdadm.

to use this function mdadm needs to be started with

mdadm --monitor --scan --program=md_script

where md_script is a bash script containing eg:

#!/bin/bash
# MD monitor script
#
EVENT=$1
MD=$2
DEV=$3
/sbin/md_monitor -c "${EVENT}:${MD}@${DEV}"

a default md_script is installed at /usr/share/misc/md_notify_device.sh.

it is recommended to use an /etc/mdadm.conf configuration file when using md_monitor to monitor MD arrays. to enable automatic device assembly into MD arrays, the configuration file should include the lines:

POLICY action=re-add
AUTO -all

it is recommended to include these lines when using md_monitor ⭐

💜 VERSIONS 💜

this manual page documents md_monitor version 4.26.

🍓 FILES 🍓

• /usr/share/misc/md_notify_device.sh — default md_monitor script
• /etc/mdadm.conf — MD configuration file

🦋 SEE ALSO 🦋

mdadm(8), mdadm.conf(7)

---

okay and that's the whole page, bestie 💖 sourced from md_monitor 6.6, dated Thu Nov 5 2015 — iconic vintage energy, we stan a classic 🎀✨ go forth and monitor those arrays, iykyk 🫶💅