SMP_UTILS(8)

💖 SMP_UTILS(8) — the smp_* girlies 💖

okay bestie, welcome to the smp_utils family, aka the smp_* squad 🫶 this is the manual for the whole crew, so buckle up.

---

✨ NAME ✨

smp_* — invoke a SAS Serial Management Protocol (SMP) function. that's the whole vibe, no cap 💅

---

🌸 SYNOPSIS 🌸

smp_* [--expected=EX] [--help] [--hex] [--interface=PARAMS] [--raw]
[--sa=SAS_ADDR] [--verbose] [--version] SMP_DEVICE[,N]

---

🦋 DESCRIPTION 🦋

Serial Attached SCSI (SAS) is a transport — you might also hear it called an interconnect — used by storage systems. a SAS system is basically a little friend group made up of:

• Host Bus Adapters (HBAs) — these typically contain SCSI initiators, they're the main characters 🎀
• disks — referred to in SCSI as both targets and logical units
• and optionally some switching hardware called "expanders" ✨

here's the tea though: expanders are not SCSI devices, so the old protocol literally could not talk to them. so they had to invent a whole new protocol to control and monitor them. its full name is the SAS Serial Management Protocol, abbreviated SMP. iconic behavior honestly.

smp_utils is a package of utilities. each utility sends an SMP function request to a SMP_DEVICE (which is an SMP target). some utilities may invoke the same function more than once. if an error happens, an error message goes to stderr. if no error happens, the response is:

• decoded (this is the default 🥰), or
• output in ASCII hex (when you give --hex), or
• output in binary to stdout (when you give --raw).

if SMP_DEVICE[,N] isn't given, then the value in the environment variable SMP_UTILS_DEVICE gets used instead.

this package was originally written for Linux and has been ported to FreeBSD and Solaris. she gets around 💅

---

🐧 LINUX INTERFACE 🐧

right now there are multiple interfaces that let SMP functions get passed through to an SMP target.

one method is to have a SMP_DEVICE that is actually the SMP initiator (e.g. /dev/mptctl,0). in that case the SMP target's SAS address must be supplied with the --sa=SAS_ADDR option. no skipping that one bestie.

another method is to have a SMP_DEVICE that represents the SMP target. in that case no SAS_ADDRESS needs to be given, since it's implicit. she already knows who she is 💅

each utility in smp_utils attempts to work out which interface it's been given by examining the SMP_DEVICE file. there are three interfaces supported currently:

aac

this specifies the aacraid SAS pass-through associated with Adaptec/PMC RAID products. the SMP_DEVICE[,N] argument takes the form /dev/aac[N[,E_ID]] where:

• N is the raid controller number (typically 0), and
• E_ID is the expander identifier (typically 0)

both default to 0, so /dev/aac is equivalent to /dev/aac0 and /dev/aac0,0.

the N is the unique_id found in /sys/class/scsi_host<HN>/unique_id. the HN is the host number, which is the first number to appear on each line in the lsscsi utility, which by default uses one line to list each accessible SCSI device (typically SCSI or ATA disks).

the E_ID is the expander identifier which can be found with the Adaptec/PMC arcconf utility using the form arcconf expanderlist <ControllerNum>. the <ControllerNum>s start at 1.

if an aac RAID controller is present then the /dev/acc device node will be created by the first smp utility to use this interface. (yes, /dev/acc, that's what the page says, we don't question the page 🫶)

mpt

this specifies the MPT fusion SAS pass-through.

• the mptsas driver uses the /dev/mptctl device node (character device major 10, minor 220)
• the mpt2sas driver uses the /dev/mpt2ctl device node (major 10, minor 221)
• for the mpt3sas driver the corresponding device node is /dev/mpt3ctl

the modprobe mptctl or modprobe mpt2ctl command may be needed.

if there are multiple mpt fusion controllers (HBAs) in the computer, then you're gonna need to specify which one to use with the syntax /dev/mptctl,<n> where <n> is the "ioc_num". this number can be found with dmesg after the mptsas driver is registered and appears as a suffix to the driver name (e.g. mpt2sas0). it can also be found in /sys/class/scsi_host/host<n>/unique_id.

⚠️ when this interface is used the --sa=SAS_ADDR option must be given to specify the SAS address of the SMP target. mandatory. no negotiation.

sgv4 (sg)

this interface is more generic and supported by several SAS HBA drivers including mptsas (and mpt2sas). it was introduced in the Linux 2.6.24 kernel.

the SMP functions are passed to the kernel via the bsg driver using a format known as "SCSI Generic Version 4", which is where this interface gets its name: sgv4, or just sg. the SAS transport layer within the SCSI sub-system unpacks the SMP requests and forwards them to SAS low level drivers that support this interface.

the SMP_DEVICE is either:

• a member of the /sys/class/bsg directory (e.g. /sys/class/bsg/expander-6:0), or
• a device node made for the bsg driver (e.g. /dev/bsg/expander-6:0)

such device nodes are dynamic (i.e. they don't have fixed major and minor numbers) and should correspond to the major and minor numbers found in the sys/class/bsg/<smp_target_device>/dev file.

---

🌊 FREEBSD INTERFACE 🌊

the CAM subsystem has been enhanced in FreeBSD 9 to pass-through SMP requests and return the corresponding responses. however, CAM does not directly access expander devices because they are not SCSI devices 😭 it makes the assumption that each SAS expander has an integrated SES (enclosure) device and that's what gets addressed. this assumption seems to be true for SAS-2 expanders but not some SAS-1 expanders.

so invocations look like this:

# smp_discover /dev/ses0

where /dev/ses0 is a SES device associated with a SAS expander. easy peasy 💖

---

☀️ SOLARIS INTERFACE ☀️

the USMP pass-through mechanism is used. invocations look like this:

# smp_rep_manufacturer /dev/smp/expd0

---

🌈 ENVIRONMENT VARIABLES 🌈

if the device name isn't given, then the SMP_UTILS_DEVICE environment variable gets checked, and if it's present its contents are used as the device name.

if the SAS address (of the SMP target) isn't given and it's required (i.e. it's not implicit in the device name), then the SMP_UTILS_SAS_ADDR environment variable gets checked, and if it's present its contents are used as the SAS address. SAS addresses are usually given in hex, indicated by a leading 0x or a trailing h.

a device slot number (dsn) is important for establishing the relationship between an expander phy and a SES array element. newer expanders (e.g. SAS-3) support dsn_s in the DISCOVER (and DISCOVER LIST) functions. these can be shown, if available, with the --dsn option to the smp_discover and smp_discover_list utilities.. to ease typing that option often, the SMP_UTILS_DSN environment variable, if present, has the same effect.

if both an environment variable and the corresponding command line option is given and they contradict, then the command line options take precedence. periodt.

---

💅 COMMON OPTIONS 💅

mandatory arguments to long options are mandatory for short options as well. if an option takes a numeric argument then that argument is assumed to be decimal unless otherwise indicated (e.g. with a leading "0x" or a trailing "h").

• -E, --expected=EX — revision 4a of the SAS-2 draft introduced an 'expected expander change count' field in some SMP requests. the idea is to detect other SMP initiators trying to change the state of an expander 👀 the value EX is from 0 to 65535 inclusive, with 0 being the default value. when EX is greater than zero, then if the value doesn't match the expander change count of the SMP target (i.e. the expander) when the request arrives, then the target ignores the request and sets a function result of "invalid expander change count" in the response. she said not today 💀
• -h, --help — output the usage message for the utility then exit.
• -H, --hex — output the response in hexadecimal. this does not include the trailing CRC field.
• -I, --interface=PARAMS — interface specific parameters. this option is usually not needed since the interface type is guessed by a utility based on the characteristics of the given SMP_DEVICE argument or what's in the corresponding environment variables. PARAMS is of the form: INTF[,force]. if the guess doesn't work then the interface can be specified by giving an INTF of either 'mpt' or 'sgv4'. sanity checks are still performed and a utility may refuse if it doesn't agree with the given INTF. if you're really sure then adding a ,force will force the utility to use the given interface. (use that power responsibly bestie 🙈)
• -r, --raw — send the response to stdout in binary. this does not include the trailing CRC field. all error messages are sent to stderr.
• -s, --sa=SAS_ADDR — specifies the SAS address of the SMP target device. typically this is an expander. this option may not be needed if the SMP_DEVICE has the target's SAS address associated with it. the SAS_ADDR is in decimal, but most SAS addresses are shown in hexadecimal. to give a number in hexadecimal either prefix it with '0x' or put a trailing 'h' on it. if this option isn't given then the value in the environment variable SMP_UTILS_SAS_ADDR is used.
• -v, --verbose — increase the verbosity of the output. can be used multiple times. stack it, queen 👑
• -V, --version — print the version string and then exit.

---

okay that's part 1 bestie, and there's more coming so don't go anywhere 🫶✨

💀 EXIT STATUS 💀

okay so bestie, if you're writing a script that calls these utilities, you're gonna wanna know how to tell if things went slay or if it's giving disaster. The exit status tells you success (0) or failure (1 or more) — here's the whole roster, no skips:

• 0 — success ✨ literally the dream, everything worked.
• 1 - 63 — these are reserved for SMP function result codes. Wanna read the receipts? Go check the SAS-2 (or later) draft, in the section on the application layer, then drill down further: management application layer → then SMP functions. And here are some common function result codes you'll actually run into: 1 [unknown SMP function], 2 [SMP function failed], 16 [phy does not exist], 17 [index does not exist], 18 [phy does not support SATA], 19 [unknown phy operation], 22 [phy vacant] and 35 [zone lock violation]. iykyk 💅
• 91 — syntax error, bestie. Either illegal options, options with bad arguments, or a combination of options that is just not permitted. The group chat said no.
• 92 — the utility can't open, close or use the SMP_DEVICE you gave it. Maybe the file name is wrong, maybe it's a permissions situation (rude). Try adding the --verbose option, it might spill more tea.
• 93 — resource problem. Typically that means it tried to allocate memory (ram) and it just... didn't happen. 😭
• 97 — the response to an SMP function failed sanity checks. Not slay.
• 99 — any error that doesn't fit into values 1 to 97 can land here. This includes transport and operating system errors.
• 126 — the utility was found but couldn't be executed. Could happen if the executable doesn't have execute permissions.
• 127 — this is the exit status for utility not found. Happens when a script calls a utility in this package but the PATH environment variable hasn't been properly set up, so the script just cannot find the executable. the way that happens every single time you forget your PATH... 🙈
• 128 + <signum> — if a signal kills a utility, the exit status is 128 plus the signal number. For example, if a segmentation fault occurs then the utility is typically killed by SIGSEGV, which according to man 7 signal has an associated signal number of 11; so the exit status will be 139.
• 255 — the utility tried to yield an exit status of 255 or larger. That should not happen; it's given here for completeness.

---

🕵️ NOTES 🕵️

ngl bestie, finding the SAS address of an expander can be a genuine challenge in some environments. An enclosure containing one or more expanders may have the expander SAS address(es) printed on the back of the device — a bit like Ethernet MAC addresses. iconic honestly.

In the Linux 2.6 kernel series the expander SAS address may well be in the sysfs tree, but it is not always easy to find. Doing this search may help:

# find /sys -name "*expander*"

That should show the suffix on any expanders that have been detected. Then a command like cat /sys/class/sas_device/expander-6:0/sas_address should show its SAS address. 🎀

Another approach is to work backwards from SCSI devices (i.e. logical units). The protocol specific port log page (log page 18h) contains fields for the "attached SAS address". The sg_logs utility from the sg3_utils package could be used like this:

# sg_logs --page=18h /dev/sdb

Any given "attached SAS address" is either a HBA, an expander, or 0 indicating that port is not connected. An expander is indicated by "attached device type: expander device". A SAS disk's target port identifiers (also known as SAS addresses), device name and logical unit name (all NAA 5 format) can be found with the sg_vpd utility (e.g. sg_vpd -i <disk_dev>). The sdparm utility can provide the same information (e.g. sdparm -i <disk_dev>).

A SAS expander is often associated with a SCSI Enclosure Services (SES) device, sometimes on the same silicon, attached via a virtual phy to the expander. That SES device may be able to access and control an attached enclosure or backplane via SGPIO or I2C on sideband signals (e.g. in a SFF-8087 cable). To interact with a SES device, see the sg_ses utility.

Often expander phys are grouped in fours on the same connector (e.g. SFF-8088). Care needs to be taken when multiple expanders are interconnected. An enclosure universal port is one in which the "table to table supported" attribute is set (in the REPORT GENERAL response) and the associated phys have the table routing attribute (in the DISCOVER response). Enclosure universal ports were introduced in SAS-2 and have few restrictions when used to interconnect expanders or connect SAS or SATA devices. An enclosure out port is one in which the "table to table supported" attribute is clear and the associated phys have the table routing attribute. An enclosure in port is one in which the associated phys have the subtractive routing attribute. When universal ports are not available, an expander interconnect should be between an in port and an out port. the more you know ✨

---

🌈 EXAMPLES 🌈

See the "Examples" section in http://sg.danny.cz/sg/smp_utils.html .

---

👑 CONFORMING TO 👑

SAS has multiple generations, and bestie, we're gonna run through all of them. The early standards are: the original SAS (ANSI INCITS 376-2003), SAS 1.1 (INCITS 417-2006) and SAS-2 (ANSI INCITS 457-2010). SAS-2.1 work was split into an electrical and physical layers document (standardized as SAS-2.1 ANSI INCITS 478-2011) with the upper level layers placed in a document called the SAS Protocol Layer, and it was standardized as SPL ANSI INCITS 476-2011. Next came SPL-2, which was standardized as SPL-2 ANSI INCITS 505-2013. Then came SPL-3, which was standardized as SPL-3 ANSI INCITS 492-2015. SPL-4 is near standardization and its most recent draft is spl4r13.pdf, while SPL-5 work has started and its most recent draft is spl5r03.pdf. To lessen confusion, the multiple generations of SAS will be referred to in these man pages as SAS 1, 1.1, 2, 2.1 (SPL), 3 (SPL-2 and SPL-3), 4 (SPL-4) and 5 (SPL-5). Roughly speaking, for the electrical and physical layers standards, SAS-1 runs at 3 Gbps, SAS-2 at 6 Gbps, SAS-3 at 12 Gbps and SAS-4 at 22.5 Gbps. Drafts, including those just prior to standardization, can be found at the http://www.t10.org site (e.g. spl-r07.pdf and spl2r04c.pdf). INCITS policy now requires a registration to view these drafts — a break from t10.org tradition. 💀

The two utilities for reading and writing to GPIO registers, smp_read_gpio and smp_write_gpio, are defined in the Small Form Factor document SFF-8485 found at http://www.sffcommittee.com . "Enhanced" versions of the corresponding SMP functions have been mentioned in some drafts but no definitions have been published and the references have been removed in more recent drafts.

In this section of each utility's man page is the first standard in which the associated SMP function appeared and whether there have been significant additions in later standards.

The COVERAGE file in the smp_utils source tarball shows a table of all SMP function names defined in the drafts, the versions of those standards in which those SMP functions first appeared, and the corresponding smp_utils utility names. A lot of extra SMP functions have been added in SAS-2 associated with zoning. zoning era fr fr

---

✍️ AUTHORS ✍️

Written by Douglas Gilbert. 💖

---

🐛 REPORTING BUGS 🐛

Report bugs to <dgilbert at interlog dot com>.

---

⚖️ COPYRIGHT ⚖️

Copyright © 2006-2020 Douglas Gilbert

This software is distributed under a FreeBSD license. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

---

🫶 SEE ALSO 🫶

• sg_logs, sg_vpd, sg_ses (sg3_utils)
• sdparm (sdparm)
• lsscsi (lsscsi)

---

okay that's the whole page bestie, you made it to the end 🎀 this lil masterpiece is from smp_utils-0.99, dated March 2020 — and it's SMP_UTILS(8), so yes, section 8, admin utilities, main character energy only. now go forth and slay those expanders 💅✨🦋