mpi.conf(5)

💖 mpi.conf — the Slurm config file for MPI plugins 💖

okay bestie, welcome to mpi.conf(5), which is a section 5 (file formats) manual page from the Slurm Configuration File crew. this is where you go when you want to tell Slurm's MPI plugins how to act. pull up a chair, we're doing config files 🌸

---

✨ NAME ✨

mpi.conf — the Slurm configuration file that lets you configure MPI plugins. that's the whole job. slay.

---

🦋 DESCRIPTION 🦋

so here's the tea: mpi.conf is an ASCII file that defines parameters controlling how MPI plugins behave. rn the config file can literally only be used to configure the <cyan>PMIx</cyan> plugin, but it's built to be extendable to other MPI plugins too, no cap. and where does it live? always in the same directory as the slurm.conf. also — this file is optional. you don't have to have one. it's giving low-maintenance queen 💅

a few house rules, bestie:

• Parameter names are case insensitive. type PMIXTIMEOUT, pmixTimeout, pmix_timeout (jk that one isn't a thing, don't do that) — casing does not matter.
• Any text following a # in the config file is treated as a comment through the end of that line. vent freely 🫶
• Changes to the config file take effect upon restart of Slurm daemons, daemon receipt of the SIGHUP signal, or execution of the command scontrol reconfigure — unless otherwise noted.

⚠️ ok but the "unless otherwise noted" is doing heavy lifting there. don't just assume everything reloads instantly, bestie.

you can view your settings in the output of scontrol show config (vibe check your own config, iconic). this config file can be included when using "configless" mode. information from mpi.conf is read at startup or upon reconfigure by slurmctld and slurmd. and instances of slurmstepd for batch steps will receive information about the requested plugin from slurmd — so the stepd gets the gossip passed down the chain 📣

---

🎀 PARAMETERS 🎀

okay this is the main character section. one bullet per parameter, every single one, let's go 💖

• PMIxCliTmpDirBase=<path> — the directory you want PMIx to use for temporary files. defaults to not being set.
• PMIxCollFence={mixed|tree|ring} — defines the type of fence to use for collecting inter-node data. defaults to not being set. see also PMIxFenceBarrier.
• PMIxDebug={0|1} — turn on debug logging for the PMIx plugin. defaults to 0. (0 = quiet, 1 = tell me everything 🔥)
• PMIxDirectConn={true|false} — disables direct launching of tasks. default is "true". yeah, it reads backwards, that's just how it is, we don't question the ancestors 💀
• PMIxDirectConnEarly={true|false} — allows early connections to a parent node. defaults to "false".
• PMIxDirectConnUCX={true|false} — allows PMIx to use UCX for communication. defaults to "false".
• PMIxDirectSameArch={true|false} — enables additional communication optimizations when PMIxDirectConn is set to true, and it assumes all the job's nodes have the same architecture. defaults to "false".
• PMIxEnv=<environment variables> — a semicolon separated list of environment variables to be set in job environments, for PMIx to use. defaults to not being set.
• PMIxFenceBarrier={true|false} — defines whether to fence inter-node communication for data collection. default is "false". see also PMIxCollFence.
• PMIxNetDevicesUCX=<device type> — the type of network device to use for communication. this will set the UCX_NET_DEVICES environment variable in only the slurmstepd environment. defaults to not being set. (only the stepd, bestie — not everything, ONLY the stepd 🫶)
• PMIxShareServerTopology={true|false} — allows the PMIx server to share its copy of the local node topology of the job with clients. if set to true, the PMIx server will set any necessary key-value pairs in the job-level information provided to each client. the topology will not be shared via shared memory. defaults to "false".

📌 NOTE: this is only supported by PMIx v4.x+. don't come crying to me on an older PMIx, i warned you 😭

• PMIxTimeout=<time> — the maximum time (in seconds) allowed for communication between hosts to take place. defaults to 300 seconds.
• PMIxTlsUCX=<tl1>[,<tl2>...] — sets the UCX_TLS variable in the slurmstepd environment, which restricts the transports to use. the accepted values are defined in the UCX documentation and may vary between installations. multiple values can be set and must be separated by commas. if not set, UCX tries to use all available transports and selects the best ones according to their performance capabilities and scale. defaults to not being set.

---

👑 COPYING 👑

Copyright (C) 2022 SchedMD LLC. 🫶

This file is part of Slurm, a resource management program. For details, see <https://slurm.schedmd.com/>.

Slurm is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Slurm is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

---

🌈 SEE ALSO 🌈

• slurm.conf(5)

---

and that's the whole page, bestie, you ate it 💅✨ this was mpi.conf(5) from Slurm 26.05, filed under "Slurm Configuration File" — go forth and configure your MPI plugins, you absolute icon 🎀🦋💖