mpi-selector(1)

📦 version 1.0.3, in opensuse-leap 16.0

✨ NAME ✨

Okay bestie, welcome to the mpi-selector(1) page from the User Commands manual. mpi-selector is a simple site-wide/per-user MPI selection tool for clusters with more than one MPI implementation installed. ✨

🌈 SYNOPSIS 🌈

Commands for end users 💖

mpi-selector [options] --list

mpi-selector [options] --set <name>

mpi-selector [options] --unset

mpi-selector [options] --query

mpi-selector [options] --version

Commands for MPI implementations 🛠️

mpi-selector [options] --register <name> --source-dir <dir>

mpi-selector [options] --unregister <name>

🦋 DESCRIPTION 🦋

The mpi-selector command is a simplistic tool to select one of multiple MPI implementations. It lets system administrators set a site-wide default MPI implementation while also letting users set their own default MPI implementation — thereby overriding the system-wide default. No cap. 💅

Note that both the site-wide and per-user defaults are independent from each other; a system administrator may choose not to have a site-wide default while a user may choose to have a personal default — and vice versa.

The system is affected by having system-wide shell startup files that look first at the user's MPI preferences. If found, the MPI implementation indicated by the user's preferences is set up in the current environment. If not found, look for a site-wide default. If found, the MPI implementation indicated by the site-wide default is set up in the current environment. If not found, exit silently.

End Users / System Administrators 👑

The mpi-selector command provides four main actions:

  • List which MPI implementations are available
  • Set a default (either on a per-user or site-wide basis)
  • Unset a default (either on a per-user or site-wide basis)
  • Query what the current default is

A common scenario is that a system administrator sets a site-wide default for a supported MPI implementation that most users will use. Power users then change their per-user defaults to use a different MPI implementation.

Another common scenario is for power users to frequently use mpi-selector to swap back and forth between multiple different MPI implementations.

NOTE: The mpi-selector command only changes the defaults for new shells. Specifically, after you invoke the mpi-selector command to change the default MPI implementation, this change does not take effect until you start a new shell. This is intentional. See the "KNOWN LIMITATIONS" section, below. 🫶

MPI Implementations 🧚

MPI implementations register themselves with mpi-selector when they are installed and unregister themselves when they are uninstalled. Each MPI installation provides two files that set up the environment for itself:

  • mpivars.sh: file sourceable by Bourne-like shells (sh, bash, etc.)
  • mpivars.csh: file sourceable by C-like shells (csh, tcsh, etc.)

These files are expected to be in a single directory and "registered" with mpi-selector using the --register and --source-dir options. mpi-selector will copy these files to its own internal store; it is safe to remove the originals after the mpi-selector registration completes successfully. 🎀

The <name> argument to --register must be simplistic — it cannot contain any shell special characters (not even if they are escaped), nor can it contain spaces. The intent is to provide simple names that users can type without escaping or quoting. Names not conforming to these rules will be rejected and the registration will fail.

When an MPI implementation is uninstalled, it should unregister with mpi-selector via the --unregister option.

💅 OPTIONS 💅

  • --list — List which MPI implementations are available.
  • --no — Assume "no" to any interactive questions asked.
  • --query — See what the current default is. If specified with no options, whichever default has precedence — if any — will be shown. If specified with --user, only show the per-user default (if there is one). If specified with --system, only show the site-wide default (if there is one).
  • --register — Register a new MPI implementation. Must be combined with the --source-dir option.
  • --set <name> — Set the default MPI implementation. May be combined with --system or --user (--user is the default and does not need to be specified).
  • --source-dir — Specify the location where mpivars.* files should be copied from. Only meaningful when used with the --register option.
  • --system — When used with --set or --unset, specifies to work with the site-wide default (vs. the per-user default). When used with --query, it specifies to specifically query the site-wide default.
  • --unregister — Unregister an MPI implementation.
  • --user — When used with --set or --unset, specifies to work with the per-user default (vs. the site-wide default). When used with --query, it specifies to specifically query the per-user default.
  • --unset — Unset the default MPI implementation. May be combined with --system or --user (--user is the default and does not need to be explicitly specified).
  • --verbose — Be verbose.
  • --version — Return the version of mpi-selector.
  • --yes — Assume "yes" to any interactive questions asked.

✨ EXAMPLES ✨

okay bestie, here’s the practical part — the four main actions that system administrators and end users invoke are: listing which MPI implementations are available, setting a default, unsetting a default, and querying what the current default is. slay, let’s go 💅

💖 Examples for End Users / System Administrators

🌸 Listing which MPI implementations are available

The --list option to the mpi-selector command shows a simple list of which MPI implementations are available:

shell$ mpi-selector --list
mympi-1.2.3
mympi-4.5.6
othermpi-7.8.9
shell$

🎀 Setting a default

By default, MPI selections are performed on a per-user basis with the --set option, using a name from the list of available MPI implementations (which can be obtained via the --list command):

shell$ mpi-selector --set mympi-4.5.6
shell$
⚠️ Note that the default takes effect in the next shell that is started; it does NOT take effect in the current shell! literally not the same shell, bestie 😭

If a default MPI is already set, setting a new default will cause an interactive confirmation prompt. This interactive prompt can be avoided by using the --yes option, which assumes a "yes" answer to all questions:

shell$ mpi-selector --set mympi-4.5.6
shell$ mpi-selector --set mympi-4.5.6 --yes
shell$

If the --system option is used, the site-wide default is modified instead of the per-user default. Since this option typically requires writing files into protected areas, root access may be required.

shell# mpi-selector --set mympi-4.5.6 --system
shell#

🦋 Unsetting a default

Unset the current default with the --unset option:

shell$ mpi-selector --unset
shell$

Similar to --set, the --system option can be used to unset the site-wide default:

shell# mpi-selector --unset --system
shell#

✨ Querying what the current default is

The --query option can be used to see what the current MPI implementation is (more specifically, what the MPI implementation will be for the next shell that is started). It indicates both which MPI is the default and at what level the default was set (per-user vs. site-wide):

shell$ mpi-selector --set mympi-1.2.3
shell$ mpi-selector --query
default:mympi-1.2.3
level:user
shell$

Note that if there is no per-user default, the system default will be shown:

shell# mpi-selector --set othermpi-7.8.9 --system

shell$ mpi-selector --unset
shell$ mpi-selector --query
default:othermpi-7.8.9
level:system
shell$ mpi-selector --set mympi-1.2.3
shell$ mpi-selector --query
default:mympi-1.2.3
level:user
shell$

If there is no per-user default and no site-wide default, --query will return silently:

shell$ mpi-selector --query
shell$

🧚 Examples for MPI Implementations

Registering and unregistering typically writes files into protected areas, and therefore usually requires root access.

If there are no MPI implementations registered, --list will return silently:

shell# mpi-selector --list
shell#

An MPI with mpivars.sh and mpivars.csh in /opt/mympi/bin can be registered as follows:

shell# mpi-selector --register mympi-1.2.3 \
       --source-dir /opt/mympi/bin
shell# mpi-selector --list
mympi-1.2.3
shell#

Note that re-registering the same <name> will cause an interactive confirmation prompt; the --yes option can be supplied to assume "yes" to all questions asked:

shell# mpi-selector --list
mympi-1.2.3
shell# mpi-selector --register mympi-1.2.3 \
       --source-dir /opt/mympi/bin --yes
mympi-1.2.3 is already registered.
Overwriting previously registered files.
shell# mpi-selector --list
mympi-1.2.3
shell#

Unregistering is also simple:

shell# mpi-selector --list
mympi-1.2.3
shell# mpi-selector --unregister mympi-1.2.3
shell# mpi-selector --list
shell#

🌈 Registering and Unregistering in RPMs

Registering and unregistering via RPM is unfortunately more complicated than it needs to be because of the following issues:

  1. Although RPM obeys dependency ordering of rpm -i a b c. That is, c will be installed before a if a requires c. Regardless, RPM's must know a) that the mpi-selector command is installed, and b) be able to find it in its path.
  2. RPM does not obey dependency ordering of rpm -e a b c. That is, c may be uninstalled before a, even if a requires c. Hence, the mpi-select command may disappear before an RPM using the mpi-select command in a scriptlet is uninstalled.
  3. "Updating" RPMs will first uninstall the old RPM and then re-install the new one.

Additionally, the staged installations (such as the OFED installer) require telling the mpi-selector command additional information so that various internal data files can be found.

In general, MPI installations via RPMs should register during the %post scriptlet and unregister during the %preun scriptlet (not during the %postun scriptlet!).

If RPMs "require" the mpi-selector RPM, they can be assured that the mpi-selector command will exist and be installed properly, but they still need to be able to find mpi-selector in their PATH. Hence, if mpi-selector is not installed into a default PATH location, the %post scriptlet won't be able to find it, and the registration call will fail. The simplest workaround (at least for the moment) is to set the PATH to where mpi-selector is installed before installing any RPMs that use it.

With that in mind, here is a possible %post scriptlet for OFED-installed RPMS:

mpi-selector --register <name> --source-dir <source_dir> \
    --yes --silent

Note the following:

  1. The --yes option forces an overwrite if, for some reason, a previous MPI of the name name is already registered.
  2. The --silent option makes mpi-selector run silently, since RPMs are supposed to install with no output.

Here is a possible %preun scriptlet for OFED-installed RPMs:

mpi-selector --unregister <name> --yes || \
    /bin/true > /dev/null 2> /dev/null

Note the following:

  1. We use %preun instead of %postun because of RPM's upgrade behavior.
  2. Since RPM does not honor dependencies when uninstalling, it is possible that mpi-selector is no longer installed, and therefore the command may fail. However, since mpi-selector is no longer installed, we don't care that it failed (i.e., there's nothing to unregister from), so just redirect all output to /dev/null and ensure that the return code from the overall command is "true" (RPM will abort if any individual scriptlet command fails).

💀 KNOWN LIMITATIONS

The main known limitation of mpi-selector is that it only affects future shells -- running it does not affect the current shell. After you run mpi-selector to set a new default MPI (regardless of whether it is a system-level or user-specific default), that default will not take effect until you start a new shell -- even though --query will report the new default.

This behavior is because mpi-selector defaults are only read during shell startup. It was an intentional design decision -- mpi-selector is intended to be a simplistic tool, and an all-encompassing solution.

Other solutions for modifying the current environment exist, such as the Environment Modules package (<http://modules.sourceforge.net/>) and SoftEnv from Argonne National Laboratory (and probably others). Using these tools, you can immediately change the environment of the current shell (to include switching to use a different MPI implementation). As such, these already-existing, mature tools are better suited for such usage patterns; mpi-selector is not intended to replace them.

For rsh/ssh-based parallel environments, switching defaults frequently should be done with care. Specifically, rsh/ssh-based launchers may depend on a common environment across all nodes (e.g., to find helper executables and/or libraries for a specific MPI). Consider the following example:

shell$ mpi-selector --set mympi-1.2.3
shell$ mpicc my_mpi_app.c -o my_mpi_app
shell$ mpirun -np 32 --hostfile myhosts my_mpi_app

While my_mpi_app is starting, it may be dangerous to switch the mpi-selector default (perhaps in a different window) because the rsh and/or ssh commands currently executing may be relying on finding the same MPI implementation on all nodes. Changing the default while the application is launching may cause a different MPI implementation to be found on some nodes, thereby causing undefined behavior. not the vibe 😭

📁 FILES

  • $HOME/.mpi-selector — Location of per-user default selection
  • /etc/sysconfig/mpi-selector — Location of site-wide default selection.
  • /etc/mpi-selector/data/data — Directory containing registered MPI shell startup files.

👑 AUTHOR

Written by Jeff Squyres.

🐛 REPORTING BUGS

Send bug reports to the OpenFabrics general mailing list (see <http://www.openfabrics.org/>). This is a high-volume mailing list, so be sure to put "mpi-selector" in the subject to ensure that it is not missed.

Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.

This is free software; see source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

and that’s the whole tea, bestie 🫶 source/version 1.0.3, dated December 29, 2009, serving mpi-selector(1) realness. bye and slay responsibly 💅✨

📦 where this page comes from