LAMBOOT(1)

💖 LAMBOOT 💖

okay so bestie, welcome to lamboot from the LAM TOOLS lineup — this is the glow-up that starts your LAM multicomputer. yeah. we're booting a whole multicomputer. it's giving main character energy fr fr. ✨

---

🌸 NAME 🌸

lamboot — Start a LAM multicomputer. 👑

---

🎀 SYNOPSIS 🎀

lamboot [-b] [-d] [-h] [-H] [-l] [-s] [-v] [-V] [-x] [-nn] [-np] [-c
       conf file] [-prefix /lam/install/path/] [-sessionprefix value]
       [-sessionsuffix value] [-withlamprefixpath value] [-ssi key
       value] [bhost]

like. look at all those flags. she's got options, literally. 💅

---

✨ OPTIONS ✨

• -b — assume the local shell and the remote shell are the same, bestie. that means only one remote shell invocation gets used for each node. if you don't pass -b, then two remote shell invocations are used for each node. no cap. 🦋
• -d — turn on debugging output. heads up tho: this also implies -v. 😭
• -h — print the command help menu. the classic cry for help, and honestly? valid. 🙈
• -l — delay hostname-to-IP-address resolution. she's taking her time, and that's okay. ⭐
• -prefix — use the LAM installation specified in /lam/install/path/. ⚠️ not compatible with LAM/MPI versions prior to 7.1 — so don't come crying to me if you're on an ancient build, i warned you. 💀
• -s — close stdio on the local node. bye bye, input/output. 🫶
• -ssi key value — send arguments to various SSI modules. peek the "SSI" section, below. 👀
• -v — be verbose. she's a talker and we love that for her. 🗣️
• -x — run in fault tolerant mode. tough girl energy. 🔥
• -H — do not display the command header. no intro, straight to business. 💅
• -nn — don't add "-n" to the remote agent command line. 🙅♀️
• -np — do not force the execution of $HOME/.profile on remote hosts. let them live. 🌈
• -session-prefix value — set the session prefix, overriding LAM_MPI_SESSION_PREFIX. 👑
• -session-suffix value — set the session suffix, overriding LAM_MPI_SESSION_SUFFIX. 💖
• -withlamprefixpath value — override the internal installation path. ⚠️ for internal use only — do not use unless you know what you are doing. like, genuinely. put it down. 🙈

---

🌟 ENVIRONMENT VARIABLES 🌟

• LAM_MPI_SESSION_PREFIX
• LAM_MPI_SESSION_SUFFIX

so you can change the session directory that LAM/MPI uses, which is normally shaped like this:

tmpdir/lam-username@hostname[-suffix]

• tmpdir — this gets set to LAM_MPI_SESSION_PREFIX if it's set. otherwise, it falls back to whatever TMPDIR is. and if neither of those is set? default is /tmp. periodt. ✨
• suffix — this one can be overridden by the LAM_MPI_SESSION_SUFFIX environment variable. if LAM_MPI_SESSION_SUFFIX is not set and LAM is running under a supported batch scheduling system, then $suffix will be a value that's unique to the currently running job. iykyk. 🧚

---

💖 DESCRIPTION 💖

okay bestie, listen up, this is the part where we spill the tea on what lamboot actually DOES ✨

So the lamboot tool starts the LAM software on every single machine you listed in the boot schema, bhost. That boot schema is where you tell it the hostnames of the nodes you want in your run-time MPI environment, and optionally how many CPUs LAM is allowed to use on each one. And honestly? You might wanna run recon(1) first to double check LAM can even start. Just saying. 🫶

---

Starting LAM is a three step procedure, no cap:

1. hboot(1) gets invoked on each of the specified machines.
2. Then each machine allocates a dynamic port and sends it back to lamboot, which collects them all up.
3. Then lamboot hands every machine the full list of machines/ports so they can form a fully connected topology. ✨

And if any machine couldn't start, OR if a timeout period expires before step one finishes? lamboot invokes lamwipe(1) to terminate LAM and reports the error. 💀

The bhost file is a LAM boot schema written in the host file syntax. Go check bhost(5) for that. Instead of doing it via command line, you can also specify a boot schema in the LAMBHOST environment variable. And if you don't do either? A default file, lam-bhost.def, gets used. LAM searches for bhost first in the local directory and then in the installation directory under etc/.

On top of all that, lamboot also uses a process schema for the individual LAM nodes. A process schema (peep conf(5)) is basically a description of the processes that make up the operating system on a node. Generally the sysadmin maintains this file — LAM/MPI users will generally not need to touch it. But you can customize LAM with a private process schema if you're feeling spicy. 🌶️

🌸 The bhost file 🌸

The format of the bhost file is documented over in the bhost(5) man page. iykyk.

lamboot will resolve all names in bhost on the node where lamboot was invoked (the origin node). After that, LAM only uses IP addresses, not names. So specifically: the name resolution config on all other nodes is not used. Which means the origin node has to be able to resolve every name in bhost into addresses that are reachable by all the other nodes. Periodt.

A common mistake is listing localhost (or any name that resolves to the special address 127.0.0.1 — the loopback TCP/IP device) in a bhost file that also contains other nodes. In that case, 127.0.0.1 gets sent to every other node as the address of the origin node. If those other nodes try to use 127.0.0.1 to contact the origin node, they'll actually be contacting themselves, and would eventually timeout and fail. 💀

The IP addresses grabbed from bhost are used for LAM's meta messages: startup and shutdown of jobs, out-of-band messages used for coordination, etc. The traffic amount is fairly low (unless you're using the "lamd" mode of MPI message passing, in which case ALL MPI traffic also uses LAM's meta messages for transport — peep mpirun(1)). When using the TCP RPI, these IP addresses are also used for MPI message passing via direct sockets between each pair of nodes.

A super common case: a "master" node has multiple network interface cards (NICs) — one hooked up to a public network, one hooked up to a private network where the parallel jobs are gonna run. To include the master node in a bhost file, you list the IP name (or address) of the NIC on the private network in bhost. That way all the other nodes can reach the master node on the private network. 🎀

Another example: some setups have multiple TCP/IP NICs in each node of a parallel job. One NIC is "slow" (like, 10Mbps), the other is "fast" (like, 100Mbps). You wanna let LAM take advantage of the higher bandwidth on the "fast" network for MPI messages. So in that case, bhost should list the IP names (or addresses) of all the "fast" NICs. BUT — if the LAM RPI doesn't use TCP/IP (like the Myrinet/GM RPI), then bhost should probably list the "slow" NICs instead, so that LAM's meta message traffic doesn't cause overhead and potentially drag down performance on the "fast" network for other high-performance apps.

🦋 Delaying hostname lookups 🦋

Normally, name resolution of hostnames happens on the machines where lamboot is invoked. This is for optimization reasons — so the list of hostnames only needs resolving once (potentially minimizing DNS or other hostname-lookup network traffic).

BUT. In some non-uniform networking environments, that's not enough, because each host may have a different IP address on each of its peers. For example, host A might have address Z on host B, but address Y on host C. 🙈

The -l option to lamboot makes LAM distribute hostnames to each node instead of a fully resolved set of IP addresses. So each node where LAM is booted does its own name resolution on the list of hostnames. Slay.

⭐ SSI (System Services Interface) ⭐

The -ssi switch lets you pass parameters to various SSI modules. LAM's SSI modules are described in detail in lamssi(7). SSI modules have direct impact on MPI programs because they let you set tunable parameters at run time (like which boot device driver to use, what parameters to pass to that driver, etc.).

The -ssi switch takes two arguments: key and value. The key argument generally says which SSI module gets the value. For example, the key "boot" is used to select which RPI gets used for starting processes on remote nodes. The value argument is, well, the value that gets passed. Like:

lamboot -ssi boot tm

• Tells LAM to use the "tm" boot module for native launching in PBSPro / OpenPBS environments (the tm boot module does not require a boot schema).

lamboot -ssi boot rsh -ssi rsh_agent "ssh -x" boot_schema

• Tells LAM to use the "rsh" boot module, and tells the rsh module to use "ssh -x" as the specific agent to launch executables on remote nodes.

And so on. LAM's boot SSI modules are described in lamssi_boot(7). That page should be consulted for the specific actions taken by each boot module, and how to tweak their run-time behavior. 📖

The -ssi switch can be used multiple times to specify different key and/or value arguments. If you specify the same key more than once, the values get concatenated with a comma (",") separating them.

Heads up bestie: the -ssi switch is just a shortcut for setting environment variables. You can get the exact same effect by setting corresponding environment variables before running lamboot. The form of the environment variables LAM sets is: LAM_MPI_SSI_key=value.

Also note: the -ssi switch overrides any previously set environment variables. And unknown key arguments still get set as environment variables — they are not checked (by lamwipe) for correctness. Illegal or incorrect value arguments may or may not be reported — it depends on the specific SSI module. 🙃

💅 Remote Executable Invocation 💅

All the tweakable aspects of launching executables on remote nodes during lamboot are discussed in lamssi(7) and lamssi_boot(7). Topics include (but are not limited to): discovery of remote shell, run-time overrides of the agent used to launch remote executables (like rsh and ssh), etc. 🫶

🌈 Closing stdio 🌈

The stdio of each LAM daemon on a remote host launched by lamboot is closed by default. Normally, the stdio of the LAM daemon launched on the local host stays open so the internal LAM tstdio(3) package works properly. BUT sometimes you really do wanna close the stdio of the local LAM daemon too. For example:

rsh somenode lamboot -s hostfile

Why? Because rsh waits for two conditions before exiting: lamboot to exit, AND stdout / stderr to be closed. Without -s, stdout / stderr wouldn't get closed, and rsh (and ssh) will hang even though lamboot already completed. 💀

-s makes the stdout / stderr of the local LAM daemon get closed upon invocation, which lets rsh finish up. Using -s won't affect lamboot in any other way — but it will prevent the tstdio(3) package from working properly. So pick your struggle. 😭

🧚 Fault Tolerance 🧚

If the -x option is given, LAM runs in fault tolerant mode. In this mode, nodes exchange heart beat'' messages periodically to make sure all nodes are running and the links connecting them are operational. When a node's heart beats stop, it gets declared dead'' and all LAM nodes (and processes) are notified. This lets users write fault tolerant applications that can degrade gracefully, or fully recover by replacing the defunct node with another (peep lamgrow(1)). And since this mode introduces a performance penalty, it is not activated by default. ⚠️

---

✨ EXAMPLES ✨

lamboot -v

• Start LAM on the machines described in the default boot schema. Report on important steps as they're done. 👑

lamboot -d hostfile

• Start LAM on the machines described in file hostfile. Provide incredibly detailed reports on what's happening at each stage in the boot process. 🔍

lamboot mynodes

• Start LAM on the machines described in the boot schema mynodes. Operate silently. 🤫

---

🎀 FILES 🎀

• laminstalldir/etc/lam-bhost.def — default boot schema file, where "laminstalldir" is the directory where LAM/MPI was installed
• laminstalldir/etc/lam-conf.lamd — default process schema file for LAM nodes

---

👑 SEE ALSO 👑

recon(1), lamwipe(1), hboot(1), tstdio(3), bhost(5), conf(5), lam-helpfile(5), lamssi(7), lamssi_boot(7)

---

and that's a wrap bestie!! this whole page is from LAM 7.1.4, dated July, 2007, and it's LAMBOOT(1) — iconic, historic, lowkey legendary. mwah 💖✨💅