MPI_COMM_SPAWN(3)

📦
version 5.0.10 · 7 versions
, in ubuntu stonking

👑 MPI_COMM_SPAWN(3) — the Open MPI glow-up 💅

okay so bestie, welcome to MPI_Comm_spawn from Open MPI — yes the manual page is literally called MPI_COMM_SPAWN(3), we love a consistent queen. this is part 1 of a long one so buckle up, hydrate, and let's get into it ✨

🌸 NAME 🌸

MPI_Comm_spawn — Spawns a number of identical binaries.

like… it makes a whole lil squad of the same program appear. iconic behavior fr fr 👑

✨ SYNTAX ✨

💖 C Syntax

#include <mpi.h>

int MPI_Comm_spawn(const char *command, char *argv[], int maxprocs,
     MPI_Info info, int root, MPI_Comm comm,
     MPI_Comm *intercomm, int array_of_errcodes[])

💖 Fortran Syntax

USE MPI
! or the older form: INCLUDE 'mpif.h'
MPI_COMM_SPAWN(COMMAND, ARGV, MAXPROCS, INFO, ROOT, COMM,
     INTERCOMM, ARRAY_OF_ERRCODES, IERROR)

     CHARACTER*(*) COMMAND, ARGV(*)
     INTEGER INFO, MAXPROCS, ROOT, COMM, INTERCOMM,
     ARRAY_OF_ERRCODES(*), IERROR

💖 Fortran 2008 Syntax

USE mpi_f08
MPI_Comm_spawn(command, argv, maxprocs, info, root, comm, intercomm,
             array_of_errcodes, ierror)
     CHARACTER(LEN=*), INTENT(IN) :: command, argv(*)
     INTEGER, INTENT(IN) :: maxprocs, root
     TYPE(MPI_Info), INTENT(IN) :: info
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Comm), INTENT(OUT) :: intercomm
     INTEGER :: array_of_errcodes(*)
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

🎀 INPUT PARAMETERS 🎀

  • command — Name of program to be spawned (string, significant only at root). so like, the name of the lil program you're summoning 🪄 only the root process's opinion on this actually counts.
  • argv — Arguments to command (array of strings, significant only at root). the lil argument besties you're passing along 🍓 and again… only root's version of this matters, no cap.
  • maxprocs — Maximum number of processes to start (integer, significant only at root). the absolute max number of processes we're starting up 💫 root says it, root means it.
  • info — A set of key-value pairs telling the runtime system where and how to start the processes (handle, significant only at root). literally the vibe check for the runtime: where + how to launch these processes 📋 and yep… root-only energy.
  • root — Rank of process in which previous arguments are examined (integer). the rank of the process whose copy of the earlier args actually gets read 👀 everybody else's are just… there.
  • comm — Intracommunicator containing group of spawning processes (handle). the intracommunicator holding the group of processes doing the spawning 💜 the friend group, basically.

🦋 OUTPUT PARAMETER 🦋

  • intercomm — Intercommunicator between original group and the newly spawned group (handle). the intercommunicator linking the OG group to the freshly spawned group 🤝 the group chat bridge, if you will.
  • array_of_errcodes — One code per process (array of integers). one error code for each process — everybody gets their own lil report card 📝
  • ierror — Fortran only: Error status (integer). Fortran girlies get an error status integer, periodt 🍑

💖 DESCRIPTION 💖

okay so MPI_Comm_spawn literally tries to start maxprocs identical copies of the MPI program you name in command, gets them talking to you, and hands back an intercommunicator. The new lil babies are called children 🍼. They get their own MPI_COMM_WORLD — completely separate from the parents' one, no shared custody bestie.

MPI_Comm_spawn is collective over comm, and it also might not return until MPI_Init has been called over in the children. And same energy the other way: MPI_Init in the children may not return until ALL the parents have called MPI_Comm_spawn. So in that sense, MPI_Comm_spawn on the parent side + MPI_Init on the child side form one collective operation over the union of parent and child processes. Togetherness 💅

The intercommunicator you get back from MPI_Comm_spawn has the parent processes in the local group and the child processes in the remote group. The ordering of processes in the local and remote groups is the same as the ordering of the group of the comm in the parents and of MPI_COMM_WORLD of the children, respectively. (Yes the original really does say "the same as the as the", we love a typo era 😭.) And the children can grab that intercommunicator themselves with MPI_Comm_get_parent.

Quick heads up: the MPI standard allows an implementation to use the MPI_UNIVERSE_SIZE attribute of MPI_COMM_WORLD to specify how many processes will be active in a program. Although this implementation of the MPI standard does define MPI_UNIVERSE_SIZE, it does not let you set its value. So if you try to set the value of MPI_UNIVERSE_SIZE? You get an error message. No cap, it will simply refuse 🙈.

🌸 The command Argument 🌸

command is a string holding the name of the program to be spawned. The string is null-terminated in C. In Fortran, leading and trailing spaces get stripped. MPI looks for the file first in the working directory of the spawning process. Priorities, bestie.

🦋 The argv Argument 🦋

argv is an array of strings containing the arguments that get passed to the program. The first element of argv is the first argument passed to command — NOT the command itself, unlike what's conventional in some contexts. The argument list is terminated by NULL in C and by an empty string in Fortran (and yes, it is the MPI application's responsibility to make sure the last entry of the argv array is an empty string — the compiler will NOT just auto-insert it for you, she's not your mom 💀).

In Fortran, leading and trailing spaces are always stripped, so a string made of nothing but spaces counts as an empty string. The constant MPI_ARGV_NULL can be used in C and Fortran to mean "empty argument list". In C, that constant is the same as NULL.

In C, the MPI_Comm_spawn argv differs from main's argv in two ways:

  1. It's shifted by one element. Specifically, argv[0] of main contains the name of the program (given by command). argv[1] of main corresponds to argv[0] in MPI_Comm_spawn, argv[2] of main to argv[1] of MPI_Comm_spawn, and so on and so on ✨
  2. argv of MPI_Comm_spawn must be null-terminated, so its length can be figured out.

Passing an argv of MPI_ARGV_NULL to MPI_Comm_spawn results in main receiving an argc of 1 and an argv whose element 0 is the name of the program.

✨ The maxprocs Argument ✨

Open MPI tries to spawn maxprocs processes. If it can't spawn maxprocs processes, it raises an error of class MPI_ERR_SPAWN. If MPI can spawn the specified number of processes, MPI_Comm_spawn returns successfully and the number of spawned processes, m, is given by the size of the remote group of intercomm.

A spawn call with the default behavior is called hard. A spawn call for which fewer than maxprocs processes may be returned is called soft.

🎀 The info Argument 🎀

The info argument is an opaque handle of type MPI_Info in C and INTEGER in Fortran. It's basically a container for a bunch of user-specified (key,value) pairs. key and value are strings (null-terminated char * in C, character*(*) in Fortran). Routines to create and manipulate the info argument are described in Section 4.10 of the MPI-2 standard.

For the SPAWN calls, info provides additional, implementation-dependent instructions to MPI and the runtime system on how to start processes. An application may pass MPI_INFO_NULL in C or Fortran. Portable programs that don't need detailed control over process locations should use MPI_INFO_NULL.

The following keys for info are recognized in Open MPI. (The reserved values mentioned in Section 5.3.4 of the MPI-2 standard are not implemented. rip to them 🪦)

  • host — char *. Host on which the process should be spawned. See the orte_host man page for an explanation of how this will be used.
  • hostfile — char *. Hostfile containing the hosts on which the processes are to be spawned. See the orte_hostfile man page for an explanation of how this will be used.
  • add-host — char *. Add the specified host to the list of hosts known to this job and use it for the associated process. This will be used similarly to the -host option.
  • add-hostfile — char *. Hostfile containing hosts to be added to the list of hosts known to this job and use it for the associated process. This will be used similarly to the -hostfile option.
  • wdir — char *. Directory where the executable is located. If files are to be pre-positioned, then this location is the desired working directory at time of execution — if not specified, then it will automatically be set to ompi_preload_files_dest_dir.
  • ompi_prefix — char *. Same as the --prefix command line argument to mpirun.
  • ompi_preload_binary — bool. If set to true, pre-position the specified executable onto the remote host. A destination directory must also be provided.
  • ompi_preload_files — char *. A comma-separated list of files that are to be pre-positioned in addition to the executable. Note that this option does not depend upon ompi_preload_binary — files can be moved to the target even if an executable is not moved.
  • ompi_stdin_target — char *. Comma-delimited list of ranks to receive stdin when forwarded.
  • ompi_non_mpi — bool. If set to true, launching a non-MPI application; the returned communicator will be MPI_COMM_NULL. Failure to set this flag when launching a non-MPI application will cause both the child and parent jobs to "hang". (do not do this bestie, the hang is real 😭)
  • ompi_param — char *. Pass an OMPI MCA parameter to the child job. If that parameter already exists in the environment, the value will be overwritten by the provided value.
  • mapper — char *. Mapper to be used for this job.
  • map_by — char *. Mapping directive indicating how processes are to be mapped (slot, node, socket, etc.).
  • rank_by — char *. Ranking directive indicating how processes are to be ranked (slot, node, socket, etc.).
  • bind_to — char *. Binding directive indicating how processes are to be bound (core, slot, node, socket, etc.).
  • path — char *. List of directories to search for the executable.
  • npernode — char *. Number of processes to spawn on each node of the allocation.
  • pernode — bool. Equivalent to npernode of 1.
  • ppr — char *. Spawn specified number of processes on each of the identified object type.
  • env — char *. Newline-delimited list of envars to be passed to the spawned procs.

Okay so bool info keys are actually strings, but they get evaluated like this: if the string value is a number, it's converted to an integer and cast to a boolean (meaning zero integers are false and non-zero values are true). If the string value is (case-insensitive) "yes" or "true", the boolean is true. If the string value is (case-insensitive) "no" or "false", the boolean is false. Every other string value is unrecognized, and therefore false. girlypop the vibes must be exact 💅

💜 The root Argument 💜

All arguments before the root argument are examined only on the process whose rank in comm is equal to root. The value of these arguments on other processes is just ignored. Main character energy — only root gets a say.

🌈 The array_of_errcodes Argument 🌈

array_of_errcodes is an array of length maxprocs in which MPI reports the status of the processes that MPI was requested to start. If all maxprocs processes were spawned, array_of_errcodes is filled in with the value MPI_SUCCESS. If any of the processes are not spawned, array_of_errcodes is filled in with the value MPI_ERR_SPAWN. In C or Fortran, an application may pass MPI_ERRCODES_IGNORE if it's not interested in the error codes.

⭐ NOTES ⭐

Completion of MPI_Comm_spawn in the parent does not necessarily mean that MPI_Init has been called in the children. (although the returned intercommunicator can be used immediately, so like, go off 🫶)

🚨 Important 🚨

ok bestie, real talk, read this first 💀 In Open MPI, this function only works properly when the MPI job is launched via mpirun / mpiexec. Using this function in other run-time environments is currently unsupported, and may result in undefined behavior (e.g., the MPI job may hang). Not "oops teehee" undefined behavior — the kind where your job just sits there staring at you forever. 🙈

Almost all MPI routines return an error value; C routines as the return result of the function and Fortran routines in the last argument.

Before the error value is returned, the current MPI error handler associated with the communication object (e.g., communicator, window, file) is called. If no communication object is associated with the MPI call, then the call is considered attached to MPI_COMM_SELF and will call the associated MPI error handler. When MPI_COMM_SELF is not initialized (i.e., before MPI_Init / MPI_Init_thread, after MPI_Finalize, or when using the Sessions Model exclusively) the error raises the initial error handler. The initial error handler can be changed by calling MPI_Comm_set_errhandler on MPI_COMM_SELF when using the World model, or the mpi_initial_errhandler CLI argument to mpiexec or info key to MPI_Comm_spawn / MPI_Comm_spawn_multiple. If no other appropriate error handler has been set, then the MPI_ERRORS_RETURN error handler is called for MPI I/O functions and the MPI_ERRORS_ABORT error handler is called for all other MPI functions.

Open MPI includes three predefined error handlers that can be used:

  • MPI_ERRORS_ARE_FATAL — Causes the program to abort all connected MPI processes. girl, it's over. 💀
  • MPI_ERRORS_ABORT — An error handler that can be invoked on a communicator, window, file, or session. When called on a communicator, it acts as if MPI_Abort was called on that communicator. If called on a window or file, acts as if MPI_Abort was called on a communicator containing the group of processes in the corresponding window or file. If called on a session, aborts only the local process.
  • MPI_ERRORS_RETURN — Returns an error code to the application. the chill one, she just hands you the receipt 🧾

MPI applications can also implement their own error handlers by calling:

  • MPI_Comm_create_errhandler then MPI_Comm_set_errhandler
  • MPI_File_create_errhandler then MPI_File_set_errhandler
  • MPI_Session_create_errhandler then MPI_Session_set_errhandler or at MPI_Session_init
  • MPI_Win_create_errhandler then MPI_Win_set_errhandler

Note that MPI does not guarantee that an MPI program can continue past an error.

See the MPI man page for a full list of MPI error codes.

See the Error Handling section of the MPI-3.1 standard for more information.

🌈 See also 🌈

  • MPI_Comm_spawn_multiple
  • MPI_Comm_get_parent
  • mpirun(1)

2003-2026, The Open MPI Community

and that's a wrap on the whole page, bestie!! 🎀 this was page MPI_COMM_SPAWN(3) from the Open MPI crew, dated Sep 10, 2026 — go forth, spawn responsibly, and remember: launch it with mpirun / mpiexec or she simply will not behave ✨ slay 💅

📦 where this page comes from