MPI_IALLREDUCE(3)

📦

version 5.0.10 · 4 versions

👑 MPI_IALLREDUCE(3) 👑

okay so bestie, welcome to the Open MPI family reunion 💖 this page covers three besties who do the exact same glow-up: MPI_Allreduce, MPI_Iallreduce, and MPI_Allreduce_init.

and what do they do? they combine values from all processes and distribute the result back to all processes. that's it. that's the whole vibe. everyone contributes, everyone gets the answer back. slay. ✨

📜 SYNTAX 📜

🩷 C Syntax

#include <mpi.h>

int MPI_Allreduce(const void *sendbuf, void *recvbuf, int count,
                  MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)

int MPI_Iallreduce(const void *sendbuf, void *recvbuf, int count,
                   MPI_Datatype datatype, MPI_Op op, MPI_Comm comm,
                   MPI_Request *request)

int MPI_Allreduce_init(const void *sendbuf, void *recvbuf, int count,
                       MPI_Datatype datatype, MPI_Op op, MPI_Comm comm,
                       MPI_Info info, MPI_Request *request)

🦋 Fortran Syntax

USE MPI
! or the older form: INCLUDE 'mpif.h'
MPI_ALLREDUCE(SENDBUF, RECVBUF, COUNT, DATATYPE, OP, COMM, IERROR)
     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER COUNT, DATATYPE, OP, COMM, IERROR

MPI_IALLREDUCE(SENDBUF, RECVBUF, COUNT, DATATYPE, OP, COMM, REQUEST, IERROR)
     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER COUNT, DATATYPE, OP, COMM, REQUEST, IERROR

MPI_ALLREDUCE_INIT(SENDBUF, RECVBUF, COUNT, DATATYPE, OP, COMM, INFO, REQUEST, IERROR)
     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER COUNT, DATATYPE, OP, COMM, INFO, REQUEST, IERROR

🌸 Fortran 2008 Syntax

USE mpi_f08
MPI_Allreduce(sendbuf, recvbuf, count, datatype, op, comm, ierror)
     TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
     TYPE(*), DIMENSION(..) :: recvbuf
     INTEGER, INTENT(IN) :: count
     TYPE(MPI_Datatype), INTENT(IN) :: datatype
     TYPE(MPI_Op), INTENT(IN) :: op
     TYPE(MPI_Comm), INTENT(IN) :: comm
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Iallreduce(sendbuf, recvbuf, count, datatype, op, comm, request,
             ierror)
     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN) :: count
     TYPE(MPI_Datatype), INTENT(IN) :: datatype
     TYPE(MPI_Op), INTENT(IN) :: op
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Allreduce_init(sendbuf, recvbuf, count, datatype, op, comm, info, request,
             ierror)
     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN) :: count
     TYPE(MPI_Datatype), INTENT(IN) :: datatype
     TYPE(MPI_Op), INTENT(IN) :: op
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Info), INTENT(IN) :: info
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

📥 INPUT PARAMETERS 📥

sendbuf — the starting address of the send buffer. it's a choice type, so you've got options bestie 💅
count — how many elements are chilling in the send buffer (integer).
datatype — the datatype of the elements in the send buffer (handle).
op — the operation you're applying (handle).
comm — the communicator (handle).
info — Info (handle, persistent only). yes, only the persistent girlies get one.

📤 OUTPUT PARAMETERS 📤

recvbuf — the starting address of the receive buffer (choice).
request — Request (handle, non-blocking only).
ierror — Fortran only: error status (integer).

💖 DESCRIPTION 💖

literally the same as MPI_Reduce, except the result shows up in the receive buffer of all the group members, not just one. main character energy for everybody, no gatekeeping 🫶

Example 1: a routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at all nodes (compare with Example 2, with MPI_Reduce, below).

SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
REAL a(m), b(m,n)    ! local slice of array
REAL c(n)            ! result
REAL sum(n)
INTEGER n, comm, i, j, ierr

! local sum
DO j= 1, n
  sum(j) = 0.0
  DO i = 1, m
    sum(j) = sum(j) + a(i)*b(i,j)
  END DO
END DO

! global sum
CALL MPI_ALLREDUCE(sum, c, n, MPI_REAL, MPI_SUM, comm, ierr)

! return result at all nodes
RETURN

Example 2: a routine that computes the product of a vector and an array that are distributed across a group of processes and returns the answer at node zero.

SUBROUTINE PAR_BLAS2(m, n, a, b, c, comm)
REAL a(m), b(m,n)    ! local slice of array
REAL c(n)            ! result
REAL sum(n)
INTEGER n, comm, i, j, ierr

! local sum
DO j= 1, n
  sum(j) = 0.0
  DO i = 1, m
    sum(j) = sum(j) + a(i)*b(i,j)
  END DO
END DO

! global sum
CALL MPI_REDUCE(sum, c, n, MPI_REAL, MPI_SUM, 0, comm, ierr)

! return result at node zero (and garbage at the other nodes)
RETURN

🔄 USE OF IN-PLACE OPTION 🔄

when the communicator is an intracommunicator, you can do an all-reduce in-place — meaning the output buffer is also the input buffer, we love a multitasking queen. just use the variable MPI_IN_PLACE as the value of sendbuf at all processes. all of them. no exceptions bestie.

ngl, MPI_IN_PLACE is a special kind of value — it comes with the same restrictions on its use as MPI_BOTTOM. so don't get cute with it.

and because the in-place option turns the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT. read that again. INOUT. 💀

🧚 WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR 🧚

when the communicator is an inter-communicator, the reduce operation happens in two phases, okay? phase one: the data is reduced from all the members of the first group and received by all the members of the second group. then phase two: the data is reduced from all the members of the second group and received by all the members of the first. the whole operation exhibits a symmetric, full-duplex behavior. it's giving very balanced, very equal, very iconique.

when the communicator is an intra-communicator, these groups are literally the same thing, so the operation only happens in a single phase. simpler queen. ✨

🎀 NOTES ON COLLECTIVE OPERATIONS 🎀

the reduction functions (MPI_Op) do not return an error value. so if they detect an error, all they can do is either call MPI_Abort or silently skip the problem. that's it, that's the whole toolbox 😭

so if you change the error handler from MPI_ERRORS_ARE_FATAL to something else — for example, MPI_ERRORS_RETURN — then no error may be indicated. yep. you might just get nothing. stay safe out there.

😭 ERRORS 😭

almost all MPI routines return an error value — C routines hand it back as the return result of the function, and Fortran routines tuck it into the last argument.

before that error value gets returned, the current MPI error handler associated with the communication object (e.g., communicator, window, file) gets called. and if there's no communication object associated with the MPI call, then the call counts as attached to MPI_COMM_SELF and will call the associated MPI error handler.

when MPI_COMM_SELF is not initialized — meaning before MPI_Init/MPI_Init_thread, after MPI_Finalize, or when using the Sessions Model exclusively — the error raises the initial error handler. and you can change that initial error handler by calling MPI_Comm_set_errhandler on MPI_COMM_SELF when using the World model, or with the mpi_initial_errhandler CLI argument to mpiexec, or the info key to MPI_Comm_spawn/MPI_Comm_spawn_multiple.

and if no other appropriate error handler has been set? then MPI_ERRORS_RETURN gets called for MPI I/O functions, and MPI_ERRORS_ABORT gets called for all other MPI functions. okay so that's the fallback plan, noted. 📝

Open MPI ships with three predefined error handlers you can use:

MPI_ERRORS_ARE_FATAL — causes the program to abort all connected MPI processes. everyone goes down together, no survivors 💀
MPI_ERRORS_ABORT — an error handler that can be invoked on a communicator, window, file, or session. when called on a communicator, it acts as if MPI_Abort was called on that communicator. if called on a window or file, it acts as if MPI_Abort was called on a communicator containing the group of processes in the corresponding window or file. if called on a session, it aborts only the local process.
MPI_ERRORS_RETURN — returns an error code to the application. polite!

MPI applications can also implement their own error handlers by calling:

MPI_Comm_create_errhandler then MPI_Comm_set_errhandler
MPI_File_create_errhandler then MPI_File_set_errhandler
MPI_Session_create_errhandler then MPI_Session_set_errhandler or at MPI_Session_init
MPI_Win_create_errhandler then MPI_Win_set_errhandler

fr fr, note that MPI does not guarantee that an MPI program can continue past an error. so like, don't count on it bestie.

see the MPI man page for a full list of MPI error codes, and see the Error Handling section of the MPI-3.1 standard for more information.

🫶 SIGN-OFF 🫶

and that's the page!! copyright is 2003-2026, The Open MPI Community, this version is dated Sep 10, 2026, and it's page MPI_IALLREDUCE(3) of the Open MPI manual. go reduce something, you absolute legend 👑💅✨

📦 where this page comes from

ubuntu stonking: openmpi-doc 5.0.10-6 · other
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms