MPI_Allreduce(3)

💖 MPI_Allreduce — the group chat, but for numbers (from the MPI manual, section 3) ✨

okay so bestie. imagine everyone in the group chat has a number, and you want EVERYONE to know the final answer. that's this. MPI_Allreduce combines values from all processes and distributes the result back to all processes. no one gets left on read. 😌

---

🌈 SYNOPSIS 🌈

int MPI_Allreduce(const void *sendbuf, void *recvbuf, int count,
MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)

int MPI_Allreduce_c(const void *sendbuf, void *recvbuf, MPI_Count count,
MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)

two whole flavors, bestie, the classic and the _c one with the big-boy MPI_Count. iconic. 👑

---

🫶 INPUT PARAMETERS 🫶

• sendbuf — starting address of send buffer (choice). it's giving "here's my contribution to the group project" 🎀
• count — number of elements in send buffer (non-negative integer). must be >= 0, we do not do negative counts here 💅
• datatype — data type of elements of send buffer (handle)
• op — operation (handle). like, what are we doing to these numbers, MPI_SUM? combin', babes 🔥
• comm — communicator (handle). the group chat itself

🎀 OUTPUT PARAMETERS 🎀

• recvbuf — starting address of receive buffer (choice). where the final answer lands, for everybody, equally. main character energy for every single process fr fr 💜

---

💀 NOTES ON COLLECTIVE OPERATIONS 💀

the reduction functions (MPI_Op) do not return an error value. that's the whole vibe and honestly it's a little scary.

so because of that, if the functions detect an error, all they can do is either call MPI_Abort or silently skip the problem. yes, silently. no cap, no error message, just pretending nothing happened 🙈

so like, if you change the error handler from MPI_ERRORS_ARE_FATAL to something else — for example MPI_ERRORS_RETURN — then no error may be indicated. you asked for the error and the error did not come. tragic.

and why?? the reason for this is the performance problems in ensuring that all collective routines return the same error value. it's a whole thing. periodt. ✨

---

🛡️ THREAD AND INTERRUPT SAFETY 🛡️

this routine is thread-safe 💪 that means it may be safely used by multiple threads without needing any user-provided thread locks. no extra locking homework, we love that.

however — and this is the plot twist — the routine is not interrupt safe 😭 typically this is due to the use of memory allocation routines such as malloc or other non-MPICH runtime routines that are themselves not interrupt-safe. so like, safe with your threads, not safe with your interrupts. iykyk.

---

🧚 NOTES FOR FORTRAN 🧚

all MPI routines in Fortran (except for MPI_WTIME and MPI_WTICK) have an additional argument ierr at the end of the argument list. ierr is an integer and has the same meaning as the return value of the routine in C.

in Fortran, MPI routines are subroutines, and they are invoked with the call statement. (so call-girl behavior only 🌸)

all MPI objects (e.g., MPI_Datatype, MPI_Comm) are of type INTEGER in Fortran.

---

⚠️ ERRORS ⚠️

all MPI routines (except MPI_Wtime and MPI_Wtick) return an error value: C routines as the value of the function, and Fortran routines in the last argument. before the value is returned, the current MPI error handler is called.

by default, this error handler aborts the MPI job. like, whole program, gone, no survivors 💀

the error handler may be changed with MPI_Comm_set_errhandler (for communicators), MPI_File_set_errhandler (for files), and MPI_Win_set_errhandler (for RMA windows). the MPI-1 routine MPI_Errhandler_set may be used but its use is deprecated — so like, it's giving flip phone, it works but bestie, why 🫠

the predefined error handler MPI_ERRORS_RETURN may be used to cause error values to be returned. note that MPI does not guarantee that an MPI program can continue past an error; however, MPI implementations will attempt to continue whenever possible. so it's a "we'll try" not a "we promise" 💅

here's the error lineup, one bullet each because we respect the list:

• MPI_SUCCESS — no error; MPI routine completed successfully. we ate and left no crumbs 👑
• MPI_ERR_BUFFER — invalid buffer pointer. usually a null buffer where one is not valid. (a null buffer is not it, babe)
• MPI_ERR_COMM — invalid communicator. a common error is to use a null communicator in a call (not even allowed in MPI_Comm_rank). yes, that's how serious this is.
• MPI_ERR_COUNT — invalid count argument. count arguments must be non-negative; a count of zero is often valid. zero is fine, negative is not 🚫
• MPI_ERR_OP — invalid operation. MPI operations (objects of type MPI_Op) must either be one of the predefined operations (e.g., MPI_SUM) or created with MPI_Op_create.
• MPI_ERR_TYPE — invalid datatype argument. additionally, this error can occur if an uncommitted MPI_Datatype (see MPI_Type_commit) is used in a communication call. commit your types first, bestie, that's the rule 🌸
• MPI_ERR_OTHER — other error; use MPI_Error_string to get more information about this error code.

---

okay that's the whole page, ily, go reduce something 💖 this was MPI_Allreduce(3) from the MPI manual, dated 4/10/2026, and that's a wrap, periodt ✨🎀🦋