MPI_IALLTOALLW(3)

📦

version 5.0.10 · 4 versions

💖 MPI_IALLTOALLW(3) — Open MPI 💖

okay so bestie, welcome to the Open MPI page for MPI_IALLTOALLW(3). This is the all-to-all where every single block can have its own datatype, which is highkey the most extra energy in the whole collective family. We're obsessed. 🦋✨

✨ NAME ✨

MPI_Alltoallw <#mpi-alltoallw>, MPI_Ialltoallw, MPI_Alltoallw_init <# mpi-alltoallw-init> — all processes send data of different types to, and receive data of different types from, all processes. It's giving full custom-type exchange, no cap. 💅

🌈 SYNTAX 🌈

🍓 C Syntax 🍓

Here's the C glow-up:

#include <mpi.h>

int MPI_Alltoallw(const void *sendbuf, const int sendcounts[],
     const int sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const int rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm)

int MPI_Ialltoallw(const void *sendbuf, const int sendcounts[],
     const int sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const int rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm,
     MPI_Request *request)

int MPI_Alltoallw_init(const void *sendbuf, const int sendcounts[],
     const int sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const int rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm, MPI_Info info,
     MPI_Request *request)

🎀 Fortran Syntax 🎀

Fortran besties, this one's for you:

USE MPI
! or the older form: INCLUDE 'mpif.h'
MPI_ALLTOALLW(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SDISPLS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RDISPLS(*), RECVTYPES(*)
     INTEGER COMM, IERROR

MPI_IALLTOALLW(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, REQUEST, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SDISPLS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RDISPLS(*), RECVTYPES(*)
     INTEGER COMM, REQUEST, IERROR

MPI_ALLTOALLW_INIT(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, INFO, REQUEST, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SDISPLS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RDISPLS(*), RECVTYPES(*)
     INTEGER COMM, INFO, REQUEST, IERROR

🦋 Fortran 2008 Syntax 🦋

And the Fortran 2008 version, because we stay current:

USE mpi_f08
MPI_Alltoallw(sendbuf, sendcounts, sdispls, sendtypes, recvbuf, recvcounts,
             rdispls, recvtypes, comm, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
     TYPE(*), DIMENSION(..) :: recvbuf
     INTEGER, INTENT(IN) :: sendcounts(*), sdispls(*), recvcounts(*),
     rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN) :: sendtypes(*)
     TYPE(MPI_Datatype), INTENT(IN) :: recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Ialltoallw(sendbuf, sendcounts, sdispls, sendtypes, recvbuf,
             recvcounts, rdispls, recvtypes, comm, request, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN), ASYNCHRONOUS :: sendcounts(*), sdispls(*),
     recvcounts(*), rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN), ASYNCHRONOUS :: sendtypes(*),
     recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Alltoallw_init(sendbuf, sendcounts, sdispls, sendtypes, recvbuf,
             recvcounts, rdispls, recvtypes, comm, fIinfo, request, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN), ASYNCHRONOUS :: sendcounts(*), sdispls(*),
     recvcounts(*), rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN), ASYNCHRONOUS :: sendtypes(*),
     recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Info), INTENT(IN) :: info
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

🎀 INPUT PARAMETERS 🎀

Let's go through the inputs, bestie:

sendbuf — the starting address of your send buffer. That's where the data is coming from. 💖
sendcounts — an integer array. Entry i says how many elements you're sending to rank i. Slay.
sdispls — an integer array. Entry i says the displacement (in bytes, offset from sendbuf) from which to send data to rank i. Bytes, not elements — don't play yourself. 💅
sendtypes — a datatype array. Entry i says the datatype to use when sending data to rank i. Custom types, we love to see it.
recvcounts — an integer array. Entry j says how many elements to receive from rank j. Receiving era.
rdispls — an integer array. Entry j says the displacement (in bytes, offset from recvbuf) to which data from rank j should be written. Again: bytes. 👑
recvtypes — a datatype array. Entry j says the datatype to use when receiving data from rank j.
comm — the communicator over which data is to be exchanged. The group chat, basically.
info — Info (handle, persistent only). Yes bestie, only the persistent one. 📌

💖 OUTPUT PARAMETERS 💖

recvbuf — address of the receive buffer. The inbox.
request — request (handle, non-blocking only). Non-blocking only, so don't try this with the blocking one. 🚫
ierror — Fortran only: error status. Fortran girlies get the error code. 💜

💜 DESCRIPTION 💜

MPI_Alltoallw <#mpi-alltoallw> is a generalized collective operation where all processes send data to and receive data from all other processes. It adds flexibility to MPI_Alltoallv <#mpi-alltoallv> by letting you specify the datatype of individual data blocks (in addition to displacement and element count). Its operation can be thought of like this: each process performs 2*n* (where n is the number of processes in communicator comm) independent point-to-point communications, including communication with itself. Iconic? Yes.

MPI_Comm_size(comm, &n);
for (i = 0, i < n; i++)
    MPI_Send(sendbuf + sdispls[i], sendcounts[i],
        sendtypes[i], i, ..., comm);
for (i = 0, i < n; i++)
    MPI_Recv(recvbuf + rdispls[i], recvcounts[i],
        recvtypes[i], i, ..., comm);

Process j sends the k-th block of its local sendbuf to process k, which places the data in the j-th block of its local recvbuf.

When a pair of processes exchanges data, each may pass different element count and datatype arguments as long as the sender specifies the same amount of data to send (in bytes) as the receiver expects to receive. The vibe has to match byte-for-byte, fr fr.

Note that process i may send a different amount of data to process j than it receives from process j. Also, a process may send entirely different amounts and types of data to different processes in the communicator. It's giving personalized data packages for every rank. 🎁

🦋 WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR 🦋

When the communicator is an inter-communicator, the gather operation happens in two phases. First, the data is gathered from all the members of the first group and received by all the members of the second group. Then the data is gathered from all the members of the second group and received by all the members of the first. The operation exhibits a symmetric, full-duplex behavior. It's a two-way slay. 💅

The first group defines the root process. The root process uses MPI_ROOT as the value of root. All other processes in the first group use MPI_PROC_NULL as the value of root. All processes in the second group use the rank of the root process in the first group as the value of root. Got it? Root rules apply. 👑

When the communicator is an intra-communicator, these groups are the same, and the operation occurs in a single phase. Simple. No cap.

💅 USE OF IN-PLACE OPTION 💅

When the communicator is an intracommunicator, you can perform an all-to-all operation in-place — that means the output buffer is used as the input buffer. Use the variable MPI_IN_PLACE as the value of sendbuf. In this case, sendcounts, sdispls, and sendtypes are ignored. The input data of each process is assumed to be in the area where that process would receive its own contribution to the receive buffer. It's giving "I'll just receive myself, thanks." 🙈

⭐ NOTES ⭐

The specification of counts, types, and displacements should not cause any location to be written more than once. Read that again. No double-writing, bestie. 😭

All arguments on all processes are significant. The comm argument, in particular, must describe the same communicator on all processes. No mismatched group chats. 🚫

The offsets of sdispls and rdispls are measured in bytes. Compare this to MPI_Alltoallv <#mpi-alltoallv>, where these offsets are measured in units of sendtype and recvtype, respectively. So don't mix up your byte offsets with your type offsets. Iykyk. 👑

💖 ERRORS 💖

okay bestie, so error handling. Almost all MPI routines return an error value — C routines as the return result of the function, and Fortran routines in the last argument. everyone gets a lil error value, it's giving equal opportunity. ✨

Before that error value comes back at you, the current MPI error handler associated with the communication object (like a communicator, window, or file) gets called. If there's no communication object associated with the MPI call, then the call is considered attached to MPI_COMM_SELF and it'll call the associated MPI error handler. 🫶

And here's the spicy part: when MPI_COMM_SELF is not initialized (like before MPI_Init/MPI_Init_thread, after MPI_Finalize, or when you're using the Sessions Model exclusively) the error raises the initial error handler. That initial error handler can be changed by calling MPI_Comm_set_errhandler on MPI_COMM_SELF when you're using the World model, or via the mpi_initial_errhandler CLI argument to mpiexec or the info key to MPI_Comm_spawn/MPI_Comm_spawn_multiple. If no other appropriate error handler has been set, then the MPI_ERRORS_RETURN error handler is called for MPI I/O functions and the MPI_ERRORS_ABORT error handler is called for all other MPI functions. so like, it defaults to dramatic, got it. 😭

Open MPI ships with three predefined error handlers you can use, and they're iconic:

MPI_ERRORS_ARE_FATAL — Causes the program to abort all connected MPI processes. no survivors, fr fr 💀
MPI_ERRORS_ABORT — An error handler that can be invoked on a communicator, window, file, or session. When called on a communicator, it acts as if MPI_Abort was called on that communicator. If called on a window or file, acts as if MPI_Abort was called on a communicator containing the group of processes in the corresponding window or file. If called on a session, it aborts only the local process. context-aware chaos, we love to see it
MPI_ERRORS_RETURN — Returns an error code to the application. chill one, hands it back to you 🎀

MPI applications can also implement their own error handlers by calling, in these lil make-it-yourself combos:

MPI_Comm_create_errhandler then MPI_Comm_set_errhandler
MPI_File_create_errhandler then MPI_File_set_errhandler
MPI_Session_create_errhandler then MPI_Session_set_errhandler or at MPI_Session_init
MPI_Win_create_errhandler then MPI_Win_set_errhandler

⚠️ Note that MPI does not guarantee that an MPI program can continue past an error. like, it might just be over bestie, don't build your whole plan around surviving it. 🙈

See the MPI man page for a full list of MPI error codes. See the Error Handling section of the MPI-3.1 standard for more information. 📚

👑 Copyright 👑

2003-2026, The Open MPI Community

and that's a wrap on MPI_IALLTOALLW(3) bestie 🦋✨ — this lil masterpiece is dated Sep 10, 2026, brought to you by the Open MPI girlies, and it ate. go forth and scatter your data in style, no cap. 💅💜

📦 where this page comes from

ubuntu stonking: openmpi-doc 5.0.10-6 · other
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms