MPI_INEIGHBOR_ALLTOALLW(3)

📦

version 5.0.10 · 4 versions

💅 MPI_INEIGHBOR_ALLTOALLW 💅

okay bestie sit down, we're reading the Open MPI manual page for MPI_Ineighbor_alltoallw and its two sibling functions, and ngl it's kind of a lot but we're gonna get through it together. grab a snack. 🍓

✨ NAME ✨

So there's a whole little family here, three functions, one page:

MPI_Neighbor_alltoallw
MPI_Ineighbor_alltoallw
MPI_Neighbor_alltoallw_init

and what they do, like as a group, is: all processes send data of different types to, and receive data of different types from, all processes. yes really. different types!! every single neighbor!! iconic behavior honestly. 👑

🛠️ SYNTAX 🛠️

C Syntax

#include <mpi.h>

int MPI_Neighbor_alltoallw(const void *sendbuf, const int sendcounts[],
     const MPI_Aint sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const MPI_Aint rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm)

int MPI_Ineighbor_alltoallw(const void *sendbuf, const int sendcounts[],
     const MPI_Aint sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const MPI_Aint rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm, MPI_Request *request)

int MPI_Neighbor_alltoallw_init(const void *sendbuf, const int sendcounts[],
     const MPI_Aint sdispls[], const MPI_Datatype sendtypes[],
     void *recvbuf, const int recvcounts[], const MPI_Aint rdispls[],
     const MPI_Datatype recvtypes[], MPI_Comm comm, MPI_Info info, MPI_Request *request)

Fortran Syntax

USE MPI
! or the older form: INCLUDE 'mpif.h'
MPI_NEIGHBOR_ALLTOALLW(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RECVTYPES(*)
     INTEGER(KIND=MPI_ADDRESS_KIND) SDISPLS(*), RDISPLS(*)
     INTEGER COMM, IERROR

MPI_INEIGHBOR_ALLTOALLW(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, REQUEST, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RECVTYPES(*)
     INTEGER(KIND=MPI_ADDRESS_KIND) SDISPLS(*), RDISPLS(*)
     INTEGER COMM, REQUEST, IERROR

MPI_NEIGHBOR_ALLTOALLW_INIT(SENDBUF, SENDCOUNTS, SDISPLS, SENDTYPES,
     RECVBUF, RECVCOUNTS, RDISPLS, RECVTYPES, COMM, INFO, REQUEST, IERROR)

     <type>  SENDBUF(*), RECVBUF(*)
     INTEGER SENDCOUNTS(*), SENDTYPES(*)
     INTEGER RECVCOUNTS(*), RECVTYPES(*)
     INTEGER(KIND=MPI_ADDRESS_KIND) SDISPLS(*), RDISPLS(*)
     INTEGER COMM, INFO, REQUEST, IERROR

Fortran 2008 Syntax

USE mpi_f08
MPI_Neighbor_alltoallw(sendbuf, sendcounts, sdispls, sendtypes, recvbuf,
             recvcounts, rdispls, recvtypes, comm, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
     TYPE(*), DIMENSION(..) :: recvbuf
     INTEGER, INTENT(IN) :: sendcounts(*), recvcounts(*)
     INTEGER(KIND=MPI_ADDRESS_KIND), INTENT(IN) :: sdispls(*), rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN) :: sendtypes(*), recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Ineighbor_alltoallw(sendbuf, sendcounts, sdispls, sendtypes, recvbuf,
             recvcounts, rdispls, recvtypes, comm, request, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN), ASYNCHRONOUS :: sendcounts(*), recvcounts(*)
     INTEGER(KIND=MPI_ADDRESS_KIND), INTENT(IN), ASYNCHRONOUS ::
     sdispls(*), rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN), ASYNCHRONOUS :: sendtypes(*),
     recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

MPI_Neighbor_alltoallw_init(sendbuf, sendcounts, sdispls, sendtypes, recvbuf,
             recvcounts, rdispls, recvtypes, comm, info, request, ierror)

     TYPE(*), DIMENSION(..), INTENT(IN), ASYNCHRONOUS :: sendbuf
     TYPE(*), DIMENSION(..), ASYNCHRONOUS :: recvbuf
     INTEGER, INTENT(IN), ASYNCHRONOUS :: sendcounts(*), recvcounts(*)
     INTEGER(KIND=MPI_ADDRESS_KIND), INTENT(IN), ASYNCHRONOUS ::
     sdispls(*), rdispls(*)
     TYPE(MPI_Datatype), INTENT(IN), ASYNCHRONOUS :: sendtypes(*),
     recvtypes(*)
     TYPE(MPI_Comm), INTENT(IN) :: comm
     TYPE(MPI_Info), INTENT(IN) :: info
     TYPE(MPI_Request), INTENT(OUT) :: request
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

like yes, the ASYNCHRONOUS keywords are doing the most but they're important, don't touch them. 🙈

📥 INPUT PARAMETERS 📥

sendbuf — the starting address of the send buffer. that's where the goods leave from. 📦
sendcounts — an integer array, and entry i says how many elements to send to neighbor i.
sdispls — an integer array, entry i is the displacement (in bytes, offset from sendbuf) to send data to neighbor i from.
sendtypes — a datatype array, entry i is the datatype to use when sending data to neighbor i. yes you pick per-neighbor. main character flexibility. ✨
recvcounts — an integer array, entry j is how many elements to receive from neighbor j.
rdispls — an integer array, entry j is the displacement (in bytes, offset from recvbuf) where data from neighbor j should be written.
recvtypes — a datatype array, entry j is the datatype to use when receiving data from neighbor j.
comm — the communicator over which all this data is gonna be exchanged. the group chat, basically. 🫶
info — Info (handle, persistent only).

📤 OUTPUT PARAMETERS 📤

recvbuf — address of the receive buffer. where the gifts land. 🎁
request — Request (handle, non-blocking only).
ierror — Fortran only: error status.

🌈 DESCRIPTION 🌈

MPI_Neighbor_alltoallw is a generalized collective operation where literally every process sends data to and receives data from all of its neighbors. she's giving all-to-all but make it neighbor-aware.

so how is this different from MPI_Neighbor_alltoallv? bestie it adds flexibility 💅 — it lets you specify the datatype of individual data blocks, not just displacement and element count. so you get per-block type control. obsessed.

the way to think about it: each process performs 2n independent point-to-point communications, where n is the number of neighbors in the topology of communicator comm. and the neighbors plus the buffer layout are both determined by the topology of comm. no cap.

here's the vibe in code, verbatim:

MPI_Cart_get(comm, maxdims, dims, periods, coords);
for (dim = 0, i = 0 ; dim < dims ; ++dim) {
    MPI_Cart_shift(comm, dim, 1, &r0, &r1);
    MPI_Isend(sendbuf + sdispls[i]  * extent(sendtype),
              sendcount, sendtypes[i], r0, ..., comm, ...);
    MPI_Irecv(recvbuf + rdispls[i] * extent(recvtype),
              recvcount, recvtypes[i], r0, ..., comm, ...);
    ++i;
    MPI_Isend(sendbuf + sdispls[i] * extent(sendtype),
              sendcount, sendtypes[i], r1, ..., comm, &req[i]);
    MPI_Irecv(recvbuf + rdispls[i] * extent(recvtype),
              recvcount, recvtypes[i], r1, ..., comm, ...);
    ++i;
}

MPI_Wait_all (...);

MPI_Comm_size(comm, &n);
for (i = 0, i < n; i++)
  MPI_Send(sendbuf + sdispls[i], sendcounts[i],
       sendtypes[i], i, ..., comm);
for (i = 0, i < n; i++)
  MPI_Recv(recvbuf + rdispls[i], recvcounts[i],
       recvtypes[i], i, ..., comm);

so process j sends the k-th block of its local sendbuf to neighbor k, and that neighbor drops the data into the j-th block of its local recvbuf. cute little handoff. 🤝

when a pair of processes exchanges data, each one may pass different element count and datatype arguments — as long as the sender specifies the same amount of data to send (in bytes) as the receiver expects to receive. that's the deal. bytes must match, everything else is your business.

also note: process i may send a different amount of data to process j than it receives from process j. and a process may send entirely different amounts and types of data to different processes in the communicator. it's giving chaotic good. she contains multitudes. 🔥

📝 NOTES 📝

bestie read these, they're the rules not the vibes.

The MPI_IN_PLACE option for sendbuf is not meaningful for this operation. don't do it. it's not it.
The specification of counts, types, and displacements should not cause any location to be written more than once. seriously — writing the same spot twice is an undefined-behavior situation and you do not want that smoke.
All arguments on all processes are significant. The comm argument, in particular, must describe the same communicator on all processes. everybody has to be on the same page, fr fr.
The offsets of sdispls and rdispls are measured in bytes. compare this to MPI_Neighbor_alltoallv, where those offsets are measured in units of sendtype and recvtype, respectively. so yeah. bytes here, not types. this is the classic gotcha, write it on your hand. 🖊️

🔥 ERRORS 🔥

okay bestie this part is important so lock in 💅

Almost all MPI routines return an error value — and like, where does it go? For the C routines it's the return result of the function, and for Fortran routines it's the last argument. classic.

But listen — before that error value ever gets returned, the current MPI error handler that's associated with the communication object (think communicator, window, file, whatever) gets called. ✨

And if there's no communication object associated with the MPI call? Then the call is considered attached to MPI_COMM_SELF and it'll call the error handler associated with that. 🫶

Now here's the plot twist: when MPI_COMM_SELF is not initialized — meaning before MPI_Init/MPI_Init_thread, or after MPI_Finalize, or when you're using the Sessions Model exclusively — the error raises the initial error handler. 😭

And you can change that initial error handler, no cap:

by calling MPI_Comm_set_errhandler on MPI_COMM_SELF when you're using the World model, or
with the mpi_initial_errhandler CLI argument to mpiexec, or
with the info key to MPI_Comm_spawn/MPI_Comm_spawn_multiple.

If no other appropriate error handler has been set, then the MPI_ERRORS_RETURN error handler gets called for MPI I/O functions, and the MPI_ERRORS_ABORT error handler gets called for all other MPI functions. periodt. ⭐

💖 the three predefined error handlers in Open MPI 💖

Open MPI ships three of them and they're iconic:

MPI_ERRORS_ARE_FATAL — Causes the program to abort all connected MPI processes. 💀
MPI_ERRORS_ABORT — An error handler that can be invoked on a communicator, window, file, or session. When called on a communicator, it acts as if MPI_Abort was called on that communicator. If called on a window or file, it acts as if MPI_Abort was called on a communicator containing the group of processes in the corresponding window or file. If called on a session, it aborts only the local process.
MPI_ERRORS_RETURN — Returns an error code to the application. (the "just tell me what went wrong, I'll handle it" girlie 🎀)

🌸 making your own error handler 🌸

MPI applications can also implement their own error handlers, and the way you do that is by calling:

MPI_Comm_create_errhandler then MPI_Comm_set_errhandler
MPI_File_create_errhandler then MPI_File_set_errhandler
MPI_Session_create_errhandler then MPI_Session_set_errhandler or at MPI_Session_init
MPI_Win_create_errhandler then MPI_Win_set_errhandler

😭 heads up: note that MPI does not guarantee that an MPI program can continue past an error. like, fr fr, don't count on it.

See the MPI man page for a full list of MPI error codes. 👑

See the Error Handling section of the MPI-3.1 standard for more information.

🦋 See also:

MPI_Neighbor_alltoall
MPI_Neighbor_alltoallv
MPI_Cart_create
MPI_Graph_create
MPI_Dist_graph_create

💜 Copyright 💜

2003-2026, The Open MPI Community

and that's the whole page bestie, we ate and left no crumbs 💅✨ this version is the MPI_INEIGHBOR_ALLTOALLW(3) page dated Sep 10, 2026, straight from The Open MPI Community — go forth and neighbor-alltoall-w your little heart out, ily 🫶🌈💖

📦 where this page comes from

ubuntu stonking: openmpi-doc 5.0.10-6 · other
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms