mpi4py.util.pkl5

Added in version 3.1.0.

pickle protocol 5 (see PEP 574) introduced support for out-of-band buffers, allowing for more efficient handling of certain object types with large memory footprints.

MPI for Python uses the traditional in-band handling of buffers. This approach is appropriate for communicating non-buffer Python objects, or buffer-like objects with small memory footprints. For point-to-point communication, in-band buffer handling allows for the communication of a pickled stream with a single MPI message, at the expense of additional CPU and memory overhead in the pickling and unpickling steps.

The mpi4py.util.pkl5 module provides communicator wrapper classes reimplementing pickle-based point-to-point and collective communication methods using pickle protocol 5. Handling out-of-band buffers necessarily involves multiple MPI messages, thus increasing latency and hurting performance in case of small size data. However, in case of large size data, the zero-copy savings of out-of-band buffer handling more than offset the extra latency costs. Additionally, these wrapper methods overcome the infamous 2 GiB message count limit (MPI-1 to MPI-3).

Note

Support for pickle protocol 5 is available in the pickle module within the Python standard library since Python 3.8. Previous Python 3 releases can use the pickle5 backport, which is available on PyPI and can be installed with:

python -m pip install pickle5

class mpi4py.util.pkl5.Request

Request.

Custom request class for nonblocking communications.

Note

Request is not a subclass of mpi4py.MPI.Request

Free()

Free a communication request.

Return type:

None

free()

Free a communication request.

Return type:

None

cancel()

Cancel a communication request.

Return type:

None

get_status(status=None)

Non-destructive test for the completion of a request.

Parameters:

status (Status | None)

Return type:

bool

test(status=None)

Test for the completion of a request.

Parameters:

status (Status | None)

Return type:

tuple[bool, Any | None]

wait(status=None)

Wait for a request to complete.

Parameters:

status (Status | None)

Return type:

Any

classmethod get_status_all(requests, statuses=None)

Non-destructive test for the completion of all requests.

Classmethod:

classmethod testall(requests, statuses=None)

Test for the completion of all requests.

Classmethod:

classmethod waitall(requests, statuses=None)

Wait for all requests to complete.

Classmethod:

class mpi4py.util.pkl5.Message

Message.

Custom message class for matching probes.

Note

Message is not a subclass of mpi4py.MPI.Message

free()

Do nothing.

Return type:

None

recv(status=None)

Blocking receive of matched message.

Parameters:

status (Status | None)

Return type:

Any

irecv()

Nonblocking receive of matched message.

Return type:

Request

classmethod probe(comm, source=ANY_SOURCE, tag=ANY_TAG, status=None)

Blocking test for a matched message.

Classmethod:

classmethod iprobe(comm, source=ANY_SOURCE, tag=ANY_TAG, status=None)

Nonblocking test for a matched message.

Classmethod:

class mpi4py.util.pkl5.Comm

Communicator.

Base communicator wrapper class.

send(obj, dest, tag=0)

Blocking send in standard mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

None

bsend(obj, dest, tag=0)

Blocking send in buffered mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

None

ssend(obj, dest, tag=0)

Blocking send in synchronous mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

None

isend(obj, dest, tag=0)

Nonblocking send in standard mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

Request

ibsend(obj, dest, tag=0)

Nonblocking send in buffered mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

Request

issend(obj, dest, tag=0)

Nonblocking send in synchronous mode.

Parameters:

• obj (Any)

• dest (int)

• tag (int)

Return type:

Request

recv(buf=None, source=ANY_SOURCE, tag=ANY_TAG, status=None)

Blocking receive.

Parameters:

• buf (Buffer | None)

• source (int)

• tag (int)

• status (Status | None)

Return type:

Any

irecv(buf=None, source=ANY_SOURCE, tag=ANY_TAG)

Nonblocking receive.

Warning

This method cannot be supported reliably and raises RuntimeError.

Parameters:

• buf (Buffer | None)

• source (int)

• tag (int)

Return type:

Request

sendrecv(sendobj, dest, sendtag=0, recvbuf=None, source=ANY_SOURCE, recvtag=ANY_TAG, status=None)

Send and receive.

Parameters:

• sendobj (Any)

• dest (int)

• sendtag (int)

• recvbuf (Buffer | None)

• source (int)

• recvtag (int)

• status (Status | None)

Return type:

Any

mprobe(source=ANY_SOURCE, tag=ANY_TAG, status=None)

Blocking test for a matched message.

Parameters:

• source (int)

• tag (int)

• status (Status | None)

Return type:

Message

improbe(source=ANY_SOURCE, tag=ANY_TAG, status=None)

Nonblocking test for a matched message.

Parameters:

• source (int)

• tag (int)

• status (Status | None)

Return type:

Message | None

bcast(obj, root=0)

Broadcast.

Added in version 3.1.0.

Parameters:

• obj (Any)

• root (int)

Return type:

Any

gather(sendobj, root=0)

Gather.

Added in version 4.0.0.

Parameters:

• sendobj (Any)

• root (int)

Return type:

list[Any] | None

scatter(sendobj, root=0)

Scatter.

Added in version 4.0.0.

Parameters:

• sendobj (Sequence[Any] | None)

• root (int)

Return type:

Any

allgather(sendobj)

Gather to All.

Added in version 4.0.0.

Parameters:

sendobj (Any)

Return type:

list[Any]

alltoall(sendobj)

All to All Scatter/Gather.

Added in version 4.0.0.

Parameters:

sendobj (Sequence[Any])

Return type:

list[Any]

class mpi4py.util.pkl5.Intracomm

Intracommunicator.

Intracommunicator wrapper class.

class mpi4py.util.pkl5.Intercomm

Intercommunicator.

Intercommunicator wrapper class.

Examples

test-pkl5-1.py

import numpy as np
from mpi4py import MPI
from mpi4py.util import pkl5

comm = pkl5.Intracomm(MPI.COMM_WORLD)  # comm wrapper
size = comm.Get_size()
rank = comm.Get_rank()
dst = (rank + 1) % size
src = (rank - 1) % size

sobj = np.full(1024**3, rank, dtype='i4')  # > 4 GiB
sreq = comm.isend(sobj, dst, tag=42)
robj = comm.recv (None, src, tag=42)
sreq.Free()

assert np.min(robj) == src
assert np.max(robj) == src

test-pkl5-2.py

import numpy as np
from mpi4py import MPI
from mpi4py.util import pkl5

comm = pkl5.Intracomm(MPI.COMM_WORLD)  # comm wrapper
size = comm.Get_size()
rank = comm.Get_rank()
dst = (rank + 1) % size
src = (rank - 1) % size

sobj = np.full(1024**3, rank, dtype='i4')  # > 4 GiB
sreq = comm.isend(sobj, dst, tag=42)

status = MPI.Status()
rmsg = comm.mprobe(status=status)
assert status.Get_source() == src
assert status.Get_tag() == 42
rreq = rmsg.irecv()
robj = rreq.wait()

sreq.Free()
assert np.max(robj) == src
assert np.min(robj) == src