rapidsmpf::coll::SparseAlltoall Class Reference

Sparse all-to-all collective over explicit source and destination peer sets. More...

#include <sparse_alltoall.hpp>

Public Member Functions
	SparseAlltoall (std::shared_ptr< Communicator > comm, OpID op_id, BufferResource *br, std::vector< Rank > srcs, std::vector< Rank > dsts, std::function< void()> &&finished_callback=nullptr)
	Construct a sparse all-to-all collective instance. More...
	SparseAlltoall (SparseAlltoall const &)=delete
SparseAlltoall &	operator= (SparseAlltoall const &)=delete
	SparseAlltoall (SparseAlltoall &&)=delete
SparseAlltoall &	operator= (SparseAlltoall &&)=delete
std::shared_ptr< Communicator > const &	comm () const noexcept
	Gets the communicator associated with this SparseAlltoall. More...
void	insert (Rank dst, PackedData &&packed_data)
	Insert data to send to a destination rank. More...
void	insert_finished ()
	Indicate that no more data will be inserted for any destination. More...
void	wait (std::chrono::milliseconds timeout=std::chrono::milliseconds{-1})
	Wait for local completion. More...
std::vector< PackedData >	extract (Rank src)
	Extract all received messages from a source rank. More...

Detailed Description

Sparse all-to-all collective over explicit source and destination peer sets.

Each rank may send zero or more messages to ranks listed in dsts and receives zero or more messages from ranks listed in srcs. Sender order is defined by the local order of calls to insert(dst, ...) for each destination rank.

This object is logically collective over the communicator and identified by op_id. Local extraction is only valid after wait() has completed.

Definition at line 35 of file sparse_alltoall.hpp.

Constructor & Destructor Documentation

SparseAlltoall()

rapidsmpf::coll::SparseAlltoall::SparseAlltoall	(	std::shared_ptr< Communicator >	comm,
		OpID	op_id,
		BufferResource *	br,
		std::vector< Rank >	srcs,
		std::vector< Rank >	dsts,
		std::function< void()> &&	finished_callback = nullptr
	)

Construct a sparse all-to-all collective instance.

Parameters

comm	Communicator for the collective.
op_id	Collective operation identifier.
br	Buffer resource used for allocations.
srcs	Ranks this rank will receive from.
dsts	Ranks this rank will send to.
finished_callback	Optional callback invoked exactly once when the collective is locally complete. The callback should be fast and non-blocking. Ideally it should only be used to signal a thread to do the actual work of extraction. Note in particular that the callback should not extract any data.

Exceptions

std::out_of_range	If either srcs or dsts have invalid values. All source and destination ranks must be in [0, ..., comm->nranks()), and not equal to the current rank.
std::invalid_argument	If the rank lists are not unique.
std::logic_error	If the communicator or buffer resource pointers are null.

Note

It is safe to reuse the op_id as soon as wait has completed locally or the finished_callback has been invoked.

The caller promises that inserted buffers are stream-ordered with respect to their own stream, and extracted buffers are likewise guaranteed to be stream- ordered with respect to their own stream.

Collectively the src and dst pairs of participating ranks must be consistent (not checked for). That is if rank-A advertises that rank-B is in its dst set, rank-B must advertise that rank-A is in its src set. If we ever need to relax this restriction we could have each rank advertise its send set and bootstrap the two-sided information using the non-blocking consensus algorithm of Hoefler, Siebert, and Lumsdaine, ACM SIGPLAN (2010), https://dl.acm.org/doi/10.1145/1837853.1693476.

Member Function Documentation

comm()

std::shared_ptr<Communicator> const& rapidsmpf::coll::SparseAlltoall::comm ( ) const

noexcept

Gets the communicator associated with this SparseAlltoall.

Returns

Shared pointer to communicator.

extract()

std::vector<PackedData> rapidsmpf::coll::SparseAlltoall::extract

(

Rank

src

)

Extract all received messages from a source rank.

The returned vector is ordered by the sender's local insertion order.

Parameters

src	Source rank. Must be present in the constructor's srcs.

Returns

All messages received from src.

Note

Concurrent extraction is supported, behaviour is undefined if two threads attempt to extract data from the same source.

Exceptions

std::logic_error

If extracting before the collective is complete.

insert()

void rapidsmpf::coll::SparseAlltoall::insert	(	Rank	dst,
		PackedData &&	packed_data
	)

Insert data to send to a destination rank.

The order the destination rank obtains the sent data is given by the insertion order on the send side. If inserting concurrently to the same destination, the caller must establish a total order of the insertions, otherwise the reconstruction order on the receive side is unspecified.

Note

Concurrent insertion by multiple threads is supported.

the caller must ensure that insert_finished() is called after all insert() calls have completed.

Parameters

dst	Destination rank. Must be present in the constructor's dsts.
packed_data	Packed payload and metadata to send.

insert_finished()

void rapidsmpf::coll::SparseAlltoall::insert_finished

(

)

Indicate that no more data will be inserted for any destination.

Must be called exactly once.

Note

If multiple threads are insert()ing, you must establish a happens-before relationship between the completion of all insert()s and the final call to insert_finished().

wait()

void rapidsmpf::coll::SparseAlltoall::wait

(

std::chrono::milliseconds

timeout = std::chrono::milliseconds{-1}

)

Wait for local completion.

Parameters

timeout

Optional timeout. Negative values mean no timeout.

Exceptions

std::runtime_error

If the timeout is reached.

---

The documentation for this class was generated from the following file:

• /__w/rapidsmpf/rapidsmpf/cpp/include/rapidsmpf/coll/sparse_alltoall.hpp