Abstract base class for a communication mechanism between nodes. More...

#include <communicator.hpp>

Inheritance diagram for rapidsmpf::Communicator:

Classes
class	Future
	Abstract base class for asynchronous operation within the communicator. More...

class	Logger
	A logger base class for handling different levels of log messages. More...

Public Member Functions
virtual Rank	rank () const =0
	Retrieves the rank of the current node. More...

virtual Rank	nranks () const =0
	Retrieves the total number of ranks. More...

virtual std::unique_ptr< Future >	send (std::unique_ptr< std::vector< std::uint8_t >> msg, Rank rank, Tag tag)=0
	Sends a host message to a specific rank. More...

virtual std::unique_ptr< Future >	send (std::unique_ptr< Buffer > msg, Rank rank, Tag tag)=0
	Sends a message (device or host) to a specific rank. Use `release_data` to obtain the data buffer again once the future is completed. More...

virtual std::unique_ptr< Future >	recv (Rank rank, Tag tag, std::unique_ptr< Buffer > recv_buffer)=0
	Receives a message from a specific rank to a buffer. Use `release_data` to extract the data out of the buffer once the future is completed. More...

virtual std::unique_ptr< Future >	recv_sync_host_data (Rank rank, Tag tag, std::unique_ptr< std::vector< std::uint8_t >> synced_buffer)=0
	Receives a message from a specific rank to an allocated (synchronized) host buffer. Use `release_sync_host_data` to extract the data out of the buffer once the future is completed. More...

virtual std::pair< std::unique_ptr< std::vector< std::uint8_t > >, Rank >	recv_any (Tag tag)=0
	Receives a message from any rank (blocking). More...

virtual std::unique_ptr< std::vector< std::uint8_t > >	recv_from (Rank src, Tag tag)=0
	Receives a message from a specific rank (blocking). More...

virtual std::pair< std::vector< std::unique_ptr< Future > >, std::vector< std::size_t > >	test_some (std::vector< std::unique_ptr< Future >> &future_vector)=0
	Tests for completion of multiple futures. More...

virtual std::vector< std::size_t >	test_some (std::unordered_map< std::size_t, std::unique_ptr< Communicator::Future >> const &future_map)=0
	Tests for completion of multiple futures in a map. More...

virtual bool	test (std::unique_ptr< Communicator::Future > &future)=0
	Test for completion of a single future. More...

virtual std::vector< std::unique_ptr< Buffer > >	wait_all (std::vector< std::unique_ptr< Communicator::Future >> &&futures)=0
	Wait for completion of all futures and return their data buffers. More...

virtual std::unique_ptr< Buffer >	wait (std::unique_ptr< Future > future)=0
	Wait for a future to complete and return the data buffer. More...

virtual std::unique_ptr< Buffer >	release_data (std::unique_ptr< Communicator::Future > future)=0
	Retrieves data associated with a completed future. More...

virtual std::unique_ptr< std::vector< std::uint8_t > >	release_sync_host_data (std::unique_ptr< Communicator::Future > future)=0
	Retrieves synchronized host data associated with a completed future. When the future is completed, the the host data is valid, and ready, but not stream-ordered. More...

virtual std::shared_ptr< Communicator::Logger > const &	logger ()=0
	Retrieves the logger associated with this communicator. More...

virtual std::shared_ptr< ProgressThread > const &	progress_thread () const =0
	Retrieves the progress thread associated with this communicator. More...

virtual std::string	str () const =0
	Provides a string representation of the communicator. More...

Detailed Description

Abstract base class for a communication mechanism between nodes.

Provides an interface for sending and receiving messages between nodes, supporting asynchronous operations, GPU data transfers, and custom logging. Implementations must define the virtual methods to enable specific communication backends.

The API of the Communicator is not stream-ordered (since the concrete libraries we use to implement communication patterns are not stream-ordered). A consequence of this is that the user must ensure that any stream-ordered work on buffers is complete before passing a buffer into send or recv. As a corollary, after completing a future, the extracted Buffer is valid on its stream (it has no stream-ordered work queued up).

Sends and receives are matched on (rank, tag) pairs. The concrete implementation must provide that there is no message overtaking.

Definition at line 188 of file communicator.hpp.

Member Function Documentation

◆ logger()

virtual std::shared_ptr<Communicator::Logger> const& rapidsmpf::Communicator::logger ( )

pure virtual

Retrieves the logger associated with this communicator.

Returns: Shared pointer to the logger.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ nranks()

virtual Rank rapidsmpf::Communicator::nranks ( ) const

pure virtual

Retrieves the total number of ranks.

Returns: The total number of ranks.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ progress_thread()

virtual std::shared_ptr<ProgressThread> const& rapidsmpf::Communicator::progress_thread ( ) const

pure virtual

Retrieves the progress thread associated with this communicator.

Returns: Shared pointer to the progress thread.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ rank()

virtual Rank rapidsmpf::Communicator::rank ( ) const

pure virtual

Retrieves the rank of the current node.

Returns: The rank of the node.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ recv()

virtual std::unique_ptr<Future> rapidsmpf::Communicator::recv	(	Rank	rank,
		Tag	tag,
		std::unique_ptr< Buffer >	recv_buffer
	)

pure virtual

Receives a message from a specific rank to a buffer. Use release_data to extract the data out of the buffer once the future is completed.

Parameters

rank	The source rank.
tag	Message tag for identification.
recv_buffer	The receive buffer.

Returns: A unique pointer to a Future representing the asynchronous operation.

Exceptions

std::invalid_argument	If `recv_buffer` is `nullptr`.
std::logic_error	If `recv_buffer` is not ready (see warning for more details).

Warning: The caller is responsible to ensure the underlying Buffer allocation is already valid before calling, for example, when a CUDA allocation and/or copy are done asynchronously. Specifically, the caller should ensure Buffer::is_ready() returns true before calling this function. Providing a non-ready buffer leads to an irrecoverable condition.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ recv_any()

virtual std::pair<std::unique_ptr<std::vector<std::uint8_t> >, Rank> rapidsmpf::Communicator::recv_any ( Tag tag )

pure virtual

Receives a message from any rank (blocking).

Parameters

tag	Message tag for identification.

Returns: A pair containing the message data (host memory) and the rank of the sender.

Note: If no message is available this is indicated by returning a nullptr in the first slot of the pair.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ recv_from()

virtual std::unique_ptr<std::vector<std::uint8_t> > rapidsmpf::Communicator::recv_from	(	Rank	src,
		Tag	tag
	)

pure virtual

Receives a message from a specific rank (blocking).

Parameters

src	The source rank from which to receive the message.
tag	Message tag for identification.

Returns: A unique pointer to a vector containing the received message data (host memory).

Note: If no message is available, this function returns a nullptr.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ recv_sync_host_data()

virtual std::unique_ptr<Future> rapidsmpf::Communicator::recv_sync_host_data	(	Rank	rank,
		Tag	tag,
		std::unique_ptr< std::vector< std::uint8_t >>	synced_buffer
	)

pure virtual

Receives a message from a specific rank to an allocated (synchronized) host buffer. Use release_sync_host_data to extract the data out of the buffer once the future is completed.

Parameters

rank	The source rank.
tag	Message tag for identification.
synced_buffer	The receive buffer.

Returns: A unique pointer to a Future representing the asynchronous operation.

Exceptions

std::invalid_argument If synced_buffer is nullptr.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ release_data()

virtual std::unique_ptr<Buffer> rapidsmpf::Communicator::release_data ( std::unique_ptr< Communicator::Future > future )

pure virtual

Retrieves data associated with a completed future.

Parameters

future The completed future.

Returns: A unique pointer to the data buffer.

Exceptions

std::runtime_error if the future has no data.

◆ release_sync_host_data()

virtual std:: unique_ptr<std::vector<std::uint8_t> > rapidsmpf::Communicator::release_sync_host_data ( std::unique_ptr< Communicator::Future > future )

pure virtual

Retrieves synchronized host data associated with a completed future. When the future is completed, the the host data is valid, and ready, but not stream-ordered.

Parameters

future The completed future.

Returns: A unique pointer to the synchronized host data.

Exceptions

std::runtime_error if the future has no data.

◆ send() [1/2]

virtual std::unique_ptr<Future> rapidsmpf::Communicator::send	(	std::unique_ptr< Buffer >	msg,
		Rank	rank,
		Tag	tag
	)

pure virtual

Sends a message (device or host) to a specific rank. Use release_data to obtain the data buffer again once the future is completed.

Parameters

msg	Unique pointer to the message data (Buffer).
rank	The destination rank.
tag	Message tag for identification.

Returns: A unique pointer to a Future representing the asynchronous operation.

Exceptions

std::invalid_argument	If `msg` is `nullptr`.
std::logic_error	If `msg` is not ready (see warning for more details).

Warning: The caller is responsible to ensure the underlying Buffer allocation and data are already valid before calling, for example, when a CUDA allocation and/or copy are done asynchronously. Specifically, the caller should ensure Buffer::is_ready() returns true before calling this function. Providing a non-ready buffer leads to an irrecoverable condition.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ send() [2/2]

virtual std::unique_ptr<Future> rapidsmpf::Communicator::send	(	std::unique_ptr< std::vector< std::uint8_t >>	msg,
		Rank	rank,
		Tag	tag
	)

pure virtual

Sends a host message to a specific rank.

This is used to send data that resides in host memory and is guaranteed to be valid at the time of the call.

Use release_sync_host_data to obtain the data buffer again once the future is completed.

Parameters

msg	Unique pointer to the message data (host memory).
rank	The destination rank.
tag	Message tag for identification.

Returns: A unique pointer to a Future representing the asynchronous operation.

Exceptions

std::invalid_argument If msg is nullptr.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ str()

virtual std::string rapidsmpf::Communicator::str ( ) const

pure virtual

Provides a string representation of the communicator.

Returns: A string describing the communicator.

Implemented in rapidsmpf::ucxx::UCXX, rapidsmpf::Single, and rapidsmpf::MPI.

◆ test()

virtual bool rapidsmpf::Communicator::test ( std::unique_ptr< Communicator::Future > & future )

pure virtual

Test for completion of a single future.

Parameters

future Future to test

Returns: True if the future is completed. After test returns true, it is safe to call release_data().

◆ test_some() [1/2]

virtual std::vector<std::size_t> rapidsmpf::Communicator::test_some ( std::unordered_map< std::size_t, std::unique_ptr< Communicator::Future >> const & future_map )

pure virtual

Tests for completion of multiple futures in a map.

Parameters

future_map Map of futures identified by keys.

Returns: Keys of completed futures.

◆ test_some() [2/2]

virtual std:: pair<std::vector<std::unique_ptr<Future> >, std::vector<std::size_t> > rapidsmpf::Communicator::test_some ( std::vector< std::unique_ptr< Future >> & future_vector )

pure virtual

Tests for completion of multiple futures.

Parameters

[in,out] future_vector Vector of Future objects. Completed futures are erased from the vector.

Returns: Pair of completed futures and indices of input vector that were completed.

◆ wait()

virtual std::unique_ptr<Buffer> rapidsmpf::Communicator::wait ( std::unique_ptr< Future > future )

pure virtual

Wait for a future to complete and return the data buffer.

Parameters

future The future to wait for completion of.

Returns: A unique pointer to the GPU data buffer (or nullptr if the future had no data).

◆ wait_all()

virtual std::vector<std::unique_ptr<Buffer> > rapidsmpf::Communicator::wait_all ( std::vector< std::unique_ptr< Communicator::Future >> && futures )

pure virtual

Wait for completion of all futures and return their data buffers.

Parameters

futures Futures to wait for completion of, consumed.

Returns: A vector of the contained data buffers.

The documentation for this class was generated from the following file:

/__w/rapidsmpf/rapidsmpf/cpp/include/rapidsmpf/communicator/communicator.hpp

Classes

Public Member Functions

Detailed Description

Member Function Documentation

◆ logger()

◆ nranks()

◆ progress_thread()

◆ rank()

◆ recv()

◆ recv_any()

◆ recv_from()

◆ recv_sync_host_data()

◆ release_data()

◆ release_sync_host_data()

◆ send() [1/2]

◆ send() [2/2]

◆ str()

◆ test()

◆ test_some() [1/2]

◆ test_some() [2/2]

◆ wait()

◆ wait_all()