rapidsmpf::shuffler::detail::FinishCounter Class Reference

Helper to tally the finish status of a shuffle. More...

#include <finish_counter.hpp>

Public Types
using	FinishedCallback = std::function< void(PartID)>
	Callback function type called when a partition is finished. More...

Public Member Functions
	FinishCounter (Rank nranks, std::vector< PartID > const &local_partitions, FinishedCallback &&finished_callback=nullptr)
	Construct a finish counter. More...
void	move_goalpost (PartID pid, ChunkID nchunks)
	Move the goalpost for a specific rank and partition. More...
void	add_finished_chunk (PartID pid)
	Add a finished chunk to a partition counter. More...
bool	all_finished () const
	Returns whether all partitions are finished (non-blocking). More...
PartID	wait_any (std::optional< std::chrono::milliseconds > timeout={})
	Returns the partition ID of a finished partition that hasn't been waited on (blocking). Optionally a timeout (in ms) can be provided. More...
void	wait_on (PartID pid, std::optional< std::chrono::milliseconds > timeout={})
	Wait for a specific partition to be finished (blocking). Optionally a timeout (in ms) can be provided. More...
std::string	str () const
	Returns a description of this instance. More...

Detailed Description

Helper to tally the finish status of a shuffle.

The FinishCounter class tracks the completion of shuffle operations across different ranks and partitions. Each rank maintains a counter for tracking how many chunks have been received for each partition.

Definition at line 45 of file finish_counter.hpp.

Member Typedef Documentation

FinishedCallback

using rapidsmpf::shuffler::detail::FinishCounter::FinishedCallback = std::function<void(PartID)>

Callback function type called when a partition is finished.

The callback receives the partition ID of the finished partition.

Warning

A callback must be fast and non-blocking and should not call any of the wait* methods. And be very careful if acquiring locks. Ideally it should be used to signal a separate thread to do the actual processing.

Definition at line 56 of file finish_counter.hpp.

Constructor & Destructor Documentation

FinishCounter()

rapidsmpf::shuffler::detail::FinishCounter::FinishCounter	(	Rank	nranks,
		std::vector< PartID > const &	local_partitions,
		FinishedCallback &&	finished_callback = nullptr
	)

Construct a finish counter.

Parameters

nranks	The total number of ranks participating in the shuffle.
local_partitions	The partition IDs local to the current rank.
finished_callback	The callback to notify when a partition is finished (optional).

Member Function Documentation

add_finished_chunk()

void rapidsmpf::shuffler::detail::FinishCounter::add_finished_chunk

(

PartID

pid

)

Add a finished chunk to a partition counter.

This function increments the finished chunk counter for a specific partition. When the number of finished chunks matches the goalpost, the partition is marked as finished.

Parameters

pid	The partition ID to update.

Exceptions

std::logic_error

If the partition has already reached the goalpost.

all_finished()

bool rapidsmpf::shuffler::detail::FinishCounter::all_finished

(

)

const

Returns whether all partitions are finished (non-blocking).

Returns

True if all partitions are finished, otherwise False.

move_goalpost()

void rapidsmpf::shuffler::detail::FinishCounter::move_goalpost	(	PartID	pid,
		ChunkID	nchunks
	)

Move the goalpost for a specific rank and partition.

This function sets the number of chunks that need to be received from a specific rank and partition. It should only be called once per rank and partition.

Parameters

pid	The partition ID the goalpost is assigned to.
nchunks	The number of chunks required. (Requires nchunks > 0)

Exceptions

std::logic_error

If the goalpost is moved more than once for the same rank and partition, or if nchunks is 0.

str()

std::string rapidsmpf::shuffler::detail::FinishCounter::str

(

)

const

Returns a description of this instance.

Returns

The description.

wait_any()

PartID rapidsmpf::shuffler::detail::FinishCounter::wait_any

(

std::optional< std::chrono::milliseconds >

timeout = {}

)

Returns the partition ID of a finished partition that hasn't been waited on (blocking). Optionally a timeout (in ms) can be provided.

This function blocks until a partition is finished and ready to be processed. If the timeout is set and a partition is not available by the time, a std::runtime_error will be thrown.

Parameters

timeout

Optional timeout (ms) to wait.

Returns

The partition ID of a finished partition.

Exceptions

std::out_of_range	If all partitions have already been waited on.
std::runtime_error	If timeout was set and no partitions have been finished by the expiration.

Note

The caller needs to be careful when using wait_any alongside is_finished. For example, is_finished() will return true once all partitions have been finished, regardless of how many partitions were waited on.

wait_on()

void rapidsmpf::shuffler::detail::FinishCounter::wait_on	(	PartID	pid,
		std::optional< std::chrono::milliseconds >	timeout = {}
	)

Wait for a specific partition to be finished (blocking). Optionally a timeout (in ms) can be provided.

This function blocks until the desired partition is finished and ready to be processed. If the timeout is set and the requested partition is not available by the time, a std::runtime_error will be thrown.

Parameters

pid	The desired partition ID.
timeout	Optional timeout (ms) to wait.

Exceptions

std::out_of_range	If the desired partition is unavailable.
std::runtime_error	If timeout was set and requested partition has been finished by the expiration.

Note

The caller needs to be careful when using wait_on alongside is_finished. For example, is_finished() will return true once all partitions have been finished, regardless of how many partitions were waited on.

---

The documentation for this class was generated from the following file:

• /__w/rapidsmpf/rapidsmpf/cpp/include/rapidsmpf/shuffler/finish_counter.hpp