CUDA Streams

group cuda_streams

Functions

inline bool operator==(cuda_stream_view lhs, cuda_stream_view rhs)

Equality comparison operator for streams.

Parameters:

  • lhs – The first stream view to compare

  • rhs – The second stream view to compare

Returns:

true if equal, false if unequal

inline bool operator!=(cuda_stream_view lhs, cuda_stream_view rhs)

Inequality comparison operator for streams.

Parameters:

  • lhs – The first stream view to compare

  • rhs – The second stream view to compare

Returns:

true if unequal, false if equal

inline std::ostream &operator<<(std::ostream &os, cuda_stream_view stream)

Output stream operator for printing / logging streams.

Parameters:
Returns:

std::ostream& The output ostream

Variables

static constexpr cuda_stream_view cuda_stream_default = {}

Static cuda_stream_view of the default stream (stream 0), for convenience.

static const cuda_stream_view cuda_stream_legacy{cudaStreamLegacy}

Static cuda_stream_view of cudaStreamLegacy, for convenience.

static const cuda_stream_view cuda_stream_per_thread{cudaStreamPerThread}

Static cuda_stream_view of cudaStreamPerThread, for convenience.

class cuda_stream
#include <cuda_stream.hpp>

Owning wrapper for a CUDA stream.

Provides RAII lifetime semantics for a CUDA stream.

Public Functions

cuda_stream(cuda_stream&&) = default

Move constructor (default)

A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.

cuda_stream &operator=(cuda_stream&&) = default

Move copy assignment operator (default)

A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.

Returns:

A reference to this cuda_stream

inline cuda_stream()

Construct a new cuda stream object.

Throws:

rmm::cuda_error – if stream creation fails

inline bool is_valid() const

Returns true if the owned stream is non-null.

Returns:

true If the owned stream has not been explicitly moved and is therefore non-null.

Returns:

false If the owned stream has been explicitly moved and is therefore null.

inline cudaStream_t value() const

Get the value of the wrapped CUDA stream.

Returns:

cudaStream_t The wrapped CUDA stream.

inline explicit operator cudaStream_t() const noexcept

Explicit conversion to cudaStream_t.

inline cuda_stream_view view() const

Creates an immutable, non-owning view of the wrapped CUDA stream.

Returns:

rmm::cuda_stream_view The view of the CUDA stream

inline operator cuda_stream_view() const

Implicit conversion to cuda_stream_view.

Returns:

A view of the owned stream

inline void synchronize() const

Synchronize the owned CUDA stream.

Calls cudaStreamSynchronize().

Throws:

rmm::cuda_error – if stream synchronization fails

inline void synchronize_no_throw() const noexcept

Synchronize the owned CUDA stream. Does not throw if there is an error.

Calls cudaStreamSynchronize() and asserts if there is an error.

class cuda_stream_pool
#include <cuda_stream_pool.hpp>

A pool of CUDA streams.

Provides efficient access to collection of CUDA stream objects.

Successive calls may return a cuda_stream_view of identical streams. For example, a possible implementation is to maintain a circular buffer of cuda_stream objects.

Public Functions

inline explicit cuda_stream_pool(std::size_t pool_size = default_size)

Construct a new cuda stream pool object of the given non-zero size.

Throws:

logic_error – if pool_size is zero

Parameters:

pool_size – The number of streams in the pool

inline rmm::cuda_stream_view get_stream() const noexcept

Get a cuda_stream_view of a stream in the pool.

This function is thread safe with respect to other calls to the same function.

Returns:

rmm::cuda_stream_view

inline rmm::cuda_stream_view get_stream(std::size_t stream_id) const

Get a cuda_stream_view of the stream associated with stream_id. Equivalent values of stream_id return a stream_view to the same underlying stream.

This function is thread safe with respect to other calls to the same function.

Parameters:

stream_id – Unique identifier for the desired stream

Returns:

rmm::cuda_stream_view

inline std::size_t get_pool_size() const noexcept

Get the number of streams in the pool.

This function is thread safe with respect to other calls to the same function.

Returns:

the number of streams in the pool

Public Static Attributes

static constexpr std::size_t default_size = {16}

Default stream pool size.

class cuda_stream_view
#include <cuda_stream_view.hpp>

Strongly-typed non-owning wrapper for CUDA streams with default constructor.

This wrapper is simply a “view”: it does not own the lifetime of the stream it wraps.

Public Functions

constexpr cuda_stream_view(cuda_stream_view const&) = default

Default copy constructor.

constexpr cuda_stream_view(cuda_stream_view&&) = default

Default move constructor.

constexpr cuda_stream_view &operator=(cuda_stream_view const&) = default

Default copy assignment operator.

Returns:

cuda_stream_view& Reference to the assigned object

constexpr cuda_stream_view &operator=(cuda_stream_view&&) = default

Default move assignment operator.

Returns:

cuda_stream_view& Reference to the assigned object

inline constexpr cuda_stream_view(cudaStream_t stream) noexcept

Constructor from a cudaStream_t.

Parameters:

stream – The underlying stream for this view

inline constexpr cuda_stream_view(cuda::stream_ref stream) noexcept

Implicit conversion from stream_ref.

Parameters:

stream – The underlying stream for this view

inline constexpr cudaStream_t value() const noexcept

Get the wrapped stream.

Returns:

cudaStream_t The underlying stream referenced by this cuda_stream_view

inline constexpr operator cudaStream_t() const noexcept

Implicit conversion to cudaStream_t.

Returns:

cudaStream_t The underlying stream referenced by this cuda_stream_view

inline constexpr operator cuda::stream_ref() const noexcept

Implicit conversion to stream_ref.

Returns:

stream_ref The underlying stream referenced by this cuda_stream_view

inline bool is_per_thread_default() const noexcept

true if the wrapped stream is the CUDA per-thread default stream

Returns:

true if the wrapped stream is the CUDA per-thread default stream

inline bool is_default() const noexcept

true if the wrapped stream is explicitly the CUDA legacy default stream

Returns:

true if the wrapped stream is explicitly the CUDA legacy default stream

inline void synchronize() const

Synchronize the viewed CUDA stream.

Calls cudaStreamSynchronize().

Throws:

rmm::cuda_error – if stream synchronization fails

inline void synchronize_no_throw() const noexcept

Synchronize the viewed CUDA stream. Does not throw if there is an error.

Calls cudaStreamSynchronize() and asserts if there is an error.