RAII construct for device memory allocation. More...

#include <device_buffer.hpp>

Public Member Functions
	device_buffer (device_buffer const &other)=delete

device_buffer &	operator= (device_buffer const &other)=delete

	device_buffer ()
	Default constructor creates an empty `device_buffer`

	device_buffer (std::size_t size, cuda_stream_view stream, device_async_resource_ref mr=mr::get_current_device_resource_ref())
	Constructs a new device buffer of `size` uninitialized bytes. More...

	device_buffer (void const *source_data, std::size_t size, cuda_stream_view stream, device_async_resource_ref mr=mr::get_current_device_resource_ref())
	Construct a new device buffer by copying from a raw pointer to an existing host or device memory allocation. More...

	device_buffer (device_buffer const &other, cuda_stream_view stream, device_async_resource_ref mr=mr::get_current_device_resource_ref())
	Construct a new `device_buffer` by deep copying the contents of another `device_buffer`, optionally using the specified stream and memory resource. More...

	device_buffer (device_buffer &&other) noexcept
	Constructs a new `device_buffer` by moving the contents of another `device_buffer` into the newly constructed one. More...

device_buffer &	operator= (device_buffer &&other) noexcept
	Move assignment operator moves the contents from `other`. More...

	~device_buffer () noexcept
	Destroy the device buffer object. More...

void	reserve (std::size_t new_capacity, cuda_stream_view stream)
	Increase the capacity of the device memory allocation. More...

void	resize (std::size_t new_size, cuda_stream_view stream)
	Resize the device memory allocation. More...

void	shrink_to_fit (cuda_stream_view stream)
	Forces the deallocation of unused memory. More...

void const *	data () const noexcept
	Const pointer to the device memory allocation. More...

void *	data () noexcept
	Pointer to the device memory allocation. More...

std::size_t	size () const noexcept
	The number of bytes. More...

std::int64_t	ssize () const noexcept
	The signed number of bytes. More...

bool	is_empty () const noexcept
	Whether or not the buffer currently holds any data. More...

std::size_t	capacity () const noexcept
	Returns actual size in bytes of device memory allocation. More...

cuda_stream_view	stream () const noexcept
	The stream most recently specified for allocation/deallocation. More...

void	set_stream (cuda_stream_view stream) noexcept
	Sets the stream to be used for deallocation. More...

rmm::device_async_resource_ref	memory_resource () const noexcept
	The resource used to allocate and deallocate. More...

Detailed Description

RAII construct for device memory allocation.

This class allocates untyped and uninitialized device memory using a device_async_resource_ref. If not explicitly specified, the memory resource returned from get_current_device_resource_ref() is used.

Note: Unlike std::vector or thrust::device_vector, the device memory allocated by a device_buffer is uninitialized. Therefore, it is undefined behavior to read the contents of data() before first initializing it.

Examples:

//Allocates at least 100 bytes of device memory using the default memory
//resource and default stream.
device_buffer buff(100);
 
// allocates at least 100 bytes using the custom memory resource and
// specified stream
custom_memory_resource mr;
cuda_stream_view stream = cuda_stream_view{};
device_buffer custom_buff(100, stream, &mr);
 
// deep copies `buff` into a new device buffer using the specified stream
device_buffer buff_copy(buff, stream);
 
// moves the memory in `from_buff` to `to_buff`. Deallocates previously allocated
// to_buff memory on `to_buff.stream()`.
device_buffer to_buff(std::move(from_buff));
 
// deep copies `buff` into a new device buffer using the specified stream
device_buffer buff_copy(buff, stream);
 
// shallow copies `buff` into a new device_buffer, `buff` is now empty
device_buffer buff_move(std::move(buff));
 
// Default construction. Buffer is empty
device_buffer buff_default{};
 
// If the requested size is larger than the current size, resizes allocation to the new size and
// deep copies any previous contents. Otherwise, simply updates the value of `size()` to the
// newly requested size without any allocations or copies. Uses the specified stream.
buff_default.resize(100, stream);

Constructor & Destructor Documentation

◆ device_buffer() [1/4]

rmm::device_buffer::device_buffer	(	std::size_t	size,
		cuda_stream_view	stream,
		device_async_resource_ref	mr = `mr::get_current_device_resource_ref()`
	)

inlineexplicit

Constructs a new device buffer of size uninitialized bytes.

Exceptions

rmm::bad_alloc If allocation fails.

Parameters

size	Size in bytes to allocate in device memory.
stream	CUDA stream on which memory may be allocated if the memory resource supports streams.
mr	Memory resource to use for the device memory allocation.

◆ device_buffer() [2/4]

rmm::device_buffer::device_buffer	(	void const *	source_data,
		std::size_t	size,
		cuda_stream_view	stream,
		device_async_resource_ref	mr = `mr::get_current_device_resource_ref()`
	)

inline

Construct a new device buffer by copying from a raw pointer to an existing host or device memory allocation.

Note: This function does not synchronize stream. source_data is copied on stream, so the caller is responsible for correct synchronization to ensure that source_data is valid when the copy occurs. This includes destroying source_data in stream order after this function is called, or synchronizing or waiting on stream after this function returns as necessary.

Exceptions

rmm::bad_alloc	If creating the new allocation fails.
rmm::logic_error	If `source_data` is null, and `size != 0`.
rmm::cuda_error	if copying from the device memory fails.

Parameters

source_data	Pointer to the host or device memory to copy from.
size	Size in bytes to copy.
stream	CUDA stream on which memory may be allocated if the memory resource supports streams.
mr	Memory resource to use for the device memory allocation

◆ device_buffer() [3/4]

rmm::device_buffer::device_buffer	(	device_buffer const &	other,
		cuda_stream_view	stream,
		device_async_resource_ref	mr = `mr::get_current_device_resource_ref()`
	)

inline

Construct a new device_buffer by deep copying the contents of another device_buffer, optionally using the specified stream and memory resource.

Note: Only copies other.size() bytes from other, i.e., if other.size() != other.capacity(), then the size and capacity of the newly constructed device_buffer will be equal to other.size().; This function does not synchronize stream. other is copied on stream, so the caller is responsible for correct synchronization to ensure that other is valid when the copy occurs. This includes destroying other in stream order after this function is called, or synchronizing or waiting on stream after this function returns as necessary.

Exceptions

rmm::bad_alloc	If creating the new allocation fails.
rmm::cuda_error	if copying from `other` fails.

Parameters

other	The `device_buffer` whose contents will be copied
stream	The stream to use for the allocation and copy
mr	The resource to use for allocating the new `device_buffer`

◆ device_buffer() [4/4]

rmm::device_buffer::device_buffer ( device_buffer && other )

inlinenoexcept

Constructs a new device_buffer by moving the contents of another device_buffer into the newly constructed one.

After the new device_buffer is constructed, other is modified to be a valid, empty device_buffer, i.e., data() returns nullptr, and size() and capacity() are zero.

Parameters

other The device_buffer whose contents will be moved into the newly constructed one.

◆ ~device_buffer()

rmm::device_buffer::~device_buffer ( )

inlinenoexcept

Destroy the device buffer object.

Note: If the memory resource supports streams, this destructor deallocates using the stream most recently passed to any of this device buffer's methods.

Member Function Documentation

◆ capacity()

std::size_t rmm::device_buffer::capacity ( ) const

inlinenoexcept

Returns actual size in bytes of device memory allocation.

The invariant size() <= capacity() holds.

Returns: The actual size in bytes of the device memory allocation

◆ data() [1/2]

void const* rmm::device_buffer::data ( ) const

inlinenoexcept

Const pointer to the device memory allocation.

Returns: Const pointer to the device memory allocation

◆ data() [2/2]

void* rmm::device_buffer::data ( )

inlinenoexcept

Pointer to the device memory allocation.

Returns: Pointer to the device memory allocation

◆ is_empty()

bool rmm::device_buffer::is_empty ( ) const

inlinenoexcept

Whether or not the buffer currently holds any data.

Returns: Whether or not the buffer currently holds any data

If is_empty() == true, the device_buffer may still hold an allocation if capacity() > 0.

◆ memory_resource()

rmm::device_async_resource_ref rmm::device_buffer::memory_resource ( ) const

inlinenoexcept

The resource used to allocate and deallocate.

Returns: The resource used to allocate and deallocate

◆ operator=()

device_buffer& rmm::device_buffer::operator= ( device_buffer && other )

inlinenoexcept

Move assignment operator moves the contents from other.

This device_buffer's current device memory allocation will be deallocated on stream().

If a different stream is required, call set_stream() on the instance before assignment. After assignment, this instance's stream is replaced by the other.stream().

Parameters

other The device_buffer whose contents will be moved.

Returns: A reference to this device_buffer

◆ reserve()

void rmm::device_buffer::reserve	(	std::size_t	new_capacity,
		cuda_stream_view	stream
	)

inline

Increase the capacity of the device memory allocation.

If the requested new_capacity is less than or equal to capacity(), no action is taken.

If new_capacity is larger than capacity(), a new allocation is made on stream to satisfy new_capacity, and the contents of the old allocation are copied on stream to the new allocation. The old allocation is then freed. The bytes from [size(), new_capacity) are uninitialized.

Exceptions

rmm::bad_alloc	If creating the new allocation fails
rmm::cuda_error	if the copy from the old to new allocation fails

Parameters

new_capacity	The requested new capacity, in bytes
stream	The stream to use for allocation and copy

◆ resize()

void rmm::device_buffer::resize	(	std::size_t	new_size,
		cuda_stream_view	stream
	)

inline

Resize the device memory allocation.

If the requested new_size is less than or equal to capacity(), no action is taken other than updating the value that is returned from size(). Specifically, no memory is allocated nor copied. The value capacity() remains the actual size of the device memory allocation.

Note: shrink_to_fit() may be used to force the deallocation of unused capacity().

If new_size is larger than capacity(), a new allocation is made on stream to satisfy new_size, and the contents of the old allocation are copied on stream to the new allocation. The old allocation is then freed. The bytes from [old_size, new_size) are uninitialized.

The invariant size() <= capacity() holds.

Exceptions

rmm::bad_alloc	If creating the new allocation fails
rmm::cuda_error	if the copy from the old to new allocation fails

Parameters

new_size	The requested new size, in bytes
stream	The stream to use for allocation and copy

◆ set_stream()

void rmm::device_buffer::set_stream ( cuda_stream_view stream )

inlinenoexcept

Sets the stream to be used for deallocation.

If no other rmm::device_buffer method that allocates memory is called after this call with a different stream argument, then stream will be used for deallocation in the rmm::device_uvector destructor. However, if either of resize() or shrink_to_fit() is called after this, the later stream parameter will be stored and used in the destructor.

Parameters

stream The stream to use for deallocation

◆ shrink_to_fit()

void rmm::device_buffer::shrink_to_fit ( cuda_stream_view stream )

inline

Forces the deallocation of unused memory.

Reallocates and copies on stream stream the contents of the device memory allocation to reduce capacity() to size().

If size() == capacity(), no allocations or copies occur.

Exceptions

rmm::bad_alloc	If creating the new allocation fails
rmm::cuda_error	If the copy from the old to new allocation fails

Parameters

stream The stream on which the allocation and copy are performed

◆ size()

std::size_t rmm::device_buffer::size ( ) const

inlinenoexcept

The number of bytes.

Returns: The number of bytes

◆ ssize()

std::int64_t rmm::device_buffer::ssize ( ) const

inlinenoexcept

The signed number of bytes.

Returns: The signed number of bytes

◆ stream()

cuda_stream_view rmm::device_buffer::stream ( ) const

inlinenoexcept

The stream most recently specified for allocation/deallocation.

Returns: The stream most recently specified for allocation/deallocation

The documentation for this class was generated from the following file:

device_buffer.hpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ device_buffer() [1/4]

◆ device_buffer() [2/4]

◆ device_buffer() [3/4]

◆ device_buffer() [4/4]

◆ ~device_buffer()

Member Function Documentation

◆ capacity()

◆ data() [1/2]

◆ data() [2/2]

◆ is_empty()

◆ memory_resource()

◆ operator=()

◆ reserve()

◆ resize()

◆ set_stream()

◆ shrink_to_fit()

◆ size()

◆ ssize()

◆ stream()