Public Member Functions | List of all members
rmm::mr::device_memory_resource Class Referenceabstract

Base class for all libcudf device memory allocation. More...

#include <device_memory_resource.hpp>

Inheritance diagram for rmm::mr::device_memory_resource:
Inheritance graph

Public Member Functions

 device_memory_resource (device_memory_resource const &)=default
device_memory_resourceoperator= (device_memory_resource const &)=default
 device_memory_resource (device_memory_resource &&) noexcept=default
device_memory_resourceoperator= (device_memory_resource &&) noexcept=default
void * allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Allocates memory of size at least bytes. More...
void deallocate (void *ptr, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Deallocate memory pointed to by p. More...
bool is_equal (device_memory_resource const &other) const noexcept
 Compare this resource to another. More...
virtual bool supports_streams () const noexcept=0
 Query whether the resource supports use of non-null CUDA streams for allocation/deallocation. More...
virtual bool supports_get_mem_info () const noexcept=0
 Query whether the resource supports the get_mem_info API. More...
std::pair< std::size_t, std::size_t > get_mem_info (cuda_stream_view stream) const
 Queries the amount of free and total memory for the resource. More...

Detailed Description

Base class for all libcudf device memory allocation.

This class serves as the interface that all custom device memory implementations must satisfy.

There are two private, pure virtual functions that all derived classes must implement: do_allocate and do_deallocate. Optionally, derived classes may also override is_equal. By default, is_equal simply performs an identity comparison.

The public, non-virtual functions allocate, deallocate, and is_equal simply call the private virtual functions. The reason for this is to allow implementing shared, default behavior in the base class. For example, the base class' allocate function may log every allocation, no matter what derived class implementation is used.

The allocate and deallocate APIs and implementations provide stream-ordered memory allocation. This allows optimizations such as re-using memory deallocated on the same stream without the overhead of stream synchronization.

A call to allocate(bytes, stream_a) (on any derived class) returns a pointer that is valid to use on stream_a. Using the memory on a different stream (say stream_b) is Undefined Behavior unless the two streams are first synchronized, for example by using cudaStreamSynchronize(stream_a) or by recording a CUDA event on stream_a and then calling cudaStreamWaitEvent(stream_b, event).

The stream specified to deallocate() should be a stream on which it is valid to use the deallocated memory immediately for another allocation. Typically this is the stream on which the allocation was last used before the call to deallocate(). The passed stream may be used internally by a device_memory_resource for managing available memory with minimal synchronization, and it may also be synchronized at a later time, for example using a call to cudaStreamSynchronize().

For this reason, it is Undefined Behavior to destroy a CUDA stream that is passed to deallocate(). If the stream on which the allocation was last used has been destroyed before calling deallocate() or it is known that it will be destroyed, it is likely better to synchronize the stream (before destroying it) and then pass a different stream to deallocate() (e.g. the default stream).

A device_memory_resource should only be used when the active CUDA device is the same device that was active when the device_memory_resource was created. Otherwise behavior is undefined.

Creating a device_memory_resource for each device requires care to set the current device before creating each resource, and to maintain the lifetime of the resources as long as they are set as per-device resources. Here is an example loop that creates unique_ptrs to pool_memory_resource objects for each device and sets them as the per-device resource for that device.

std::vector<unique_ptr<pool_memory_resource>> per_device_pools;
for(int i = 0; i < N; ++i) {
set_per_device_resource(cuda_device_id{i}, &per_device_pools.back());

Member Function Documentation

◆ allocate()

void* rmm::mr::device_memory_resource::allocate ( std::size_t  bytes,
cuda_stream_view  stream = cuda_stream_view{} 

Allocates memory of size at least bytes.

The returned pointer will have at minimum 256 byte alignment.

If supported, this operation may optionally be executed on a stream. Otherwise, the stream is ignored and the null stream is used.

<tt>rmm::bad_alloc</tt>When the requested bytes cannot be allocated on the specified stream.
bytesThe size of the allocation
streamStream on which to perform allocation
void* Pointer to the newly allocated memory

◆ deallocate()

void rmm::mr::device_memory_resource::deallocate ( void *  ptr,
std::size_t  bytes,
cuda_stream_view  stream = cuda_stream_view{} 

Deallocate memory pointed to by p.

p must have been returned by a prior call to allocate(bytes,stream) on a device_memory_resource that compares equal to *this, and the storage it points to must not yet have been deallocated, otherwise behavior is undefined.

If supported, this operation may optionally be executed on a stream. Otherwise, the stream is ignored and the null stream is used.

pPointer to be deallocated
bytesThe size in bytes of the allocation. This must be equal to the value of bytes that was passed to the allocate call that returned p.
streamStream on which to perform deallocation

◆ get_mem_info()

std::pair<std::size_t, std::size_t> rmm::mr::device_memory_resource::get_mem_info ( cuda_stream_view  stream) const

Queries the amount of free and total memory for the resource.

streamthe stream whose memory manager we want to retrieve
a pair containing the free memory in bytes in .first and total amount of memory in .second

◆ is_equal()

bool rmm::mr::device_memory_resource::is_equal ( device_memory_resource const &  other) const

Compare this resource to another.

Two device_memory_resources compare equal if and only if memory allocated from one device_memory_resource can be deallocated from the other and vice versa.

By default, simply checks if *this and other refer to the same object, i.e., does not check if they are two objects of the same class.

otherThe other resource to compare to
If the two resources are equivalent

◆ supports_get_mem_info()

virtual bool rmm::mr::device_memory_resource::supports_get_mem_info ( ) const
pure virtualnoexcept

◆ supports_streams()

virtual bool rmm::mr::device_memory_resource::supports_streams ( ) const
pure virtualnoexcept

The documentation for this class was generated from the following file:
device_memory_resource * set_per_device_resource(cuda_device_id device_id, device_memory_resource *new_mr)
Set the device_memory_resource for the specified device.
Definition: per_device_resource.hpp:164