Classes | Public Member Functions | Protected Types | Protected Member Functions | List of all members
rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType > Class Template Reference

Base class for a stream-ordered memory resource. More...

#include <stream_ordered_memory_resource.hpp>

Inheritance diagram for rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >:
Inheritance graph
[legend]
Collaboration diagram for rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >:
Collaboration graph
[legend]

Classes

struct  stream_event_pair
 

Public Member Functions

 stream_ordered_memory_resource (stream_ordered_memory_resource const &)=delete
 
 stream_ordered_memory_resource (stream_ordered_memory_resource &&)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource const &)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource &&)=delete
 
- Public Member Functions inherited from rmm::mr::detail::crtp< PoolResource >
PoolResource & underlying ()
 
PoolResource const & underlying () const
 
- Public Member Functions inherited from rmm::mr::device_memory_resource
 device_memory_resource (device_memory_resource const &)=default
 
device_memory_resourceoperator= (device_memory_resource const &)=default
 
 device_memory_resource (device_memory_resource &&) noexcept=default
 
device_memory_resourceoperator= (device_memory_resource &&) noexcept=default
 
void * allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Allocates memory of size at least bytes. More...
 
void deallocate (void *ptr, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Deallocate memory pointed to by p. More...
 
bool is_equal (device_memory_resource const &other) const noexcept
 Compare this resource to another. More...
 
virtual bool supports_streams () const noexcept=0
 Query whether the resource supports use of non-null CUDA streams for allocation/deallocation. More...
 
virtual bool supports_get_mem_info () const noexcept=0
 Query whether the resource supports the get_mem_info API. More...
 
std::pair< std::size_t, std::size_t > get_mem_info (cuda_stream_view stream) const
 Queries the amount of free and total memory for the resource. More...
 

Protected Types

using free_list = FreeListType
 
using block_type = typename free_list::block_type
 
using lock_guard = std::lock_guard< std::mutex >
 
using split_block = std::pair< block_type, block_type >
 Pair representing a block that has been split for allocation.
 

Protected Member Functions

void insert_block (block_type const &block, cuda_stream_view stream)
 Returns the block b (last used on stream stream_event) to the pool. More...
 
void insert_blocks (free_list &&blocks, cuda_stream_view stream)
 
std::mutex & get_mutex ()
 Get the mutex object. More...
 
void * do_allocate (std::size_t size, cuda_stream_view stream) override
 Allocates memory of size at least bytes. More...
 
void do_deallocate (void *ptr, std::size_t size, cuda_stream_view stream) override
 Deallocate memory pointed to by p. More...
 

Detailed Description

template<typename PoolResource, typename FreeListType>
class rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >

Base class for a stream-ordered memory resource.

This base class uses CRTP (https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern) to provide static polymorphism to enable defining suballocator resources that maintain separate pools per stream. All of the stream-ordering logic is contained in this class, but the logic to determine how memory pools are managed and the type of allocation is implented in a derived class and in a free list class.

For example, a coalescing pool memory resource uses a coalescing_free_list and maintains data structures for allocated blocks and has functions to allocate and free blocks and to expand the pool.

Classes derived from stream_ordered_memory_resource must implement the following four methods, documented separately:

  1. std::size_t get_maximum_allocation_size() const
  2. block_type expand_pool(std::size_t size, free_list& blocks, cuda_stream_view stream)
  3. split_block allocate_from_block(block_type const& b, std::size_t size)
  4. block_type free_block(void* p, std::size_t size) noexcept

Member Function Documentation

◆ do_allocate()

template<typename PoolResource , typename FreeListType >
void* rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::do_allocate ( std::size_t  size,
cuda_stream_view  stream 
)
inlineoverrideprotectedvirtual

Allocates memory of size at least bytes.

The returned pointer has at least 256B alignment.

Exceptions
<tt>std::bad_alloc</tt>if the requested allocation could not be fulfilled
Parameters
sizeThe size in bytes of the allocation
streamThe stream in which to order this allocation
Returns
void* Pointer to the newly allocated memory

Implements rmm::mr::device_memory_resource.

◆ do_deallocate()

template<typename PoolResource , typename FreeListType >
void rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::do_deallocate ( void *  ptr,
std::size_t  size,
cuda_stream_view  stream 
)
inlineoverrideprotectedvirtual

Deallocate memory pointed to by p.

Exceptions
nothing
Parameters
pPointer to be deallocated
sizeThe size in bytes of the allocation to deallocate
streamThe stream in which to order this deallocation

Implements rmm::mr::device_memory_resource.

◆ get_mutex()

template<typename PoolResource , typename FreeListType >
std::mutex& rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::get_mutex ( )
inlineprotected

Get the mutex object.

Returns
std::mutex

◆ insert_block()

template<typename PoolResource , typename FreeListType >
void rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::insert_block ( block_type const &  block,
cuda_stream_view  stream 
)
inlineprotected

Returns the block b (last used on stream stream_event) to the pool.

Parameters
blockThe block to insert into the pool.
streamThe stream on which the memory was last used.

The documentation for this class was generated from the following file: