Classes | Public Member Functions | Static Public Attributes | Protected Types | Protected Member Functions | List of all members
rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType > Class Template Reference

Base class for a stream-ordered memory resource. More...

#include <stream_ordered_memory_resource.hpp>

Inheritance diagram for rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >:
Inheritance graph
[legend]
Collaboration diagram for rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >:
Collaboration graph
[legend]

Classes

struct  split_block
 Get the maximum size of a single allocation supported by this suballocator memory resource. More...
 
struct  stream_event_pair
 

Public Member Functions

 stream_ordered_memory_resource (stream_ordered_memory_resource const &)=delete
 
 stream_ordered_memory_resource (stream_ordered_memory_resource &&)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource const &)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource &&)=delete
 
- Public Member Functions inherited from rmm::mr::detail::crtp< PoolResource >
PoolResource & underlying ()
 
PoolResource const & underlying () const
 
- Public Member Functions inherited from rmm::mr::device_memory_resource
void * allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Allocates memory of size at least bytes. More...
 
void deallocate (void *p, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Deallocate memory pointed to by p. More...
 
bool is_equal (device_memory_resource const &other) const noexcept
 Compare this resource to another. More...
 
virtual bool supports_streams () const noexcept=0
 Query whether the resource supports use of non-null CUDA streams for allocation/deallocation. More...
 
virtual bool supports_get_mem_info () const noexcept=0
 Query whether the resource supports the get_mem_info API. More...
 
std::pair< std::size_t, std::size_t > get_mem_info (cuda_stream_view stream) const
 Queries the amount of free and total memory for the resource. More...
 

Static Public Attributes

static constexpr size_t allocation_alignment = 256
 

Protected Types

using free_list = FreeListType
 
using block_type = typename free_list::block_type
 
using lock_guard = std::lock_guard< std::mutex >
 

Protected Member Functions

void insert_block (block_type const &b, cuda_stream_view stream)
 Split block b if necessary to return a pointer to memory of size bytes. More...
 
void insert_blocks (free_list &&blocks, cuda_stream_view stream)
 
void print_free_blocks () const
 
std::mutex & get_mutex ()
 Get the mutex object. More...
 
virtual void * do_allocate (std::size_t bytes, cuda_stream_view stream) override
 Allocates memory of size at least bytes. More...
 
virtual void do_deallocate (void *p, std::size_t bytes, cuda_stream_view stream) override
 Deallocate memory pointed to by p. More...
 

Detailed Description

template<typename PoolResource, typename FreeListType>
class rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >

Base class for a stream-ordered memory resource.

This base class uses CRTP (https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern) to provide static polymorphism to enable defining suballocator resources that maintain separate pools per stream. All of the stream-ordering logic is contained in this class, but the logic to determine how memory pools are managed and the type of allocation is implented in a derived class and in a free list class.

For example, a coalescing pool memory resource uses a coalescing_free_list and maintains data structures for allocated blocks and has functions to allocate and free blocks and to expand the pool.

Classes derived from stream_ordered_memory_resource must implement the following four methods, documented separately:

  1. size_t get_maximum_allocation_size() const
  2. block_type expand_pool(size_t size, free_list& blocks, cuda_stream_view stream)
  3. split_block allocate_from_block(block_type const& b, size_t size)
  4. block_type free_block(void* p, size_t size) noexcept

Member Function Documentation

◆ do_allocate()

template<typename PoolResource , typename FreeListType >
virtual void* rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::do_allocate ( std::size_t  bytes,
cuda_stream_view  stream 
)
inlineoverrideprotectedvirtual

Allocates memory of size at least bytes.

The returned pointer has at least 256B alignment.

Exceptions
<tt>std::bad_alloc</tt>if the requested allocation could not be fulfilled
Parameters
bytesThe size in bytes of the allocation
streamThe stream to associate this allocation with
Returns
void* Pointer to the newly allocated memory

Implements rmm::mr::device_memory_resource.

◆ do_deallocate()

template<typename PoolResource , typename FreeListType >
virtual void rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::do_deallocate ( void *  p,
std::size_t  bytes,
cuda_stream_view  stream 
)
inlineoverrideprotectedvirtual

Deallocate memory pointed to by p.

Exceptions
nothing
Parameters
pPointer to be deallocated

Implements rmm::mr::device_memory_resource.

◆ get_mutex()

template<typename PoolResource , typename FreeListType >
std::mutex& rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::get_mutex ( )
inlineprotected

Get the mutex object.

Returns
std::mutex

◆ insert_block()

template<typename PoolResource , typename FreeListType >
void rmm::mr::detail::stream_ordered_memory_resource< PoolResource, FreeListType >::insert_block ( block_type const &  b,
cuda_stream_view  stream 
)
inlineprotected

Split block b if necessary to return a pointer to memory of size bytes.

If the block is split, the remainder is returned as the remainder element in the output split_block.

Parameters
bThe block to allocate from.
sizeThe size in bytes of the requested allocation.
stream_eventThe stream and associated event on which the allocation will be used.
Returns
A split_block comprising the allocated pointer and any unallocated remainder of the input block.

Finds, frees and returns the block associated with pointer p.

Parameters
pThe pointer to the memory to free.
sizeThe size of the memory to free. Must be equal to the original allocation size.
Returns
The (now freed) block associated with p. The caller is expected to return the block to the pool.

Returns the block b (last used on stream stream_event) to the pool.

Parameters
bThe block to insert into the pool.
streamThe stream on which the memory was last used.

The documentation for this class was generated from the following file: