Public Member Functions | Static Public Attributes | Protected Types | Protected Member Functions | Protected Attributes | Friends | List of all members
rmm::mr::pool_memory_resource< Upstream > Class Template Referencefinal

A coalescing best-fit suballocator which uses a pool of memory allocated from an upstream memory_resource. More...

#include <pool_memory_resource.hpp>

Inheritance diagram for rmm::mr::pool_memory_resource< Upstream >:
Inheritance graph
[legend]
Collaboration diagram for rmm::mr::pool_memory_resource< Upstream >:
Collaboration graph
[legend]

Public Member Functions

 pool_memory_resource (Upstream *upstream_mr, thrust::optional< std::size_t > initial_pool_size=thrust::nullopt, thrust::optional< std::size_t > maximum_pool_size=thrust::nullopt)
 Construct a pool_memory_resource and allocate the initial device memory pool using upstream_mr. More...
 
 ~pool_memory_resource ()
 Destroy the pool_memory_resource and deallocate all memory it allocated using the upstream resource.
 
 pool_memory_resource (pool_memory_resource const &)=delete
 
 pool_memory_resource (pool_memory_resource &&)=delete
 
pool_memory_resourceoperator= (pool_memory_resource const &)=delete
 
pool_memory_resourceoperator= (pool_memory_resource &&)=delete
 
bool supports_streams () const noexcept override
 Queries whether the resource supports use of non-null CUDA streams for allocation/deallocation. More...
 
bool supports_get_mem_info () const noexcept override
 Query whether the resource supports the get_mem_info API. More...
 
Upstream * get_upstream () const noexcept
 Get the upstream memory_resource object. More...
 
- Public Member Functions inherited from rmm::mr::detail::stream_ordered_memory_resource< pool_memory_resource< Upstream >, detail::coalescing_free_list >
 stream_ordered_memory_resource (stream_ordered_memory_resource const &)=delete
 
 stream_ordered_memory_resource (stream_ordered_memory_resource &&)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource const &)=delete
 
stream_ordered_memory_resourceoperator= (stream_ordered_memory_resource &&)=delete
 
- Public Member Functions inherited from rmm::mr::detail::crtp< pool_memory_resource< Upstream > >
pool_memory_resource< Upstream > & underlying ()
 
pool_memory_resource< Upstream > const & underlying () const
 
- Public Member Functions inherited from rmm::mr::device_memory_resource
void * allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Allocates memory of size at least bytes. More...
 
void deallocate (void *p, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Deallocate memory pointed to by p. More...
 
bool is_equal (device_memory_resource const &other) const noexcept
 Compare this resource to another. More...
 
std::pair< std::size_t, std::size_t > get_mem_info (cuda_stream_view stream) const
 Queries the amount of free and total memory for the resource. More...
 

Static Public Attributes

static constexpr size_t allocation_alignment = 256
 
- Static Public Attributes inherited from rmm::mr::detail::stream_ordered_memory_resource< pool_memory_resource< Upstream >, detail::coalescing_free_list >
static constexpr size_t allocation_alignment
 

Protected Types

using free_list = detail::coalescing_free_list
 
using block_type = free_list::block_type
 
using lock_guard = std::lock_guard< std::mutex >
 
- Protected Types inherited from rmm::mr::detail::stream_ordered_memory_resource< pool_memory_resource< Upstream >, detail::coalescing_free_list >
using free_list = detail::coalescing_free_list
 
using block_type = typename free_list::block_type
 
using lock_guard = std::lock_guard< std::mutex >
 

Protected Member Functions

size_t get_maximum_allocation_size () const
 Get the maximum size of allocations supported by this memory resource. More...
 
block_type try_to_expand (std::size_t try_size, std::size_t min_size, cuda_stream_view stream)
 Try to expand the pool by allocating a block of at least min_size bytes from upstream. More...
 
void initialize_pool (thrust::optional< std::size_t > initial_size, thrust::optional< std::size_t > maximum_size)
 Allocate initial memory for the pool. More...
 
block_type expand_pool (std::size_t size, free_list &blocks, cuda_stream_view stream)
 Allocate space from upstream to supply the suballocation pool and return a sufficiently sized block. More...
 
std::size_t size_to_grow (std::size_t size) const
 Given a minimum size, computes an appropriate size to grow the pool. More...
 
thrust::optional< block_type > block_from_upstream (size_t size, cuda_stream_view stream)
 Allocate a block from upstream to expand the suballocation pool. More...
 
split_block allocate_from_block (block_type const &b, size_t size)
 Splits block b if necessary to return a pointer to memory of size bytes. More...
 
block_type free_block (void *p, size_t size) noexcept
 Finds, frees and returns the block associated with pointer p. More...
 
size_t pool_size () const noexcept
 Computes the size of the current pool. More...
 
void release ()
 Free all memory allocated from the upstream memory_resource.
 
void print ()
 Print debugging information about all blocks in the pool. More...
 
std::pair< std::size_t, std::size_t > free_list_summary (free_list const &blocks)
 Get the largest available block size and total free size in the specified free list. More...
 
std::pair< size_t, size_t > do_get_mem_info (cuda_stream_view stream) const override
 Get free and available memory for memory resource. More...
 
- Protected Member Functions inherited from rmm::mr::detail::stream_ordered_memory_resource< pool_memory_resource< Upstream >, detail::coalescing_free_list >
void insert_block (block_type const &b, cuda_stream_view stream)
 Split block b if necessary to return a pointer to memory of size bytes. More...
 
void insert_blocks (free_list &&blocks, cuda_stream_view stream)
 
void print_free_blocks () const
 
std::mutex & get_mutex ()
 Get the mutex object. More...
 
virtual void * do_allocate (std::size_t bytes, cuda_stream_view stream) override
 Allocates memory of size at least bytes. More...
 
virtual void do_deallocate (void *p, std::size_t bytes, cuda_stream_view stream) override
 Deallocate memory pointed to by p. More...
 

Protected Attributes

Upstream * upstream_mr_
 
std::size_t current_pool_size_ {}
 
thrust::optional< std::size_t > maximum_pool_size_ {}
 
std::set< block_type, rmm::mr::detail::compare_blocks< block_type > > upstream_blocks_
 

Friends

class detail::stream_ordered_memory_resource< pool_memory_resource< Upstream >, detail::coalescing_free_list >
 

Detailed Description

template<typename Upstream>
class rmm::mr::pool_memory_resource< Upstream >

A coalescing best-fit suballocator which uses a pool of memory allocated from an upstream memory_resource.

Allocation (do_allocate()) and deallocation (do_deallocate()) are thread-safe. Also, this class is compatible with CUDA per-thread default stream.

Template Parameters
UpstreamResourcememory_resource to use for allocating the pool. Implements rmm::mr::device_memory_resource interface.

Constructor & Destructor Documentation

◆ pool_memory_resource()

template<typename Upstream >
rmm::mr::pool_memory_resource< Upstream >::pool_memory_resource ( Upstream *  upstream_mr,
thrust::optional< std::size_t >  initial_pool_size = thrust::nullopt,
thrust::optional< std::size_t >  maximum_pool_size = thrust::nullopt 
)
inlineexplicit

Construct a pool_memory_resource and allocate the initial device memory pool using upstream_mr.

Exceptions
rmm::logic_errorif upstream_mr == nullptr
rmm::logic_errorif initial_pool_size is neither the default nor aligned to a multiple of pool_memory_resource::allocation_alignment bytes.
rmm::logic_errorif maximum_pool_size is neither the default nor aligned to a multiple of pool_memory_resource::allocation_alignment bytes.
Parameters
upstream_mrThe memory_resource from which to allocate blocks for the pool.
initial_pool_sizeMinimum size, in bytes, of the initial pool. Defaults to half of the available memory on the current device.
maximum_pool_sizeMaximum size, in bytes, that the pool can grow to. Defaults to all of the available memory on the current device.

Member Function Documentation

◆ allocate_from_block()

template<typename Upstream >
split_block rmm::mr::pool_memory_resource< Upstream >::allocate_from_block ( block_type const &  b,
size_t  size 
)
inlineprotected

Splits block b if necessary to return a pointer to memory of size bytes.

If the block is split, the remainder is returned to the pool.

Parameters
bThe block to allocate from.
sizeThe size in bytes of the requested allocation.
stream_eventThe stream and associated event on which the allocation will be used.
Returns
A pair comprising the allocated pointer and any unallocated remainder of the input block.

◆ block_from_upstream()

template<typename Upstream >
thrust::optional<block_type> rmm::mr::pool_memory_resource< Upstream >::block_from_upstream ( size_t  size,
cuda_stream_view  stream 
)
inlineprotected

Allocate a block from upstream to expand the suballocation pool.

Parameters
sizeThe size in bytes to allocate from the upstream resource
streamThe stream on which the memory is to be used.
Returns
block_type The allocated block

◆ do_get_mem_info()

template<typename Upstream >
std::pair<size_t, size_t> rmm::mr::pool_memory_resource< Upstream >::do_get_mem_info ( cuda_stream_view  stream) const
inlineoverrideprotectedvirtual

Get free and available memory for memory resource.

Exceptions
nothing
Parameters
streamto execute on
Returns
std::pair contaiing free_size and total_size of memory

Implements rmm::mr::device_memory_resource.

◆ expand_pool()

template<typename Upstream >
block_type rmm::mr::pool_memory_resource< Upstream >::expand_pool ( std::size_t  size,
free_list blocks,
cuda_stream_view  stream 
)
inlineprotected

Allocate space from upstream to supply the suballocation pool and return a sufficiently sized block.

Parameters
sizeThe minimum size to allocate
blocksThe free list (ignored in this implementation)
streamThe stream on which the memory is to be used.
Returns
block_type a block of at least size bytes

◆ free_block()

template<typename Upstream >
block_type rmm::mr::pool_memory_resource< Upstream >::free_block ( void *  p,
size_t  size 
)
inlineprotectednoexcept

Finds, frees and returns the block associated with pointer p.

Parameters
pThe pointer to the memory to free.
sizeThe size of the memory to free. Must be equal to the original allocation size.
streamThe stream-event pair for the stream on which the memory was last used.
Returns
The (now freed) block associated with p. The caller is expected to return the block to the pool.

◆ free_list_summary()

template<typename Upstream >
std::pair<std::size_t, std::size_t> rmm::mr::pool_memory_resource< Upstream >::free_list_summary ( free_list const &  blocks)
inlineprotected

Get the largest available block size and total free size in the specified free list.

This is intended only for debugging

Parameters
blocksThe free list from which to return the summary
Returns
std::pair<std::size_t, std::size_t> Pair of largest available block, total free size

◆ get_maximum_allocation_size()

template<typename Upstream >
size_t rmm::mr::pool_memory_resource< Upstream >::get_maximum_allocation_size ( ) const
inlineprotected

Get the maximum size of allocations supported by this memory resource.

Note this does not depend on the memory size of the device. It simply returns the maximum value of size_t

Returns
size_t The maximum size of a single allocation supported by this memory resource

◆ get_upstream()

template<typename Upstream >
Upstream* rmm::mr::pool_memory_resource< Upstream >::get_upstream ( ) const
inlinenoexcept

Get the upstream memory_resource object.

Returns
UpstreamResource* the upstream memory resource.

◆ initialize_pool()

template<typename Upstream >
void rmm::mr::pool_memory_resource< Upstream >::initialize_pool ( thrust::optional< std::size_t >  initial_size,
thrust::optional< std::size_t >  maximum_size 
)
inlineprotected

Allocate initial memory for the pool.

If initial_size is unset, then queries the upstream memory resource for available memory if upstream supports get_mem_info, or queries the device (using CUDA API) for available memory if not. Then attempts to initialize to half the available memory.

If initial_size is set, then tries to initialize the pool to that size.

Parameters
initial_sizeThe optional initial size for the pool
maximum_sizeThe optional maximum size for the pool

◆ pool_size()

template<typename Upstream >
size_t rmm::mr::pool_memory_resource< Upstream >::pool_size ( ) const
inlineprotectednoexcept

Computes the size of the current pool.

Includes allocated as well as free memory.

Returns
size_t The total size of the currently allocated pool.

◆ print()

template<typename Upstream >
void rmm::mr::pool_memory_resource< Upstream >::print ( )
inlineprotected

Print debugging information about all blocks in the pool.

Note
This function is intended only for use in debugging.

◆ size_to_grow()

template<typename Upstream >
std::size_t rmm::mr::pool_memory_resource< Upstream >::size_to_grow ( std::size_t  size) const
inlineprotected

Given a minimum size, computes an appropriate size to grow the pool.

Strategy is to try to grow the pool by half the difference between the configured maximum pool size and the current pool size, if the maximum pool size is set. If it is not set, try to double the current pool size.

Returns 0 if the requested size cannot be satisfied.

Parameters
sizeThe size of the minimum allocation immediately needed
Returns
size_t The computed size to grow the pool.

◆ supports_get_mem_info()

template<typename Upstream >
bool rmm::mr::pool_memory_resource< Upstream >::supports_get_mem_info ( ) const
inlineoverridevirtualnoexcept

Query whether the resource supports the get_mem_info API.

Returns
bool false

Implements rmm::mr::device_memory_resource.

◆ supports_streams()

template<typename Upstream >
bool rmm::mr::pool_memory_resource< Upstream >::supports_streams ( ) const
inlineoverridevirtualnoexcept

Queries whether the resource supports use of non-null CUDA streams for allocation/deallocation.

Returns
bool true.

Implements rmm::mr::device_memory_resource.

◆ try_to_expand()

template<typename Upstream >
block_type rmm::mr::pool_memory_resource< Upstream >::try_to_expand ( std::size_t  try_size,
std::size_t  min_size,
cuda_stream_view  stream 
)
inlineprotected

Try to expand the pool by allocating a block of at least min_size bytes from upstream.

Attempts to allocate try_size bytes from upstream. If it fails, it iteratively reduces the attempted size by half until min_size, returning the allocated block once it succeeds.

Exceptions
rmm::bad_allocif min_size bytes cannot be allocated from upstream or maximum pool size is exceeded.
Parameters
try_sizeThe initial requested size to try allocating.
min_sizeThe minimum requested size to try allocating.
streamThe stream on which the memory is to be used.
Returns
block_type a block of at least min_size bytes

The documentation for this class was generated from the following file: