All Classes Files Functions Variables Typedefs Enumerations Enumerator Friends Modules Pages
Public Member Functions | List of all members
rmm::mr::sam_headroom_memory_resource Class Referencefinal

Resource that uses system memory resource to allocate memory with a headroom. More...

#include <sam_headroom_memory_resource.hpp>

Inheritance diagram for rmm::mr::sam_headroom_memory_resource:
Inheritance graph
[legend]
Collaboration diagram for rmm::mr::sam_headroom_memory_resource:
Collaboration graph
[legend]

Public Member Functions

 sam_headroom_memory_resource (std::size_t headroom)
 Construct a headroom memory resource. More...
 
 sam_headroom_memory_resource (sam_headroom_memory_resource const &)=delete
 
 sam_headroom_memory_resource (sam_headroom_memory_resource &&)=delete
 
sam_headroom_memory_resourceoperator= (sam_headroom_memory_resource const &)=delete
 
sam_headroom_memory_resourceoperator= (sam_headroom_memory_resource &&)=delete
 
- Public Member Functions inherited from rmm::mr::device_memory_resource
 device_memory_resource (device_memory_resource const &)=default
 Default copy constructor.
 
 device_memory_resource (device_memory_resource &&) noexcept=default
 Default move constructor.
 
device_memory_resourceoperator= (device_memory_resource const &)=default
 Default copy assignment operator. More...
 
device_memory_resourceoperator= (device_memory_resource &&) noexcept=default
 Default move assignment operator. More...
 
void * allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Allocates memory of size at least bytes. More...
 
void deallocate (void *ptr, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
 Deallocate memory pointed to by p. More...
 
bool is_equal (device_memory_resource const &other) const noexcept
 Compare this resource to another. More...
 
void * allocate (std::size_t bytes, std::size_t alignment)
 Allocates memory of size at least bytes. More...
 
void deallocate (void *ptr, std::size_t bytes, std::size_t alignment)
 Deallocate memory pointed to by p. More...
 
void * allocate_async (std::size_t bytes, std::size_t alignment, cuda_stream_view stream)
 Allocates memory of size at least bytes. More...
 
void * allocate_async (std::size_t bytes, cuda_stream_view stream)
 Allocates memory of size at least bytes. More...
 
void deallocate_async (void *ptr, std::size_t bytes, std::size_t alignment, cuda_stream_view stream)
 Deallocate memory pointed to by p. More...
 
void deallocate_async (void *ptr, std::size_t bytes, cuda_stream_view stream)
 Deallocate memory pointed to by p. More...
 
bool operator== (device_memory_resource const &other) const noexcept
 Comparison operator with another device_memory_resource. More...
 
bool operator!= (device_memory_resource const &other) const noexcept
 Comparison operator with another device_memory_resource. More...
 

Detailed Description

Resource that uses system memory resource to allocate memory with a headroom.

System allocated memory (SAM) can be migrated to the GPU, but is never migrated back the host. If GPU memory is over-subscribed, this can cause other CUDA calls to fail with out-of-memory errors. To work around this problem, when using a system memory resource, we reserve some GPU memory as headroom for other CUDA calls, and only conditionally set its preferred location to the GPU if the allocation would not eat into the headroom.

Since doing this check on every allocation can be expensive, the caller may choose to use other allocators (e.g. binning_memory_resource) for small allocations, and use this allocator for large allocations only.

Constructor & Destructor Documentation

◆ sam_headroom_memory_resource()

rmm::mr::sam_headroom_memory_resource::sam_headroom_memory_resource ( std::size_t  headroom)
inlineexplicit

Construct a headroom memory resource.

Parameters
headroomSize of the reserved GPU memory as headroom

The documentation for this class was generated from the following file: