Resource that uses system memory resource to allocate memory with a headroom. More...
#include <sam_headroom_memory_resource.hpp>
Public Member Functions | |
sam_headroom_memory_resource (std::size_t headroom) | |
Construct a headroom memory resource. More... | |
sam_headroom_memory_resource (sam_headroom_memory_resource const &)=delete | |
sam_headroom_memory_resource (sam_headroom_memory_resource &&)=delete | |
sam_headroom_memory_resource & | operator= (sam_headroom_memory_resource const &)=delete |
sam_headroom_memory_resource & | operator= (sam_headroom_memory_resource &&)=delete |
![]() | |
device_memory_resource (device_memory_resource const &)=default | |
Default copy constructor. | |
device_memory_resource (device_memory_resource &&) noexcept=default | |
Default move constructor. | |
device_memory_resource & | operator= (device_memory_resource const &)=default |
Default copy assignment operator. More... | |
device_memory_resource & | operator= (device_memory_resource &&) noexcept=default |
Default move assignment operator. More... | |
void * | allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{}) |
Allocates memory of size at least bytes . More... | |
void | deallocate (void *ptr, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{}) |
Deallocate memory pointed to by p . More... | |
bool | is_equal (device_memory_resource const &other) const noexcept |
Compare this resource to another. More... | |
void * | allocate (std::size_t bytes, std::size_t alignment) |
Allocates memory of size at least bytes . More... | |
void | deallocate (void *ptr, std::size_t bytes, std::size_t alignment) |
Deallocate memory pointed to by p . More... | |
void * | allocate_async (std::size_t bytes, std::size_t alignment, cuda_stream_view stream) |
Allocates memory of size at least bytes . More... | |
void * | allocate_async (std::size_t bytes, cuda_stream_view stream) |
Allocates memory of size at least bytes . More... | |
void | deallocate_async (void *ptr, std::size_t bytes, std::size_t alignment, cuda_stream_view stream) |
Deallocate memory pointed to by p . More... | |
void | deallocate_async (void *ptr, std::size_t bytes, cuda_stream_view stream) |
Deallocate memory pointed to by p . More... | |
bool | operator== (device_memory_resource const &other) const noexcept |
Comparison operator with another device_memory_resource. More... | |
bool | operator!= (device_memory_resource const &other) const noexcept |
Comparison operator with another device_memory_resource. More... | |
Resource that uses system memory resource to allocate memory with a headroom.
System allocated memory (SAM) can be migrated to the GPU, but is never migrated back the host. If GPU memory is over-subscribed, this can cause other CUDA calls to fail with out-of-memory errors. To work around this problem, when using a system memory resource, we reserve some GPU memory as headroom for other CUDA calls, and only conditionally set its preferred location to the GPU if the allocation would not eat into the headroom.
Since doing this check on every allocation can be expensive, the caller may choose to use other allocators (e.g. binning_memory_resource
) for small allocations, and use this allocator for large allocations only.
|
inlineexplicit |
Construct a headroom memory resource.
headroom | Size of the reserved GPU memory as headroom |