rmm::mr::sam_headroom_memory_resource Class Reference

Resource that uses system memory resource to allocate memory with a headroom. More...

#include <sam_headroom_memory_resource.hpp>

Inheritance diagram for rmm::mr::sam_headroom_memory_resource:

Collaboration diagram for rmm::mr::sam_headroom_memory_resource:

Public Member Functions
	sam_headroom_memory_resource (std::size_t headroom)
	Construct a headroom memory resource. More...
	sam_headroom_memory_resource (sam_headroom_memory_resource const &)=delete
	sam_headroom_memory_resource (sam_headroom_memory_resource &&)=delete
sam_headroom_memory_resource &	operator= (sam_headroom_memory_resource const &)=delete
sam_headroom_memory_resource &	operator= (sam_headroom_memory_resource &&)=delete
Public Member Functions inherited from rmm::mr::device_memory_resource
	device_memory_resource (device_memory_resource const &)=default
	Default copy constructor.
	device_memory_resource (device_memory_resource &&) noexcept=default
	Default move constructor.
device_memory_resource &	operator= (device_memory_resource const &)=default
	Default copy assignment operator. More...
device_memory_resource &	operator= (device_memory_resource &&) noexcept=default
	Default move assignment operator. More...
void *	allocate (std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
	Allocates memory of size at least bytes. More...
void	deallocate (void *ptr, std::size_t bytes, cuda_stream_view stream=cuda_stream_view{})
	Deallocate memory pointed to by p. More...
bool	is_equal (device_memory_resource const &other) const noexcept
	Compare this resource to another. More...
void *	allocate (std::size_t bytes, std::size_t alignment)
	Allocates memory of size at least bytes. More...
void	deallocate (void *ptr, std::size_t bytes, std::size_t alignment)
	Deallocate memory pointed to by p. More...
void *	allocate_async (std::size_t bytes, std::size_t alignment, cuda_stream_view stream)
	Allocates memory of size at least bytes. More...
void *	allocate_async (std::size_t bytes, cuda_stream_view stream)
	Allocates memory of size at least bytes. More...
void	deallocate_async (void *ptr, std::size_t bytes, std::size_t alignment, cuda_stream_view stream)
	Deallocate memory pointed to by p. More...
void	deallocate_async (void *ptr, std::size_t bytes, cuda_stream_view stream)
	Deallocate memory pointed to by p. More...
bool	operator== (device_memory_resource const &other) const noexcept
	Comparison operator with another device_memory_resource. More...
bool	operator!= (device_memory_resource const &other) const noexcept
	Comparison operator with another device_memory_resource. More...

Detailed Description

Resource that uses system memory resource to allocate memory with a headroom.

System allocated memory (SAM) can be migrated to the GPU, but is never migrated back the host. If GPU memory is over-subscribed, this can cause other CUDA calls to fail with out-of-memory errors. To work around this problem, when using a system memory resource, we reserve some GPU memory as headroom for other CUDA calls, and only conditionally set its preferred location to the GPU if the allocation would not eat into the headroom.

Since doing this check on every allocation can be expensive, the caller may choose to use other allocators (e.g. binning_memory_resource) for small allocations, and use this allocator for large allocations only.

Constructor & Destructor Documentation

sam_headroom_memory_resource()

rmm::mr::sam_headroom_memory_resource::sam_headroom_memory_resource ( std::size_t  headroom )

inlineexplicit

Construct a headroom memory resource.

Parameters

headroom

Size of the reserved GPU memory as headroom

---

The documentation for this class was generated from the following file:

• sam_headroom_memory_resource.hpp