Memory resource that uses cudaMallocAsync/cudaFreeAsync for allocation/deallocation. More...

#include <cuda_async_memory_resource.hpp>

Inheritance diagram for rmm::mr::cuda_async_memory_resource:

Collaboration diagram for rmm::mr::cuda_async_memory_resource:

Public Types
enum class	allocation_handle_type : std::int32_t { none = cudaMemHandleTypeNone , posix_file_descriptor , win32 , win32_kmt = cudaMemHandleTypeWin32Kmt , fabric = 0x8 }
	Flags for specifying memory allocation handle types. More...

enum class	mempool_usage : unsigned short { hw_decompress = 0x2 }
	Flags for specifying memory pool usage. More...

Public Member Functions
	cuda_async_memory_resource (std::optional< std::size_t > initial_pool_size={}, std::optional< std::size_t > release_threshold={}, std::optional< allocation_handle_type > export_handle_type={})
	Constructs a cuda_async_memory_resource with the optionally specified initial pool size and release threshold. More...

cudaMemPool_t	pool_handle () const noexcept
	Returns the underlying native handle to the CUDA pool. More...

	cuda_async_memory_resource (cuda_async_memory_resource const &)=default
	Default copy constructor.

	cuda_async_memory_resource (cuda_async_memory_resource &&)=default
	Default move constructor.

cuda_async_memory_resource &	operator= (cuda_async_memory_resource const &)=default
	Default copy assignment operator. More...

cuda_async_memory_resource &	operator= (cuda_async_memory_resource &&)=default
	Default move assignment operator. More...

Friends
void	get_property (cuda_async_memory_resource const &, cuda::mr::device_accessible) noexcept
	Enables the `cuda::mr::device_accessible` property.

Detailed Description

Memory resource that uses cudaMallocAsync/cudaFreeAsync for allocation/deallocation.

Member Enumeration Documentation

◆ allocation_handle_type

enum rmm::mr::cuda_async_memory_resource::allocation_handle_type : std::int32_t

strong

Flags for specifying memory allocation handle types.

Note: These values are exact copies from cudaMemAllocationHandleType. We need a placeholder that can be used consistently in the constructor of cuda_async_memory_resource with all supported versions of CUDA. See the cudaMemAllocationHandleType docs at https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html and ensure the enum values are kept in sync with the CUDA documentation.; cudaMemHandleTypeFabric can be used instead of 0x8 once we require CUDA 12.4+.

Enumerator
none	Does not allow any export mechanism.
posix_file_descriptor	Allows a file descriptor to be used for exporting. Permitted only on POSIX systems.
win32	Allows a Win32 NT handle to be used for exporting. (HANDLE)
win32_kmt	Allows a Win32 KMT handle to be used for exporting. (D3DKMT_HANDLE)
fabric	Allows a fabric handle to be used for exporting. (cudaMemFabricHandle_t)

◆ mempool_usage

enum rmm::mr::cuda_async_memory_resource::mempool_usage : unsigned short

strong

Flags for specifying memory pool usage.

Note: These values are exact copies from the runtime API. See the cudaMemPoolProps docs at https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaMemPoolProps.html and ensure the enum values are kept in sync with the CUDA documentation. cudaMemPoolCreateUsageHwDecompress is currently the only supported usage flag, introduced in CUDA 12.8 and documented in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html

Enumerator
hw_decompress	If set indicates that the memory can be used as a buffer for hardware accelerated decompression.

Constructor & Destructor Documentation

◆ cuda_async_memory_resource()

rmm::mr::cuda_async_memory_resource::cuda_async_memory_resource	(	std::optional< std::size_t >	initial_pool_size = `{}`,
		std::optional< std::size_t >	release_threshold = `{}`,
		std::optional< allocation_handle_type >	export_handle_type = `{}`
	)

Constructs a cuda_async_memory_resource with the optionally specified initial pool size and release threshold.

If the pool size grows beyond the release threshold, unused memory held by the pool will be released at the next synchronization event.

Exceptions

rmm::logic_error if the CUDA version does not support cudaMallocAsync

Parameters

initial_pool_size	Optional initial size in bytes of the pool. If provided, the pool will be primed by allocating and immediately deallocating this amount of memory on the default CUDA stream.
release_threshold	Optional release threshold size in bytes of the pool. If no value is provided, the release threshold is set to the total amount of memory on the current device.
export_handle_type	Optional `cudaMemAllocationHandleType` that allocations from this resource should support interprocess communication (IPC). Default is `cudaMemHandleTypeNone` for no IPC support.

Member Function Documentation

◆ operator=() [1/2]

cuda_async_memory_resource& rmm::mr::cuda_async_memory_resource::operator= ( cuda_async_memory_resource && )

default

Default move assignment operator.

Returns: cuda_async_memory_resource& Reference to the assigned object

◆ operator=() [2/2]

cuda_async_memory_resource& rmm::mr::cuda_async_memory_resource::operator= ( cuda_async_memory_resource const & )

default

Default copy assignment operator.

Returns: cuda_async_memory_resource& Reference to the assigned object

◆ pool_handle()

cudaMemPool_t rmm::mr::cuda_async_memory_resource::pool_handle ( ) const

noexcept

Returns the underlying native handle to the CUDA pool.

Returns: cudaMemPool_t Handle to the underlying CUDA pool

The documentation for this class was generated from the following file:

cuda_async_memory_resource.hpp

Public Types

Public Member Functions

Friends