Attention

The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called cuVS. We will continue to support the vector search algorithms in RAFT during this move, but will no longer update them after the RAPIDS 24.06 (June) release. We plan to complete the migration by RAPIDS 24.10 (October) release and they will be removed from RAFT altogether in the 24.12 (December) release.

mdarray: Multi-dimensional Owning Container#

#include <raft/core/mdarray.hpp>

template<typename ...Tn> constexpr bool is_array_interface_v = is_array_interface<Tn...>::value#: \brief Boolean to determine if variadic template types Tn are raft::array_interface or derived type or any type that has a member function view() that returns either raft::host_mdspan or raft::device_mdspan

template<typename Base> class array_interface#

#include <mdarray.hpp>

Interface to implement an owning multi-dimensional array.

raft::array_interace is an interface to owning container types for mdspan. Check implementation of raft::mdarray which implements raft::array_interface using Curiously Recurring Template Pattern. This interface calls into method view() whose implementation is provided by the implementing class. view() must return an object of type raft::host_mdspan or raft::device_mdspan or any types derived from the them.

template<typename...> struct is_array_interface : public std::true_type#: #include <mdarray.hpp>

template<typename T1> struct is_array_interface<T1> : public raft::detail::is_array_interface<T1>#: #include <mdarray.hpp>

template<typename T1, typename ...Tn> struct is_array_interface<T1, Tn...> : public std::conditional_t<detail::is_array_interface_v<T1>, is_array_interface<Tn...>, std::false_type>#: #include <mdarray.hpp>

template<typename ElementType, typename Extents, typename LayoutPolicy, typename ContainerPolicy> class mdarray : public raft::array_interface<mdarray<ElementType, Extents, LayoutPolicy, ContainerPolicy>>#

#include <mdarray.hpp>

Modified from the c++ mdarray proposal.

https://isocpp.org/files/papers/D1684R0.html

mdarray is a container type for mdspan with similar template arguments. However there are some inconsistencies in between them. We have made some modificiations to fit our needs, which are listed below.

Layout policy is different, the mdarray in raft uses std::experimental::extent directly just like mdspan, while the mdarray in the reference implementation uses varidic template.
Most of the constructors from the reference implementation is removed to make sure CUDA stream is honored. Note that this class is not coupled to CUDA and therefore will only be used in the case where the device variant is used.
unique_size is not implemented, which is still working in progress in the proposal
For container policy, we adopt the alternative approach documented in the proposal [sec 2.4.3], which requires an additional make_accessor method for it to be used in mdspan. The container policy reference implementation has multiple access methods that accommodate needs for both mdarray and mdspan. This is more difficult for us since the policy might contain states that are unwanted inside a CUDA kernel. Also, on host we return a proxy to the actual value as device_ref so different access methods will have different return type, which is less desirable.
For the above reasons, copying from other mdarray with different policy type is also removed.

Public Types

using view_type = view_type_impl<element_type>#: the mdspan type returned by view method.

Public Functions

inline RAFT_MDARRAY_CTOR_CONSTEXPR mdarray(raft::resources const &handle, mapping_type const &m, container_policy_type const &cp)#: The only constructor that can create storage, raft::resources is accepted so that the device implementation can make sure the relevant CUDA stream is being used for allocation.

inline auto view() noexcept#: Get an mdspan.

inline auto view() const noexcept#: Get an mdspan<const T>

template<typename ...IndexType> inline auto operator()(IndexType&&... indices) -> std::enable_if_t<sizeof...(IndexType) == extents_type::rank() && (std::is_convertible_v<IndexType, index_type>&&...)&&std::is_constructible_v<extents_type, IndexType...>, reference>#: Indexing operator, use it sparingly since it triggers a device<->host copy.

template<typename ...IndexType> inline auto operator()(IndexType&&... indices) const -> std::enable_if_t<sizeof...(IndexType) == extents_type::rank() && (std::is_convertible_v<IndexType, index_type>&&...)&&std::is_constructible_v<extents_type, IndexType...> && std::is_constructible<mapping_type, extents_type>::value, const_reference>#: Indexing operator, use it sparingly since it triggers a device<->host copy.

inline RAFT_INLINE_FUNCTION constexpr auto extent (size_t r) const noexcept -> index_type: the extent of rank r

Device Vocabulary#

#include <raft/core/device_mdarray.hpp>

template<typename ElementType, typename Extents, typename LayoutPolicy = layout_c_contiguous, typename ContainerPolicy = device_uvector_policy<ElementType>> using raft::device_mdarray = mdarray<ElementType, Extents, LayoutPolicy, device_accessor<ContainerPolicy>>#

mdarray with device container policy

Template Parameters:

ElementType – the data type of the elements
Extents – defines the shape
LayoutPolicy – policy for indexing strides and layout ordering
ContainerPolicy – storage and accessor policy

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::device_matrix = device_mdarray<ElementType, matrix_extent<IndexType>, LayoutPolicy>#

Shorthand for c-contiguous device matrix.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::device_vector = device_mdarray<ElementType, vector_extent<IndexType>, LayoutPolicy>#

Shorthand for 1-dim device mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t> using raft::device_scalar = device_mdarray<ElementType, scalar_extent<IndexType>>#

Shorthand for 0-dim host mdarray (scalar).

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Device Factories#

#include <raft/core/device_mdarray.hpp>

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_device_matrix(raft::resources const &handle, IndexType n_rows, IndexType n_cols)#

Create a 2-dim c-contiguous device mdarray.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive resources
n_rows – [in] number or rows in matrix
n_cols – [in] number of columns in matrix

Returns:

raft::device_matrix

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_device_vector(raft::resources const &handle, IndexType n)#

Create a 1-dim device mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive cuda resources
n – [in] number of elements in vector

Returns:

raft::device_vector

template<typename ElementType, typename IndexType = std::uint32_t> auto raft::make_device_scalar(raft::resources const &handle, ElementType const &v)#

Create a device scalar from v.

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Parameters:

handle – [in] raft handle for managing expensive cuda resources
v – [in] scalar to wrap on device

Returns:

raft::device_scalar

Host Vocabulary#

#include <raft/core/host_mdarray.hpp>

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::host_matrix = host_mdarray<ElementType, matrix_extent<IndexType>, LayoutPolicy>#

Shorthand for c-contiguous host matrix.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::host_vector = host_mdarray<ElementType, vector_extent<IndexType>, LayoutPolicy>#

Shorthand for 1-dim host mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t> using raft::host_scalar = host_mdarray<ElementType, scalar_extent<IndexType>>#

Shorthand for 0-dim host mdarray (scalar).

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Host Factories#

#include <raft/core/host_mdarray.hpp>

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous, size_t... Extents> auto make_host_mdarray(raft::resources &res, extents<IndexType, Extents...> exts)#

Create a host mdarray.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

res – [in] raft handle for managing expensive resources
exts – [in] dimensionality of the array (series of integers)

Returns:

raft::host_mdarray

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto make_host_matrix(raft::resources &res, IndexType n_rows, IndexType n_cols)#

Create a 2-dim c-contiguous host mdarray.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

res – [in] raft handle for managing expensive resources
n_rows – [in] number or rows in matrix
n_cols – [in] number of columns in matrix

Returns:

raft::host_matrix

template<typename ElementType, typename IndexType = std::uint32_t> auto make_host_scalar(raft::resources &res, ElementType const &v)#

Create a host scalar from v.

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Parameters:

res – [in] raft handle for managing expensive resources
v – [in] scalar type to wrap

Returns:

raft::host_scalar

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto make_host_vector(raft::resources &res, IndexType n)#

Create a 1-dim host mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

res – [in] raft handle for managing expensive resources
n – [in] number of elements in vector

Returns:

raft::host_vector

Managed Vocabulary#

#include <raft/core/managed_mdarray.hpp>

template<typename ElementType, typename Extents, typename LayoutPolicy = layout_c_contiguous, typename ContainerPolicy = managed_uvector_policy<ElementType>> using raft::managed_mdarray = mdarray<ElementType, Extents, LayoutPolicy, managed_accessor<ContainerPolicy>>#

mdarray with managed container policy

Template Parameters:

ElementType – the data type of the elements
Extents – defines the shape
LayoutPolicy – policy for indexing strides and layout ordering
ContainerPolicy – storage and accessor policy

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::managed_matrix = managed_mdarray<ElementType, matrix_extent<IndexType>, LayoutPolicy>#

Shorthand for c-contiguous managed matrix.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::managed_vector = managed_mdarray<ElementType, vector_extent<IndexType>, LayoutPolicy>#

Shorthand for 1-dim managed mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t> using raft::managed_scalar = managed_mdarray<ElementType, scalar_extent<IndexType>>#

Shorthand for 0-dim host mdarray (scalar).

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Managed Factories#

#include <raft/core/managed_mdarray.hpp>

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_managed_matrix(raft::resources const &handle, IndexType n_rows, IndexType n_cols)#

Create a 2-dim c-contiguous managed mdarray.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive resources
n_rows – [in] number or rows in matrix
n_cols – [in] number of columns in matrix

Returns:

raft::managed_matrix

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_managed_vector(raft::resources const &handle, IndexType n)#

Create a 1-dim managed mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive cuda resources
n – [in] number of elements in vector

Returns:

raft::managed_vector

template<typename ElementType, typename IndexType = std::uint32_t> auto raft::make_managed_scalar(raft::resources const &handle, ElementType const &v)#

Create a managed scalar from v.

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Parameters:

handle – [in] raft handle for managing expensive cuda resources
v – [in] scalar to wrap on managed

Returns:

raft::managed_scalar

Pinned Vocabulary#

#include <raft/core/pinned_mdarray.hpp>

template<typename ElementType, typename Extents, typename LayoutPolicy = layout_c_contiguous, typename ContainerPolicy = pinned_vector_policy<ElementType>> using raft::pinned_mdarray = mdarray<ElementType, Extents, LayoutPolicy, pinned_accessor<ContainerPolicy>>#

mdarray with pinned container policy

Template Parameters:

ElementType – the data type of the elements
Extents – defines the shape
LayoutPolicy – policy for indexing strides and layout ordering
ContainerPolicy – storage and accessor policy

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::pinned_matrix = pinned_mdarray<ElementType, matrix_extent<IndexType>, LayoutPolicy>#

Shorthand for c-contiguous pinned matrix.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> using raft::pinned_vector = pinned_mdarray<ElementType, vector_extent<IndexType>, LayoutPolicy>#

Shorthand for 1-dim pinned mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

template<typename ElementType, typename IndexType = std::uint32_t> using raft::pinned_scalar = pinned_mdarray<ElementType, scalar_extent<IndexType>>#

Shorthand for 0-dim host mdarray (scalar).

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Pinned Factories#

#include <raft/core/pinned_mdarray.hpp>

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_pinned_matrix(raft::resources const &handle, IndexType n_rows, IndexType n_cols)#

Create a 2-dim c-contiguous pinned mdarray.

Template Parameters:

ElementType – the data type of the matrix elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive resources
n_rows – [in] number or rows in matrix
n_cols – [in] number of columns in matrix

Returns:

raft::pinned_matrix

template<typename ElementType, typename IndexType = std::uint32_t, typename LayoutPolicy = layout_c_contiguous> auto raft::make_pinned_vector(raft::resources const &handle, IndexType n)#

Create a 1-dim pinned mdarray.

Template Parameters:

ElementType – the data type of the vector elements
IndexType – the index type of the extents
LayoutPolicy – policy for strides and layout ordering

Parameters:

handle – [in] raft handle for managing expensive cuda resources
n – [in] number of elements in vector

Returns:

raft::pinned_vector

template<typename ElementType, typename IndexType = std::uint32_t> auto raft::make_pinned_scalar(raft::resources const &handle, ElementType const &v)#

Create a pinned scalar from v.

Template Parameters:

ElementType – the data type of the scalar element
IndexType – the index type of the extents

Parameters:

handle – [in] raft handle for managing expensive cuda resources
v – [in] scalar to wrap on pinned

Returns:

raft::pinned_scalar