Files
file	contiguous_split.hpp
	Table APIs for contiguous_split, pack, unpack, and metadata.

file	packed_types.hpp
	Packed table and column types for serialization.

file	contiguous_split.hpp
	Table APIs for contiguous_split, pack, unpack, and metadata.

file	copying.hpp
	Column APIs for gather, scatter, split, slice, etc.

Classes
class	cudf::chunked_pack
	Perform a chunked "pack" operation of the input `table_view` using a user provided buffer of size `user_buffer_size`. More...

struct	cudf::packed_columns
	Column data in a serialized format. More...

struct	cudf::packed_table
	A table with its data stored in a contiguous, serialized format. More...

Functions
std::vector< packed_table >	cudf::contiguous_split (cudf::table_view const &input, std::vector< size_type > const &splits, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref mr=cudf::get_current_device_resource_ref())
	Performs a deep-copy split of a `table_view` into a vector of `packed_table` where each `packed_table` is using a single contiguous block of memory for all of the split's column data. More...

packed_columns	cudf::pack (cudf::table_view const &input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref mr=cudf::get_current_device_resource_ref())
	Deep-copy a `table_view` into a serialized contiguous memory format. More...

std::size_t	cudf::packed_size (cudf::table_view const &input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref temp_mr=cudf::get_current_device_resource_ref())
	Compute the size in bytes of the contiguous memory buffer needed to pack the input table. More...

std::vector< uint8_t >	cudf::pack_metadata (table_view const &table, uint8_t const *contiguous_buffer, size_t buffer_size)
	Produce the metadata used for packing a table stored in a contiguous buffer. More...

table_view	cudf::unpack (packed_columns const &input)
	Deserialize the result of `cudf::pack`. More...

table_view	cudf::unpack (uint8_t const metadata, uint8_t const gpu_data)
	Deserialize the result of `cudf::pack`. More...

std::vector< column_view >	cudf::split (column_view const &input, host_span< size_type const > splits, rmm::cuda_stream_view stream=cudf::get_default_stream())
	Splits a `column_view` into a set of `column_view`s according to a set of indices derived from expected splits. More...

std::vector< column_view >	cudf::split (column_view const &input, std::initializer_list< size_type > splits, rmm::cuda_stream_view stream=cudf::get_default_stream())
	Splits a `column_view` into a set of `column_view`s according to a set of indices derived from expected splits. More...

std::vector< table_view >	cudf::split (table_view const &input, host_span< size_type const > splits, rmm::cuda_stream_view stream=cudf::get_default_stream())
	Splits a `table_view` into a set of `table_view`s according to a set of indices derived from expected splits. More...

std::vector< table_view >	cudf::split (table_view const &input, std::initializer_list< size_type > splits, rmm::cuda_stream_view stream=cudf::get_default_stream())
	Splits a `table_view` into a set of `table_view`s according to a set of indices derived from expected splits. More...

Detailed Description

Function Documentation

◆ contiguous_split()

std::vector<packed_table> cudf::contiguous_split	(	cudf::table_view const &	input,
		std::vector< size_type > const &	splits,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`,
		rmm::device_async_resource_ref	mr = `cudf::get_current_device_resource_ref()`
	)

Performs a deep-copy split of a table_view into a vector of packed_table where each packed_table is using a single contiguous block of memory for all of the split's column data.

The memory for the output views is allocated in a single contiguous rmm::device_buffer returned in the packed_table. There is no top-level owning table.

The returned views of input are constructed from a vector of indices, that indicate where each split should occur. The ith returned table_view is sliced as [0, splits[i]) if i=0, else [splits[i], input.size()) if i is the last view and [splits[i-1], splits[i]] otherwise.

For all i it is expected splits[i] <= splits[i+1] <= input.size(). For a splits size N, there will always be N+1 splits in the output.

Note: It is the caller's responsibility to ensure that the returned views do not outlive the viewed device memory contained in the all_data field of the returned packed_table.

Example:
input:   [{10, 12, 14, 16, 18, 20, 22, 24, 26, 28},
          {50, 52, 54, 56, 58, 60, 62, 64, 66, 68}]
splits:  {2, 5, 9}
output:  [{{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}},
          {{50, 52}, {54, 56, 58}, {60, 62, 64, 66}, {68}}]

Exceptions

std::out_of_range	if `splits` has end index > size of `input`.
std::out_of_range	When the value in `splits` is not in the range [0, input.size()).
std::invalid_argument	When the values in the `splits` are 'strictly decreasing'.

Parameters

input	View of a table to split
splits	A vector of indices where the view will be split
stream	CUDA stream used for device memory operations and kernel launches
mr	An optional memory resource to use for all returned device allocations

Returns: The set of requested views of input indicated by the splits and the viewed memory buffer

◆ pack()

packed_columns cudf::pack	(	cudf::table_view const &	input,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`,
		rmm::device_async_resource_ref	mr = `cudf::get_current_device_resource_ref()`
	)

Deep-copy a table_view into a serialized contiguous memory format.

The metadata from the table_view is copied into a host vector of bytes and the data from the table_view is copied into a device_buffer. Pass the output of this function into cudf::unpack to deserialize.

Parameters

input	View of the table to pack
stream	CUDA stream used for device memory operations and kernel launches
mr	An optional memory resource to use for all returned device allocations

Returns: packed_columns A struct containing the serialized metadata and data in contiguous host and device memory respectively

◆ pack_metadata()

std::vector<uint8_t> cudf::pack_metadata	(	table_view const &	table,
		uint8_t const *	contiguous_buffer,
		size_t	buffer_size
	)

Produce the metadata used for packing a table stored in a contiguous buffer.

The metadata from the table_view is copied into a host vector of bytes which can be used to construct a packed_columns or packed_table structure. The caller is responsible for guaranteeing that all of the columns in the table point into contiguous_buffer.

Parameters

table	View of the table to pack
contiguous_buffer	A contiguous buffer of device memory which contains the data referenced by the columns in `table`
buffer_size	The size of `contiguous_buffer`

Returns: Vector of bytes representing the metadata used to unpack a packed_columns struct

◆ packed_size()

std::size_t cudf::packed_size	(	cudf::table_view const &	input,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`,
		rmm::device_async_resource_ref	temp_mr = `cudf::get_current_device_resource_ref()`
	)

Compute the size in bytes of the contiguous memory buffer needed to pack the input table.

This function computes the total contiguous size that would be required to pack the input table using pack() or chunked_pack, without actually performing the packing operation. This is useful for pre-allocating memory or determining if a table will fit in available memory.

Parameters

input	View of the table to compute the packed size for
stream	CUDA stream used for device memory operations and kernel launches
temp_mr	An optional memory resource to use for temporary allocations

Returns: The size in bytes required to store the packed table data (not including metadata)

◆ split() [1/4]

std::vector<column_view> cudf::split	(	column_view const &	input,
		host_span< size_type const >	splits,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`
	)

Splits a column_view into a set of column_views according to a set of indices derived from expected splits.

The returned view's of input are constructed from vector of splits, which indicates where the split should occur. The ith returned column_view is sliced as [0, splits[i]) if i=0, else [splits[i], input.size()) if i is the last view and [splits[i-1], splits[i]] otherwise.

For all i it is expected splits[i] <= splits[i+1] <= input.size() For a splits size N, there will always be N+1 splits in the output

Note: It is the caller's responsibility to ensure that the returned views do not outlive the viewed device memory.

Example:
input:   {10, 12, 14, 16, 18, 20, 22, 24, 26, 28}
splits:  {2, 5, 9}
output:  {{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}}

Exceptions

std::out_of_range	if `splits` has end index > size of `input`.
std::out_of_range	When the value in `splits` is not in the range [0, input.size()).
std::invalid_argument	When the values in the `splits` are 'strictly decreasing'.

Parameters

input	View of column to split
splits	Indices where the view will be split
stream	CUDA stream used for device memory operations and kernel launches

Returns: The set of requested views of input indicated by the splits

◆ split() [2/4]

std::vector<column_view> cudf::split	(	column_view const &	input,
		std::initializer_list< size_type >	splits,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`
	)

Splits a column_view into a set of column_views according to a set of indices derived from expected splits.

The returned view's of input are constructed from vector of splits, which indicates where the split should occur. The ith returned column_view is sliced as [0, splits[i]) if i=0, else [splits[i], input.size()) if i is the last view and [splits[i-1], splits[i]] otherwise.

For all i it is expected splits[i] <= splits[i+1] <= input.size() For a splits size N, there will always be N+1 splits in the output

Note: It is the caller's responsibility to ensure that the returned views do not outlive the viewed device memory.

Example:
input:   {10, 12, 14, 16, 18, 20, 22, 24, 26, 28}
splits:  {2, 5, 9}
output:  {{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}}

Exceptions

std::out_of_range	if `splits` has end index > size of `input`.
std::out_of_range	When the value in `splits` is not in the range [0, input.size()).
std::invalid_argument	When the values in the `splits` are 'strictly decreasing'.

Parameters

input	View of column to split
splits	Indices where the view will be split
stream	CUDA stream used for device memory operations and kernel launches

Returns: The set of requested views of input indicated by the splits

◆ split() [3/4]

std::vector<table_view> cudf::split	(	table_view const &	input,
		host_span< size_type const >	splits,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`
	)

Splits a table_view into a set of table_views according to a set of indices derived from expected splits.

The returned views of input are constructed from vector of splits, which indicates where the split should occur. The ith returned table_view is sliced as [0, splits[i]) if i=0, else [splits[i], input.size()) if i is the last view and [splits[i-1], splits[i]] otherwise.

For all i it is expected splits[i] <= splits[i+1] <= input.size() For a splits size N, there will always be N+1 splits in the output

Note: It is the caller's responsibility to ensure that the returned views do not outlive the viewed device memory.

Example:
input:   [{10, 12, 14, 16, 18, 20, 22, 24, 26, 28},
          {50, 52, 54, 56, 58, 60, 62, 64, 66, 68}]
splits:  {2, 5, 9}
output:  [{{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}},
          {{50, 52}, {54, 56, 58}, {60, 62, 64, 66}, {68}}]

Exceptions

std::out_of_range	if `splits` has end index > size of `input`.
std::out_of_range	When the value in `splits` is not in the range [0, input.size()).
std::invalid_argument	When the values in the `splits` are 'strictly decreasing'.

Parameters

input	View of a table to split
splits	Indices where the view will be split
stream	CUDA stream used for device memory operations and kernel launches

Returns: The set of requested views of input indicated by the splits

◆ split() [4/4]

std::vector<table_view> cudf::split	(	table_view const &	input,
		std::initializer_list< size_type >	splits,
		rmm::cuda_stream_view	stream = `cudf::get_default_stream()`
	)

Splits a table_view into a set of table_views according to a set of indices derived from expected splits.

The returned views of input are constructed from vector of splits, which indicates where the split should occur. The ith returned table_view is sliced as [0, splits[i]) if i=0, else [splits[i], input.size()) if i is the last view and [splits[i-1], splits[i]] otherwise.

For all i it is expected splits[i] <= splits[i+1] <= input.size() For a splits size N, there will always be N+1 splits in the output

Note: It is the caller's responsibility to ensure that the returned views do not outlive the viewed device memory.

Example:
input:   [{10, 12, 14, 16, 18, 20, 22, 24, 26, 28},
          {50, 52, 54, 56, 58, 60, 62, 64, 66, 68}]
splits:  {2, 5, 9}
output:  [{{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}},
          {{50, 52}, {54, 56, 58}, {60, 62, 64, 66}, {68}}]

Exceptions

std::out_of_range	if `splits` has end index > size of `input`.
std::out_of_range	When the value in `splits` is not in the range [0, input.size()).
std::invalid_argument	When the values in the `splits` are 'strictly decreasing'.

Parameters

input	View of a table to split
splits	Indices where the view will be split
stream	CUDA stream used for device memory operations and kernel launches

Returns: The set of requested views of input indicated by the splits

◆ unpack() [1/2]

table_view cudf::unpack ( packed_columns const & input )

Deserialize the result of cudf::pack.

Converts the result of a serialized table into a table_view that points to the data stored in the contiguous device buffer contained in input.

It is the caller's responsibility to ensure that the table_view in the output does not outlive the data in the input.

No new device memory is allocated in this function.

Parameters

input The packed columns to unpack

Returns: The unpacked table_view

◆ unpack() [2/2]

table_view cudf::unpack	(	uint8_t const *	metadata,
		uint8_t const *	gpu_data
	)

Deserialize the result of cudf::pack.

Converts the result of a serialized table into a table_view that points to the data stored in the contiguous device buffer contained in gpu_data using the metadata contained in the host buffer metadata.

It is the caller's responsibility to ensure that the table_view in the output does not outlive the data in the input.

No new device memory is allocated in this function.

Parameters

metadata	The host-side metadata buffer resulting from the initial pack() call
gpu_data	The device-side contiguous buffer storing the data that will be referenced by the resulting `table_view`

Returns: The unpacked table_view

Files

Classes

Functions

Detailed Description

Function Documentation

◆ contiguous_split()

◆ pack()

◆ pack_metadata()

◆ packed_size()

◆ split() [1/4]

◆ split() [2/4]

◆ split() [3/4]

◆ split() [4/4]

◆ unpack() [1/2]

◆ unpack() [2/2]