IO Utilities#

group IO Utilities

Functions

std::unique_ptr<cudf::io::datasource::buffer> fetch_footer_to_host(cudf::io::datasource &datasource)#

Fetches a host buffer of Parquet footer bytes from the input data source.

Parameters:

datasource – Input data source

Returns:

Host buffer containing footer bytes

std::unique_ptr<cudf::io::datasource::buffer> fetch_page_index_to_host(cudf::io::datasource &datasource, byte_range_info const page_index_bytes)#

Fetches a host buffer of Parquet page index from the input data source.

Parameters:

  • datasource – Input datasource

  • page_index_bytes – Byte range of page index

Returns:

Host buffer containing page index bytes

std::tuple<std::vector<rmm::device_buffer>, std::vector<cudf::device_span<uint8_t const>>, std::future<void>> fetch_byte_ranges_to_device_async(cudf::io::datasource &datasource, cudf::host_span<byte_range_info const> byte_ranges, rmm::cuda_stream_view stream, rmm::device_async_resource_ref mr)#

Fetches a list of byte ranges from a datasource into device buffers.

Parameters:

  • datasource – Input datasource

  • byte_ranges – Byte ranges to fetch

  • stream – CUDA stream

  • mr – Device memory resource

Returns:

A tuple containing the device buffers, the device spans of the fetched data, and a future to wait on the read tasks