Column Factories#

group column_factories

Functions

std::unique_ptr<column> make_empty_column(data_type type)#

Creates an empty column of the specified type.

An empty column contains zero elements and no validity mask.

Parameters:: type – [in] The column data type
Returns:: Empty column with desired type

std::unique_ptr<column> make_empty_column(type_id id)#

Creates an empty column of the specified type.

An empty column contains zero elements and no validity mask.

Parameters:: id – [in] The column type id
Returns:: Empty column with specified type

std::unique_ptr<column> make_numeric_column(data_type type, size_type size, mask_state state = mask_state::UNALLOCATED, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified numeric data_type with an optional null mask.

Note

null_count() is determined by the requested null mask state

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a numeric type
cudf::logic_error – if size < 0

Parameters:

type – [in] The desired numeric element type
size – [in] The number of elements in the column
state – [in] Optional, controls allocation/initialization of the column’s null mask. By default, no null mask is allocated.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed numeric column

template<typename B> std::unique_ptr<column> make_numeric_column(data_type type, size_type size, B &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified numeric data_type with a null mask.

Note

null_count is optional and will be computed if not provided.

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a numeric type

Parameters:

type – [in] The desired numeric element type
size – [in] The number of elements in the column
null_mask – [in] Null mask to use for this column.
null_count – [in] Optional number of nulls in the null_mask.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed numeric column

std::unique_ptr<column> make_fixed_point_column(data_type type, size_type size, mask_state state = mask_state::UNALLOCATED, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified fixed_point data_type with an optional null mask.

Note

The column’s null count is determined by the requested null mask state.

Throws:

cudf::logic_error – if type is not a fixed_point type.
cudf::logic_error – if size < 0

Parameters:

type – [in] The desired fixed_point element type.
size – [in] The number of elements in the column.
state – [in] Optional, controls allocation/initialization of the. column’s null mask. By default, no null mask is allocated.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory.

Returns:

Constructed fixed-point type column

template<typename B> std::unique_ptr<column> make_fixed_point_column(data_type type, size_type size, B &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified fixed_point data_type with a null mask.

Note

null_count is optional and will be computed if not provided.

Throws:

cudf::logic_error – if type is not a fixed_point type.

Parameters:

type – [in] The desired fixed_point element type.
size – [in] The number of elements in the column.
null_mask – [in] Null mask to use for this column.
null_count – [in] Optional number of nulls in the null_mask.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory.

Returns:

Constructed fixed-point type column

std::unique_ptr<column> make_timestamp_column(data_type type, size_type size, mask_state state = mask_state::UNALLOCATED, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified timestamp data_type with an optional null mask.

Note

null_count() is determined by the requested null mask state

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a timestamp type
cudf::logic_error – if size < 0

Parameters:

type – [in] The desired timestamp element type
size – [in] The number of elements in the column
state – [in] Optional, controls allocation/initialization of the column’s null mask. By default, no null mask is allocated.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed timestamp type column

template<typename B> std::unique_ptr<column> make_timestamp_column(data_type type, size_type size, B &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified timestamp data_type with a null mask.

Note

null_count is optional and will be computed if not provided.

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a timestamp type

Parameters:

type – [in] The desired timestamp element type
size – [in] The number of elements in the column
null_mask – [in] Null mask to use for this column.
null_count – [in] Optional number of nulls in the null_mask.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed timestamp type column

std::unique_ptr<column> make_duration_column(data_type type, size_type size, mask_state state = mask_state::UNALLOCATED, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified duration data_type with an optional null mask.

Note

null_count() is determined by the requested null mask state

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a duration type
cudf::logic_error – if size < 0

Parameters:

type – [in] The desired duration element type
size – [in] The number of elements in the column
state – [in] Optional, controls allocation/initialization of the column’s null mask. By default, no null mask is allocated.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed duration type column

template<typename B> std::unique_ptr<column> make_duration_column(data_type type, size_type size, B &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified duration data_type with a null mask.

Note

null_count is optional and will be computed if not provided.

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a duration type

Parameters:

type – [in] The desired duration element type
size – [in] The number of elements in the column
null_mask – [in] Null mask to use for this column.
null_count – [in] Optional number of nulls in the null_mask.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed duration type column

std::unique_ptr<column> make_fixed_width_column(data_type type, size_type size, mask_state state = mask_state::UNALLOCATED, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified fixed width data_type with an optional null mask.

Note

null_count() is determined by the requested null mask state

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a fixed width type
cudf::logic_error – if size < 0

Parameters:

type – [in] The desired fixed width type
size – [in] The number of elements in the column
state – [in] Optional, controls allocation/initialization of the column’s null mask. By default, no null mask is allocated.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed fixed-width type column

template<typename B> std::unique_ptr<column> make_fixed_width_column(data_type type, size_type size, B &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct column with sufficient uninitialized storage to hold size elements of the specified fixed width data_type with a null mask.

Note

null_count is optional and will be computed if not provided.

Throws:

std::bad_alloc – if device memory allocation fails
cudf::logic_error – if type is not a fixed width type

Parameters:

type – [in] The desired fixed width element type
size – [in] The number of elements in the column
null_mask – [in] Null mask to use for this column.
null_count – [in] Optional number of nulls in the null_mask.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory

Returns:

Constructed fixed-width type column

std::unique_ptr<column> make_strings_column(cudf::device_span<thrust::pair<char const*, size_type> const> strings, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a STRING type column given a device span of pointer/size pairs.

The total number of char bytes must not exceed the maximum size of size_type. The string characters are expected to be UTF-8 encoded sequence of char bytes. Use the strings_column_view class to perform strings operations on this type of column.

Note

null_count() and null_bitmask are determined if a pair contains a null string. That is, for each pair, if .first is null, that string is considered null. Likewise, a string is considered empty (not null) if .first is not null and .second is 0. Otherwise the .first member must be a valid device address pointing to .second consecutive bytes.

Throws:

std::bad_alloc – if device memory allocation fails

Parameters:

strings – [in] The device span of pointer/size pairs. Each pointer must be a device memory address or nullptr (indicating a null string). The size must be the number of bytes.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used for allocation of the column’s null_mask and children columns’ device memory.

Returns:

Constructed strings column

std::vector<std::unique_ptr<column>> make_strings_column_batch(std::vector<cudf::device_span<thrust::pair<char const*, size_type> const>> const &input, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a batch of STRING type columns given an array of device spans of pointer/size pairs.

This function has input/output expectation similar to the make_strings_column() API that accepts only one device span of pointer/size pairs. The difference is that, this is designed to create many strings columns at once with minimal overhead of multiple kernel launches and stream synchronizations.

Parameters:

input – Array of device spans of pointer/size pairs, where each pointer is a device memory address or nullptr (indicating a null string), and size is string length (in bytes)
stream – CUDA stream used for device memory operations and kernel launches
mr – Device memory resource used for memory allocation of the output columns

Returns:

Array of constructed strings columns

std::unique_ptr<column> make_strings_column(cudf::device_span<string_view const> string_views, string_view const null_placeholder, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a STRING type column given a device span of string_view.

Note

For each string_view, if .data() is null_placeholder.data(), that string is considered null. Likewise, a string is considered empty (not null) if .data() is not null_placeholder.data() and .size_bytes() is 0. Otherwise the .data() must be a valid device address pointing to .size_bytes() consecutive bytes. The null_count() for the output column will be equal to the number of input string_views that are null.

Throws:

std::bad_alloc – if device memory allocation fails

Parameters:

string_views – [in] The span of string_view. Each string_view must point to a device memory address or null_placeholder (indicating a null string). The size must be the number of bytes.
null_placeholder – [in] string_view indicating null string in given list of string_views.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used for allocation of the column’s null_mask and children columns’ device memory.

Returns:

Constructed strings column

std::unique_ptr<column> make_strings_column(size_type num_strings, std::unique_ptr<column> offsets_column, rmm::device_buffer &&chars_buffer, size_type null_count, rmm::device_buffer &&null_mask)#

Construct a STRING type column given offsets column, chars columns, and null mask and null count.

The columns and mask are moved into the resulting strings column.

Parameters:

num_strings – The number of strings the column represents.
offsets_column – The column of offset values for this column. The number of elements is one more than the total number of strings so the offset[last] - offset[0] is the total number of bytes in the strings vector.
chars_buffer – The buffer of char bytes for all the strings for this column. Individual strings are identified by the offsets and the nullmask.
null_count – The number of null string entries.
null_mask – The bits specifying the null strings in device memory. Arrow format for nulls is used for interpreting this bitmask.

Returns:

Constructed strings column

std::unique_ptr<cudf::column> make_lists_column(size_type num_rows, std::unique_ptr<column> offsets_column, std::unique_ptr<column> child_column, size_type null_count, rmm::device_buffer &&null_mask, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a LIST type column given offsets column, child column, null mask and null count.

The columns and mask are moved into the resulting lists column.

List columns are structured similarly to strings columns. They contain a set of offsets which represents the lengths of the lists in each row, and a “child” column of data that is referenced by the offsets. Since lists are a nested type, the child column may itself be further nested.

When child column at depth N+1 is itself a list, the offsets column at depth N references the offsets column for depth N+1. When the child column at depth N+1 is a leaf type (int, float, etc), the offsets column at depth N references the data for depth N+1.

Example:
List<int>
input:              {{1, 2}, {3, 4, 5}}
offsets (depth 0)   {0, 2, 5}
data    (depth 0)
offsets (depth 1)
data    (depth 1)   {1, 2, 3, 4, 5}

Example:
List<List<int>>
input:              { {{1, 2}}, {{3, 4, 5}, {6, 7}} }
offsets (depth 0)   {0, 1, 3}
data    (depth 0)
offsets (depth 1)   {0, 2, 5, 7}
data    (depth 1)
offsets (depth 2)
data    (depth 2)   {1, 2, 3, 4, 5, 6, 7}

Parameters:

num_rows – [in] The number of lists the column represents.
offsets_column – [in] The column of offset values for this column. Each value should represent the starting offset into the child elements that corresponds to the beginning of the row, with the first row starting at 0. The length of row N can be determined by subtracting offsets[N+1] - offsets[N]. The total number of offsets should be 1 longer than the # of rows in the column.
child_column – [in] The column of nested data referenced by the lists represented by the offsets_column. Note: the child column may itself be further nested.
null_count – [in] The number of null list entries.
null_mask – [in] The bits specifying the null lists in device memory. Arrow format for nulls is used for interpreting this bitmask.
stream – [in] Optional stream for use with all memory allocation and device kernels
mr – [in] Optional resource to use for device memory allocation of the column’s null_mask and children.

Returns:

Constructed lists column

std::unique_ptr<cudf::column> make_structs_column(size_type num_rows, std::vector<std::unique_ptr<column>> &&child_columns, size_type null_count, rmm::device_buffer &&null_mask, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a STRUCT column using specified child columns as members.

Specified child/member columns and null_mask are adopted by resultant struct column.

A struct column requires that all specified child columns have the same number of rows. A struct column’s row count equals that of any/all of its child columns. A single struct row at any index is comprised of all the individual child column values at the same index, in the order specified in the list of child columns.

The specified null mask governs which struct row has a null value. This is orthogonal to the null values of individual child columns.

Parameters:

num_rows – [in] The number of struct values in the struct column.
child_columns – [in] The list of child/members that the struct is comprised of.
null_count – [in] The number of null values in the struct column.
null_mask – [in] The bits specifying the null struct values in the column.
stream – [in] Optional stream for use with all memory allocation and device kernels.
mr – [in] Optional resource to use for device memory allocation.

Returns:

Constructed structs column

std::unique_ptr<column> make_column_from_scalar(scalar const &s, size_type size, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a column with size elements that are all equal to the given scalar.

The output column will have the same type as s.type() The output column will contain all null rows if s.invalid()==false The output column will be empty if size==0. For LIST scalars, the column hierarchy from s is preserved.

Parameters:

s – [in] The scalar to use for values in the column.
size – [in] The number of rows for the output column.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory.

Returns:

Constructed column whose rows all contain the scalar value

std::unique_ptr<column> make_dictionary_from_scalar(scalar const &s, size_type size, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a dictionary column with size elements that are all equal to the given scalar.

The output column will have keys of type s.type() The output column will be empty if size==0.

Throws:

cudf::logic_error – if s.is_valid()==false

Parameters:

s – [in] The scalar to use for values in the column.
size – [in] The number of rows for the output column.
stream – [in] CUDA stream used for device memory operations and kernel launches.
mr – [in] Device memory resource used to allocate the returned column’s device memory.

Returns:

Constructed dictionary column

std::unique_ptr<column> make_dictionary_column(column_view const &keys_column, column_view const &indices_column, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a dictionary column by copying the provided keys and indices.

It is expected that keys_column.has_nulls() == false. It is assumed the elements in keys_column are unique and are in a strict, total order. Meaning, keys_column[i] is ordered before keys_column[i+1] for all i in [0,n-1) where n is the number of keys.

The indices values must be in the range [0,keys_column.size()).

The null_mask and null count for the output column are copied from the indices column. If element i in indices_column is null, then element i in the returned dictionary column will also be null.

k = ["a","c","d"]
i = [1,0,null,2,2]
d = make_dictionary_column(k,i)
d is now {["a","c","d"],[1,0,undefined,2,2]} bitmask={1,1,0,1,1}

The null_mask and null count for the output column are copied from the indices column.

Throws:

cudf::logic_error – if keys_column contains nulls
cudf::logic_error – if indices_column type is not INT32

Parameters:

keys_column – Column of unique, ordered values to use as the new dictionary column’s keys.
indices_column – Indices to use for the new dictionary column.
stream – CUDA stream used for device memory operations and kernel launches.
mr – Device memory resource used to allocate the returned column’s device memory.

Returns:

New dictionary column.

std::unique_ptr<column> make_dictionary_column(std::unique_ptr<column> keys_column, std::unique_ptr<column> indices_column, rmm::device_buffer &&null_mask, size_type null_count, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a dictionary column by taking ownership of the provided keys and indices columns.

The keys_column and indices columns must contain no nulls. It is assumed the elements in keys_column are unique and are in a strict, total order. Meaning, keys_column[i] is ordered before keys_column[i+1] for all i in [0,n-1) where n is the number of keys.

The indices values must be in the range [0,keys_column.size()).

Throws:

cudf::logic_error – if keys_column or indices_column contains nulls
cudf::logic_error – if indices_column type is not an unsigned integer type

Parameters:

keys_column – Column of unique, ordered values to use as the new dictionary column’s keys.
indices_column – Indices to use for the new dictionary column.
null_mask – Null mask for the output column.
null_count – Number of nulls for the output column.
stream – CUDA stream used for device memory operations and kernel launches.
mr – Device memory resource used to allocate the returned column’s device memory.

Returns:

New dictionary column.

std::unique_ptr<column> make_dictionary_column(std::unique_ptr<column> keys_column, std::unique_ptr<column> indices_column, rmm::cuda_stream_view stream = cudf::get_default_stream(), rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref())#

Construct a dictionary column by taking ownership of the provided keys and indices columns.

The keys_column must contain no nulls and is assumed to have elements that are unique and are in a strict, total order. Meaning, keys_column[i] is ordered before keys_column[i+1] for all i in [0,n-1) where n is the number of keys.

The indices_column can be any integer type and should contain the null-mask to be used for the output column. The indices values must be in the range [0,keys_column.size()).

Throws:

cudf::logic_error – if keys_column contains nulls

Parameters:

keys_column – Column of unique, ordered values to use as the new dictionary column’s keys.
indices_column – Indices values and null-mask to use for the new dictionary column.
stream – CUDA stream used for device memory operations and kernel launches.
mr – Device memory resource used to allocate the returned column’s device memory.

Returns:

New dictionary column.