Public Member Functions | List of all members
nvtext::bpe_merge_pairs Struct Reference

The table of merge pairs for the BPE encoder. More...

#include <byte_pair_encoding.hpp>

Public Member Functions

 bpe_merge_pairs (std::unique_ptr< cudf::column > &&input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref mr=cudf::get_current_device_resource_ref())
 Construct a new bpe merge pairs object. More...
 
 bpe_merge_pairs (cudf::strings_column_view const &input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref mr=cudf::get_current_device_resource_ref())
 Construct a new bpe merge pairs object. More...
 

Detailed Description

The table of merge pairs for the BPE encoder.

To create an instance, call nvtext::load_merge_pairs

Definition at line 39 of file byte_pair_encoding.hpp.

Constructor & Destructor Documentation

◆ bpe_merge_pairs() [1/2]

nvtext::bpe_merge_pairs::bpe_merge_pairs ( std::unique_ptr< cudf::column > &&  input,
rmm::cuda_stream_view  stream = cudf::get_default_stream(),
rmm::device_async_resource_ref  mr = cudf::get_current_device_resource_ref() 
)

Construct a new bpe merge pairs object.

Parameters
inputThe input file containing the BPE merge pairs
streamCUDA stream used for device memory operations and kernel launches.
mrDevice memory resource used to allocate the device memory

◆ bpe_merge_pairs() [2/2]

nvtext::bpe_merge_pairs::bpe_merge_pairs ( cudf::strings_column_view const &  input,
rmm::cuda_stream_view  stream = cudf::get_default_stream(),
rmm::device_async_resource_ref  mr = cudf::get_current_device_resource_ref() 
)

Construct a new bpe merge pairs object.

Parameters
inputThe input column of strings
streamCUDA stream used for device memory operations and kernel launches.
mrDevice memory resource used to allocate the device memory

The documentation for this struct was generated from the following file: