ngrams_tokenize.hpp File Reference

APIs for tokenizing strings and generating ngrams from the resulting tokens. More...

#include <cudf/column/column.hpp>
#include <cudf/scalar/scalar.hpp>
#include <cudf/strings/strings_column_view.hpp>
#include <cudf/utilities/export.hpp>
#include <cudf/utilities/memory_resource.hpp>

Go to the source code of this file.

Namespaces
	nvtext
	NVText APIs.

Functions
std::unique_ptr< cudf::column >	nvtext::ngrams_tokenize (cudf::strings_column_view const &input, cudf::size_type ngrams, cudf::string_scalar const &delimiter, cudf::string_scalar const &separator, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::device_async_resource_ref mr=cudf::get_current_device_resource_ref())
	Returns a single column of strings by tokenizing the input strings column and then producing ngrams of each string. More...

Detailed Description

APIs for tokenizing strings and generating ngrams from the resulting tokens.

Definition in file ngrams_tokenize.hpp.