nvtext Directory Reference

Files
file	byte_pair_encoding.hpp [code]
	APIs for byte pair encoding (BPE) tokenization of strings columns.
file	deduplicate.hpp [code]
	APIs for identifying and removing duplicate substrings in a strings column.
file	edit_distance.hpp [code]
	APIs for computing the Levenshtein edit distance between strings.
file	generate_ngrams.hpp [code]
	APIs for generating character and word ngrams from strings columns.
file	jaccard.hpp [code]
	APIs for computing the Jaccard similarity between strings in two columns.
file	minhash.hpp [code]
	APIs for computing MinHash values of strings columns.
file	ngrams_tokenize.hpp [code]
	APIs for tokenizing strings and generating ngrams from the resulting tokens.
file	normalize.hpp [code]
	APIs for normalizing whitespace and characters within strings columns.
file	replace.hpp [code]
	APIs for replacing tokens within strings columns.
file	stemmer.hpp [code]
	APIs for stemming words using the Porter stemming algorithm measure.
file	tokenize.hpp [code]
	APIs for tokenizing and counting tokens in strings columns.
file	wordpiece_tokenize.hpp [code]
	APIs for WordPiece tokenization of strings columns using a vocabulary.