WordPieceTokenizer

Constructor

WordPieceVocabulary(vocabulary)	A vocabulary object used to tokenize input text.
WordPieceVocabulary.tokenize(text[, ...])	Produces tokens for the input strings.

SubwordTokenizer(hash_file[, do_lower_case])	Run CUDA BERT subword tokenizer on cuDF strings column.
SubwordTokenizer.__call__(text, max_length, ...)	Run CUDA BERT subword tokenizer on cuDF strings column.