Uses of Interface
opennlp.tools.tokenize.SubwordTokenizer

Packages that use SubwordTokenizer

Package	Description
opennlp.tools.tokenize

Uses of SubwordTokenizer in opennlp.tools.tokenize

Classes in opennlp.tools.tokenize that implement SubwordTokenizer

Modifier and Type	Class	Description
final class	WordpieceEncoder	A SubwordTokenizer implementing the BERT tokenization stages: basic tokenization (control removal, whitespace normalization, CJK isolation, optional lower casing with accent stripping, punctuation isolation) followed by greedy longest-match wordpiece segmentation.