bert_tokenizer

[ API doc | crates.io ]

The crate provides the port of the original BERT tokenizer from the Google BERT repository.

License

MIT license. See LICENSE file for full license.