A collection of tokenizers for Natural Language Processing.
This crate is distributed under the terms of both the MIT License and the Apache License (version 2.0). See the files and for details.