nlpO3

Thai Natural Language Processing library in Rust, with Python and Node bindings. Formerly oxidized-thainlp.

Features

• Thai word tokenizer
  • use maximal-matching dictionary-based tokenization algorithm and honor Thai Character Cluster boundaries
  • 2x faster than similar pure Python implementation (PyThaiNLP's newmm)
  • support custom dictionary
  • default dictionary included (62,000 words, a copy from PyThaiNLP)

Usage

Command line interface

• nlpo3-cli

bash echo "ฉันกินข้าว" | nlpo3 segment

Bindings

• Node.js
• Python

```python from nlpo3 import segment

segment("สวัสดีครับ") ```

As Rust library

In Cargo.toml:

```toml [dependencies]

...

nlpo3 = "1.2.0" ```

Build

Requirements

• Rust 2018 Edition

Steps

Generic test: bash cargo test

Build API document and open it to check: bash cargo doc --open

Build (remove --release to keep debug information): bash cargo build --release

Check target/ for build artifacts.

Issues

Please report issues at https://github.com/PyThaiNLP/nlpo3/issues