lex

This library aids in lexing programming languages.

Design

This library provides a single implementation for lexing programming languages that is general enough to fit most modern programming languages. To keep things simple the implementation is not modifiable, though this may come in later versions.

Usage

You will first need to lex your source code:

use lex::{Lex, Lexer};
...
let source: &str = ...
let result: Lex = Lexer::new().lex(source);

You should then be able to use the Kind enum to differentiate different tokens in your tokenized source code.