LEMON parser generator modified to generate Rust code.
Lemon source and SQLite3 grammar were last synced as of september 2020.
%token_destructor: Code to execute to destroy token data%default_destructor: Code for the default non-terminal destructor%destructor: Code which executes whenever this symbol is
popped from the stack during error processinghttps://www.codeproject.com/Articles/1056460/Generating-a-High-Speed-Parser-Part-Lemon https://www.sqlite.org/lemon.html
SQLite lexer and SQLite parser have been ported from C to Rust. The parser generates an AST.
Lexer/Parser: - Keep track of position (line, column). - Streamable (stop at the end of statement). - Resumable (restart after the end of statement).
Lexer and parser have been tested with the following scripts: * https://github.com/bkiers/sqlite-parser/tree/master/src/test/resources * https://github.com/codeschool/sqlite-parser/tree/master/test/sql/official-suite which can be updated with script in https://github.com/codeschool/sqlite-parser/tree/master/test/misc
TODO: - [ ] Check generated AST (reparse/reinject) - [ ] [If a keyword in double quotes is used in a context where it cannot be resolved to an identifier but where a string literal is allowed, then the token is understood to be a string literal instead of an identifier.](https://sqlite.org/lang_keywords.html) - [ ] Tests - [ ] Do not panic while parsing - [ ] CREATE VIRTUAL TABLE args - [ ] Zero copy (at least tokens)
#line directiveParseAlloc/ParseFree anymore %extra_argument is not supported.lemon_rust does the same thing
but with an old version of lemon. And it seems not possible to use yystack
as a stack because items may be access randomly and the top+1 item can be used.
lalrpop would be the perfect alternative but it does not support fallback/streaming (see this issue) and compilation/generation is slow.