grex is a library as well as a command-line utility that is meant to simplify the often complicated and tedious task of creating regular expressions. It does so by automatically generating regular expressions from user-provided test cases. The produced expressions are Perl-compatible regular expressions (PCRE) which are also compatible with the regular expression parser in the regex crate. Other regular expression parsers or respective libraries from other programming languages have not been tested so far.
This project has started as a Rust port of the JavaScript tool regexgen written by Devon Govett. Although a lot of further useful features could be added to it, its development was apparently ceased several years ago. The plan is now to add these new features to grex as Rust really shines when it comes to command-line tools. grex offers all features that regexgen provides, and more.
The philosophy of this project is to generate the most specific regular expression possible by default which exactly matches the given input only and nothing else. With the use of command-line flags (in the CLI tool) or preprocessing methods (in the library), more generalized expressions can be created.
{min,max} quantifier notation| operator? quantifierPre-compiled 64-Bit binaries are available within the package managers Scoop (for Windows) and Homebrew (for macOS).
scoop install grex
brew tap pemistahl/formulas
brew install grex
Alternatively, you can download the self-contained executable for your platform above and put it in a place of your choice. grex is also hosted on crates.io, the official Rust package registry. If you are a Rust developer and already have the Rust toolchain installed, you can install by compiling from source using cargo, the Rust package manager:
cargo install grex
In order to use grex as a library, simply add it as a dependency to your Cargo.toml file:
toml
[dependencies]
grex = "1.0.0"
``` $ grex -h
grex 1.0.0 © 2019-2020 Peter M. Stahl pemistahl@gmail.com Licensed under the Apache License, Version 2.0 Downloadable from https://crates.io/crates/grex Source code at https://github.com/pemistahl/grex
grex generates regular expressions from user-provided test cases.
USAGE:
grex [FLAGS] ... --file
FLAGS: -d, --digits Converts any Unicode decimal digit to \d -D, --non-digits Converts any character which is not a Unicode decimal digit to \D -s, --spaces Converts any Unicode whitespace character to \s -S, --non-spaces Converts any character which is not a Unicode whitespace character to \S -w, --words Converts any Unicode word character to \w -W, --non-words Converts any character which is not a Unicode word character to \W -r, --repetitions Detects repeated non-overlapping substrings and converts them to {min,max} quantifier notation -e, --escape Replaces all non-ASCII characters with unicode escape sequences --with-surrogates Converts astral code points to surrogate pairs if --escape is set -c, --colorize Provides syntax highlighting for the resulting regular expression -h, --help Prints help information -v, --version Prints version information
OPTIONS:
-f, --file
ARGS: ... One or more test cases separated by blank space
```
```rust use grex::RegExpBuilder;
let regexp = RegExpBuilder::from(&["a", "aa", "aaa"]).build(); assert_eq!(regexp, "^a(aa?)?$"); ```
```rust use grex::{Feature, RegExpBuilder};
let regexp = RegExpBuilder::from(&["a", "aa", "aaa"]) .withconversionof(&[Feature::Repetition]) .build(); assert_eq!(regexp, "^a{1,3}$"); ```
```rust use grex::{Feature, RegExpBuilder};
let regexp = RegExpBuilder::from(&["a", "aa", "123"]) .withconversionof(&[Feature::Digit, Feature::Word]) .build(); assert_eq!(regexp, "^(\d\d\d|\w\w|\w)$"); ```
```rust use grex::RegExpBuilder;
let regexp = RegExpBuilder::from(&["You smell like 💩."]) .withescapingofnonasciichars(false) .build(); asserteq!(regexp, "^You smell like \u{1f4a9}\.$"); ```
Old versions of JavaScript do not support unicode escape sequences for the astral code planes
(range U+010000 to U+10FFFF). In order to support these symbols in JavaScript regular
expressions, the conversion to surrogate pairs is necessary. More information on that matter
can be found here.
```rust use grex::RegExpBuilder;
let regexp = RegExpBuilder::from(&["You smell like 💩."]) .withescapednonasciichars(true) .build(); assert_eq!(regexp, "^You smell like \u{d83d}\u{dca9}\.$"); ```
```rust use grex::{Feature, RegExpBuilder};
let regexp = RegExpBuilder::from(&["You smell like 💩💩💩."]) .withconversionof(&[Feature::Repetition]) .withescapednonasciichars(false) .build(); assert_eq!(regexp, "^You smel{2} like \u{1f4a9}{3}\.$"); ```
```rust use grex::{Feature, RegExpBuilder};
let regexp = RegExpBuilder::from(&["a", "aa", "123"]) .withconversionof(&[Feature::Repetition, Feature::Digit, Feature::Word]) .build(); assert_eq!(regexp, "^(\w{1,2}|\d{3})$"); ```
⚠ The method with_syntax_highlighting() may only be used if the resulting regular expression is meant to
be printed to the console. The regex string representation returned from enabling
this setting cannot be fed into the regex crate.
```rust use grex::RegExpBuilder;
let regexp = RegExpBuilder::from(&["a", "aa", "123"]) .withsyntaxhighlighting() .build(); ```
The following examples show the various supported regex syntax features:
``` $ grex a b c ^[a-c]$
$ grex a c d e f ^[ac-f]$
$ grex a b x de ^(de|[abx])$
$ grex a b bc ^(bc?|a)$
$ grex [a-z] ^[a-z]$
$ grex -r b ba baa baaa ^b(a{1,3})?$
$ grex -r b ba baa baaaa ^b(a{1,2}|a{4})?$
$ grex y̆ a z ^(y̆|[az])$ Note: Grapheme y̆ consists of two Unicode symbols: U+0079 (Latin Small Letter Y) U+0306 (Combining Breve).
$ grex "I ♥ cake" "I ♥ cookies" ^I ♥ c(ookies|ake)$ Note: Input containing blank space must be surrounded by quotation marks. ```
The string "I ♥♥♥ 36 and ٣ and 💩💩." serves as input for the following examples using the command-line notation:
``` $ grex ^I ♥♥♥ 36 and ٣ and 💩💩.$
$ grex -e ^I \u{2665}\u{2665}\u{2665} 36 and \u{663} and \u{1f4a9}\u{1f4a9}.$
$ grex -e --with-surrogates ^I \u{2665}\u{2665}\u{2665} 36 and \u{663} and \u{d83d}\u{dca9}\u{d83d}\u{dca9}.$
$ grex -d ^I ♥♥♥ \d\d and \d and 💩💩.$
$ grex -s ^I\s♥♥♥\s36\sand\s٣\sand\s💩💩.$
$ grex -w ^\w ♥♥♥ \w\w \w\w\w \w \w\w\w 💩💩.$
$ grex -D ^\D\D\D\D\D\D36\D\D\D\D\D٣\D\D\D\D\D\D\D\D$
$ grex -S ^\S \S\S\S \S\S \S\S\S \S \S\S\S \S\S\S$
$ grex -dsw ^\w\s♥♥♥\s\d\d\s\w\w\w\s\d\s\w\w\w\s💩💩.$
$ grex -dswW ^\w\s\W\W\W\s\d\d\s\w\w\w\s\d\s\w\w\w\s\W\W\W$
$ grex -r ^I ♥{3} 36 and ٣ and 💩{2}.$
$ grex -er ^I \u{2665}{3} 36 and \u{663} and \u{1f4a9}{2}.$
$ grex -er --with-surrogates ^I \u{2665}{3} 36 and \u{663} and (\u{d83d}\u{dca9}){2}.$
$ grex -dr ^I ♥{3} \d(\d and ){2}💩{2}.$
$ grex -rs ^I\s♥{3}\s36\sand\s٣\sand\s💩{2}.$
$ grex -rw ^\w ♥{3} \w(\w \w{3} ){2}💩{2}.$
$ grex -Dr ^\D{6}36\D{5}٣\D{8}$
$ grex -rS ^\S \S(\S{2} ){2}\S{3} \S \S{3} \S{3}$
$ grex -rW ^I\W{5}36\Wand\W٣\Wand\W{4}$
$ grex -drsw ^\w\s♥{3}\s\d(\d\s\w{3}\s){2}💩{2}.$
$ grex -drswW ^\w\s\W{3}\s\d(\d\s\w{3}\s){2}\W{3}$ ```
In order to build the source code yourself, you need the stable Rust toolchain installed on your machine so that cargo, the Rust package manager is available.
git clone https://github.com/pemistahl/grex.git
cd grex
cargo build
The source code is accompanied by an extensive test suite consisting of unit tests, integration tests and property tests. For running the unit and integration tests, simply say:
cargo test
Property tests are disabled by default with the #[ignore] annotation because they are
very long-running. They are used for automatically generating test cases for regular
expression conversion. If a test case is found that produces a wrong conversion, it is
shrinked to the shortest test case possible that still produces a wrong result.
This is a very useful tool for finding bugs. If you want to run these tests, say:
cargo test -- --ignored
A deterministic finite automaton (DFA) is created from the input strings.
The number of states and transitions between states in the DFA is reduced by applying Hopcroft's DFA minimization algorithm.
The minimized DFA is expressed as a system of linear equations which are solved with Brzozowski's algebraic method, resulting in the final regular expression.
In case you want to contribute something to grex even though it's in a very early stage of development, then I encourage you to do so nevertheless. Do you have ideas for cool features? Or have you found any bugs so far? Feel free to open an issue or send a pull request. It's very much appreciated. :-)