This Rust program is used to perform simple integration tests on programs through a JSON configuration file.
Usage: tiny-integration-tester [options]
| Arg | Long name | Description | Platforms | |-----|-------------|---------------------------------------|-----------| | -f | --file | set json tests file name | ALL | | -t | --timeout | set default timeout(ms) for all tests | *NIX ONLY | | -o, | --output | display output of commands by default | *NIX ONLY | | -h | --help | print this help menu | ALL | | -v | --version | print the program version | ALL |
This program has been tested on Linux. Although it compiles for Windows, features such as timeout and the ability to display outputs are not supported
After creating a new project with the command cargo new hello
, create a file named test.json like this :
[
{
"command" : "./target/debug/hello",
"stdout" : "Hello, world!\n"
},
{
"command" : "./target/debug/hello",
"stdout" : "Hello, dummy!\n"
}
]
test.json is the default json filename.
Then simply run cargo build then tiny-integration-tester
:
Executing command: "./target/debug/hello" with args: []
OK!
Executing command: "./target/debug/hello" with args: []
FAIL!
YOU GOT: Output {
status: ExitStatus(
unix_wait_status(
0,
),
),
stdout: "Hello, world!\n",
stderr: "",
}
BUT DESIRED OUTPUT IS: Output {
status: ExitStatus(
unix_wait_status(
0,
),
),
stdout: "Hello, dummy!\n",
stderr: "",
}
result: 1 / 2
You can set these values into the json test file :
| Field | Definition | Type | Comment | |-------------|--------------------------|---------------------------|-------------------------| | command | command to run | String | Mandatory | | stdout | FD stdout | String | | | stderr | FD stderr | String | | | stdin | FD stdin | String | | | args | command arguments | Array of String | | | envs | command envs args | Array of (String, String) | json tuple is [1, 2] | | envclear | clear environment | Boolean | clear all env if true | | status | process exit status | Hexadecimal String | 0xHHLL depends of OS | | test | false if it is no a test | Boolean | no timeout effect | | terminate | true if exit in failure | Boolean | exit tester on failure | | timeout | kill process after X ms | Unsigned Integer | UNIX ONLY. 0 = infinity | | showoutput | true for showing output | Boolean | UNIX ONLY. |
Here, for instance, we run the command cargo build --release
, setting arguments to - "args" : ["build", "--release"]
,waiting for the command to succeed - "status" : "0x0". There is no time limit - "timeout" : 0, and the output will be
displayed on the screen - "showoutput" : true. Finally, if an error occurs, we stop
the tests - "terminate" : true :
text
[
{
"command" : "cargo",
"args" : ["build", "--release"],
"status" : "0x0",
"timeout" : 0,
"terminate" : true,
"show_output" : true
}
]
Same example as below :
text
[
{
"command" : "echo",
"args" : ["hello", "world"],
"status" : "0x0",
"show_output" : true
},
{
"command" : "cargo",
"args" : ["test", "--release"],
"status" : "0x0",
"timeout" : 0,
"terminate" : true,
"show_output" : true
}
]
Also, we check if stdout or stderr match a given pattern :
text
[
{
"command" : "./target/release/fibonacci",
"args" : ["3"],
"stdout" : "fibo(3) = 2\n",
"timeout" : 1000
},
{
"command" : "./target/release/fibonacci",
"args" : ["1", ""],
"status" : "0xff00",
"stderr" : "Usage: target/debug/fibonacci POSITIF_NUMBER\n"
},
{
"command" : "./target/release/fibonacci",
"args" : ["7"],
"stdout" : "fibo(7) = 13\n"
},
{
"timeout" : 500,
"command" : "./target/release/fibonacci",
"args" : ["15"],
"status" : "0x0",
"stdout" : "fibo(15) = 610\n"
},
{
"command" : "./target/release/fibonacci",
"args" : ["164neuf"],
"status" : "0xff00",
"stderr" : "Bad number format\n"
},
{
"command" : "./target/release/fibonacci",
"status" : "0xff00",
"stderr" : "Bad number format\n"
}
]
We can set test to false so that it does not count as a test :
text
[
{
"command" : "rustc",
"args" : ["readline-tester.rs"],
"test" : false
},
{
"command" : "rustc",
"args" : ["env-tester.rs"],
"test" : false
}
]
We can also record data for stdin or manage environment variables with envclear and envs :
text
[
{
"command" : "./readline-tester",
"status" : "0x0",
"stdin" : "carrots are cooked\nbananas\n",
"stdout" : "1 carrots are cooked\n2 bananas\n"
},
{
"command" : "./env-tester",
"stdout" : "1 = foo\n2 = bar\n",
"env_clear" : true,
"envs" : [["1", "foo"], ["2", "bar"], ["CLUTTER_IM_MODULE", "dummy"]],
"show_output" : true
}
]
Be careful with trailing commas at the end in a JSON file.