bigml-parallel
: A CLI tool for running WhizzML scripts in parallel
This is a tool for parallel processing of BigML resources (typically sources or datasets) using WhizzML scripts. It's still somewhat experimental. To install, download binaries from the releases page.
This tool will output the resulting BigML execution objects as JSON structs, one per line, in no particular order. It runs up to --max-tasks
BigML executions at a time.
```txt
Execute WhizzML script in parallel over one or more BigML resources
USAGE:
bigml-parallel [OPTIONS] --script
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-i, --input ...
Extra inputs to our WhizzML script, specified as
"name=value". These will be parsed as JSON if possible,
or treated as strings otherwise
-J, --max-tasks
How many BigML tasks should we use at a time? [default:
2]
-n, --name
The name to use for our execution objects
-o, --output ...
Expected outputs to our WhizzML script, specified as
"name"
-R, --resource-input-name
The input name used to pass the dataset [default:
resource]
-r, --resource ...
The resource IDs to process. (Alternatively, pipe
resource IDs on standard input, one per line.)
--retry-count
How many times should we retry a failed execution
matching --retry-on? [default: 0]
--retry-on
A regular expression specifying which WhizzML script
errors should be retried
-s, --script