Fluvio Jolt JSON library

Overview

JSON to JSON transformation where the "specification" for the transform is itself a JSON document.

Port of Java Jolt library written in Rust.

Usage Example

Add fluvio-jolt crate to your Cargo.toml file: toml [dependencies] fluvio-jolt = { version = "0.1"}

Then, for example, if you want to repack your JSON record, you can do the following: ```rust use serdejson::{json, Value}; use fluviojolt::{transform, TransformSpec};

let input: Value = serdejson::fromstr(r#" { "id": 1, "name": "John Smith", "account": { "id": 1000, "type": "Checking" } } "#).unwrap();

let spec: TransformSpec = serdejson::fromstr(r#"[ { "operation": "shift", "spec": { "name": "data.name", "account": "data.account" } } ]"#).unwrap();

let output = transform(input, &spec);

assert_eq!(output, json!({ "data" : { "name": "John Smith", "account": { "id": 1000, "type": "Checking" } } })); ```

Supported Operations

1. shift: copy data from the input tree and put it the output tree
2. default: apply default values to the tree
3. remove: remove data from the tree

Specification

Composes a list of operation specifications. Each operation has its own DSL (Domain Specific Language) in order to facilitate its narrow job.

``` use fluvio_jolt::TransformSpec;

let spec: TransformSpec = serdejson::fromstr(r#"[ { "operation": "shift", "spec": { "name": "data.name", "account": "data.account" } } ]"#).unwrap(); ```

Shift operation

Specifies where the data from the input JSON should be placed in the output JSON, or in other words, how the input JSON/data should be shifted around to make the output JSON/data.

At a base level, a single shift operation is a mapping from an input path to an output path, similar to the mv command in Unix, mv /var/data /var/backup/data.

The input path is a JSON tree structure, and the output path is flattened "dot notation" path notation.

For example, given this simple input JSON:

{
    "id": 1,
    "name": "John Smith",
    "account": {
        "id": 1000,
        "type": "Checking"
    }
}

{
    "id": "data.id",
    "name": "data.name",
    "account": "data.account"
}

would produce the following output JSON:

{
    "data" : {
        "id": 1,
        "name": "John Smith",
        "account": {
            "id": 1000,
            "type": "Checking"
        }
    }
}

Wildcards

The shift specification on the keys level supports wildcards and conditions:
1. * - match everything
2. name1|name2|nameN - match any of the specified names

& lookup allows referencing the values captured by the * or |.
It allows for specs to be more compact. For example, for this input:

{
    "id": 1,
    "name": "John Smith",
    "account": {
        "id": 1000,
        "type": "Checking"
    }
}

{
    "data" : {
        "id": 1,
        "name": "John Smith",
        "account": {
            "id": 1000,
            "type": "Checking"
        }
    }
}

the spec with wildcards would be:

{
    "*": "data.&0"
}

If you want only id and name in the output, the spec is:

{
    "id|name": "data.&0"
}

& wildcard also allows to dereference any level of the path of given node:

{
    "foo": {
        "bar" : {
            "baz": "new_location.&0.&1.&2" // &0 = baz, &1 = bar, &2 = foo
            }
        }
    }
}

for the input:

{
    "foo": {
      "bar": {
        "baz": "value"
      }
    }
  }

will produce:

{
    "new_location": {
      "baz": {
        "bar": {
          "foo": "value"
        }
      }
    }
}

Default operation

Applies default values if the value is not present in the input JSON.

For example, given this simple input JSON:

{
    "phones": {
        "mobile": 01234567,
        "country": "US"
    }
}

{
    "phones": {
        "mobile": 0000000,
        "code": "+1"
    }
}

the output JSON will be:

{
    "phones": {
        "mobile": 01234567,
        "country": "US",
        "code": "+1"
    }
}

As you can see, the field mobile remains not affected while the code has a default '+1' value.

Remove operation

Removes content from the input JSON. The spec structure matches the input JSON structure. The value of fields is ignored.

For example, given this simple input JSON:

{
    "phones": {
        "mobile": 01234567,
        "country": "US"
    }
}

{
    "phones": {
        "country": ""
    }
}

the output JSON will be:

{
    "phones": {
        "mobile": 01234567
    }
}

Contributing

If you'd like to contribute to the project, please read our Contributing guide.

License

This project is licensed under the Apache license.