tfrecord-rust

The crate provides the functionality to serialize and deserialize TFRecord data format from TensorFlow.

Features

• Provide both high level EasyExample type as well as low level Vec<u8> bytes {,de}serialization.
• Support async/await syntax. It's easy to work with futures-rs.
• Interoperability with serde.

Usage

Append this line to your Cargo.toml.

tfrecord = "0.1.0"

The crate provides several cargo features that you can conditionally compile modules.

• serde: Enable interoperability with serde to serialize and deserialize example types.
• async_: Enable async/awat API.
• full: Enable all features above.

Documentation

See docs.rs for the API.

Example

File reading example

This is a snipplet copied from examples/tfrecord_info.rs.

```rust use tfrecord::{EasyExampleReader, EasyFeature, Error, RecordReaderInit};

fn main() -> Result<(), Error> { // use init pattern to construct the tfrecord reader let reader: EasyExampleReader<_> = RecordReaderInit { checkintegrity: true, } .open(&*INPUTTFRECORD_PATH)?;

// print header
println!("example_no\tfeature_no\tname\ttype\tsize");

// enumerate examples
for (example_index, result) in reader.enumerate() {
    let example = result?;

    // enumerate features in an example
    for (feature_index, (name, feature)) in example.into_iter().enumerate() {
        print!("{}\t{}\t{}\t", example_index, feature_index, name);

        match feature {
            EasyFeature::BytesList(list) => {
                println!("bytes\t{}", list.len());
            }
            EasyFeature::FloatList(list) => {
                println!("float\t{}", list.len());
            }
            EasyFeature::Int64List(list) => {
                println!("int64\t{}", list.len());
            }
            EasyFeature::None => {
                println!("none");
            }
        }
    }
}

Ok(())

} ```

Work with async/await syntax

The snipplet from examples/tfrecord_info_async.rs demonstrates the integration with async-std.

```rust use futures::stream::TryStreamExt; use std::{fs::File, io::BufWriter, path::PathBuf}; use tfrecord::{EasyFeature, Error, RecordStreamInit};

[async_std::main]

async fn main() -> Result<(), Error> { // use init pattern to construct the tfrecord stream let stream = RecordStreamInit { checkintegrity: true, } .easyexamplesopen(&*INPUTTFRECORD_PATH) .await?;

// print header
println!("example_no\tfeature_no\tname\ttype\tsize");

// enumerate examples
stream
    .try_fold(0, |example_index, example| {
        async move {
            // enumerate features in an example
            for (feature_index, (name, feature)) in example.into_iter().enumerate() {
                print!("{}\t{}\t{}\t", example_index, feature_index, name);

                match feature {
                    EasyFeature::BytesList(list) => {
                        println!("bytes\t{}", list.len());
                    }
                    EasyFeature::FloatList(list) => {
                        println!("float\t{}", list.len());
                    }
                    EasyFeature::Int64List(list) => {
                        println!("int64\t{}", list.len());
                    }
                    EasyFeature::None => {
                        println!("none");
                    }
                }
            }

            Ok(example_index + 1)
        }
    })
    .await?;

Ok(())

} ```

More examples

Also, we suggest visiting the test code for more detailed usage.

License

MIT license. See LICENSE file for full license.