Parquet Derive

A crate for deriving RecordWriter for arbitrary, simple structs. This does not generate writers for arbitrarily nested structures. It only works for primitives and a few generic structures and various levels of reference. Please see features checklist for what is currently supported.

Derive also has some support for the chrono time library. You must must enable the chrono feature to get this support.

Usage

Add this to your Cargo.toml: toml [dependencies] parquet = "4.0.0-SNAPSHOT" parquet_derive = "4.0.0-SNAPSHOT"

and this to your crate root: ```rust extern crate parquet;

[macrouse] extern crate parquetderive;

```

Example usage of deriving a RecordWriter for your struct:

```rust use parquet; use parquet::record::RecordWriter;

[derive(ParquetRecordWriter)]

struct ACompleteRecord<'a> { pub abool: bool, pub astr: &'a str, pub astring: String, pub aborrowedstring: &'a String, pub maybeastr: Option<&'a str>, pub magicnumber: i32, pub lowqualitypi: f32, pub highqualitypi: f64, pub maybepi: Option, pub maybebest_pi: Option, }

// Initialize your parquet file let mut writer = SerializedFileWriter::new(file, schema, props).unwrap(); let mut rowgroup = writer.nextrow_group().unwrap();

// Build up your records let chunks = vec![ACompleteRecord{...}];

// The derived RecordWriter takes over here (&chunks[..]).writetorowgroup(&mut rowgroup);

writer.closerowgroup(row_group).unwrap(); writer.close().unwrap(); ```

Features

• [X] Support writing String, &str, bool, i32, f32, f64, Vec<u8>
• [ ] Support writing dictionaries
• [X] Support writing logical types like timestamp
• [X] Derive definition_levels for Option
• [ ] Derive definition levels for nested structures
• [ ] Derive writing tuple struct
• [ ] Derive writing tuple container types

Requirements

• Same as parquet-rs

Test

Testing a *_derive crate requires an intermediate crate. Go to parquet_derive_test and run cargo test for unit tests.

Docs

To build documentation, run cargo doc --no-deps. To compile and view in the browser, run cargo doc --no-deps --open.

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0.