DataFusion Proto

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.

This crate is a submodule of DataFusion that provides a protocol buffer format for representing query plans and expressions.

Serializing Expressions

Based on examples/expr_serde.rs

```rust use datafusioncommon::Result; use datafusionexpr::{col, lit, Expr}; use datafusion_proto::bytes::Serializeable;

fn main() -> Result<()> { // Create a new Expr a < 32 let expr = col("a").lt(lit(5i32));

// Convert it to an opaque form
let bytes = expr.to_bytes()?;

// Decode bytes from somewhere (over network, etc.)
let decoded_expr = Expr::from_bytes(&bytes)?;
assert_eq!(expr, decoded_expr);
Ok(())

} ```

Serializing Logical Plans

Based on examples/logicalplanserde.rs

```rust use datafusion::prelude::*; use datafusioncommon::Result; use datafusionproto::bytes::{logicalplanfrombytes, logicalplantobytes};

[tokio::main]

async fn main() -> Result<()> { let ctx = SessionContext::new(); ctx.registercsv("t1", "testdata/test.csv", CsvReadOptions::default()) .await ?; let plan = ctx.table("t1")?.tologicalplan()?; let bytes = logicalplantobytes(&plan)?; let logicalroundtrip = logicalplanfrombytes(&bytes, &ctx)?; asserteq!(format!("{:?}", plan), format!("{:?}", logicalroundtrip)); Ok(()) } ```

Serializing Physical Plans

Based on examples/physicalplanserde.rs

```rust use datafusion::prelude::*; use datafusioncommon::Result; use datafusionproto::bytes::{physicalplanfrombytes,physicalplantobytes};

[tokio::main]

async fn main() -> Result<()> { let ctx = SessionContext::new(); ctx.registercsv("t1", "testdata/test.csv", CsvReadOptions::default()) .await ?; let logicalplan = ctx.table("t1")?.tologicalplan()?; let physicalplan = ctx.createphysicalplan(&logicalplan).await?; let bytes = physicalplantobytes(physicalplan.clone())?; let physicalroundtrip = physicalplanfrombytes(&bytes, &ctx)?; asserteq!(format!("{:?}", physicalplan), format!("{:?}", physicalround_trip)); Ok(()) }

```