⛓️gzp

Build Status license Version info

Multithreaded gzip encoding.

Why?

This crate provides a near drop in replacement for Write that has will compress chunks of data in parallel and write to an underlying writer in the same order that the bytes were handed to the writer. This allows for much faster compression of Gzip data.

Usage

toml [dependencies] gzp = { version = "*", features = ["zlib-ng-compat"] }

Examples

Simple example

```rust use std::{env, fs::File, io::Write};

use gzp::ParGz;

fn main() { let file = env::args().skip(1).next().unwrap(); let writer = File::create(file).unwrap(); let mut pargz = ParGz::builder(writer).build(); pargz.writeall(b"This is a first test line\n").unwrap(); pargz.writeall(b"This is a second test line\n").unwrap(); pargz.finish().unwrap(); } ```

An updated version of pgz.

```rust use gzp::ParGz; use std::io::{Read, Write};

fn main() { let chunksize = 64 * (1 << 10) * 2;

let stdout = std::io::stdout();
let mut writer = ParGz::builder(stdout).build();

let stdin = std::io::stdin();
let mut stdin = stdin.lock();

let mut buffer = Vec::with_capacity(chunksize);
loop {
    let mut limit = (&mut stdin).take(chunksize as u64);
    limit.read_to_end(&mut buffer).unwrap();
    if buffer.is_empty() {
        break;
    }
    writer.write_all(&buffer).unwrap();
    buffer.clear();
}
writer.finish().unwrap();

} ```

Notes

Future todos