rust-brotli

Project Requirements

Direct no-stdlib port of the C brotli decompressor to Rust

no dependency on the Rust stdlib: this library would be ideal for decompressing within a rust kernel among other things.

This is useful to see how C and Rust compare in an apples-to-apples comparison where the same algorithms and data structures and optimizations are employed.

Compression Usage

Rust brotli currently supports compression levels 0 - 11 They should be bitwise identical to the brotli C compression engine at compression levels 0-9 Recommended lgwindowsize is between 20 and 22

With the io::Read abstraction

rust let mut input = brotli::CompressorReader::new(&mut io::stdin(), 4096 /* buffer size */, quality as u32, lg_window_size as u32); then you can simply read input as you would any other io::Read class

With the io::Write abstraction

rust let mut writer = brotli::Compressor::new(&mut io::stdout(), 4096 /* buffer size */, quality as u32, lg_window_size as u32);

With the Stream Copy abstraction

rust match brotli::BrotliCompress(&mut io::stdin(), &mut io::stdout(), quality as u32, lg_window_size as u32) { Ok(_) => {}, Err(e) => panic!("Error {:?}", e), }

Decompression Usage

With the io::Read abstraction

rust let mut input = brotli::Decompressor::new(&mut io::stdin(), 4096 /* buffer size */); then you can simply read input as you would any other io::Read class

With the io::Write abstraction

rust let mut writer = brotli::DecompressorWriter::new(&mut io::stdout(), 4096 /* buffer size */);

With the Stream Copy abstraction

rust match brotli::BrotliDecompress(&mut io::stdin(), &mut io::stdout()) { Ok(_) => {}, Err(e) => panic!("Error {:?}", e), }

With manual memory management

There are 3 steps to using brotli without stdlib

1. setup the memory manager
2. setup the BrotliState
3. in a loop, call BrotliDecompressStream

in Detail

```rust // at global scope declare a MemPool type -- in this case we'll choose the heap to // avoid unsafe code, and avoid restrictions of the stack size

declarestackallocator_struct!(MemPool, heap);

// at local scope, make a heap allocated buffers to hold uint8's uint32's and huffman codes let mut u8buffer = defineallocatormemorypool!(4096, u8, [0; 32 * 1024 * 1024], heap); let mut u32buffer = defineallocatormemorypool!(4096, u32, [0; 1024 * 1024], heap); let mut hcbuffer = defineallocatormemorypool!(4096, HuffmanCode, [0; 4 * 1024 * 1024], heap); let heapu8allocator = HeapPrealloc::::newallocator(4096, &mut u8buffer, bzero); let heapu32allocator = HeapPrealloc::::newallocator(4096, &mut u32buffer, bzero); let heaphcallocator = HeapPrealloc::::newallocator(4096, &mut hcbuffer, bzero);

// At this point no more syscalls are going to be needed since everything can come from the allocators.

// Feel free to activate SECCOMP jailing or other mechanisms to secure your application if you wish.

// Now it's possible to setup the decompressor state let mut brotlistate = BrotliState::new(heapu8allocator, heapu32allocator, heaphc_allocator);

// at this point the decompressor simply needs an input and output buffer and the ability to track // the available data left in each buffer loop { result = BrotliDecompressStream(&mut availablein, &mut inputoffset, &input.slice(), &mut availableout, &mut outputoffset, &mut output.slicemut(), &mut written, &mut brotlistate);

// just end the decompression if result is BrotliResult::ResultSuccess or BrotliResult::ResultFailure

} ```

This interface is the same interface that the C brotli decompressor uses

Also feel free to use custom allocators that invoke Box directly. This example illustrates a mechanism to avoid subsequent syscalls after the initial allocation