This crate aims to implement the LZ4 compression and decompression algorithm, as well as the framing format used for LZ4 files, in pure Rust with no unsafe code. The output perfectly matches the C reference implementation byte for byte. At the time of writing, this is also the fastest no-unsafe implementation that I'm aware of.
The lz4 crate calls into the C library.
The compress crate has an implementation that relies on unsafe
.
And the redox implementation (more on that below) is slower, in some cases substantially.
Decompressor status: Beta. Works well, and is blazingly fast (at least as fast as the official C implementation in my tests).
Compressor status: Alpha. You can expect it to produce perfect (i.e. identical to what the C library produces) output for all configurations. The API may still change a little. The example named "dolz4" is a compressor but is currently lacking a CLI. You have to configure it by changing the code. Performance is good, but takes ~2-3x as long as the C implementation. The current bottleneck appears to be an abundance of range checks when writing output (~25% of cycles spent in there) which also cause the compiler to completely trip over itself and sometimes emit a sequence of copyfromslice calls for 1-byte and 4-byte writes to the output array. Help wanted.
If you take a look at the git history, this is strictly speaking a fork from @johannesvollmer. He took redox-os' lz4 compression, and reworked it to be usable as a library crate.
However after noticing performance issues, I have gradually rewritten both the compressor and the decompressor from scratch to closely resemble the 2400-line mess that is the official C implementation (and that's just the raw format without framing!). Admittedly they pack plenty of optimizations in there (lots of intentionally reading beyond buffer boundaries for the sake of performance), but I'm proud to say that I achieved similar performance in just 400 lines of competely safe Rust.