A Relative-Lempel Ziv (RLZ) based LZ compressor that compresses against a large static dictionary.
This code implements the RLZ compressor, as described in:
bibtex
@article{DBLP:journals/pvldb/HoobinPZ11,
author = {Christopher Hoobin and
Simon J. Puglisi and
Justin Zobel},
title = {Relative Lempel-Ziv Factorization for Efficient Storage
and Retrieval of Web Collections},
journal = {Proc. {VLDB} Endow.},
volume = {5},
number = {3},
pages = {265--273},
year = {2011},
}
bibtex
@article{DBLP:journals/corr/PetriMNW16,
author = {Matthias Petri and
Alistair Moffat and
P. C. Nagesh and
Anthony Wirth},
title = {Access Time Tradeoffs in Archive Compression},
journal = {CoRR},
volume = {abs/1602.08829},
year = {2016},
url = {http://arxiv.org/abs/1602.08829},
eprinttype = {arXiv},
}
bibtex
@inproceedings{DBLP:conf/www/LiaoPMW16,
author = {Kewen Liao and
Matthias Petri and
Alistair Moffat and
Anthony Wirth},
title = {Effective Construction of Relative Lempel-Ziv
Dictionaries},
booktitle = {Proceedings of {WWW}},
pages = {807--816},
publisher = {{ACM}},
year = {2016},
}
The Relative Lempel-Ziv (RLZ) scheme is a hybrid of several phrase-based compression mechanisms. Encoding is based on a fixed-text dictionary, with all substrings within the dictionary available for use as factors, in the style of LZ77. But the dictionary is constructed in a semi-static manner, and hence needs to be representative of the entire text being coded if compression effectiveness is not to be compromised. Furthermore, because RLZ is intended for large web-based archives when constructing the dictionary, it is infeasible to have the whole input text in memory.
```rust use rlz::RlzCompressor; use rlz::Dictionary;
let dict = Dictionary::from(&b"banana"[..]); let rlzcompressor = RlzCompressor::builder().buildfrom_dict(dict);
let mut output = Vec::new();
let text = b"banana$aba"; let encodedlen = rlzcompressor.encode(&text[..],&mut output).unwrap(); asserteq!(encodedlen,output.len());
let mut storeddecoder = Vec::new(); rlzcompressor.store(&mut storeddecoder).unwrap(); let loadeddecoder = RlzCompressor::load(&stored_decoder[..]).unwrap();
let mut recovered = Vec::new(); loadeddecoder.decode(&output[..],&mut recovered).unwrap(); asserteq!(recovered,text); ```
MIT