OneIO

OneIO is a Rust library that provides unified simple IO interface for reading and writing to and from data files from different sources and compressions.

Usage and Feature Flags

Enable all compression algorithms, and handle remote files (default) toml oneio = "0.4"

Select from supported feature flags toml oneio = {version = "0.4", features = ["remote", "gz"]}

Supported feature flags: - all (default): all flags (["gz", "bz", "lz", "remote"]) - remote: allow reading from remote files - gz: support gzip files - bz: support bzip2 files - lz: support lz4 files

OneIO Reader

The returned reader implements BufRead, and handles decompression from the following types: - gzip: files ending with gz or gzip - bzip2: files ending with bz or bz2 - lz4: files ending with lz4 or lz

It also handles reading from remote or local files transparently.

Examples

Read all into string: ```rust const TEST_TEXT: &str = "OneIO test file. This is a test.";

let mut reader = oneio::getreader("https://spaces.bgpkit.org/oneio/testdata.txt.gz").unwrap();

let mut text = "".tostring(); reader.readtostring(&mut text).unwrap(); asserteq!(text.asstr(), TESTTEXT); ```

Read into lines: ```rust use std::io::BufRead; const TEST_TEXT: &str = "OneIO test file. This is a test.";

let reader = oneio::getreader("https://spaces.bgpkit.org/oneio/testdata.txt.gz").unwrap(); let lines = reader.lines().into_iter().map(|line| line.unwrap()).collect::>();

asserteq!(lines.len(), 2); asserteq!(lines[0].asstr(), "OneIO test file."); asserteq!(lines[1].as_str(), "This is a test."); ```

OneIO Writer

[get_writer] returns a generic writer that implements [Write], and handles decompression from the following types: - gzip: files ending with gz or gzip - bzip2: files ending with bz or bz2

Note: lz4 writer is not currently supported.

Example

```rust let toreadfile = "https://spaces.bgpkit.org/oneio/testdata.txt.gz"; let towritefile = "/tmp/testwrite.txt.bz2";

// read text from remote gzip file let mut text = "".tostring(); oneio::getreader(toreadfile).unwrap().readtostring(&mut text).unwrap();

// write the same text to a local bz2 file let mut writer = oneio::getwriter(towritefile).unwrap(); writer.writeall(text.as_ref()).unwrap(); drop(writer);

// read from the newly-generated bz2 file let mut newtext = "".tostring(); oneio::getreader(towritefile).unwrap().readtostring(&mut newtext).unwrap();

// compare the decompressed content of the remote and local files asserteq!(text.asstr(), newtext.asstr()); std::fs::removefile(towrite_file).unwrap(); ```