Wrapper that uses the Range HTTP header to resume get requests.
It's a thin wrapper around reqwest. It's a work in progress – wrapping functionality is copied across on an as-needed basis. Feel free to open a PR/issue if you need something.
```rust use async_compression::futures::bufread::GzipDecoder; use futures::{io::BufReader, AsyncBufReadExt, StreamExt, TryStreamExt}; use std::io;
async fn main() { let url = "http://commoncrawl.s3.amazonaws.com/crawl-data/CC-MAIN-2018-30/warc.paths.gz"; let body = reqwestresume::get(url.parse().unwrap()).await.unwrap(); // Content-Encoding isn't set, so decode manually let body = body .bytesstream() .maperr(|e| io::Error::new(io::ErrorKind::Other, e)); let body = BufReader::new(body.intoasyncread()); let mut body = GzipDecoder::new(body); // Content-Encoding isn't set, so decode manually body.multiplemembers(true);
let mut lines = BufReader::new(body).lines();
while let Some(line) = lines.next().await {
println!("{}", line.unwrap());
}
} ```
Licensed under either of
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.