dyer

Overview

[dyer] is designed for reliable, flexible and fast Request-Response based service, including data processing, web-crawling and so on, providing some friendly, interoperable, comprehensive features without compromising speed.

Features

• asynchronous, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
• Event-driven, once you set the initials and recursive generator, dyer will handle the rest of it interoperably.
• User-friendly and flexible, dyer offers high-level, flexible, easy to use wrappers and APIs what does a lot for you.

Feature Flag

To reduce code redundancy and speed up compilation, dyer use feature flag to mark the necessary modules/functions, Currently here are some supported Features:

• xpath: Enable parse the html response with xpath
• compression: Enable HTTP Compression: br, deflate, gzip
• proxy: Enable use proxies
• full: Enable all features

Guide

Get started by installing [dyer-cli] and looking over the [examples].

The Cookbook gives a detailed view of dyer.