Prometheus Web Exporter

Prometheus Web Exporter collects information about status of web pages from internet.

It send a request to a webpage and runs one or more css queries on the result. Number of results for each query becomes a metric on premetheus.

Configuration

Web configuration is done with a file named webexporter.yaml with the same directory with the executable here is an example fonfiguration for webexporterl.yaml fie

``` yaml ipaddress: "0.0.0.0" port: 3030 metricspath: "metrics" targets: - url: "https://www.rust-lang.org/" queries: - "#language-values div.flex-none section" - "header h1" - "footer div.attribution"

• url: "https://www.rust-lang.org/invalid-page-with-404-response" queries:

  • "div.flex"
  • "div"

• url: "https://www.page-does-not-exist.io/" queries:

  • "div"

• url: "https://www.rust-lang.org/invalid-css-query" queries:

  • "XX" ```

Configuration above will generate metrics like following:

txt web_exporter_response_duration_milliseconds{url="https://www.rust-lang.org/", status=200, error=0} 640 web_exporter_response_response_size_bytes{url="https://www.rust-lang.org/", status=200, error=0} 19220 web_exporter_query_count{url="https://www.rust-lang.org/", query="#language-values div.flex-none section", status=200, error=0} 3 web_exporter_query_count{url="https://www.rust-lang.org/", query="header h1", status=200, error=0} 1 web_exporter_query_count{url="https://www.rust-lang.org/", query="footer div.attribution", status=200, error=0} 1 web_exporter_response_duration_milliseconds{url="https://www.rust-lang.org/invalid-page-with-404-response", status=404, error=0} 292 web_exporter_response_response_size_bytes{url="https://www.rust-lang.org/invalid-page-with-404-response", status=404, error=0} 8244 web_exporter_query_count{url="https://www.rust-lang.org/invalid-page-with-404-response", query="div.flex", status=404, error=0} 6 web_exporter_query_count{url="https://www.rust-lang.org/invalid-page-with-404-response", query="div", status=404, error=0} 14 web_exporter_response_duration_milliseconds{url="https://www.page-does-not-exist.io/", status=0, error=1} 270 web_exporter_response_response_size_bytes{url="https://www.page-does-not-exist.io/", status=0, error=1} 0 web_exporter_response_duration_milliseconds{url="https://www.rust-lang.org/invalid-css-query", status=404, error=0} 306 web_exporter_response_response_size_bytes{url="https://www.rust-lang.org/invalid-css-query", status=404, error=0} 8244 web_exporter_query_count{url="https://www.rust-lang.org/invalid-css-query", query="**XX**", status=404, error=0} 0 web_exporter_scrape_duration_milliseconds 641

Logging:

Logging configuration can be provided through environment variables. To run the project with info logging level you can run it like this

bash $ WEB_EXPORTER_LOG_LEVEL=info ./prometheus_web_exporter

How to run web_exporter

RUN WITH DOCKER

docker run \ --rm -ti -d \ -p 3030:3030 \ --name web_exporter \ -v $(pwd)/web_exporter.yaml:/usr/local/prometheus_web_exporter/web_exporter.yaml \ huseyinyilmaz/web_exporter:latest