Telemetry library

This is a library for common telemetry functionality, especially subscribers for Tokio tracing libraries. The subscribers enable writing trace data to Jaeger, distributed tracing, common logs and metrics destinations, etc.

Getting started is easy. In your app:

rust let config = telemetry::TelemetryConfig { service_name: "my_app".into(), ..Default::default() }; let guard = telemetry::init(config);

It is important to retain the guard until the end of the program. Assign it in the main fn and keep it, for once it drops then log output will stop.

You can also run the example and see output in ANSI color:

cargo run --example easy-init

Stdout vs file output

By default, logs (but not spans) are formatted for human readability and output to stdout, with key-value tags at the end of every line. RUST_LOG can be configured for custom logging output, including filtering.

By setting log_file in the config, one can write log output to a daily-rotated file.

Tracing and span output

Detailed span start and end logs can be generated by defining the json_log_output config variable. Note that this causes all output to be in JSON format, which is not as human-readable, so it is not enabled by default. This output can easily be fed to backends such as ElasticSearch for indexing, alerts, aggregation, and analysis.

NOTE: JSON output requires the json crate feature to be enabled.

Jaeger (seeing distributed traces)

To see nested spans visualized with Jaeger, do the following:

  1. Run this to get a local Jaeger container: docker run -d -p6831:6831/udp -p6832:6832/udp -p16686:16686 jaegertracing/all-in-one:latest
  2. Set enable_tracing config setting to true
  3. Run your app
  4. Browse to http://localhost:16686/ and select the service you configured using service_name

NOTE: separate spans (which are not nested) are not connected as a single trace for now.

Jaeger subscriber is enabled by default but is protected by the jaeger feature flag. If you'd like to leave out the Jaeger dependencies, you can turn off the default-features in your dependency:

telemetry = { url = "...", default-features = false }

Live async inspection / Tokio Console

Tokio-console is an awesome CLI tool designed to analyze and help debug Rust apps using Tokio, in real time! It relies on a special subscriber.

  1. Build your app using a special flag: RUSTFLAGS="--cfg tokio_unstable" cargo build
  2. Enable the tokio-console feature for this crate.
  3. Set the tokio_console config setting when running your app
  4. Clone the console repo and cargo run to launch the console