BGPKIT Broker

BGPKIT Broker is an online data API service that allows users to search for publicly available BGP archive files by time, collector, project, or data type. The service indexes the archives in close to real-time (delay is less than 5 minutes). Currently, we are indexing BGP table dump and updates files from RIPE RIS and RouteViews.

This Rust library provides SDK access to the BGPKIT Broker API as well as a command-line tool to start a self-hosted broker instance. Current BGPKIT Broker API is available at https://api.bgpkit.com/docs.

BGPKIT Broker is used in production at Cloudflare Radar powering its routing page and projects like BGP hijack detection and route leak detection.

Broker Rust SDK

Usage

Add the following dependency line to your project's Cargo.toml file: yaml bgpkit-broker = "0.7.0-beta.1"

Example

You can run the follow example with cargo run --example query (source code).

```rust use bgpkit_broker::{BgpkitBroker, BrokerItem};

pub fn main() { let broker = BgpkitBroker::new() .tsstart("1634693400") .tsend("1634693400");

// method 1: create iterator from reference (so that you can reuse the broker object)
// same as `&broker.into_iter()`
for item in &broker {
    println!("{}", item);
}

// method 2: create iterator from the broker object (taking ownership)
let items = broker.into_iter().collect::<Vec<BrokerItem>>();

assert_eq!(items.len(), 106);

} ```

bgpkit-broker CLI Tool

bgpkit-broker is a command-line application that packages many functionalities to allow users to self-host a BGPKIT Broker instance with ease.

Install with cargo install bgpkit-broker@0.7.0-beta.1 --features cli or check out the main branch and run cargo install --path . --features cli.

bgpkit-broker has the following subcommands:

```text A library and command-line to provide indexing and searching functionalities for public BGP data archive files over time.

Usage: bgpkit-broker [OPTIONS]

Commands: serve Serve the Broker content via RESTful API update Update the Broker database config Print out current configuration bootstrap Bootstrap the Broker database backup Export broker database to parquet file search Search MRT files in Broker db help Print this message or the help of the given subcommand(s)

Options: --no-log disable logging --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -h, --help Print help -V, --version Print version ```

serve

bgpkit-broker serve is the main command to start the BGPKIT Broker service. It will start a web server that serves the API endpoints. It will also periodically update the local database unless the --no-update flag is set.

```text Serve the Broker content via RESTful API

Usage: bgpkit-broker serve [OPTIONS]

Options: -i, --update-interval update interval in seconds [default: 300] --no-log disable logging --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -h, --host host address [default: 0.0.0.0] -p, --port port number [default: 40064] -r, --root root path, useful for configuring docs UI [default: /] --no-update disable updater service --no-api disable API service --full-bootstrap do a full database bootstrap from duckdb or parquet file -h, --help Print help -V, --version Print version ```

update

bgpkit-broker update triggers a local database update manually. This command cannot be run at the same time as serve because the active API will lock the database file.

```text Update the Broker database

Usage: bgpkit-broker update [OPTIONS]

Options: --no-log disable logging --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -h, --help Print help -V, --version Print version ```

config

bgpkit-broker config displays current configuration, e.g. local database path, update interval, etc.

```text Print out current configuration

Usage: bgpkit-broker config [OPTIONS]

Options: --no-log disable logging --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -h, --help Print help -V, --version Print version ```

backup

bgpkit-broker update runs a database backup and export the database to a duckdb file and a parquet file. This can be run while serve is running.

```text Export broker database to parquet file

Usage: bgpkit-broker backup [OPTIONS]

Options: --no-log disable logging --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -h, --help Print help -V, --version Print version ```

search

bgpkit-broker search queries for MRT files using the default production API unless specified otherwise.

```text Search MRT files in Broker db

Usage: bgpkit-broker search [OPTIONS]

Options: --no-log disable logging -t, --ts-start Start timestamp --bootstrap-parquet bootstrap from parquet file instead of DuckDB file -T, --ts-end End timestamp -p, --project filter by route collector projects, i.e. route-views or riperis -c, --collector-id filter by collector IDs, e.g. 'rrc00', 'route-views2. use comma to separate multiple collectors -d, --data-type filter by data types, i.e. 'update', 'rib' --page page number --page-size page size -u, --url
-j, --json print out search results in JSON format instead of Markdown table -h, --help Print help -V, --version Print version ```

Data Provider

If you have publicly available data and want to be indexed BGPKIT Broker service, please send us an email at data@bgpkit.com. Our back-end service is designed to be flexible and should be able to adapt to most data archiving approaches.