This is a Prometheus exporter for Solana originally based on the Golang original by CertusOne but now providing additional functionality. It is the basis for Grafana dashboards and status alerts.
The tool exports the following metrics:
solana_active_validators: Total number of active validators.solana_validator_delinquent: Whether a validator is delinquent.solana_validator_activated_stake: Activated stake of a validator.solana_validator_last_vote: Last voted slot of a validator.solana_validator_root_slot: The root slot of a validator.solana_transaction_count: Total number of confirmed transactions since genesis.solana_slot_height: Last confirmed slot height.solana_current_epoch: Current epoch.solana_current_epoch_first_slot: Current epoch's first slot.solana_current_epoch_last_slot: Current epoch's last slot.solana_active_validators_isp_count: ISP of active validators.solana_active_validators_isp_stake: ISP of active validators grouped by stake.solana_active_validators_dc_stake: Datacenter of active validators grouped by stake.solana_leader_slots: Validated and skipped leader slots per validator.solana_skipped_slot_percent: Skipped slot percentage per validator.cargo build --release or cargo installBy default solana-exporter takes input config from ~/.solana-exporter/config.toml. If it doesn't
exist it can be generated with template values. Here is a demo example of a config file that works with the public mainnet RPC server and doesn't require running a validator or opening a MaxMind account:
rpc = 'https://api.mainnet-beta.solana.com/'
target = '127.0.0.1:9179'
pubkey_whitelist = []
Here is a production config template for monitoring the entire network via the local validator RPC port and getting server location data from MaxMind: ``` rpc = 'http://localhost:8899/' target = '127.0.0.1:9179' pubkey_whitelist = []
[maxmind] username = 'username' password = 'password' ```
Run this as a systemd service by a non-root user with a script like this one: ``` [Unit] Description=Solana Exporter After=solana.service Requires=solana.service
[Service] User=solana Restart=always RestartSec=20 ExecStart=/home/solana/.cargo/bin/solana-exporter
[Install] WantedBy=multi-user.target ```
If querying a public RPC port, solana-exporter can be run from anywhere, not necessarily from the
validator machine. If querying a private RPC port, install Prometheus on the validator machine. Add
the following snippet to the scrape_configs section of the prometheus.yml config file:
- job_name: solana
static_configs:
- targets: ['localhost:9179']
Restart Prometheus. Now the solana-exporter metrics should be available to view at
http://localhost:9179/metrics. If running on the validator machine, it is highly advisable to only
open the metrics ports to the Grafana machine. This can be achieved with nft. Here is an example /etc/nftables.conf:
```
flush ruleset
table inet filter { chain input { type filter hook input priority 0; # allow connection to Prometheus datasource only locally and from Grafana ip saddr { 127.0.0.1, 193.111.199.108 } tcp dport 9090 accept tcp dport 9090 drop # allow connection to Prometheus exporter endpoints only internally ip saddr != 127.0.0.1 tcp dport 9100 drop ip saddr != 127.0.0.1 tcp dport 9179 drop } chain forward { type filter hook forward priority 0; } chain output { type filter hook output priority 0; } } ```
Note the order of commands. An accept clause should appear before the corresponding drop clause.
When solana-exporter is used on a mainnet validator node, Grafana must always run on a different
machine to circumvent potential DDoS attacks on the validator. In the Grafana dashboard, add the
Prometheus data source http://<Validator IP>:9090. Then import the default
dashboard using that data source.
To display pie charts we use a Grafana pie chart plugin. Prior to Grafana v8 it needed to be installed in order for the pie charts to be displayed. Starting from v8 pie charts are included in the core distribution.
A starting point can be our default dashboard.
Some repetitive lines are ellipsed for brevity in the example below. ```
solanaactivevalidators{status="current"} 561 solanaactivevalidators{status="delinquent"} 51
solanaactivevalidatorsdcstake{dcidentifier="11427-US-Austin"} 9672542164238 solanaactivevalidatorsdcstake{dcidentifier="11524-US-Portland"} 35172007429 solanaactivevalidatorsdcstake{dc_identifier="12212-CA-Toronto"} 407342367161475 ...
solanaactivevalidatorsispcount{ispname="Advanced Solutions LLC"} 1 solanaactivevalidatorsispcount{ispname="Amazon.com"} 48 solanaactivevalidatorsispcount{isp_name="CAIW Internet"} 1 ...
solanaactivevalidatorsispstake{ispname="Advanced Solutions LLC"} 230372233054571 solanaactivevalidatorsispstake{ispname="Amazon.com"} 97432578281165840 solanaactivevalidatorsispstake{isp_name="CAIW Internet"} 352505951070073 ...
solanacurrentepoch 186
solanacurrentepochfirstslot 80665865
solanacurrentepochlastslot 81097865
solanaleaderslots{pubkey="12CUDzb3oe8RBQ4tYGqsuPsCbsVE4KWfktXRihXf8Ggq",status="skipped"} 54 solanaleaderslots{pubkey="12CUDzb3oe8RBQ4tYGqsuPsCbsVE4KWfktXRihXf8Ggq",status="validated"} 146 solanaleaderslots{pubkey="12oRmi8YDbqpkn326MdjwFeZ1bh3t7zVw8Nra2QK2SnR",status="skipped"} 35 solanaleaderslots{pubkey="12oRmi8YDbqpkn326MdjwFeZ1bh3t7zVw8Nra2QK2SnR",status="validated"} 217 ... ```