cargo-llvm-cov

crates.io license rustc build status

Cargo subcommand to easily use LLVM source-based code coverage.

This is a wrapper around rustc -C instrument-coverage and provides:

Table of Contents:

Usage

Basic usage

Click to show a complete list of options

```console $ cargo llvm-cov --help cargo-llvm-cov Cargo subcommand to easily use LLVM source-based code coverage (-C instrument-coverage).

Use -h for short descriptions and --help for more details.

USAGE: cargo llvm-cov [OPTIONS] [-- ...] [SUBCOMMAND]

ARGS: ... Arguments for the test binary

OPTIONS: --json Export coverage data in "json" format

        If --output-path is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov export -format=text`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.

    --lcov
        Export coverage data in "lcov" format

        If --output-path is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov export -format=lcov`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.

    --text
        Generate coverage report in “text” format

        If --output-path or --output-dir is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov show -format=text`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.

    --html
        Generate coverage report in "html" format

        If --output-dir is not specified, the report will be generated in `target/llvm-cov/html`
        directory.

        This internally calls `llvm-cov show -format=html`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.

    --open
        Generate coverage reports in "html" format and open them in a browser after the
        operation.

        See --html for more.

    --summary-only
        Export only summary information for each file in the coverage data

        This flag can only be used together with either --json or --lcov.

    --output-path <PATH>
        Specify a file to write coverage data into.

        This flag can only be used together with --json, --lcov, or --text. See --output-dir for
        --html and --open.

    --output-dir <DIRECTORY>
        Specify a directory to write coverage report into (default to `target/llvm-cov`).

        This flag can only be used together with --text, --html, or --open. See also
        --output-path.

    --failure-mode <any|all>
        Fail if `any` or `all` profiles cannot be merged (default to `any`)

    --ignore-filename-regex <PATTERN>
        Skip source code files with file paths that match the given regular expression

    --hide-instantiations
        Hide instantiations from report

    --no-cfg-coverage
        Unset cfg(coverage)

    --no-report
        Run tests, but don't generate coverage report

    --fail-under-lines <MIN>
        Exit with a status of 1 if the total line coverage is less than MIN percent

    --show-missing-lines
        Show lines with no coverage

    --doctests
        Including doc tests (unstable)

        This flag is unstable. See <https://github.com/taiki-e/cargo-llvm-cov/issues/2> for
        more.

    --no-run
        Generate coverage report without running tests

    --no-fail-fast
        Run all tests regardless of failure

-q, --quiet
        Display one character per test instead of one line

    --lib
        Test only this package's library unit tests

    --bin <NAME>
        Test only the specified binary

    --bins
        Test all binaries

    --example <NAME>
        Test only the specified example

    --examples
        Test all examples

    --test <NAME>
        Test only the specified test target

    --tests
        Test all tests

    --bench <NAME>
        Test only the specified bench target

    --benches
        Test all benches

    --all-targets
        Test all targets

    --doc
        Test only this library's documentation (unstable)

        This flag is unstable because it automatically enables --doctests flag. See
        <https://github.com/taiki-e/cargo-llvm-cov/issues/2> for more.

-p, --package <SPEC>
        Package to run tests for

    --workspace
        Test all packages in the workspace

        [aliases: all]

    --exclude <SPEC>
        Exclude packages from both the test and report

    --exclude-from-test <SPEC>
        Exclude packages from the test (but not from the report)

    --exclude-from-report <SPEC>
        Exclude packages from the report (but not from the test)

-j, --jobs <N>
        Number of parallel jobs, defaults to # of CPUs

    --release
        Build artifacts in release mode, with optimizations

    --profile <PROFILE-NAME>
        Build artifacts with the specified profile

    --features <FEATURES>
        Space or comma separated list of features to activate

    --all-features
        Activate all available features

    --no-default-features
        Do not activate the `default` feature

    --target <TRIPLE>
        Build for the target triple

        When this option is used, coverage for proc-macro and build script will not be displayed
        because cargo does not pass RUSTFLAGS to them.

-v, --verbose
        Use verbose output

        Use -vv (-vvv) to propagate verbosity to cargo.

    --color <WHEN>
        Coloring

        [possible values: auto, always, never]

    --remap-path-prefix
        Use --remap-path-prefix for workspace root

        Note that this does not fully compatible with doctest.

    --manifest-path <PATH>
        Path to Cargo.toml

    --frozen
        Require Cargo.lock and cache are up to date

    --locked
        Require Cargo.lock is up to date

    --offline
        Run without accessing the network

-Z <FLAG>
        Unstable (nightly-only) flags to Cargo

-h, --help
        Print help information

-V, --version
        Print version information

SUBCOMMANDS: run Run a binary or example and generate coverage report show-env Output the environment set by cargo-llvm-cov to build Rust projects clean Remove artifacts that cargo-llvm-cov has generated in the past nextest Run tests with cargo nextest help Print this message or the help of the given subcommand(s) ```

By default, run tests (via cargo test), and print the coverage summary to stdout.

sh cargo llvm-cov

To run cargo run instead of cargo test, use run subcommand.

sh cargo llvm-cov run

With html report (the report will be generated to target/llvm-cov/html directory):

sh cargo llvm-cov --html open target/llvm-cov/html/index.html

or

sh cargo llvm-cov --open

With plain text report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --text | less -R

With json report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --json --output-path cov.json

With lcov report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --lcov --output-path lcov.info

You can get a coverage report in a different format based on the results of a previous run by using --no-run.

sh cargo llvm-cov --html # run tests and generate html report cargo llvm-cov --no-run --lcov # generate lcov report

Merge coverages generated under different test conditions

You can merge the coverages generated under different test conditions by using --no-report and --no-run.

sh cargo llvm-cov clean --workspace # remove artifacts that may affect the coverage results cargo llvm-cov --no-report --features a cargo llvm-cov --no-report --features b cargo llvm-cov --no-run --lcov # generate report without tests

Get coverage of external tests

cargo test, cargo run, and cargo nextest are available as builtin, but cargo-llvm-cov can also be used for arbitrary binaries built using cargo (including other cargo subcommands or external tests that use make, [xtask], etc.)

```sh source <(cargo llvm-cov show-env --export-prefix) # Set the environment variables needed to get coverage. cargo llvm-cov clean --workspace # remove artifacts that may affect the coverage results cargo build # build rust binaries

commands using binaries in target/debug/*, including cargo test and other cargo subcommands

...

cargo llvm-cov --no-run --lcov # generate report without tests ```

Exclude file from coverage

To exclude specific file patterns from the report, use the --ignore-filename-regex option.

sh cargo llvm-cov --open --ignore-filename-regex build

Exclude function from coverage

To exclude the specific function from coverage, use the #[no_coverage] attribute.

Since #[no_coverage] is unstable, it is recommended to use it together with cfg(coverage) set by cargo-llvm-cov.

```rust

![cfgattr(coverage, feature(nocoverage))]

[cfgattr(coverage, nocoverage)]

fn excludefromcoverage() { // ... } ```

Continuous Integration

Here is an example of GitHub Actions workflow that uploads coverage to [Codecov].

```yaml name: Coverage

on: [pull_request, push]

jobs: coverage: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install Rust run: rustup toolchain install stable --component llvm-tools-preview - name: Install cargo-llvm-cov uses: taiki-e/install-action@cargo-llvm-cov - name: Generate code coverage run: cargo llvm-cov --all-features --workspace --lcov --output-path lcov.info - name: Upload coverage to Codecov uses: codecov/codecov-action@v1 with: token: ${{ secrets.CODECOVTOKEN }} # not required for public repos files: lcov.info failciiferror: true ```

Note: Currently, only line coverage is available on Codecov. This is because -C instrument-coverage does not support branch coverage and Codecov does not support region coverage. See also [#8], [#12], and [#20].

Installation

Prerequisites

cargo-llvm-cov requires llvm-tools-preview:

sh rustup component add llvm-tools-preview

Running cargo-llvm-cov requires rustc 1.60+.

From source

sh cargo install cargo-llvm-cov

Currently, installing cargo-llvm-cov requires rustc 1.54+.

From prebuilt binaries

You can download prebuilt binaries from the Release page. Prebuilt binaries are available for macOS, Linux (gnu and musl), and Windows (static executable).

On GitHub Actions

You can use taiki-e/install-action to install prebuilt binaries on Linux, macOS, and Windows. This makes the installation faster and may avoid the impact of problems caused by upstream changes.

yaml - uses: taiki-e/install-action@cargo-llvm-cov

When used with [nextest]:

yml - uses: taiki-e/install-action@cargo-llvm-cov - uses: taiki-e/install-action@nextest

Via Homebrew

You can install cargo-llvm-cov using Homebrew tap on macOS and Linux:

sh brew install taiki-e/tap/cargo-llvm-cov

Via AUR (ArchLinux)

You can install cargo-llvm-cov from AUR:

sh paru -S cargo-llvm-cov

Note: AUR package is maintained by community, not maintainer of cargo-llvm-cov.

Known limitations

See also the code-coverage-related issues reported in rust-lang/rust.

Related Projects

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.