cargo-llvm-cov

crates.io license rustc build status

Cargo subcommand to easily use LLVM source-based code coverage.

This is a wrapper around rustc -Z instrument-coverage and provides:

Table of Contents:

Installation

Prerequisites

cargo-llvm-cov requires nightly toolchain and llvm-tools-preview:

sh rustup component add llvm-tools-preview --toolchain nightly

From source

sh cargo install cargo-llvm-cov

cargo-llvm-cov relies on unstable compiler flags so it requires a nightly toolchain to be installed, though does not require nightly to be the default toolchain or the one with which cargo-llvm-cov itself is executed. If the default toolchain is one other than nightly, running cargo llvm-cov will find and use nightly anyway.

From prebuilt binaries

You can download prebuilt binaries from the Release page. Prebuilt binaries are available for macOS, Linux (gnu and musl), and Windows.

Via Homebrew

You can install cargo-llvm-cov using Homebrew tap on macOS and Linux:

sh brew install taiki-e/tap/cargo-llvm-cov

Usage

Click to show a complete list of options

```console $ cargo llvm-cov --help cargo-llvm-cov

Cargo subcommand to easily use LLVM source-based code coverage (-Z instrument-coverage).

Use -h for short descriptions and --help for more details.

USAGE: cargo llvm-cov [OPTIONS] [-- ...]

ARGS: ... Arguments for the test binary

OPTIONS: -h, --help Print help information

-V, --version
        Print version information

    --json
        Export coverage data in "json" format

        If --output-path is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov export -format=text`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.

    --lcov
        Export coverage data in "lcov" format

        If --output-path is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov export -format=lcov`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.

    --text
        Generate coverage report in “text” format

        If --output-path or --output-dir is not specified, the report will be printed to stdout.

        This internally calls `llvm-cov show -format=text`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.

    --html
        Generate coverage report in "html" format

        If --output-dir is not specified, the report will be generated in `target/llvm-cov/html`
        directory.

        This internally calls `llvm-cov show -format=html`. See
        <https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.

    --open
        Generate coverage reports in "html" format and open them in a browser after the
        operation.

        See --html for more.

    --summary-only
        Export only summary information for each file in the coverage data

        This flag can only be used together with either --json or --lcov.

    --output-path <PATH>
        Specify a file to write coverage data into.

        This flag can only be used together with --json, --lcov, or --text. See --output-dir for
        --html and --open.

    --output-dir <DIRECTORY>
        Specify a directory to write coverage report into (default to `target/llvm-cov`).

        This flag can only be used together with --text, --html, or --open. See also --output-
        path.

    --ignore-filename-regex <PATTERN>
        Skip source code files with file paths that match the given regular expression

    --doctests
        Including doc tests (unstable)

    --no-report
        Run tests, but don't generate coverage report

    --no-run
        Generate coverage report without running tests

    --no-fail-fast
        Run all tests regardless of failure

    --lib
        Test only this package's library unit tests

    --bin <NAME>...
        Test only the specified binary

    --bins
        Test all binaries

    --example <NAME>...
        Test only the specified example

    --examples
        Test all examples

    --test <NAME>...
        Test only the specified test target

    --tests
        Test all tests

    --bench <NAME>...
        Test only the specified bench target

    --benches
        Test all benches

    --all-targets
        Test all targets

-p, --package <SPEC>...
        Package to run tests for

    --workspace
        Test all packages in the workspace [aliases: all]

    --exclude <SPEC>...
        Exclude packages from the test

    --release
        Build artifacts in release mode, with optimizations

    --profile <PROFILE-NAME>
        Build artifacts with the specified profile

    --features <FEATURES>...
        Space or comma separated list of features to activate

    --all-features
        Activate all available features

    --no-default-features
        Do not activate the `default` feature

    --target <TRIPLE>
        Build for the target triple

        When this option is used, coverage for proc-macro and build script will not be displayed
        because cargo does not pass RUSTFLAGS to them.

    --manifest-path <PATH>
        Path to Cargo.toml

-v, --verbose
        Use verbose output

        Use -vv (-vvv) to propagate verbosity to cargo.

    --color <WHEN>
        Coloring [possible values: auto, always, never]

    --frozen
        Require Cargo.lock and cache are up to date

    --locked
        Require Cargo.lock is up to date

    --offline
        Run without accessing the network

-Z <FLAG>...
        Unstable (nightly-only) flags to Cargo

```

By default, run tests, and print the coverage summary to stdout.

sh cargo llvm-cov

With html report (the report will be generated to target/llvm-cov/html directory):

sh cargo llvm-cov --html open target/llvm-cov/html/index.html

or

sh cargo llvm-cov --open

With plain text report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --text | less -R

With json report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --json --output-path cov.json

With lcov report (if --output-path is not specified, the report will be printed to stdout):

sh cargo llvm-cov --lcov --output-path lcov.info

You can merge the coverages generated under different test conditions by using --no-report and --no-run.

sh cargo clean cargo llvm-cov --no-report --features a cargo llvm-cov --no-report --features b cargo llvm-cov --no-run --lcov

Continuous Integration

Here is an example of GitHub Actions workflow that uploads coverage to [Codecov].

```yaml name: Coverage

on: [pull_request, push]

jobs: coverage: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Install Rust run: rustup toolchain install nightly --component llvm-tools-preview - name: Install cargo-llvm-cov run: curl -LsSf https://github.com/taiki-e/cargo-llvm-cov/releases/latest/download/cargo-llvm-cov-x8664-unknown-linux-gnu.tar.gz | tar xzf - -C ~/.cargo/bin - name: Generate code coverage run: cargo llvm-cov --all-features --workspace --lcov --output-path lcov.info - name: Upload coverage to Codecov uses: codecov/codecov-action@v1 with: token: ${{ secrets.CODECOVTOKEN }} # not required for public repos files: lcov.info failciif_error: true ```

NOTE: Currently, only line coverage is available on Codecov. This is because -Z instrument-coverage does not support branch coverage and Codecov does not support region coverage. See also [#8], [#12], and [#20].

Known limitations

See also the code-coverage-related issues reported in rust-lang/rust.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.