Investigate code coverage

[thibault-martinez]

No description provided.

[Adam-Gleave]

I have written a quick bash script to make use of Cargo's source-based-coverage functionality. Currently this runs all unit tests, but not doctests, since there is currently an open issue regarding the accuracy of their results: rust-lang/rust#79417.

The commands will output an HTML report in a coverage directory in the root of the repository:

#! /bin/bash

# Ensure past coverage files are deleted
find . -name "*.profraw" -type f -delete
find . -name "*.profdata" -type f -delete
rm -r coverage
mkdir coverage

# Run tests with profiling instrumentation
RUSTFLAGS="-Zinstrument-coverage" LLVM_PROFILE_FILE="bee-%m.profraw" cargo test --tests

# Merge all .profraw files into "bee.profdata"
cargo profdata -- merge */bee-*.profraw -o bee.profdata

# Generate and export the coverage report to lcov format
cargo cov -- export \
    $( \
      for file in \
        $( \
          RUSTFLAGS="-Zinstrument-coverage" \
            cargo test --tests --no-run --message-format=json \
              | jq -r "select(.profile.test == true) | .filenames[]" \
              | grep -v dSYM - \
        ); \
      do \
        printf "%s %s " -object $file; \
      done \
    ) \
  --instr-profile=bee.profdata \
  --ignore-filename-regex="/.cargo|rustc|/.rustup" \
  --format=lcov --Xdemangler=rustfilt \
  >> coverage/coverage.info

# Generate report html
genhtml coverage/coverage.info --legend --title="Bee Coverage" --output-directory=coverage

This will ignore any code for external dependencies or the Rust codebase, with the option --ignore-filename-regex="/.cargo|rustc|/.rustup", so the only code that gets checked is Bee source code, with no dependencies. On the chrysalis-pt-2 branch, this means some Bee crates do not get covered (since they are pulled from the dev branch and placed in ~/.cargo/git – this will not happen on the dev branch).

Results

Running these commands against commit 117ab6a gives these summary results:

• Line coverage: 46.0%
• Function coverage: 62.8%