Skip to content

Implement Modified Condition/Decision Coverage#123409

Merged
bors merged 4 commits intorust-lang:masterfrom
ZhuUx:master
Apr 20, 2024
Merged

Implement Modified Condition/Decision Coverage#123409
bors merged 4 commits intorust-lang:masterfrom
ZhuUx:master

Conversation

@ZhuUx
Copy link

@ZhuUx ZhuUx commented Apr 3, 2024

This is an implementation based on llvm backend support (>= 18) by @evodius96 and branch coverage support by @Zalathar.

Major changes:

  • Add -Zcoverage-options=mcdc as switch. Now coverage options accept either no-branch, branch, or mcdc. mcdc also enables branch because it is essential to work.
  • Add coverage mapping for MCDCBranch and MCDCDecision. Note that MCDCParameter evolves from llvm 18 to llvm 19. The mapping in rust side mainly references to 19 and is casted to 18 types in llvm wrapper.
  • Add wrapper for mcdc instrinc functions from llvm. And inject associated statements to mir.
  • Add BcbMappingKind::Decision, I'm not sure is it proper but can't find a better way temporarily.
  • Let coverage-dump support parsing MCDCBranch and MCDCDecision from llvm ir.
  • Add simple tests to check whether mcdc works.
  • Same as clang, currently rustc does not generate instrument for decision with more than 6 condtions or only 1 condition due to considerations of resource.

Implementation Details

  1. To get information about conditions and decisions, MCDCState in BranchInfoBuilder is used during hir lowering to mir. For expressions with logical op we call Builder::visit_coverage_branch_operation to record its sub conditions, generate condition ids for them and save their spans (to construct the span of whole decision). This process mainly references to the implementation in clang and is described in comments over MCDCState::record_conditions. Also true marks and false marks introduced by branch coverage are used to detect where the decision evaluation ends: the next id of the condition == 0.
  2. Once the MCDCState::decision_stack popped all recorded conditions, we can ensure that the decision is checked over and push it into decision_spans. We do not manually insert decision span to avoid complexity from then_else_break in nested if scopes.
  3. When constructing CoverageSpans, add condition info to BcbMappingKind::Branch and decision info to BcbMappingKind::Decision. If the branch mapping has non-zero condition id it will be transformed to MCDCBranch mapping and insert CondBitmapUpdate statements to its evaluated blocks. While decision bcb mapping will insert TestVectorBitmapUpdate in all its end blocks.

Usage

 echo "[build]\nprofiler=true" >> config.toml
./x build --stage 1
./x test tests/coverage/mcdc_if.rs

to build the compiler and run tests.

export PATH=path/to/llvm-build:$PATH
rustup toolchain link mcdc build/host/stage1
cargo +mcdc rustc --bin foo -- -Cinstrument-coverage -Zcoverage-options=mcdc
cd target/debug
LLVM_PROFILE_FILE="foo.profraw" ./foo
llvm-profdata merge -sparse foo.profraw -o foo.profdata   
llvm-cov show ./foo -instr-profile=foo.profdata --show-mcdc

to check "foo" code.

Problems to solve

For now decision mapping will insert statements to its all end blocks, which may be optimized by inserting a final block of the decision. To do this we must also trace the evaluated value at each end of the decision and join them separately.

This implementation is not heavily tested so there should be some unrevealed issues. We are going to check our rust products in the next. Please let me know if you had any suggestions or comments.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.