Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage: Add condition coverage #124402

Closed

Conversation

RenjiSann
Copy link
Contributor

This MR adds condition-coverage as defined in #124120 .

This is done by removing the "optimization" that prevents the compiler to create a branch for the last operand of a boolean expression that is not an if-else block decision.

@Lambdaris
@Zalathar
@ZhuUx

@rustbot
Copy link
Collaborator

rustbot commented Apr 26, 2024

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jieyouxu (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

  • @rustbot author: the review is finished, PR author should check the comments and take action accordingly
  • @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 26, 2024
@rust-log-analyzer

This comment has been minimized.

@rustbot
Copy link
Collaborator

rustbot commented Apr 26, 2024

Some changes occurred in coverage tests.

cc @Zalathar

@RenjiSann RenjiSann force-pushed the renji/branch-on-bool-last-assign branch 2 times, most recently from 7a80098 to 290a532 Compare April 26, 2024 13:39
@jieyouxu
Copy link
Member

The changes look reasonable to me. However, I'm not very familiar with MIR building and if the change to conditional lowering is desirable, so I would like another compiler reviewer to also take a look at this PR.

r? compiler

@Zalathar
Copy link
Contributor

I plan to take a look at this sometime this week.

@Zalathar
Copy link
Contributor

@rustbot label +A-code-coverage

@rustbot rustbot added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Apr 28, 2024
@Zalathar
Copy link
Contributor

The main thing that worries me about this change that it adds a big chunk of coverage-specific complexity right in the middle of MIR building, which is something I've generally been trying very hard to avoid.

I suspect there is probably a less-disruptive way to get the desired behaviour here, perhaps via one of:

  • Calling out to some separate coverage-specific code, in a way that is obviously a no-op when (condition) coverage is not enabled.
  • Instrumenting bool expressions in a more general way, so that little or no special-case code is needed here specifically.

@Zalathar
Copy link
Contributor

Also see #124507 for my proposed handling of the coverage level in -Zcoverage-options.

@RenjiSann
Copy link
Contributor Author

The main thing that worries me about this change that it adds a big chunk of coverage-specific complexity right in the middle of MIR building, which is something I've generally been trying very hard to avoid.

I suspect there is probably a less-disruptive way to get the desired behavior here, perhaps via one of:

  • Calling out to some separate coverage-specific code, in a way that is obviously a no-op when (condition) coverage is not enabled.
  • Instrumenting bool expressions in a more general way, so that little or no special-case code is needed here specifically.

I was concerned by the same thing, that's why I added a big comment to document what is happening.
However, I am struggling to find a way to not impact MIR building.
Currently, the instrumentation inserts block markers to insert instructions, and everything happens in the then_else_break_inner context.
Branch and MCDC coverage rely on the fact that there are 2 possible outcomes which can be instrumented.
I feel like changing this behavior would involve consequent refactoring to the current instrumentation, so I assumed this implementation was the lesser evil.

I take your point and share your concern, but I probably lack experience with the compiler codebase to see a better alternative.

@michaelwoerister
Copy link
Member

r? mir

@RenjiSann RenjiSann force-pushed the renji/branch-on-bool-last-assign branch from 290a532 to c138d40 Compare April 29, 2024 16:12
@RenjiSann
Copy link
Contributor Author

Rebase on master + splitting tests commits to get before/after diff (diff)

@bors
Copy link
Contributor

bors commented Apr 30, 2024

☔ The latest upstream changes (presumably #124507) made this pull request unmergeable. Please resolve the merge conflicts.

@RenjiSann RenjiSann force-pushed the renji/branch-on-bool-last-assign branch from c138d40 to c06499a Compare April 30, 2024 08:27
@rust-log-analyzer

This comment has been minimized.

@RenjiSann RenjiSann force-pushed the renji/branch-on-bool-last-assign branch from c06499a to b95334e Compare April 30, 2024 09:18
@RenjiSann RenjiSann force-pushed the renji/branch-on-bool-last-assign branch from b95334e to bf51676 Compare April 30, 2024 13:06
@RenjiSann
Copy link
Contributor Author

Rebase on #124399

@Zalathar
Copy link
Contributor

Zalathar commented May 3, 2024

I take your point and share your concern, but I probably lack experience with the compiler codebase to see a better alternative.

I spent some time trying to find an alternative, and #124644 shows what I was able to come up with.

(You were right that it's not easy! I went down several dead-ends before finally arriving at this approach.)

@RenjiSann
Copy link
Contributor Author

RenjiSann commented May 3, 2024

superseded by #124644, which implement the same logic while being less intrusive in the main MIR building code.

@RenjiSann RenjiSann closed this May 3, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request May 31, 2024
coverage: Optionally instrument the RHS of lazy logical operators

(This is an updated version of rust-lang#124644 and rust-lang#124402. Fixes rust-lang#124120.)

When `||` or `&&` is used outside of a branching context (such as the condition of an `if`), the rightmost value does not directly influence any branching decision, so branch coverage instrumentation does not treat it as its own true-or-false branch.

That is a correct and useful interpretation of “branch coverage”, but might be undesirable in some contexts, as described at rust-lang#124120. This PR therefore adds a new coverage level `-Zcoverage-options=condition` that behaves like branch coverage, but also adds additional branch instrumentation to the right-hand-side of lazy boolean operators.

---

As discussed at rust-lang#124120 (comment), this is mainly intended as an intermediate step towards fully-featured MC/DC instrumentation. It's likely that we'll eventually want to remove this coverage level (rather than stabilize it), either because it has been incorporated into MC/DC instrumentation, or because it's getting in the way of future MC/DC work. The main appeal of landing it now is so that work on tracking conditions can proceed concurrently with other MC/DC-related work.

`@rustbot` label +A-code-coverage
jieyouxu added a commit to jieyouxu/rust that referenced this pull request May 31, 2024
coverage: Optionally instrument the RHS of lazy logical operators

(This is an updated version of rust-lang#124644 and rust-lang#124402. Fixes rust-lang#124120.)

When `||` or `&&` is used outside of a branching context (such as the condition of an `if`), the rightmost value does not directly influence any branching decision, so branch coverage instrumentation does not treat it as its own true-or-false branch.

That is a correct and useful interpretation of “branch coverage”, but might be undesirable in some contexts, as described at rust-lang#124120. This PR therefore adds a new coverage level `-Zcoverage-options=condition` that behaves like branch coverage, but also adds additional branch instrumentation to the right-hand-side of lazy boolean operators.

---

As discussed at rust-lang#124120 (comment), this is mainly intended as an intermediate step towards fully-featured MC/DC instrumentation. It's likely that we'll eventually want to remove this coverage level (rather than stabilize it), either because it has been incorporated into MC/DC instrumentation, or because it's getting in the way of future MC/DC work. The main appeal of landing it now is so that work on tracking conditions can proceed concurrently with other MC/DC-related work.

``@rustbot`` label +A-code-coverage
jieyouxu added a commit to jieyouxu/rust that referenced this pull request May 31, 2024
coverage: Optionally instrument the RHS of lazy logical operators

(This is an updated version of rust-lang#124644 and rust-lang#124402. Fixes rust-lang#124120.)

When `||` or `&&` is used outside of a branching context (such as the condition of an `if`), the rightmost value does not directly influence any branching decision, so branch coverage instrumentation does not treat it as its own true-or-false branch.

That is a correct and useful interpretation of “branch coverage”, but might be undesirable in some contexts, as described at rust-lang#124120. This PR therefore adds a new coverage level `-Zcoverage-options=condition` that behaves like branch coverage, but also adds additional branch instrumentation to the right-hand-side of lazy boolean operators.

---

As discussed at rust-lang#124120 (comment), this is mainly intended as an intermediate step towards fully-featured MC/DC instrumentation. It's likely that we'll eventually want to remove this coverage level (rather than stabilize it), either because it has been incorporated into MC/DC instrumentation, or because it's getting in the way of future MC/DC work. The main appeal of landing it now is so that work on tracking conditions can proceed concurrently with other MC/DC-related work.

```@rustbot``` label +A-code-coverage
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request May 31, 2024
coverage: Optionally instrument the RHS of lazy logical operators

(This is an updated version of rust-lang#124644 and rust-lang#124402. Fixes rust-lang#124120.)

When `||` or `&&` is used outside of a branching context (such as the condition of an `if`), the rightmost value does not directly influence any branching decision, so branch coverage instrumentation does not treat it as its own true-or-false branch.

That is a correct and useful interpretation of “branch coverage”, but might be undesirable in some contexts, as described at rust-lang#124120. This PR therefore adds a new coverage level `-Zcoverage-options=condition` that behaves like branch coverage, but also adds additional branch instrumentation to the right-hand-side of lazy boolean operators.

---

As discussed at rust-lang#124120 (comment), this is mainly intended as an intermediate step towards fully-featured MC/DC instrumentation. It's likely that we'll eventually want to remove this coverage level (rather than stabilize it), either because it has been incorporated into MC/DC instrumentation, or because it's getting in the way of future MC/DC work. The main appeal of landing it now is so that work on tracking conditions can proceed concurrently with other MC/DC-related work.

````@rustbot```` label +A-code-coverage
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request May 31, 2024
Rollup merge of rust-lang#125756 - Zalathar:branch-on-bool, r=oli-obk

coverage: Optionally instrument the RHS of lazy logical operators

(This is an updated version of rust-lang#124644 and rust-lang#124402. Fixes rust-lang#124120.)

When `||` or `&&` is used outside of a branching context (such as the condition of an `if`), the rightmost value does not directly influence any branching decision, so branch coverage instrumentation does not treat it as its own true-or-false branch.

That is a correct and useful interpretation of “branch coverage”, but might be undesirable in some contexts, as described at rust-lang#124120. This PR therefore adds a new coverage level `-Zcoverage-options=condition` that behaves like branch coverage, but also adds additional branch instrumentation to the right-hand-side of lazy boolean operators.

---

As discussed at rust-lang#124120 (comment), this is mainly intended as an intermediate step towards fully-featured MC/DC instrumentation. It's likely that we'll eventually want to remove this coverage level (rather than stabilize it), either because it has been incorporated into MC/DC instrumentation, or because it's getting in the way of future MC/DC work. The main appeal of landing it now is so that work on tracking conditions can proceed concurrently with other MC/DC-related work.

````@rustbot```` label +A-code-coverage
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants