Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infrastructure: Move Regression coverage to PR comment #1517

Merged
merged 1 commit into from
Sep 29, 2020

Conversation

nschonni
Copy link
Contributor

@nschonni nschonni commented Sep 7, 2020

Currently this is a job that is allowed to fail on Travis CI. This moves it to an action that will leave a comment on the PR if any of the regression related files are changed. Additional pushes will update the same comment instead of continuing to create new ones on the PR.
I'm set it so it also runs when the CI config file is touched so you'll see it on this PR, but wouldn't on a PR that doesn't touch anything in the examples or test folder.

@nschonni
Copy link
Contributor Author

nschonni commented Sep 7, 2020

Ah, looks like the security restrictions on PRs isn't letting this preview from something coming from a fork. I don't think this will be an issue once it's in the main branch. You can see result over here though nschonni#19 (comment)

@nschonni nschonni force-pushed the coverage-comment branch 2 times, most recently from d48a8b4 to f7dac4a Compare September 7, 2020 05:07
@nschonni
Copy link
Contributor Author

nschonni commented Sep 7, 2020

Using the pull_request_target gives access to a read/write token in order to actually make the comment https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/#improvements-for-public-repository-forks

@mcking65
Copy link
Contributor

mcking65 commented Sep 7, 2020

@nschonni which PRs will this add a comment to? All? Or, just PRs that change directories where some of the examples have missing tests?

Something like this would be cool for spell check.

@nschonni
Copy link
Contributor Author

nschonni commented Sep 7, 2020

From this part of the config

  pull_request_target:
    paths:
    - "examples/**"
    - "test/**"

This job will only trigger if the PR touches files under the examples or tests folders. For other PRs that don't touch those files, the job won't even start or leave a comment

Copy link
Contributor

@spectranaut spectranaut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small change to filter out landmark example changes -- but this is awesome, thanks so much! :)

pull_request_target:
paths:
- "examples/**"
- "test/**"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should filter out examples/landmarks -- "- !examples/landmarks/**"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! Done

repo: context.repo.repo,
comment_id: botComment.id,
body: commentBody
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this replace the comment, not append? Because it we don't replace it will get really long! Seems like it from the documentation but just wanted to be sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It replaces it if it exists. You can see it in the "edit" dropdown on my test PR in my fork nschonni#19 (comment)

Copy link
Contributor

@spectranaut spectranaut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super great! Can't wait for this :)

@nschonni nschonni changed the title chore: Move Regression coverage to PR comment Infrastructure: Move Regression coverage to PR comment Sep 29, 2020
@mcking65
Copy link
Contributor

@nschonni thank you again ... looking forward to how this will bring the report to the surface.

@mcking65 mcking65 merged commit 2233e5c into w3c:master Sep 29, 2020
@mcking65 mcking65 added the Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation label Sep 29, 2020
@mcking65 mcking65 added this to the 1.2 Release 1 milestone Sep 29, 2020
@nschonni nschonni deleted the coverage-comment branch September 29, 2020 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants