-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add "Testing done" section to PR template #7218
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timja
approved these changes
Oct 4, 2022
NotMyFault
approved these changes
Oct 4, 2022
daniel-beck
approved these changes
Oct 4, 2022
This PR is now ready for merge. We will merge it after ~24 hours if there is no negative feedback. |
NotMyFault
added
the
ready-for-merge
The PR is ready to go, and it will be merged soon if there is no negative feedback
label
Oct 5, 2022
Wadeck
approved these changes
Oct 7, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Especially interesting with the Hacktoberfest period.
sdenel
pushed a commit
to sdenel/jenkins
that referenced
this pull request
Oct 9, 2022
anurag-harness
pushed a commit
to anurag-harness/jenkins
that referenced
this pull request
Oct 17, 2022
anurag-harness
added a commit
to anurag-harness/jenkins
that referenced
this pull request
Oct 17, 2022
Co-authored-by: Basil Crow <[email protected]>
anurag-harness
added a commit
to anurag-harness/jenkins
that referenced
this pull request
Oct 17, 2022
* Add "Testing done" section to PR template (jenkinsci#7218) * Use enhanced `InboundAgentRule` (jenkinsci#7192) Co-authored-by: Basil Crow <[email protected]>
14 tasks
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
ready-for-merge
The PR is ready to go, and it will be merged soon if there is no negative feedback
skip-changelog
Should not be shown in the changelog
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The illumos contributor documentation contains a wonderful page about quality (itself based on an earlier blog post by Bryan Cantrill):
To maintain FCS quality and avoid regressions, we must ensure all changes are sufficiently tested prior to integration. To this end, I have observed a lack of clear guidance regarding the scope of testing that we require or whose responsibility it is to perform testing. This change clarifies both by explicitly setting expectations for the minimum amount of testing that is required for a change to be merged. It also explicitly communicates the ideal goal of doing test-driven development (TDD) whenever possible. Finally, it explicitly sets the expectation that the burden of testing is on the author of the change, not on maintainers or other members of the community. While such heroic efforts are certainly appreciated, they should not be expected in the general case: a submission should stand on its own merits, including the thoroughness of its testing.
I have included this in its own section of the PR template, below the description section. I feel strongly that this be its own section, not a checkbox in the Submitter checklist section. This strong feeling comes from many years of using Review Board, my favorite code review tool. It has a separate text box for Testing Done, implying through the UI that it is insufficient to write a description of the change without also writing a description of its testing. And the Testing Done text box is in the past tense, implying that testing ought to have been completed prior to the author requesting that the change be integrated (in the implicit form of a non-draft review request or pull request). This is in contrast to Test Plan, the phrasing used in Phabricator, which implies an incorrect order of operations. Sadly, I cannot introduce a second text box in the GitHub PR submission page, but introducing a separate section for Testing done in the existing text box is the best I can do.
By introducing this new section, I aim to strongly encourage contributors to think deeply about testing as they introduce changes and to be explicit about the testing they have done. Maintainers can then evaluate whether or not testing is sufficient prior to accepting a submission. If gaps are discovered, they can be discussed explicitly as we drive toward integration. In this way, we can together maintain and improve the quality of our releases and ultimately the experience for our end users.
Testing done
Viewed the rendered Markdown and verified that the comment was not visible as expected.
Proposed changelog entries
N/A
Proposed upgrade guidelines
N/A
Submitter checklist
@Restricted
or have@since TODO
Javadocs, as appropriate.@Deprecated(since = "TODO")
or@Deprecated(forRemoval = true, since = "TODO")
, if applicable.eval
to ease future introduction of Content Security Policy (CSP) directives (see documentation).Desired reviewers
@mention
Maintainer checklist
Before the changes are marked as
ready-for-merge
:upgrade-guide-needed
label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).lts-candidate
to be considered (see query).