Add "Testing done" section to PR template

[basil]

The illumos contributor documentation contains a wonderful page about quality (itself based on an earlier blog post by Bryan Cantrill):

It can all be summed up by asking yourself one question: have you reviewed and tested your change every way that you know how? You should not even contemplate a putback [i.e., committing code to the main source repository] until your answer to this is an unequivocal YES. Remember: you are always empowered as an engineer to take more time to test your work. […] This is important to avoid the Quality Death Spiral. You must do your part by delivering FCS [First Customer Ship] quality all the time.

To maintain FCS quality and avoid regressions, we must ensure all changes are sufficiently tested prior to integration. To this end, I have observed a lack of clear guidance regarding the scope of testing that we require or whose responsibility it is to perform testing. This change clarifies both by explicitly setting expectations for the minimum amount of testing that is required for a change to be merged. It also explicitly communicates the ideal goal of doing test-driven development (TDD) whenever possible. Finally, it explicitly sets the expectation that the burden of testing is on the author of the change, not on maintainers or other members of the community. While such heroic efforts are certainly appreciated, they should not be expected in the general case: a submission should stand on its own merits, including the thoroughness of its testing.

I have included this in its own section of the PR template, below the description section. I feel strongly that this be its own section, not a checkbox in the Submitter checklist section. This strong feeling comes from many years of using Review Board, my favorite code review tool. It has a separate text box for Testing Done, implying through the UI that it is insufficient to write a description of the change without also writing a description of its testing. And the Testing Done text box is in the past tense, implying that testing ought to have been completed prior to the author requesting that the change be integrated (in the implicit form of a non-draft review request or pull request). This is in contrast to Test Plan, the phrasing used in Phabricator, which implies an incorrect order of operations. Sadly, I cannot introduce a second text box in the GitHub PR submission page, but introducing a separate section for Testing done in the existing text box is the best I can do.

By introducing this new section, I aim to strongly encourage contributors to think deeply about testing as they introduce changes and to be explicit about the testing they have done. Maintainers can then evaluate whether or not testing is sufficient prior to accepting a submission. If gaps are discovered, they can be discussed explicitly as we drive toward integration. In this way, we can together maintain and improve the quality of our releases and ultimately the experience for our end users.

Testing done

Viewed the rendered Markdown and verified that the comment was not visible as expected.

Proposed changelog entries

N/A

Proposed upgrade guidelines

N/A

Submitter checklist

• The Jira issue, if it exists, is well-described.
• The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples).
  • Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during upgrade.
• There is automated testing or an explanation as to why this change has no tests.
• New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadocs, as appropriate.
• New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO"), if applicable.
• New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content Security Policy (CSP) directives (see documentation).
• For dependency updates, there are links to external changelogs and, if possible, full differentials.
• For new APIs and extension points, there is a link to at least one consumer.

Desired reviewers

@mention

Maintainer checklist

Before the changes are marked as ready-for-merge:

• There are at least two (2) approvals for the pull request and no outstanding requests for change.
• Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
• Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
• Proper changelog labels are set so that the changelog can be generated automatically.
• If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
• If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

[NotMyFault]

This PR is now ready for merge. We will merge it after ~24 hours if there is no negative feedback.
Please see the merge process documentation for more information about the merge process.
Thanks!

[Wadeck]

Especially interesting with the Hacktoberfest period.