Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature #2156 release_acceptance_testing #2352

Merged
merged 2 commits into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/Release_Guide/coordinated.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,6 @@ component releases.

.. include:: release_steps/coordinated/update_dtc_website.rst
.. include:: release_steps/coordinated/finalize_release_on_github.rst
.. include:: release_steps/coordinated/release_acceptance_testing.rst
.. include:: release_steps/coordinated/update_zenodo.rst
.. include:: release_steps/coordinated/announce_release.rst
1 change: 1 addition & 0 deletions docs/Release_Guide/met_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/create_release_extra.rst
.. include:: release_steps/met/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
1 change: 1 addition & 0 deletions docs/Release_Guide/metcalcpy_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/create_release_extra.rst
.. include:: release_steps/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
.. include:: release_steps/metcalcpy/update_version_on_develop.rst
1 change: 1 addition & 0 deletions docs/Release_Guide/metdataio_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/create_release_extra.rst
.. include:: release_steps/common/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
.. include:: release_steps/metdataio/update_version_on_develop.rst
1 change: 1 addition & 0 deletions docs/Release_Guide/metexpress_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/metexpress/merge_feature_branch.rst
.. include:: release_steps/metexpress/update_dtc_website.rst
.. include:: release_steps/metexpress/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
1 change: 1 addition & 0 deletions docs/Release_Guide/metplotpy_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/create_release_extra.rst
.. include:: release_steps/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
.. include:: release_steps/metplotpy/update_version_on_develop.rst


1 change: 1 addition & 0 deletions docs/Release_Guide/metplus_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/metplus/update_version_on_develop.rst
.. include:: release_steps/metplus/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
2 changes: 1 addition & 1 deletion docs/Release_Guide/metplus_official.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Create a new vX.Y.Z official release from the develop branch.
.. include:: release_steps/create_release_on_github.rst
.. include:: release_steps/metplus/create_release_extra.rst
.. include:: release_steps/metplus/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_official.rst
.. include:: release_steps/finalize_release_on_github_official.rst
.. include:: release_steps/metplus/update_version_on_develop.rst
.. include:: release_steps/update_docs_official.rst
.. include:: release_steps/metplus/update_web_server_data.rst
Expand Down
1 change: 1 addition & 0 deletions docs/Release_Guide/metviewer_development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
.. include:: release_steps/create_release_extra.rst
.. include:: release_steps/update_dtc_website.rst
.. include:: release_steps/finalize_release_on_github_development.rst
.. include:: release_steps/release_acceptance_testing.rst
.. include:: release_steps/metviewer/update_version_on_develop.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Release Acceptance Testing
--------------------------

A single
`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
discussion in the METplus repository summarizes the external testing for all development
cycles of the METplus components included in a coordinated release.

.. dropdown:: Instructions

* Navigate to the
`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
discussion for this coordinated release.
* Review and finalize any entries still marked as **OPEN** or **FAIL** in the **Status**
column of the repository testing tables. Ideally, the status for all table entries
should be **PASS** for a coordinated release.
* Click on the **Close discussion** button at the bottom discussion.
* Do *not* **Lock conversation** on this discussion to allow external testers
to provide additional comments, as needed.
54 changes: 54 additions & 0 deletions docs/Release_Guide/release_steps/release_acceptance_testing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
Release Acceptance Testing
--------------------------

A single
`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
discussion in the METplus repository summarizes the external testing for all development
cycles of the METplus components included in a coordinated release.

.. dropdown:: Instructions

* If creating a release for the first development cycle (e.g. beta1 release),
check to see if the
`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
discussion for the next coordinated release has already been created.
* If not, create a new one to summarize external testing for all development
cycles of the METplus components.

* Locate the **Release Acceptance Testing** discussion for the last
coordinated release.
* Copy and paste its contents into a new discussion, being sure to update
the title of the discussion and empty the contents of repository testing
tables for each of the METplus components.
* Carefully review the contents and links and update them as needed.
* If needed, create a new label for the next **METplus X.Y Coordinated Release**
in the METplus repository and add that label to this discussion.

* Update the contents of the **Release Acceptance Testing** discussion for
each |projectRepo| development release.

* Locate the |projectRepo| repository testing table within the body of the discussion.
* For issues for which no external testing is required:

* Create a *single table entry* for this development cycle.
* Set the "Status" column to **PASS**.
* Set the "|projectRepo| Issue" column to a list of links for all of the issues.
* Set the "Dev Cycle" column to the current development cycle name (e.g. beta1, beta2, beta3, rc1).
* Leave the "Tester" column empty.
* Set the "Acceptance Testing Comment Link" column to **No external testing required**.

* For issues that do require external testing:

* Create a *separate table entry* for each issue.
* Set the "Status" column to **OPEN**.
* Set the "|projectRepo| Issue" column to link to the issue.
* Set the "Dev Cycle" column to the current development cycle name (e.g. beta1, beta2, beta3, rc1).
* Set the "Tester" column to a list of GitHub user name(s) to solicit their feedback.
* Leave the "Acceptance Testing Comment Link" column empty.

* Save your edits to the discussion.

* External testers are instructed to add comments to the discussion to summarize
the status of their testing. The METplus team monitors those comments and, as needed,
updates the "Status" and "Acceptance Testing Comment Link" columns of the repository
testing table for each METplus component.