Feature #2156 release_acceptance_testing (#2352)

dtcenter · Sep 19, 2023 · 7127f7c · 7127f7c
1 parent 027cbb9
commit 7127f7c
Show file tree

Hide file tree

Showing 11 changed files with 82 additions and 1 deletion.
diff --git a/docs/Release_Guide/coordinated.rst b/docs/Release_Guide/coordinated.rst
@@ -19,5 +19,6 @@ component releases.
 
 .. include:: release_steps/coordinated/update_dtc_website.rst
 .. include:: release_steps/coordinated/finalize_release_on_github.rst
+.. include:: release_steps/coordinated/release_acceptance_testing.rst
 .. include:: release_steps/coordinated/update_zenodo.rst
 .. include:: release_steps/coordinated/announce_release.rst
diff --git a/docs/Release_Guide/met_development.rst b/docs/Release_Guide/met_development.rst
@@ -17,3 +17,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/create_release_extra.rst
 .. include:: release_steps/met/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
diff --git a/docs/Release_Guide/metcalcpy_development.rst b/docs/Release_Guide/metcalcpy_development.rst
@@ -17,4 +17,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/create_release_extra.rst
 .. include:: release_steps/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
 .. include:: release_steps/metcalcpy/update_version_on_develop.rst
diff --git a/docs/Release_Guide/metdataio_development.rst b/docs/Release_Guide/metdataio_development.rst
@@ -20,4 +20,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/create_release_extra.rst
 .. include:: release_steps/common/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
 .. include:: release_steps/metdataio/update_version_on_develop.rst
diff --git a/docs/Release_Guide/metexpress_development.rst b/docs/Release_Guide/metexpress_development.rst
@@ -11,3 +11,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/metexpress/merge_feature_branch.rst
 .. include:: release_steps/metexpress/update_dtc_website.rst
 .. include:: release_steps/metexpress/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
diff --git a/docs/Release_Guide/metplotpy_development.rst b/docs/Release_Guide/metplotpy_development.rst
@@ -17,6 +17,7 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/create_release_extra.rst
 .. include:: release_steps/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
 .. include:: release_steps/metplotpy/update_version_on_develop.rst
 
 
diff --git a/docs/Release_Guide/metplus_development.rst b/docs/Release_Guide/metplus_development.rst
@@ -18,3 +18,4 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/metplus/update_version_on_develop.rst
 .. include:: release_steps/metplus/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
diff --git a/docs/Release_Guide/metplus_official.rst b/docs/Release_Guide/metplus_official.rst
@@ -20,7 +20,7 @@ Create a new vX.Y.Z official release from the develop branch.
 .. include:: release_steps/create_release_on_github.rst
 .. include:: release_steps/metplus/create_release_extra.rst
 .. include:: release_steps/metplus/update_dtc_website.rst
-.. include:: release_steps/finalize_release_on_github_official.rst	     
+.. include:: release_steps/finalize_release_on_github_official.rst
 .. include:: release_steps/metplus/update_version_on_develop.rst
 .. include:: release_steps/update_docs_official.rst
 .. include:: release_steps/metplus/update_web_server_data.rst

diff --git a/docs/Release_Guide/metviewer_development.rst b/docs/Release_Guide/metviewer_development.rst
@@ -17,4 +17,5 @@ Create a new vX.Y.Z-betaN or vX.Y.Z-rcN development release from the develop bra
 .. include:: release_steps/create_release_extra.rst
 .. include:: release_steps/update_dtc_website.rst
 .. include:: release_steps/finalize_release_on_github_development.rst
+.. include:: release_steps/release_acceptance_testing.rst
 .. include:: release_steps/metviewer/update_version_on_develop.rst
diff --git a/docs/Release_Guide/release_steps/coordinated/release_acceptance_testing.rst b/docs/Release_Guide/release_steps/coordinated/release_acceptance_testing.rst
@@ -0,0 +1,19 @@
+Release Acceptance Testing
+--------------------------
+
+A single
+`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
+discussion in the METplus repository summarizes the external testing for all development
+cycles of the METplus components included in a coordinated release.
+
+.. dropdown:: Instructions
+
+  * Navigate to the
+    `Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
+    discussion for this coordinated release.
+  * Review and finalize any entries still marked as **OPEN** or **FAIL** in the **Status**
+    column of the repository testing tables. Ideally, the status for all table entries
+    should be **PASS** for a coordinated release.
+  * Click on the **Close discussion** button at the bottom discussion.
+  * Do *not* **Lock conversation** on this discussion to allow external testers
+    to provide additional comments, as needed.
diff --git a/docs/Release_Guide/release_steps/release_acceptance_testing.rst b/docs/Release_Guide/release_steps/release_acceptance_testing.rst
@@ -0,0 +1,54 @@
+Release Acceptance Testing
+--------------------------
+
+A single
+`Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
+discussion in the METplus repository summarizes the external testing for all development
+cycles of the METplus components included in a coordinated release.
+
+.. dropdown:: Instructions
+
+  * If creating a release for the first development cycle (e.g. beta1 release),
+    check to see if the
+    `Release Acceptance Testing <https://github.com/dtcenter/METplus/discussions/categories/release-acceptance-testing>`_
+    discussion for the next coordinated release has already been created.
+  * If not, create a new one to summarize external testing for all development
+    cycles of the METplus components.
+
+    * Locate the **Release Acceptance Testing** discussion for the last
+      coordinated release.
+    * Copy and paste its contents into a new discussion, being sure to update
+      the title of the discussion and empty the contents of repository testing
+      tables for each of the METplus components.
+    * Carefully review the contents and links and update them as needed.
+    * If needed, create a new label for the next **METplus X.Y Coordinated Release**
+      in the METplus repository and add that label to this discussion.
+
+  * Update the contents of the **Release Acceptance Testing** discussion for
+    each |projectRepo| development release.
+
+    * Locate the |projectRepo| repository testing table within the body of the discussion.
+    * For issues for which no external testing is required:
+
+      * Create a *single table entry* for this development cycle.
+      * Set the "Status" column to **PASS**.
+      * Set the "|projectRepo| Issue" column to a list of links for all of the issues.
+      * Set the "Dev Cycle" column to the current development cycle name (e.g. beta1, beta2, beta3, rc1).
+      * Leave the "Tester" column empty.
+      * Set the "Acceptance Testing Comment Link" column to **No external testing required**.
+
+    * For issues that do require external testing:
+
+      * Create a *separate table entry* for each issue.
+      * Set the "Status" column to **OPEN**.
+      * Set the "|projectRepo| Issue" column to link to the issue.
+      * Set the "Dev Cycle" column to the current development cycle name (e.g. beta1, beta2, beta3, rc1).
+      * Set the "Tester" column to a list of GitHub user name(s) to solicit their feedback.
+      * Leave the "Acceptance Testing Comment Link" column empty.
+
+    * Save your edits to the discussion.
+
+  * External testers are instructed to add comments to the discussion to summarize
+    the status of their testing. The METplus team monitors those comments and, as needed,
+    updates the "Status" and "Acceptance Testing Comment Link" columns of the repository
+    testing table for each METplus component.