Candidate review start date is errantly reset when a test plan advances to candidate #973
Comments
|
This is partially fixed, but the fix introduces another problem that creates equally severe data integrity issues. Now, when a test plan version is advanced to candidate, it creates two other data problems:
These problems might be two different sides of the same coin. It seems there is some fundamental problem with the logic for the requirements related to events in the test plan life cycle and how those events define the start and end of phases in the working mode.
The logic is the same for candidate review. Recommended phase is only different in that once a test plan has a recommended version, there is no end to the phase because it is the last phase. For problem 1, look at the Alert example row on the data management page in staging. In the draft review column it says:
Obviously, review of a test plan version that was developed in 2023 could not have had draft review complete in 2022. That should say:
I figured out the Feb 26 date by looking at the version history page and found a Feb 26, 2024 deprecation date for V22.12.08 in candidate review. There are several other places where the incorrect Dec 9, 2022 date shows up instead of the Feb 26, 2024 date. In the version summary on the version history page, the first row says:
Similarly, looking at the test plan review page for V23.12.06 of alert, it says:
For problem 2, In the timeline for all versions on the version history of alert test plan, there is a row with:
That should be:
|
|
Given the impact on data, these problems are sufficiently severe that we should not deploy to production until they are resolved. |
|
It seems like some of the problems are partially corrected, but I am still finding problems with candidate review start dates in three places:
The problems on data management and test plan version history pages are inconsistent. For example, I am comparing the test plans for alert, command button, and link in staging. On the data management page, the candidate review start date in the overall status column is incorrect for alert and link but correct for command button.
The overall status column of the data management page incorrectly shows the Alert test plan started candidate review on Jan 11, 2024, but the version history correctly shows that V22.12.08 started candidate review on Dec 9, 2022. All the data on the version history page looks correct for the alert test plan. Similarly, the overall status column on the data management page for link incorrectly shows that candidate review started Jan 23, 2024. However, the version history for the link test plan correctly shows that candidate review started with V22.05.04 on May 5, 2022. Like the alert version history, all the data on the link version history page looks correct. On the other hand, the situation for command button is the opposite. The data management page shows the correct date for candidate review and the version history is incorrect. The data management page correctly shows that candidate review started May 5, 2022. However, note the following problems on the version history page. The version summary shows that V23.12.13 became candidate on May 5, 2022, but it should say some date after Dec 13, 2023. It is not possible to see what that date should be from the data available.
In the timeline for all versions, for the date May 5, 2022, it incorrectly says that "V23.12.13 Candidate Review Started". It should say that V22.05.04 started candidate review. This incorrect date is propagated to the history of the V23.12.13 version as well. On the candidate review page, there is a column titled "Candidate Phase Start Date", and it is still showing the incorrect date. Instead of showing the date that the candidate review phase started for the test plan, it is showing the date the latest version of the test plan advanced to candidate review.
We have two options to fix this problem on the candidate review page:
I am thinking option 2 of treating the bug as a feature may actually be more helpful to AT vendors. This is because AT vendors do not currently have a simple way to know when there was a change to a test plan that is in their queue. While showing only the date would not help them know what has changed, at least we can tell them that it has changed. Addressing the problem of telling them what has changed is a separate issue (not yet raised). So, to resolve the manifestation of this issue on the candidate review page, I suggest we change the column title to:
Note: this name reflects the point of view of the AT vendor. Obviously, the actual test plan content was not change on this date; it changed on the date reflected by the version number. However, the date that the most recent version advanced to candidate review is the date that the plan in review changed from the perspective of the AT vendor. |
|
@mcking65 these latest comments should be addressed by #994 once reviewed. Data Management Overall Status Column date being incorrect The Data Management page never kept track of the deprecated versions, so the earliest version date wouldn't be shown if the version it belonged to was deprecated. The PR now considers deprecated versions in that date calculation. Incorrect phase change date recorded on Command Button versions page for V23.12.13 This seems like a one-off case (although there is an oddity described in the PR for toggle button in production as well). I have suggested a way to move forward with resolving it with reviewers of that PR. I have also gone through the other test plan versions pages and haven't found any major issues other than what has already been identified by #966. Candidate review page Candidate Phase Start Date I've gone ahead with option 2. |
|
Tested in staging, and this is working beautifully now! Thank you! |
The last deployment of the app broke an important working mode requirement.
When a test plan advances to candidate, if there is an earlier version of that plan in candidate review, the candidate review start and target completion dates should not be changed. Now, the candidate review start dates are incorrect for all test plans in candidate review. In addition, target dates were changed for all test plans advanced after the latest release of the app, so they are also incorrect. We can no longer see how overdue those plans are.
The text was updated successfully, but these errors were encountered: