Fix 1.x compatibility bug with stored Tasks #5412

andrross · 2022-11-30T18:53:49Z

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one.

Closes #5376

Signed-off-by: Andrew Ross [email protected]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <[email protected]>

reta · 2022-11-30T19:12:10Z

qa/rolling-upgrade/src/test/java/org/opensearch/upgrades/SystemIndicesUpgradeIT.java

+     */
+    private static void createAndVerifyStoredTask() throws Exception {
+        // Use update by query to create an async task
+        final Request updateByQueryRequest = new Request("POST", "/test_index_old/_update_by_query");


Just curious, _reindex was replaced with _update_by_query, is there some particular reason?

Because it is done twice, once for the old cluster and again for the updated cluster. Reindex creates a new index so you can't make the exact same request twice (unless you closed or deleted the new index). Update by query is simpler because it takes no parameters and doesn't create any new resources :)

(Previously this step was only being done on the old cluster, so there was no problem. My change does this test for both the old and updated clusters.)

github-actions · 2022-11-30T19:22:14Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/7452/
CommitID: d674d14

github-actions · 2022-11-30T19:30:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

URL: https://build.ci.opensearch.org/job/gradle-check/7453/
CommitID: 319aecc
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

dreamer-89

LGTM! Thanks @andrross for this quick fix.

dreamer-89 · 2022-11-30T19:45:30Z

      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕

TEST FAILURES:
      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex
URL: https://build.ci.opensearch.org/job/gradle-check/7453/

CommitID: 319aecc
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Created #5414 to track this.

opensearch-trigger-bot · 2022-11-30T19:48:38Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-5412-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4616dfac5382062a5149c799554dda4ff601369f
# Push it to GitHub
git push --set-upstream origin backport/backport-5412-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5412-to-2.x.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <[email protected]>

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes #5376 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <[email protected]>

dblock · 2022-12-01T15:10:38Z

👏

opensearch-trigger-bot · 2022-12-03T00:41:16Z

The backport to 2.4 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.4 2.4
# Navigate to the new working tree
pushd ../.worktrees/backport-2.4
# Create a new branch
git switch --create backport/backport-5412-to-2.4
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4616dfac5382062a5149c799554dda4ff601369f
# Push it to GitHub
git push --set-upstream origin backport/backport-5412-to-2.4
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.4

Then, create a pull request where the base branch is 2.4 and the compare/head branch is backport/backport-5412-to-2.4.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit 4616dfa)

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <[email protected]>

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes #5376 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>

andrross force-pushed the task-mapping-fix branch from d674d14 to 319aecc Compare November 30, 2022 18:55

andrross changed the title ~~Fix 1.x compatiblity bug with stored Tasks~~ Fix 1.x compatibility bug with stored Tasks Nov 30, 2022

andrross marked this pull request as ready for review November 30, 2022 18:57

andrross requested review from a team and reta as code owners November 30, 2022 18:57

andrross mentioned this pull request Nov 30, 2022

Upgrade the mappings on .tasks index if needed #5377

Closed

6 tasks

reta reviewed Nov 30, 2022

View reviewed changes

reta approved these changes Nov 30, 2022

View reviewed changes

dreamer-89 approved these changes Nov 30, 2022

View reviewed changes

dreamer-89 mentioned this pull request Nov 30, 2022

[BUG] org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex flaky #5414

Closed

andrross merged commit 4616dfa into opensearch-project:main Nov 30, 2022

andrross added the backport 2.x Backport to 2.x branch label Nov 30, 2022

andrross added the backport 2.4 Backport to 2.4 branch label Dec 3, 2022

andrross deleted the task-mapping-fix branch December 16, 2022 18:35

ketanv3 mentioned this pull request Jan 10, 2023

[BUG] Tasks not added to .tasks after upgrade from 1.3 to 2.3 #5708

Closed

gaobinlong mentioned this pull request Oct 12, 2023

[BUG] Tasks API will no longer be able to store results of tasks in .tasks index after upgrade from ES 7.10 to OS 2.3 #10240

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 1.x compatibility bug with stored Tasks #5412

Fix 1.x compatibility bug with stored Tasks #5412

andrross commented Nov 30, 2022

reta Nov 30, 2022

andrross Nov 30, 2022

andrross Nov 30, 2022

github-actions bot commented Nov 30, 2022

github-actions bot commented Nov 30, 2022

dreamer-89 left a comment

dreamer-89 commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Nov 30, 2022

dblock commented Dec 1, 2022

opensearch-trigger-bot bot commented Dec 3, 2022

Fix 1.x compatibility bug with stored Tasks #5412

Fix 1.x compatibility bug with stored Tasks #5412

Conversation

andrross commented Nov 30, 2022

Check List

reta Nov 30, 2022

Choose a reason for hiding this comment

andrross Nov 30, 2022

Choose a reason for hiding this comment

andrross Nov 30, 2022

Choose a reason for hiding this comment

github-actions bot commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

dreamer-89 left a comment

Choose a reason for hiding this comment

dreamer-89 commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Nov 30, 2022

dblock commented Dec 1, 2022

opensearch-trigger-bot bot commented Dec 3, 2022