Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Jest Integration Tests.x-pack/plugins/task_manager/server/integration_tests - task state validation allow_reading_invalid_state: true should fail to update the task if the task runner returns an unknown property in the state #161081

Closed
kibanamachine opened this issue Jul 3, 2023 · 5 comments · Fixed by #163744
Closed

Failing test: Jest Integration Tests.x-pack/plugins/task_manager/server/integration_tests - task state validation allow_reading_invalid_state: true should fail to update the task if the task runner returns an unknown property in the state #161081

kibanamachine opened this issue Jul 3, 2023 · 5 comments · Fixed by #163744
Assignees
@mikecote
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Jul 3, 2023 •
edited
Loading

A test failed on a tracked branch

Error: thrown: "Exceeded timeout of 600000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."
    at it (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/integration_tests/task_state_validation.test.ts:150:5)
    at _dispatchDescribe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/index.js:91:26)
    at describe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/index.js:55:5)
    at describe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/integration_tests/task_state_validation.test.ts:80:3)
    at _dispatchDescribe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/index.js:91:26)
    at describe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/index.js:55:5)
    at Object.describe (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/integration_tests/task_state_validation.test.ts:79:1)
    at Runtime._execModule (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-runtime/build/index.js:1377:24)
    at Runtime._loadModule (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-runtime/build/index.js:989:12)
    at Runtime.requireModule (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-runtime/build/index.js:849:12)
    at jestAdapter (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:77:13)
    at runTestInternal (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-runner/build/runTest.js:367:16)
    at runTest (/var/lib/buildkite-agent/builds/kb-n2-4-spot-72593d6e7609e903/elastic/kibana-on-merge/kibana/node_modules/jest-runner/build/runTest.js:444:34)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jul 3, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Jul 3, 2023
@kibanamachine kibanamachine added the Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) label Jul 3, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Jul 3, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Jul 5, 2023
@mistic
skip flaky suite (#161081)
61a743a
@mistic
Copy link
Member

mistic commented Jul 5, 2023

Skipped.

main: 61a743a

@mikecote mikecote self-assigned this Jul 13, 2023
@mikecote mikecote mentioned this issue Aug 11, 2023
Merged
mikecote added a commit that referenced this issue Aug 16, 2023
@mikecote @kibanamachine
Fix flaky test for task state validation (#163744)
42964d0 
Fixes #161081.

In this PR, I'm attempting to reduce the flakiness of the
`task_state_validation.test.ts` jest integration test by doing the
following changes:
- Ensuring each test creates a task with a different ID in case there is
a race condition with the previous test that would delete the task as
the next test runs.
- Modifying how the test waits for the task runner to be called given
the previous approach wasn't 100% that the executor would be mocked
before the task is scheduled.

Since the flaky test runner doesn't work with jest integration tests, I
instead added a for loop in my code to run the test 25x, and made CI run
it a few times:
- Commits creating for loop and setting to 25:
78561fe
and
7168537
- Commits where CI ran 25x and succeeded:
fef05d7,
f3b2a90,
7415c93,
and
74bce2a

---------

Co-authored-by: Kibana Machine <[email protected]>
@kibanamachine kibanamachine reopened this Feb 13, 2024
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mikecote
Copy link
Contributor

Closing as flakiness no longer occurring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikecote mikecote

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
AppEx: ResponseOps - Execution & Connectors
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix flaky test for task state validation mikecote/kibana
4 participants
@mikecote @mistic @elasticmachine @kibanamachine