[ML] Functional tests - stabilize slider value selection #94313
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pheyos
added
:ml
release_note:skip
|
Pinging @elastic/ml-ui (:ml) |
|
Checking test stability in a flaky test runner job ... no failure in 50 runs ✔️ |
pheyos
added
the
auto-backport
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 11, 2021
This PR stabilizes the slider value selection during ML functional tests.
jloleysens
added a commit
that referenced
this pull request
Mar 11, 2021
…-action * 'master' of github.com:elastic/kibana: (43 commits) [Console] Update copy when showing warnings in response headers (#94270) [TSVB] Type public code. Step 1 (#93231) [ML] Functional tests - stabilize slider value selection (#94313) skip another suite blocking es promotion (#94367) [Security Solution] Eliminates a redundant external link icon (#94194) skip another suite blocking es promotion (#94367) [App Search] Role mappings migration part 1 (#94346) [Security Solution][Detections] Fix flaky indicator enrichment tests (#94241) [Workplace Search] Deduplicate icons (#94359) [ML] Add latest transform to intro text (#94039) skip test failing es promotion (#94367) [Maps] convert elasticsearch_utils to TS (#93984) [Security_Solution][Telemetry] - Update endpoint usage to use agentService (#93829) [Security Solution][Exceptions] Fixes OS adding method for exception enrichment (#94343) [ILM] Add support for frozen phase (#93068) [App Search] Fixed 2 relevance tuning bugs (#94312) remove `try` auth mode (#94287) Removing resolver functional tests (#94331) migrate warning mixin to core (#94273) [App Search] Add routes for Role Mappings (#94221) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/cold_phase/cold_phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/phase/phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/shared_fields/searchable_snapshot_field/searchable_snapshot_field.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/edit_policy.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/form/configuration_issues_context.tsx
kibanamachine
added a commit
that referenced
this pull request
Mar 11, 2021
…4399) This PR stabilizes the slider value selection during ML functional tests. Co-authored-by: Robert Oskamp <[email protected]>
pheyos
added a commit
to pheyos/kibana
that referenced
this pull request
May 10, 2021
This PR stabilizes the slider value selection during ML functional tests.
pheyos
added a commit
that referenced
this pull request
May 10, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR stabilizes the slider value selection during ML functional tests.
Details
Current situation
The logic of the
setSliderValuemethod is not perfectly accurate and with bad timing can lead to test failures, see e.g. #94084.In the final step, one can see a message like
retry.tryForTime error: slider value should be '20' (got '20'), which looks wrong, but is actually not failing the test. In fact, the finalthrowof theelseis needed to trigger the outerretry. But if the innerretrytimes out in a bad moment,currentValuemight not be set correctly when the outer retry is entered (which relies on the calculation of the inner retry).Changes with this PR
currentValuefetching andcurrentDiffcalculation to the start of the outerretry, so it's always accurate and doesn't rely on the innerretrycalculation anymore.assertSliderValueinto theelse, so it returns with a success if the last slider modification hit the target value or throws an error, which will trigger the outerretry. No assertion outside needed anymore, because eithercurrentDiff === 0(inif) orassertSliderValuewas successful (inelse), which both mean that the value has been set correctly.Closes #94084