Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Constant keyword tests failing reproducibly #53131

Closed
mark-vieira opened this issue Mar 4, 2020 · 3 comments · Fixed by #53153
Closed

[CI] Constant keyword tests failing reproducibly #53131

mark-vieira opened this issue Mar 4, 2020 · 3 comments · Fixed by #53153
Assignees
@jpountz
Labels
:Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI

Comments

@mark-vieira
Copy link
Contributor

The introduction of a constant keyword field in #49713 added some additional tests which have failed a few times now in CI. The same four tests seem to reliably fail with this error:

    java.lang.AssertionError: Failure at [constant_keyword/10_basic:106]: _shards.skipped didn't match expected value:
                   _shards.skipped: expected Integer [1] but was Integer [2]
        at __randomizedtesting.SeedInfo.seed([343B2640EC8A2EB4:BC6F199A4276434C]:0)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:405)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:382)

Here are some examples of build failures:
https://gradle-enterprise.elastic.co/s/lvbzfzye7qatw/tests/failed
https://gradle-enterprise.elastic.co/s/q56aekioba46m/tests/failed
https://gradle-enterprise.elastic.co/s/5qomgr7ai3c7e/tests/failed

I was able to locally reproduce these failures with the following:

./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.*constant_keyword*" -Dtests.seed=343B2640EC8A2EB4 -Dtests.security.manager=true -Dtests.locale=fr-CH -Dtests.timezone=NST -Dcompiler.java=13

@jpountz since you added these tests do you mind taking a look?
@mark-vieira mark-vieira added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels Mar 4, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@mark-vieira
Copy link
Contributor Author

mark-vieira commented Mar 4, 2020 •
edited
Loading

Since this reproduced for me I've muted these tests in master with 883104f.

Interestingly enough, these tests have not yet failed in 7.x and the same reproduction line above passes for me when run against 7.x.

@jpountz
Copy link
Contributor

jpountz commented Mar 4, 2020

Thanks @mark-vieira, I'm assigning myself.

@jpountz jpountz self-assigned this Mar 4, 2020
jpountz added a commit to jpountz/elasticsearch that referenced this issue Mar 5, 2020
@jpountz
Fix test failures with the new constant_keyword field.
d501abf 
This test failed because YAML tests randomly install an index template
that updates the default number of shards to 2.

Closes elastic#53131
@jpountz jpountz mentioned this issue Mar 5, 2020
Merged
jpountz added a commit that referenced this issue Mar 5, 2020
@jpountz
Fix test failures with the new constant_keyword field. (#53153)
dda18fa 
This test failed because YAML tests randomly install an index template
that updates the default number of shards to 2.

Closes #53131
jpountz added a commit that referenced this issue Mar 5, 2020
@jpountz
Fix test failures with the new constant_keyword field. (#53153)
360ac19 
This test failed because YAML tests randomly install an index template
that updates the default number of shards to 2.

Closes #53131
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jpountz jpountz

Labels
:Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix test failures with the new constant_keyword field. jpountz/elasticsearch
3 participants
@jpountz @mark-vieira @elasticmachine