Failure in SourceOnlySnapshotTests.testSourceOnlyRandom

[DaveCTurner]

./gradlew :x-pack:plugin:core:test \
  -Dtests.seed=3573246BA8A76617 \
  -Dtests.class=org.elasticsearch.snapshots.SourceOnlySnapshotTests \
  -Dtests.method="testSourceOnlyRandom" \
  -Dtests.security.manager=true \
  -Dtests.locale=el-GR \
  -Dtests.timezone=ECT \
  -Dcompiler.java=10 \
  -Druntime.java=8

yielded

FAILURE 7.07s | SourceOnlySnapshotTests.testSourceOnlyRandom <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: expected:<600> but was:<979>
   >    at __randomizedtesting.SeedInfo.seed([3573246BA8A76617:C6419F8A6436F980]:0)
   >    at org.elasticsearch.snapshots.SourceOnlySnapshotTests.testSourceOnlyRandom(SourceOnlySnapshotTests.java:93)
   >    at java.lang.Thread.run(Thread.java:748)

on 6.x (a7ea5e0)

This was found in https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+multijob-unix-compatibility/os=oraclelinux/2/console

@s1monw could you take a look please?

[elasticmachine]

Pinging @elastic/es-distributed

[jakelandis]

Reopening ... ran into this in a PR and can reliably reproduce this against head of master and 6.x head with the following:

./gradlew :x-pack:plugin:core:test -Dtests.seed=E9BB79AF0F0C53E5 -Dtests.class=org.elasticsearch.snapshots.SourceOnlySnapshotTests -Dtests.method="testSourceOnlyRandom" -Dtests.security.manager=true -Dtests.locale=pl -Dtests.timezone=Europe/Oslo -Dcompiler.java=10 -Druntime.java=8

Full log:

./gradlew :x-pack:plugin:core:test -Dtests.seed=E9BB79AF0F0C53E5 -Dtests.class=org.elasticsearch.snapshots.SourceOnlySnapshotTests -Dtests.method="testSourceOnlyRandom" -Dtests.security.manager=true -Dtests.locale=pl -Dtests.timezone=Europe/Oslo -Dcompiler.java=10 -Druntime.java=8
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.10/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing

> Configure project :benchmarks
=======================================
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 4.10
  OS Info               : Mac OS X 10.12.6 (x86_64)
  Compiler JDK Version  : 10 (Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10])
  Compiler java.home    : /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home
  Runtime JDK Version   :  8 (Oracle Corporation 1.8.0_131 [Java HotSpot(TM) 64-Bit Server VM 25.131-b11])
  Runtime java.home     : /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home
  Gradle JDK Version    : 10 (Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10])
  Gradle java.home      : /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home
  Random Testing Seed   : E9BB79AF0F0C53E5
=======================================
Suite: org.elasticsearch.snapshots.SourceOnlySnapshotTests

> Task :x-pack:plugin:core:test
   [junit4] <JUnit4> says g'day! Master seed: E9BB79AF0F0C53E5
==> Test Info: seed=E9BB79AF0F0C53E5; jvm=1; suite=1

  1> [2018-09-20T17:16:39,554][WARN ][o.e.b.JNANatives         ] [[SUITE-SourceOnlySnapshotTests-seed#[E9BB79AF0F0C53E5]]] Unable to lock JVM Memory: error=78, reason=Function not implemented
  1> [2018-09-20T17:16:39,558][WARN ][o.e.b.JNANatives         ] [[SUITE-SourceOnlySnapshotTests-seed#[E9BB79AF0F0C53E5]]] This can result in part of the JVM being swapped out.
  1> [2018-09-21T00:16:40,693][INFO ][o.e.s.SourceOnlySnapshotTests] [testSourceOnlyRandom] before test
  1> [2018-09-21T00:16:43,230][INFO ][o.e.s.SourceOnlySnapshotTests] [testSourceOnlyRandom] after test
  2> REPRODUCE WITH: ./gradlew :x-pack:plugin:core:test -Dtests.seed=E9BB79AF0F0C53E5 -Dtests.class=org.elasticsearch.snapshots.SourceOnlySnapshotTests -Dtests.method="testSourceOnlyRandom" -Dtests.security.manager=true -Dtests.locale=pl -Dtests.timezone=Europe/Oslo -Dcompiler.java=10 -Druntime.java=8
FAILURE 2.59s | SourceOnlySnapshotTests.testSourceOnlyRandom <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: expected:<543> but was:<586>
   >    at __randomizedtesting.SeedInfo.seed([E9BB79AF0F0C53E5:1A89C24EC39DCC72]:0)
   >    at org.elasticsearch.snapshots.SourceOnlySnapshotTests.testSourceOnlyRandom(SourceOnlySnapshotTests.java:96)
   >    at java.lang.Thread.run(Thread.java:748)
  2> NOTE: leaving temporary files on disk at: /Users/jake/workspace/elasticsearch/x-pack/plugin/core/build/testrun/test/J0/temp/org.elasticsearch.snapshots.SourceOnlySnapshotTests_E9BB79AF0F0C53E5-001
  2> NOTE: test params are: codec=Lucene80, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3e2230da), locale=pl, timezone=Europe/Oslo
  2> NOTE: Mac OS X 10.12.6 x86_64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=8,threads=1,free=482372896,total=514850816
  2> NOTE: All tests run in this JVM: [SourceOnlySnapshotTests]
Completed [1/1] in 4.80s, 1 test, 1 failure <<< FAILURES!


> Task :x-pack:plugin:core:test FAILED
Tests with failures:
  - org.elasticsearch.snapshots.SourceOnlySnapshotTests.testSourceOnlyRandom

   [junit4] JVM J0:     0.64 ..     6.46 =     5.82s
   [junit4] Execution time total: 6.47 sec.
   [junit4] Tests summary: 1 suite, 1 test, 1 failure

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':x-pack:plugin:core:test'.
> There were test failures: 1 suite, 1 test, 1 failure [seed: E9BB79AF0F0C53E5]

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 37s
50 actionable tasks: 2 executed, 48 up-to-date

cc @s1monw

[s1monw]

@jakelandis looking