Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] too many disjunctions to expand failure in IntervalQueryBuilderTests.testMustRewrite #3321

Open
nknize opened this issue May 13, 2022 · 2 comments
Open

[CI] too many disjunctions to expand failure in IntervalQueryBuilderTests.testMustRewrite #3321

nknize opened this issue May 13, 2022 · 2 comments
Assignees
@yigithub
Labels
bug Something isn't working CI CI related >test-failure Test failure from CI, local build, etc.

Comments

@nknize
Copy link
Collaborator

nknize commented May 13, 2022

Caught on #3318 the following failure is reproducible:

./gradlew ':server:test' --tests "org.opensearch.index.query.IntervalQueryBuilderTests.testMustRewrite" -Dtests.seed=A6236EB12532F6E5 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ms-MY -Dtests.timezone=America/Rosario -Druntime.java=17

org.opensearch.index.query.IntervalQueryBuilderTests > testMustRewrite FAILED
    java.lang.IllegalArgumentException: Too many disjunctions to expand
        at __randomizedtesting.SeedInfo.seed([A6236EB12532F6E5:12CC1D8C5D65F065]:0)
        at org.apache.lucene.queries.intervals.Disjunctions.pullUp(Disjunctions.java:45)
        at org.apache.lucene.queries.intervals.OrderedIntervalsSource.pullUpDisjunctions(OrderedIntervalsSource.java:91)
        at org.apache.lucene.queries.intervals.FilteredIntervalsSource.maxGaps(FilteredIntervalsSource.java:33)
        at org.apache.lucene.queries.intervals.Intervals.maxgaps(Intervals.java:302)
        at org.opensearch.index.query.IntervalBuilder.combineSources(IntervalBuilder.java:173)
        at org.opensearch.index.query.IntervalsSourceProvider$Combine.getSource(IntervalsSourceProvider.java:468)

Looks related to #2103. A large test query is being created that is resulting in a 1152 disjunctions which exceeds IndexSearcher.maxClauseCount (1024). With the addition of the unordered non-overlapping intervals support we should probably make the test less aggressive here. Is this something you can look at @mattweber?

Test query:

{
  "intervals" : {
    "mapped_string" : {
      "all_of" : {
        "mode" : "ORDERED",
        "max_gaps" : 3,
        "intervals" : [
          {
            "match" : {
              "query" : "ഭകധള൤൛a",
              "max_gaps" : 0,
              "mode" : "UNORDERED_NO_OVERLAP",
              "analyzer" : "simple"
            }
          },
          {
            "any_of" : {
              "intervals" : [
                {
                  "any_of" : {
                    "intervals" : [
                      {
                        "match" : {
                          "query" : "ᥭᥙᥕᥨ᥹ᥧ᥾᥮ᥨᥒᥩᥛᥭ᥼ᥔa ᱳᱸᱧᱵᱻᱡᱧᱦᱹᱪ᱾᱔ᱹᱰa ᰑᰩᰊa \uD800\uDDD2\uD800\uDDFDaa \uD801\uDCA3\uD801\uDC99\uD801\uDC88\uD801\uDC9D\uD801\uDC95\uD801\uDCACa",
                          "max_gaps" : 3,
                          "mode" : "UNORDERED_NO_OVERLAP",
                          "analyzer" : "whitespace"
                        }
                      },
                      {
                        "match" : {
                          "query" : "㆟㆟㆜a",
                          "max_gaps" : 0,
                          "mode" : "ORDERED",
                          "analyzer" : "keyword"
                        }
                      },
                      {
                        "match" : {
                          "query" : "\uD802\uDF74\uD802\uDF6A\uD802\uDF6B\uD802\uDF68\uD802\uDF72\uD802\uDF6D\uD802\uDF7Eaa ㍬㍞㎗㍠㌺㎵㎕㏲a ▎▇▀▒▏▇▒▜▁▌▊▘▍▖a \uDBCD\uDEC4aa",
                          "max_gaps" : 3,
                          "mode" : "UNORDERED",
                          "analyzer" : "whitespace"
                        }
                      },
                      {
                        "match" : {
                          "query" : "₽₴₲⃎⃁⃉₥a ⡩⢺⣔⣒⡐⢵⡥⠉⠾⢭a",
                          "max_gaps" : 4,
                          "mode" : "ORDERED",
                          "analyzer" : "whitespace"
                        }
                      },
                      {
                        "match" : {
                          "query" : "꤫ꤩꤒꤔ꤅ꤐꤔa ՉԺժնԵճՓԵa \uD808\uDD91\uD808\uDF21\uD808\uDE8C\uD808\uDC6B\uD808\uDF65\uD808\uDDD9aa \uD808\uDDA5\uD808\uDC73aa",
                          "max_gaps" : 4,
                          "mode" : "UNORDERED",
                          "analyzer" : "keyword"
                        }
                      }
                    ]
                  }
                },
                {
                  "match" : {
                    "query" : "᭿ᬋ᭝ᬽᬹa ꥕ꥀ꥞꥟ꥌ꥘꥚ꥌ꥝ꤸ꥙ꥐa",
                    "max_gaps" : 4,
                    "mode" : "ORDERED",
                    "analyzer" : "simple"
                  }
                },
                {
                  "match" : {
                    "query" : "\uD84B\uDCAE\uD84D\uDD7C\uD841\uDFB3\uD847\uDD0D\uD865\uDE21\uD867\uDFDC\uD865\uDE22a ﹈︸﹇﹃﹉﹍︵︴︷﹂﹊﹊﹅﹋a",
                    "max_gaps" : 4,
                    "mode" : "UNORDERED",
                    "analyzer" : "simple"
                  }
                },
                {
                  "match" : {
                    "query" : "ᚭᛖᚠᚦᚥa \uD802\uDE64\uD802\uDE79\uD802\uDE68\uD802\uDE69\uD802\uDE64\uD802\uDE7A\uD802\uDE7Aaa ⺹⺤⺤⺗⺳⻝⻅⺾a 鳩肍稶鈆箤纠齩鹹偷儣捒a c7\u0014D)Z% \u001C\bg\u001B.cAa",
                    "max_gaps" : 1,
                    "mode" : "UNORDERED_NO_OVERLAP",
                    "analyzer" : "whitespace"
                  }
                }
              ]
            }
          },
          {
            "any_of" : {
              "intervals" : [
                {
                  "match" : {
                    "query" : "᪇ᨯᩆ᪭᪓ᩓ᪑ᩀᨽᩢᨧ᪢᪑ᨰᨷ᩼᪩᩻a ␚␳␏␩␇␻␞␳␐␈␿␎␩a ꩍꨘꨅ꩓a",
                    "max_gaps" : 1,
                    "mode" : "UNORDERED",
                    "analyzer" : "simple"
                  }
                },
                {
                  "any_of" : {
                    "intervals" : [
                      {
                        "match" : {
                          "query" : "Ꙅ꙼ꙀꚀꙏꙍꚅꚏꙥꚉꙁꚁa a ᝽ᝩ᝽ᝠᝧ᝵ᝫ᝺ᝦ᝿a ⴌⴃⴢⴕⴤⴢⴖⴣⴋ⴫ⴂⴈa ኚዏ፭ሁጇዦቑ፽ኤቓጌፎጕዌሬa",
                          "max_gaps" : 3,
                          "mode" : "UNORDERED",
                          "analyzer" : "keyword"
                        }
                      },
                      {
                        "match" : {
                          "query" : "ýÙ��áíÁò��úÜ�çáÌa \uD803\uDE73\uD803\uDE6A\uD803\uDE6C\uD803\uDE60a ݗݸݷݜݻݔݴݬݛݺݤݤݵݾa \uD800\uDCE9\uD800\uDCB8\uD800\uDCF2\uD800\uDC93\uD800\uDCF9\uD800\uDCC7a",
                          "max_gaps" : 3,
                          "mode" : "ORDERED",
                          "analyzer" : "whitespace"
                        }
                      },
                      {
                        "match" : {
                          "query" : "ፉቃርኖ዁፪ቭኌኒጱፄa ㇠㇝㇋㇂㇘㇘㇚㇎㇋㇨a ꚎꚉꙭꚍꙟꚈꙚꙉꚞa",
                          "max_gaps" : -1,
                          "mode" : "UNORDERED",
                          "analyzer" : "simple"
                        }
                      },
                      {
                        "match" : {
                          "query" : " ᚊᚒᚇᚌ᚜ᚃᚃᚈᚖa ﴵﰄﯶﲩ﴿ﲄﲇ�ﳸﲚﮃﰝ�ﯚﮡﲱﶝﱎ�a",
                          "max_gaps" : 1,
                          "mode" : "ORDERED",
                          "analyzer" : "keyword"
                        }
                      }
                    ]
                  }
                },
                {
                  "match" : {
                    "query" : "⸝⸱⹵a ᭑᭲ᬤ᭺᭹ᬱ᭟ᬷᬓ᭟᭝᭮a",
                    "max_gaps" : 1,
                    "mode" : "UNORDERED_NO_OVERLAP",
                    "analyzer" : "simple"
                  }
                },
                {
                  "match" : {
                    "query" : "a",
                    "max_gaps" : 1,
                    "mode" : "UNORDERED",
                    "analyzer" : "whitespace"
                  }
                },
                {
                  "match" : {
                    "query" : "ㄧㄟㄆㄜㄤㄎㄅㄅㄘ㄄ㄏㄐㄘㄋ㄁ㄩㄤa",
                    "max_gaps" : 3,
                    "mode" : "UNORDERED_NO_OVERLAP",
                    "analyzer" : "simple"
                  }
                }
              ]
            }
          },
          {
            "match" : {
              "query" : "ᇱᆵᄡᅔᄖᇎᆌᆶᅋᇽᄨᆿᅥa ᅭᆕᄲᇳᅆᄬᅈᄍᇖᅾᄊᇴᆔᇾᄡᇈa ꙳ꙬꙐꙣꙂꙺꙘꚍꚆꙟꙃꚄꚍꙁꚞꙨa るぉろほゃひぅもぷゟゟしべばごんa \uD87E\uDC2E\uD87E\uDD26\uD87E\uDCE5\uD87E\uDDD2\uD87E\uDDFB\uD87E\uDCD7aa",
              "max_gaps" : -1,
              "mode" : "UNORDERED",
              "analyzer" : "whitespace"
            }
          },
          {
            "all_of" : {
              "mode" : "ORDERED",
              "max_gaps" : 2,
              "intervals" : [
                {
                  "match" : {
                    "query" : "꒽꓋꒞꒑꓇꒭꒞꒹꒵a a ꡾ꡤ꡾꡴ꡖꡝꡫ꡼ꡣꡋ꡺꡿ꡖꡌ꡼ꡟꡕꡡꡫa ⦓⧯⦓a",
                    "max_gaps" : 1,
                    "mode" : "UNORDERED",
                    "analyzer" : "simple"
                  }
                },
                {
                  "all_of" : {
                    "mode" : "ORDERED",
                    "max_gaps" : 4,
                    "intervals" : [
                      {
                        "match" : {
                          "query" : "ㇴㇳㇷㇱㇷㇲㇴㇽㇲㇲㇼa \uD802\uDE5D\uD802\uDE5D\uD802\uDE4E\uD802\uDE23\uD802\uDE52\uD802\uDE11\uD802\uDE5B\uD802\uDE55\uD802\uDE57aa",
                          "max_gaps" : 1,
                          "mode" : "UNORDERED_NO_OVERLAP",
                          "analyzer" : "simple"
                        }
                      },
                      {
                        "match" : {
                          "query" : "令﨩契兩菉廉節憐﫯煮滛郎﫳祈黎a ᙋᗴᓭ᙮ᖽᒌᘒᓋᗣᙗᘺᙞᒩa ు౻ణ౜౸ఔే౗థేa",
                          "max_gaps" : 0,
                          "mode" : "UNORDERED_NO_OVERLAP",
                          "analyzer" : "whitespace"
                        }
                      }
                    ]
                  }
                },
                {
                  "match" : {
                    "query" : "ඎ෥෕ඨඹළඉජ෽අට෽ඐනa ᤟ᤁ᥌᥇ᤗᤁᤣᤝᤙᤡ᤺a",
                    "max_gaps" : 2,
                    "mode" : "UNORDERED",
                    "analyzer" : "keyword"
                  }
                },
                {
                  "all_of" : {
                    "mode" : "ORDERED",
                    "max_gaps" : 4,
                    "intervals" : [
                      {
                        "match" : {
                          "query" : "طڡأ٬ڹڼ٨۩a ⹶⸫⸌⹥⹋⸈a ︶﹍︲︰﹃︵︸﹉︿﹇﹆︼﹌︲︽︿a a ㄲㄷㅠㄴㄾㅷㅥㅡa",
                          "max_gaps" : 3,
                          "mode" : "UNORDERED_NO_OVERLAP",
                          "analyzer" : "simple"
                        }
                      },
                      {
                        "match" : {
                          "query" : "ⴅⴑⴜ⴪ⴇⴗⴠⴍⴌⴊⴥⴄⴚ⴮ⴔⴝa ᱴ᱒᱖᱐ᱟᱽᱫᱲᱼᱫᱬᱰ᱙ᱡᱲa ೠು್೫ೲ೹೔ಹರಕುರಌ್೒೦ಟ೪ೀa ᜟᜎ᜗ᜄᜓᜅ᜛᜔ᜃᜓᜆ᜙ᜏᜀᜌ᜔᜘ᜍ᜕a ပဌႛၰအၿ၁ၰa",
                          "max_gaps" : 2,
                          "mode" : "UNORDERED_NO_OVERLAP",
                          "analyzer" : "keyword"
                        }
                      },
                      {
                        "match" : {
                          "query" : "ಲಊ೜ಮಀa ⟵⟽⟳⟵⟺⟶⟻⟲⟱⟻a \uD834\uDF42\uD834\uDF04\uD834\uDF3Baa",
                          "max_gaps" : 4,
                          "mode" : "UNORDERED",
                          "analyzer" : "keyword"
                        }
                      }
                    ]
                  }
                }
              ]
            }
          }
        ]
      },
      "boost" : 0.33333334,
      "_name" : "YBYknk0"
    }
  }
}
@nknize nknize added bug Something isn't working >test-failure Test failure from CI, local build, etc. CI CI related untriaged labels May 13, 2022
@nknize nknize mentioned this issue May 13, 2022
Merged
@ccook-ibm
Copy link
Contributor

ccook-ibm commented Jul 21, 2023 •
edited
Loading

A similar instance of the Too many disjunctions to expand exception, this time failing a different test in the same suite: #8823 (comment)


Build failed due to flaky test:

Tests with failures:
 - org.opensearch.index.query.IntervalQueryBuilderTests.testToQuery
REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.index.query.IntervalQueryBuilderTests.testToQuery" -Dtests.seed=A1E5FA5E6C37D034 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-ES -Dtests.timezone=Africa/Gaborone -Druntime.java=20

org.opensearch.index.query.IntervalQueryBuilderTests > testToQuery FAILED
    java.lang.IllegalArgumentException: Too many disjunctions to expand
        at __randomizedtesting.SeedInfo.seed([A1E5FA5E6C37D034:561EF8601DB415DE]:0)
        at org.apache.lucene.queries.intervals.Disjunctions.pullUp(Disjunctions.java:45)
        at org.apache.lucene.queries.intervals.UnorderedIntervalsSource.pullUpDisjunctions(UnorderedIntervalsSource.java:90)
        at org.apache.lucene.queries.intervals.FilteredIntervalsSource.maxGaps(FilteredIntervalsSource.java:33)
        at org.apache.lucene.queries.intervals.Intervals.maxgaps(Intervals.java:302)
        at org.opensearch.index.query.IntervalBuilder.combineSources(IntervalBuilder.java:173)
        at org.opensearch.index.query.IntervalsSourceProvider$Combine.getSource(IntervalsSourceProvider.java:444)
        at org.opensearch.index.query.IntervalQueryBuilder.doToQuery(IntervalQueryBuilder.java:164)
        at org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)
        at org.opensearch.test.AbstractQueryTestCase.testToQuery(AbstractQueryTestCase.java:448)
        at java.****/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.****/java.lang.reflect.Method.invoke(Method.java:578)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.****/java.lang.Thread.run(Thread.java:1623)

@peternied
Copy link
Member

Reproduced on PR, gradle check log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yigithub yigithub

Labels
bug Something isn't working CI CI related >test-failure Test failure from CI, local build, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@yigithub @nknize @peternied @owaiskazi19 @ccook-ibm