{"query_string":{"query":"_exists_:(foo bar)"}} default operator behavior is different between 2.4.4 and 6.2.3

[bczifra]

Elasticsearch version (bin/elasticsearch --version): 6.2.3

Plugins installed: x-pack

JVM version (java -version): 1.8.0_91

OS version (uname -a if on a Unix-like system): macOS 10.12.5

Description of the problem including expected versus actual behavior:
{"query_string":{"query":"_exists_:(foo bar)"}} used to match if a document had either field foo or field bar. Now it tries to match a field named foo bar instead.

Steps to reproduce:

POST exists/doc/1
{
  "foo": "bar"
}

#matches in 2.4.4, no match in 6.2.3
GET exists/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "_exists_:(foo bar)"
          }
        }
      ]
    }
  }
}

#matches in both
GET exists/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "_exists_:(foo OR bar)"
          }
        }
      ]
    }
  }
}

#matches in 2.4.4, no match in 6.2.3
GET exists/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "_exists_:(foo bar)",
            "default_operator": "OR"
          }
        }
      ]
    }
  }
}

It appears that this might have changed in 6.1.
Using the explain API indicates that this is how the query is treated across the various versions:

• 2.4: +(ConstantScore(ConstantScore(_field_names:foo)) ConstantScore(ConstantScore(_field_names:bar)))
• 5.x: +(ConstantScore(_field_names:foo) ConstantScore(_field_names:bar))
• 6.x: ConstantScore(DocValuesFieldExistsQuery [field=foo bar])

[elasticmachine]

Pinging @elastic/es-search-aggs

[jtibshirani]

Hi @bczifra, thanks for the very detailed issue description. I think there are two changes here that are causing the differences in the explain output, one that affects which results are returned, and another that shouldn't be relevant:

• In 6.0 there was a breaking change to make query_string queries no longer split on whitespace as if it were an operator (Make split_on_whitespace default to false #25470). So to produce the same behavior as before, an OR operator needs to be explicitly provided: "_exists_:(foo OR bar)". It’s worth noting that we ended up re-introducing the ability to split keyword fields on whitespace, as this was a useful option (Add an option to split keyword field on whitespace at query time #30691).
• As you mentioned, since 6.1 we default to using doc values for exists queries when they're available (Enhances exists queries to reduce need for _field_names #26930). I don’t think this should affect the semantics of exists queries at all.

I hope that helps explain why the difference is occurring. It would be useful to know: did you raise this issue because it could have potentially been a bug, or do you have a use case where the new behavior is hurting usability?

[javanna]

ping @bczifra can you answer the question above? Are you ok with the given explanation?

[cbuescher]

Checked with @bczifra that it is okay to close here. He will reopen if there are any follow-up questions.