Skip to content

Commit

Permalink
[SPARK-32268][TESTS][FOLLOWUP] Fix BloomFilterAggregateQuerySuite f…
Browse files Browse the repository at this point in the history
…ailed in ansi mode

### What changes were proposed in this pull request?
`Test that might_contain errors out non-constant Bloom filter` in `BloomFilterAggregateQuerySuite ` failed in ansi mode due to `Numeric <=> Binary` is [not allowed in ansi mode](apache#30260),  so the content of  `exception.getMessage` is different from that of non-ans mode.

This pr change the case to ensure that the error messages of `ansi` mode and `non-ansi` are consistent.

### Why are the changes needed?
Bug fix.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

- Pass GA
- Local Test

**Before**

```
export SPARK_ANSI_SQL_MODE=false
mvn clean test -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.BloomFilterAggregateQuerySuite
```

```
Run completed in 23 seconds, 537 milliseconds.
Total number of tests run: 8
Suites: completed 2, aborted 0
Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```

```
export SPARK_ANSI_SQL_MODE=true
mvn clean test -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.BloomFilterAggregateQuerySuite
```

```
- Test that might_contain errors out non-constant Bloom filter *** FAILED ***
  "cannot resolve 'CAST(t.a AS BINARY)' due to data type mismatch:
   cannot cast bigint to binary with ANSI mode on.
   If you have to cast bigint to binary, you can set spark.sql.ansi.enabled as false.
  ; line 2 pos 21;
  'Project [unresolvedalias('might_contain(cast(a#2424L as binary), cast(5 as bigint)), None)]
  +- SubqueryAlias t
     +- LocalRelation [a#2424L]
  " did not contain "The Bloom filter binary input to might_contain should be either a constant value or a scalar subquery expression" (BloomFilterAggregateQuerySuite.scala:171)
```

**After**
```
export SPARK_ANSI_SQL_MODE=false
mvn clean test -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.BloomFilterAggregateQuerySuite
```

```
Run completed in 26 seconds, 544 milliseconds.
Total number of tests run: 8
Suites: completed 2, aborted 0
Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

```

```
export SPARK_ANSI_SQL_MODE=true
mvn clean test -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.BloomFilterAggregateQuerySuite
```

```
Run completed in 25 seconds, 289 milliseconds.
Total number of tests run: 8
Suites: completed 2, aborted 0
Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```

Closes apache#35953 from LuciferYang/SPARK-32268-FOLLOWUP.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Yuming Wang <[email protected]>
  • Loading branch information
LuciferYang authored and wangyum committed Mar 23, 2022
1 parent 39fc7ee commit 7165123
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ class BloomFilterAggregateQuerySuite extends QueryTest with SharedSparkSession {
val exception1 = intercept[AnalysisException] {
spark.sql("""
|SELECT might_contain(cast(a as binary), cast(5 as long))
|FROM values (cast(1 as long)), (cast(2 as long)) as t(a)"""
|FROM values (cast(1 as string)), (cast(2 as string)) as t(a)"""
.stripMargin)
}
assert(exception1.getMessage.contains(
Expand All @@ -175,7 +175,7 @@ class BloomFilterAggregateQuerySuite extends QueryTest with SharedSparkSession {
val exception2 = intercept[AnalysisException] {
spark.sql("""
|SELECT might_contain((select cast(a as binary)), cast(5 as long))
|FROM values (cast(1 as long)), (cast(2 as long)) as t(a)"""
|FROM values (cast(1 as string)), (cast(2 as string)) as t(a)"""
.stripMargin)
}
assert(exception2.getMessage.contains(
Expand Down

0 comments on commit 7165123

Please sign in to comment.