ORC-1059: Align findColumns behaviour between 1.6 and 1.7 release

[pgaref]

What changes were proposed in this pull request?

When trying to bump Apache Hive to 1.7 I noticed query failures related to SArgs pushdown.
In more detail, Hive may push Partition column filters that are not part of the columns, until 1.6 recently these columns were ignored, while 1.7 throws IllegalArgumentException. ORC-741 introduced this behavior change in Apache ORC 1.7.0.

Why are the changes needed?

Align findColumns behaviour between 1.6 and 1.7 release

How was this patch tested?

Existing tests

[dongjoon-hyun]

+1, LGTM. Thank you, @pgaref .

[dongjoon-hyun]

I cherry-picked this to branch-1.7 for Apache ORC 1.7.2 release.

[dongjoon-hyun]

cc @pavibhai and @omalley

[pgaref]

Thanks for the quick review @dongjoon-hyun !
This will definitely help us bump to 1.7 quickly (e.g., next 1.7 release) but I was thinking that throwing an exception when a SArg column is not found is probably a better approach than just logging.

I suggest we keep this in mind for 1.8 (and switch back to that behaviour) where we could align downstream consumers like Hive to only push SArgs for columns that are part of the schema.

[dongjoon-hyun]

Is it okay when we handle multiple ORC files in Hive schema evolutions?

I was thinking that throwing an exception when a SArg column is not found is probably a better approach than just logging.

In many cases, Hive partitions might have different schemas. The simplest case is having new columns additionally in new partitions. If a user run a query for all partitions, SArg columns can have new columns which old partitions don't have.

Apache Spark checks the physical schema when we open a file and try to adjust the missing columns. Given this PR's description, Apache Hive doesn't, right?

For me, throwing an exception could be too intrusive and the AS-IS status (-1) would be enough.

[dongjoon-hyun]

BTW, let me try to test the status of branch-1.7 and roll a new Apache ORC release for Apache Hive, @pgaref .