feat: switch to using datafusion to parse functions

[westonpace]

My original plan was to use DF for parsing expressions (not just functions). However, I ran into some issues:

• Support timestamp literals with precision specifier apache/datafusion#7249
• Datafusion does not support nested nested references (e.g. it supports x.y but not x.y.z)
• Array literals always appear as List and not FixedSizeList
  • Datafusion's array literals initially show up as "MakeArray" calls and so they are missed by our type coercion. They turn into literals later during expression simplification.
• Functions from duckdb (e.g. is_valid) are not recognized by datafusion
  • Long term this kind of engine-to-engine conversion is best handled by substrait
  • In the short term we could maybe register these functions as DF UDFs

So this PR only uses DF to parse the functions. In addition, we keep the legacy path to handle the is_valid function for DuckDb.

Closes #1115
Closes #1784

[github-actions]

ACTION NEEDED

Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[westonpace]

I've updated #1129 with the incompatibilities I observed this time.

[wjones127]

Datafusion does not support nested nested references (e.g. it supports x.y but not x.y.z)

What do you mean? can't you just nest GetIndexedField calls?

[westonpace]

What do you mean? can't you just nest GetIndexedField calls?

Expr supports it. SqlToExpr does not.

[wjones127]

Does this fix #1115?

[wjones127]

Looks great to me :)

[westonpace]

Does this fix #1115?

Looks like it. I've added a regression test for array_contains

[wjones127]

(Sorry just merged a PR that added some conflicts. Should be easy to resolve though.)