Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support array literal with scalar function #8884

Merged
merged 3 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 7 additions & 33 deletions datafusion/sql/src/expr/value.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ use datafusion_common::{
};
use datafusion_expr::expr::ScalarFunction;
use datafusion_expr::expr::{BinaryExpr, Placeholder};
use datafusion_expr::BuiltinScalarFunction;
use datafusion_expr::{lit, Expr, Operator};
use datafusion_expr::{BuiltinScalarFunction, ScalarFunctionDefinition};
use log::debug;
use sqlparser::ast::{BinaryOperator, Expr as SQLExpr, Interval, Value};
use sqlparser::parser::ParserError::ParserError;
Expand Down Expand Up @@ -135,38 +135,12 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
elements: Vec<SQLExpr>,
schema: &DFSchema,
) -> Result<Expr> {
let mut values = Vec::with_capacity(elements.len());

for element in elements {
let value = self.sql_expr_to_logical_expr(
element,
schema,
&mut PlannerContext::new(),
)?;

match value {
Expr::Literal(_) => {
values.push(value);
}
Expr::ScalarFunction(ScalarFunction {
func_def: ScalarFunctionDefinition::BuiltIn(fun),
..
}) => {
if fun == BuiltinScalarFunction::MakeArray {
values.push(value);
} else {
return not_impl_err!(
"ScalarFunctions without MakeArray are not supported: {value}"
);
}
}
_ => {
return not_impl_err!(
"Arrays with elements other than literal are not supported: {value}"
);
}
}
}
let values = elements
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it -- less code and more functionality ❤️ !

.into_iter()
.map(|element| {
self.sql_expr_to_logical_expr(element, schema, &mut PlannerContext::new())
})
.collect::<Result<Vec<_>>>()?;

Ok(Expr::ScalarFunction(ScalarFunction::new(
BuiltinScalarFunction::MakeArray,
Expand Down
26 changes: 24 additions & 2 deletions datafusion/sqllogictest/test_files/array.slt
Original file line number Diff line number Diff line change
Expand Up @@ -456,11 +456,33 @@ AS
FROM nested_arrays_with_repeating_elements
;

# Array literal

## boolean coercion is not supported
query error
select [1, true, null]

query error DataFusion error: This feature is not implemented: ScalarFunctions without MakeArray are not supported: now()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add more to covert Struct?

SELECT [now()]
## wrapped in array_length to get deterministic results
query I
SELECT array_length([now()])
----
1

## array literal with functions
query ?
select [abs(-1.2), sin(-1), log(2), ceil(3.141)]
----
[1.2, -0.8414709848078965, 0.3010299801826477, 4.0]

## array literal with nested types
query ???
select
[struct('foo', 1)],
[struct('foo', [1,2,3])],
[struct('foo', [struct(3, 'x')])]
;
----
[{c0: foo, c1: 1}] [{c0: foo, c1: [1, 2, 3]}] [{c0: foo, c1: [{c0: 3, c1: x}]}]

query TTT
select arrow_typeof(column1), arrow_typeof(column2), arrow_typeof(column3) from arrays;
Expand Down