Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port tests in select.rs to sqllogictest #8213

Closed
Tracked by #6195
alamb opened this issue Nov 15, 2023 · 12 comments · Fixed by #8967
Closed
Tracked by #6195

Port tests in select.rs to sqllogictest #8213

alamb opened this issue Nov 15, 2023 · 12 comments · Fixed by #8967
Assignees
Labels
good first issue Good for newcomers

Comments

@alamb
Copy link
Contributor

alamb commented Nov 15, 2023

Is your feature request related to a problem or challenge?

Part of #6195. We are trying to

  1. keep the DataFusion code base clean and easy to contribute to and modify
  2. Understand what features are covered and what are not

Part of doing so is having a single location for most test coverage so new tests can be added easily and people can follow the existing patterns easily

Describe the solution you'd like

Port the tests in https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/tests/sql/select.rs to
select.slt in https://github.com/apache/arrow-datafusion/tree/main/datafusion/sqllogictest/test_files

Note you can create catalogs and schema via SQL now

Catalogs: https://arrow.apache.org/datafusion/user-guide/sql/ddl.html#create-database
Schema: https://arrow.apache.org/datafusion/user-guide/sql/ddl.html#create-schema

Notes:

  1. Instructions for running sqllogitest are here: https://github.com/apache/arrow-datafusion/tree/main/datafusion/sqllogictest

Describe alternatives you've considered

No response

Additional context

I think these are good first issues as they teach the contributor about the DataFusion codebase and tests, as well as see the end user apis in practice.

@fansehep
Copy link

i want to try this. 😄

@alamb
Copy link
Contributor Author

alamb commented Jan 21, 2024

This one I think is still a good first issue / maybe even covered by existing tests

@Tangruilin
Copy link
Contributor

@alamb maybe this can assign to me

@Tangruilin
Copy link
Contributor

I need some help,

https://github.com/apache/arrow-datafusion/blob/31b9b48b08592b7d293f46e75707aad7dadd7cbc/datafusion/core/tests/sql/select.rs#L121

Here it create a column with struct, but i do not know how to create it by sql

@Tangruilin
Copy link
Contributor

I need some help,
https://github.com/apache/arrow-datafusion/blob/31b9b48b08592b7d293f46e75707aad7dadd7cbc/datafusion/core/tests/sql/select.rs#L121

Here it create a column with struct, but i do not know how to create it by sql

@alamb

Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 23, 2024
@alamb
Copy link
Contributor Author

alamb commented Jan 23, 2024

I need some help,
https://github.com/apache/arrow-datafusion/blob/31b9b48b08592b7d293f46e75707aad7dadd7cbc/datafusion/core/tests/sql/select.rs#L121

Here it create a column with struct, but i do not know how to create it by sql

I think you can use the struct function:
https://arrow.apache.org/datafusion/user-guide/sql/scalar_functions.html#struct-functions

Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 24, 2024
Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 24, 2024
Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 24, 2024
@simicd
Copy link
Contributor

simicd commented Jan 24, 2024

Hi both - @Tangruilin seems like we we're working independently on the topic 😮 I had raised two PRs previously but seems they remained unnoticed. Guess we should try to consolidate into a single PR, feel free to take over whatever you're missing currently from #8953 and #8939 and integrate into yours if that's ok with you as well.

In #8939 you will find how I implemented the structs.

image

@Tangruilin
Copy link
Contributor

Guess we should try to consolidate into a single PR,

I have already sloved the problem of struct in my CR.
It seems my CR have sloved this issue.

Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 25, 2024
@Tangruilin
Copy link
Contributor

Hi both - @Tangruilin seems like we we're working independently on the topic 😮 I had raised two PRs previously but seems they remained unnoticed. Guess we should try to consolidate into a single PR, feel free to take over whatever you're missing currently from #8953 and #8939 and integrate into yours if that's ok with you as well.

In #8939 you will find how I implemented the structs.

image

Maybe We can use my CR?
Or do you have some suggestions?

Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 25, 2024
Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 25, 2024
Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 25, 2024
Tangruilin added a commit to Tangruilin/arrow-datafusion that referenced this issue Jan 25, 2024
@simicd
Copy link
Contributor

simicd commented Jan 25, 2024

Maybe We can use my CR?
Or do you have some suggestions?

Yes sounds good!

@alamb
Copy link
Contributor Author

alamb commented Jan 25, 2024

Yes, sorry -- I am backed up on reviews. I am hoping other committers can find more time to help. I am working to try and get the backlog down

alamb added a commit that referenced this issue Jan 25, 2024
* [task #8213]Part of  Port tests in select.rs to sqllogictest

Signed-off-by: tangruilin <[email protected]>

* test large strings as well

* Restore parameter tests

* make test deterministic

---------

Signed-off-by: tangruilin <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
4 participants