Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid creating ExecutionPlan for registered tables in Ballista #1016

Closed
rdettai opened this issue Sep 17, 2021 · 0 comments · Fixed by #1028
Closed

Avoid creating ExecutionPlan for registered tables in Ballista #1016

rdettai opened this issue Sep 17, 2021 · 0 comments · Fixed by #1028
Labels
enhancement New feature or request

Comments

@rdettai
Copy link
Contributor

rdettai commented Sep 17, 2021

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
When using the sql() method on the BallistaContext, all the tables registered on the context are transformed into execution plans. We should find a way to avoid that, because scanning datasources can be an expensive operation.

Describe the solution you'd like
I would like a way to register the tables on the Datafusion context without transforming them to ExecutionPlans.

Additional context
#1012 makes the scan operation async. This propagated to the BallistaContext.sql(), which was not expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
1 participant