You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We want to explore ibis support. There are to things we can probably do:
allow the user to get ibis in memory tables (or iterate over them) the same way arrow tables and dataframes work in the current implementation. this is very easy because we just convert arrow tables into ibis in memory tables and provide two new methods on a readablerelation. this will work for all destinations supported by our dataset
the "real" integration would be to hand over credentials from a destination to the ibis connection. The user will the get an ibis object which is a full replacement of our dataset where they can join tables and even write back to the database or create new tables and stuff like that, all the regular ibis operations. This will work for a subset of our destinations, you can see the list of ibis backends in the backends dropdown here: https://ibis-project.org/. This should also works with our duckdb filesystem thing, excluding the write stuff of course which the user would have to do with the pond or a dlt pipeline to commit the computed data somewhere.
The text was updated successfully, but these errors were encountered:
We want to explore ibis support. There are to things we can probably do:
allow the user to get ibis in memory tables (or iterate over them) the same way arrow tables and dataframes work in the current implementation. this is very easy because we just convert arrow tables into ibis in memory tables and provide two new methods on a readablerelation. this will work for all destinations supported by our dataset
the "real" integration would be to hand over credentials from a destination to the ibis connection. The user will the get an ibis object which is a full replacement of our dataset where they can join tables and even write back to the database or create new tables and stuff like that, all the regular ibis operations. This will work for a subset of our destinations, you can see the list of ibis backends in the backends dropdown here: https://ibis-project.org/. This should also works with our duckdb filesystem thing, excluding the write stuff of course which the user would have to do with the pond or a dlt pipeline to commit the computed data somewhere.
The text was updated successfully, but these errors were encountered: