supporting table type view?

[andaag]

Hi

Any plans to support table type view?

And if not, any information on how complex it would be & why it's not supported?

[ncclementi]

Hi @andaag can you tell us a bit more about what are table type views, maybe providing an example of what you would like to do? I tried to look examples in the documentation and didn't find much, but it looks like it's related to SQL. Currently, the read-support we provide is to read data into a dataframe. Is this something that is done to dataframes?

[bnaul]

@andaag this library reads data using the BigQuery Storage API which does not support views. So I don't really think this is possible unfortunately; the only option would be to materialize the view into an actual table by calling SELECT *, but since that can be very expensive I think it would be best to require the user to perform that step manually and then pass the resulting table into read_gbq.

[andaag]

We have multiple views stacked on top of each other (non materialized), and would like to sync some of these to other data sinks.

My hope was that it would be as easy at a select on the view, but if it's using the storage api I get that that's not trivial. Creating temporary buffers for it is what we do now - but it's not optimal.

I guess I could potentially use the dask jdbc driver + bigquery's jdbc driver to pull off something similar 🤔

[stanleyEIQ]

are materialized views supposed to be supported?

I saw they are a different type, MATERIALIZED_VIEW, from VIEW (Table types) so I gave it a shot. Unfortunately I get an error non-table entities cannot be read with the storage API. I guess when they say Table in the ReadSession params, they mean only Tables of TABLE type.

FWIW, I see this one SO post that says it "should" work if it is a simple materialized view using only SELECT/FROM/WHERE. They are using the BigQueryReadClient while the code is using ReadSession

For the time being, I am working around it by using expiring tables.