DbApiHook: add chunksize to get_pandas_df parameters

[wittfabian]

Description

The method "get_pandas_df" should be extended by the parameter "chunksize".

Use case / motivation

For the case that very large amounts of data are read, the "chunksize" is a good way to reduce the size of individual queries.

Links

airflow/airflow/hooks/dbapi_hook.py

Line 123 in f4faed6

return psql.read_sql(sql, con=conn, params=parameters)

https://github.com/pandas-dev/pandas/blob/0.25.x/pandas/io/sql.py#L344

[imorales-mosaico]

There are a problem with the chunksize, because the get_pandas_df close the connection in the first call of the iterator[DataFrame] returned

[carmoreno1]

There are a problem with the chunksize, because the get_pandas_df close the connection in the first call of the iterator[DataFrame] returned

Hi @imorales-mosaico is there a workaround about chunksize using Airflow?

[potiuk]

Please @imorales-mosaico - open a new Feature/Bug request with a bit better description, rather than commenting on a closed ticket (you can add Relates to: #8464 in the description).

[potiuk]

(Or Ideally submit a PR fixing the problem immediately - if there is a problem).