-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DbApiHook: Support kwargs in get_pandas_df #9730
Conversation
Static check failed: Added kwargs in BigQueryHook as well |
Looks good @zikun -> but there is a handful of other hooks that are defining this method and (hive/exasol) - and by a quick look they could also benefit from those kwargs. Would you mind adding them there as well ? |
912d118
to
fa7192e
Compare
@potiuk Added to all the overriding methods |
@@ -106,7 +106,7 @@ def get_sqlalchemy_engine(self, engine_kwargs=None): | |||
engine_kwargs = {} | |||
return create_engine(self.get_uri(), **engine_kwargs) | |||
|
|||
def get_pandas_df(self, sql, parameters=None): | |||
def get_pandas_df(self, sql, parameters=None, **kwargs): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind adding information to method docstring?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added
d9d23d8
to
e47eeb9
Compare
@turbaszek let's merge it :) |
* DbApiHook: Support kwargs in get_pandas_df * BigQueryHook: Support kwargs in get_pandas_df * PrestoHook: Support kwargs in get_pandas_df * HiveServer2Hook: Support kwargs in get_pandas_df (cherry picked from commit 8f8db89)
* DbApiHook: Support kwargs in get_pandas_df * BigQueryHook: Support kwargs in get_pandas_df * PrestoHook: Support kwargs in get_pandas_df * HiveServer2Hook: Support kwargs in get_pandas_df (cherry picked from commit 8f8db89)
* DbApiHook: Support kwargs in get_pandas_df * BigQueryHook: Support kwargs in get_pandas_df * PrestoHook: Support kwargs in get_pandas_df * HiveServer2Hook: Support kwargs in get_pandas_df (cherry picked from commit 8f8db89)
* DbApiHook: Support kwargs in get_pandas_df * BigQueryHook: Support kwargs in get_pandas_df * PrestoHook: Support kwargs in get_pandas_df * HiveServer2Hook: Support kwargs in get_pandas_df (cherry picked from commit 8f8db89)
* DbApiHook: Support kwargs in get_pandas_df * BigQueryHook: Support kwargs in get_pandas_df * PrestoHook: Support kwargs in get_pandas_df * HiveServer2Hook: Support kwargs in get_pandas_df (cherry picked from commit 8f8db89)
Support all parameters that are supported by pandas
read_sql
function: https://github.com/pandas-dev/pandas/blob/1.0.x/pandas/io/sql.py#L336-L345Closes #8468
Make sure to mark the boxes below before creating PR: [x]
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.