Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook #5671

Merged
merged 3 commits into from
Sep 6, 2019
Merged

[AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook #5671

merged 3 commits into from
Sep 6, 2019

Conversation

DPGrev
Copy link
Contributor

@DPGrev DPGrev commented Jul 27, 2019 •
edited
Loading

Make sure you have checked all steps below.

Jira

  • My PR addresses the following Airflow Jira issues and references them in the PR title.

https://issues.apache.org/jira/browse/AIRFLOW-5049

Description

  • Here are some details about my PR, including screenshots of any UI changes:

Wrong entries for the src_fmt_configs in the bigquery hook are silently ignored. This PR adds validation of these entries and throws an error when incorrect.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

A test was added for _validate_src_fmt_configs

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

@DPGrev
AIRFLOW-5049 Add validation for src_fmt_configs in bigquery hook
b0c4825 
Adds validation for the src_fmt_configs arguments in the bigquery hook. Otherwise wrong src_fmt_configs would be silently ignored which is non-desireable.
@DPGrev DPGrev changed the title AIRFLOW-5049 Add validation for src_fmt_configs in bigquery hook [AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook Jul 27, 2019
@mik-laj mik-laj added the provider:google Google (including GCP) related issues label Jul 31, 2019
@mik-laj
Copy link
Member

mik-laj commented Aug 12, 2019

Are you planning to continue working on this change? This week, I would like to deal with PR reviews related to GCP. I would be happy if you would respond to all comments

@DPGrev
Copy link
Contributor Author

DPGrev commented Aug 13, 2019

I will be picking this up later this week.

@BasPH BasPH merged commit dd36d90 into apache:master Sep 6, 2019
kaxil pushed a commit that referenced this pull request Dec 14, 2019
@DPGrev @kaxil
[AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook (#…
e627948 
…5671)

* AIRFLOW-5049 Add validation for src_fmt_configs in bigquery hook

Adds validation for the src_fmt_configs arguments in the bigquery hook. Otherwise wrong src_fmt_configs would be silently ignored which is non-desireable.

* [AIRFLOW-5049] Update - Add validation for src_fmt_configs in bigquery hook

Adds a common method for validating the src_ftm_configs

(cherry picked from commit dd36d90)
kaxil pushed a commit that referenced this pull request Dec 14, 2019
@DPGrev @kaxil
[AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook (#…
1ca9d1a 
…5671)

* AIRFLOW-5049 Add validation for src_fmt_configs in bigquery hook

Adds validation for the src_fmt_configs arguments in the bigquery hook. Otherwise wrong src_fmt_configs would be silently ignored which is non-desireable.

* [AIRFLOW-5049] Update - Add validation for src_fmt_configs in bigquery hook

Adds a common method for validating the src_ftm_configs

(cherry picked from commit dd36d90)
kaxil pushed a commit that referenced this pull request Dec 17, 2019
@DPGrev @kaxil
[AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook (#…
18ce7eb 
…5671)

* AIRFLOW-5049 Add validation for src_fmt_configs in bigquery hook

Adds validation for the src_fmt_configs arguments in the bigquery hook. Otherwise wrong src_fmt_configs would be silently ignored which is non-desireable.

* [AIRFLOW-5049] Update - Add validation for src_fmt_configs in bigquery hook

Adds a common method for validating the src_ftm_configs

(cherry picked from commit dd36d90)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk requested changes

@BasPH BasPH BasPH approved these changes

Assignees
No one assigned
Labels
provider:google Google (including GCP) related issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@DPGrev @mik-laj @potiuk @BasPH