Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-6057] Update template_fields of the PythonSensor #6656

Merged
merged 1 commit into from
Dec 29, 2019

Conversation

mustafagok
Copy link
Contributor

Make sure you have checked all steps below.

Jira

  • My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
    • https://issues.apache.org/jira/browse/AIRFLOW-6057
    • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.
    • In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal (AIP).
    • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Description

  • Here are some details about my PR, including screenshots of any UI changes:
    Allow passing Jinja templated arguments to the python_callable of the PythonSensor (just like in PythonOperator) by adding op_args and op_kwargs to the template_fields of the PythonSensor.
    Also fix the test_python_callable_keyword_arguments_are_templatized test of the PythonOperator.

Tests

  • My PR adds the following unit tests:
    • test_python_callable_arguments_are_templatized for op_args,
    • test_python_callable_keyword_arguments_are_templatized for op_kwargs

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

@codecov-io
Copy link

codecov-io commented Nov 24, 2019

Codecov Report

Merging #6656 into master will decrease coverage by 0.28%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6656      +/-   ##
==========================================
- Coverage    84.7%   84.42%   -0.29%     
==========================================
  Files         680      680              
  Lines       38556    38556              
==========================================
- Hits        32659    32549     -110     
- Misses       5897     6007     +110
Impacted Files Coverage Δ
airflow/contrib/sensors/python_sensor.py 100% <100%> (ø) ⬆️
airflow/kubernetes/volume_mount.py 44.44% <0%> (-55.56%) ⬇️
airflow/kubernetes/volume.py 52.94% <0%> (-47.06%) ⬇️
airflow/kubernetes/pod_launcher.py 45.25% <0%> (-46.72%) ⬇️
airflow/kubernetes/refresh_config.py 50.98% <0%> (-23.53%) ⬇️
...rflow/contrib/operators/kubernetes_pod_operator.py 78.75% <0%> (-20%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf29ef3...9da4568. Read the comment docs.

Add op_args and op_kwargs to the template_fields of the PythonSensor.

Update _assert_calls_equal to fix
TestPythonOperator.test_python_callable_keyword_arguments_are_templatized
method which was not working since _assert_calls_equal was not check
for kwargs.

Create TestPythonBase class which will be used as a
base test class for TestPythonOperator and TestPythonSensor classes.
It will be used to prevent code duplication.

Add unit tests to TestPythonSensor
These tests will ensure that template_fields is working:
test_python_callable_arguments_are_templatized for op_args,
test_python_callable_keyword_arguments_are_templatized for op_kwargs.
Use TestPythonBase class instead of unittest.TestCase to prevent code
duplication.
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice !

@potiuk potiuk merged commit 6614ea7 into apache:master Dec 29, 2019
potiuk pushed a commit that referenced this pull request Dec 30, 2019
Add op_args and op_kwargs to the template_fields of the PythonSensor.

(cherry picked from commit 6614ea7)
@mustafagok mustafagok deleted the AIRFLOW-6057 branch January 4, 2020 19:05
potiuk pushed a commit that referenced this pull request Jan 21, 2020
Add op_args and op_kwargs to the template_fields of the PythonSensor.

(cherry picked from commit 6614ea7)
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
Add op_args and op_kwargs to the template_fields of the PythonSensor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants