Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add note about using dag_run.conf in BashOperator #9143

Merged
merged 3 commits into from
Jun 8, 2020

Conversation

ashb
Copy link
Member

@ashb ashb commented Jun 4, 2020


Make sure to mark the boxes below before creating PR: [x]

  • Description above provides context of the change
  • Unit tests coverage for changes (not needed for documentation changes)
  • Target Github ISSUE in description if exists
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

@ashb ashb requested a review from kaxil June 4, 2020 14:00
@ashb ashb force-pushed the update-example-trigger-dag branch from 08d2452 to d10f545 Compare June 4, 2020 14:43
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc building error

==================== Error   2 ====================
 WARNING: Could not lex literal_block as "python". Highlighting skipped.

File path: /opt/airflow/docs/howto/operator/bash.rst (67)

  63 | 
  64 | Instead, you should pass this via the ``env`` kwarg and use double-quotes
  65 | inside the bash_command, as below:
  66 | 
  67 | .. code-block:: python
  68 | 
  69 |     bash_task = BashOperator(
  70 |         task_id="bash_task",
  71 |         bash_command='echo "here is the message: \\'$message\\'"',
  72 |         env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
==================================================

@ashb
Copy link
Member Author

ashb commented Jun 5, 2020

Hmmmm, I wonder what is going on between sphinx and pygmentize:

airflow ❯ pygmentize -l python -F raiseonerror < test-lex.py
    bash_task = BashOperator(
        task_id="bash_task",
        bash_command='echo "here is the message: \'$message\'"',
        env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},

works fine and doesn't error.

@ashb
Copy link
Member Author

ashb commented Jun 5, 2020

(I tried with \ first, that failed, and with \\ too wondering if sphinx was treating \\ as something. That failed too)

@ashb ashb merged commit 4d8599e into apache:master Jun 8, 2020
@ashb ashb deleted the update-example-trigger-dag branch June 8, 2020 08:33
@ashb ashb added this to the Airflow 1.10.11 milestone Jun 8, 2020
ashb added a commit that referenced this pull request Jun 8, 2020
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
(cherry picked from commit 2591294)
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
(cherry picked from commit 2591294)
@kaxil kaxil added the type:doc label Jul 1, 2020
kaxil pushed a commit that referenced this pull request Jul 1, 2020
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants