Skip to content

Commit

Permalink
Add note about using dag_run.conf in BashOperator (apache#9143)
Browse files Browse the repository at this point in the history
  • Loading branch information
ashb authored Jun 8, 2020
1 parent ef6070c commit 4d8599e
Show file tree
Hide file tree
Showing 3 changed files with 65 additions and 2 deletions.
3 changes: 2 additions & 1 deletion airflow/example_dags/example_trigger_target_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ def run_this_func(**context):

bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"',
bash_command='echo "Here is the message: $message"',
env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
dag=dag,
)
33 changes: 32 additions & 1 deletion airflow/operators/bash.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@


class BashOperator(BaseOperator):
"""
r"""
Execute a Bash script, command or set of commands.
.. seealso::
Expand Down Expand Up @@ -61,6 +61,37 @@ class BashOperator(BaseOperator):
.. code-block:: python
bash_command = "set -e; python3 script.py '{{ next_execution_date }}'"
.. warning::
Care should be taken with "user" input or when using Jinja templates in the
``bash_command``, as this bash operator does not perform any escaping or
sanitization of the command.
This applies mostly to using "dag_run" conf, as that can be submitted via
users in the Web UI. Most of the default template variables are not at
risk.
For example, do **not** do this:
.. code-block:: python
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"',
)
Instead, you should pass this via the ``env`` kwarg and use double-quotes
inside the bash_command, as below:
.. code-block:: python
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "here is the message: \'$message\'"',
env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
)
"""
template_fields = ('bash_command', 'env')
template_ext = ('.sh', '.bash',)
Expand Down
31 changes: 31 additions & 0 deletions docs/howto/operator/bash.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,37 @@ You can use :ref:`Jinja templates <jinja-templating>` to parameterize the
:start-after: [START howto_operator_bash_template]
:end-before: [END howto_operator_bash_template]


.. warning::

Care should be taken with "user" input or when using Jinja templates in the
``bash_command``, as this bash operator does not perform any escaping or
sanitization of the command.

This applies mostly to using "dag_run" conf, as that can be submitted via
users in the Web UI. Most of the default template variables are not at
risk.

For example, do **not** do this:

.. code-block:: python
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"',
)
Instead, you should pass this via the ``env`` kwarg and use double-quotes
inside the bash_command, as below:

.. code-block:: python
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "here is the message: \'$message\'"',
env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
)
Troubleshooting
---------------

Expand Down

0 comments on commit 4d8599e

Please sign in to comment.