Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize tutorials into a section #25890

Merged
merged 12 commits into from
Aug 27, 2022
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ repos:
exclude: ^airflow/_vendor/|^RELEASE_NOTES\.txt$|^airflow/www/static/css/material-icons\.css$|^images/.*$
args:
- --ignore-words=docs/spelling_wordlist.txt
- --skip=docs/*/commits.rst,airflow/providers/*/*.rst,*.lock,INTHEWILD.md,*.min.js,docs/apache-airflow/pipeline_example.csv,airflow/www/*.log
- --skip=docs/*/commits.rst,airflow/providers/*/*.rst,*.lock,INTHEWILD.md,*.min.js,docs/apache-airflow/tutorial/pipeline_example.csv,airflow/www/*.log
- --exclude-file=.codespellignorelines
- repo: local
hooks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@


"""
### ETL DAG Tutorial Documentation
This ETL DAG is demonstrating an Extract -> Transform -> Load pipeline
### DAG Tutorial Documentation
This DAG is demonstrating an Extract -> Transform -> Load pipeline
"""
# [START tutorial]
# [START import_module]
Expand All @@ -38,13 +38,13 @@

# [START instantiate_dag]
with DAG(
'tutorial_etl_dag',
'tutorial_dag',
# [START default_args]
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args={'retries': 2},
# [END default_args]
description='ETL DAG tutorial',
description='DAG tutorial',
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@
catchup=False,
tags=['example'],
)
def tutorial_taskflow_api_etl():
def tutorial_taskflow_api():
"""
### TaskFlow API Tutorial Documentation
This is a simple ETL data pipeline example which demonstrates the use of
This is a simple data pipeline example which demonstrates the use of
the TaskFlow API using three simple tasks for Extract, Transform, and Load.
Documentation that goes along with the Airflow TaskFlow API tutorial is
located
Expand Down Expand Up @@ -100,7 +100,7 @@ def load(total_order_value: float):


# [START dag_invocation]
tutorial_etl_dag = tutorial_taskflow_api_etl()
tutorial_dag = tutorial_taskflow_api()
# [END dag_invocation]

# [END tutorial]
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,14 @@
log = logging.getLogger(__name__)

if not shutil.which("virtualenv"):
log.warning(
"The tutorial_taskflow_api_etl_virtualenv example DAG requires virtualenv, please install it."
)
log.warning("The tutorial_taskflow_api_virtualenv example DAG requires virtualenv, please install it.")
else:

@dag(schedule=None, start_date=datetime(2021, 1, 1), catchup=False, tags=['example'])
def tutorial_taskflow_api_etl_virtualenv():
def tutorial_taskflow_api_virtualenv():
"""
### TaskFlow API example using virtualenv
This is a simple ETL data pipeline example which demonstrates the use of
This is a simple data pipeline example which demonstrates the use of
the TaskFlow API using three simple tasks for Extract, Transform, and Load.
"""

Expand Down Expand Up @@ -86,4 +84,4 @@ def load(total_order_value: float):
order_summary = transform(order_data)
load(order_summary["total_order_value"])

tutorial_etl_dag = tutorial_taskflow_api_etl_virtualenv()
tutorial_dag = tutorial_taskflow_api_virtualenv()
2 changes: 1 addition & 1 deletion docs/apache-airflow/concepts/taskflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ You can also use a plain value or variable to call a TaskFlow function - for exa

hello_name('Airflow users')

If you want to learn more about using TaskFlow, you should consult :doc:`the TaskFlow tutorial </tutorial_taskflow_api>`.
If you want to learn more about using TaskFlow, you should consult :doc:`the TaskFlow tutorial </tutorial/taskflow>`.

Context
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/apache-airflow/howto/docker-compose/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ The Docker Compose file uses the latest Airflow image (`apache/airflow <https://
What's Next?
============

From this point, you can head to the :doc:`/tutorial` section for further examples or the :doc:`/howto/index` section if you're ready to get your hands dirty.
From this point, you can head to the :doc:`/tutorial/index` section for further examples or the :doc:`/howto/index` section if you're ready to get your hands dirty.

.. _docker-compose-env-variables:

Expand Down
Binary file removed docs/apache-airflow/img/new_tutorial-1.png
Binary file not shown.
Binary file removed docs/apache-airflow/img/new_tutorial-2.png
Binary file not shown.
Binary file removed docs/apache-airflow/img/new_tutorial-3.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 1 addition & 2 deletions docs/apache-airflow/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,7 @@ so coding will always be required.
start
installation/index
upgrading-from-1-10/index
tutorial
tutorial_taskflow_api
tutorial/index
howto/index
ui
concepts/index
Expand Down
4 changes: 4 additions & 0 deletions docs/apache-airflow/redirects.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,7 @@ upgrading-to-2.rst upgrading-from-1-10/index.rst

# Release Notes
changelog.rst release_notes.rst

# Tutorials
tutorial.rst tutorial/index.rst
tutorial_taskflow_api.rst tutorial/taskflow.rst
2 changes: 1 addition & 1 deletion docs/apache-airflow/start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -115,4 +115,4 @@ the all-in-one ``standalone`` command, you can instead run:

What's Next?
''''''''''''
From this point, you can head to the :doc:`/tutorial` section for further examples or the :doc:`/howto/index` section if you're ready to get your hands dirty.
From this point, you can head to the :doc:`/tutorial/index` section for further examples or the :doc:`/howto/index` section if you're ready to get your hands dirty.
Loading