Add documentation for Apache Airflow Helm Chart 1.14.0 (#1025)

docs-archive/helm-chart/1.14.0/.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 056258ab2cdd715d555b535014a5087c
+tags: 645f666f9bcd5a90fca523b33c5a78b7

docs-archive/helm-chart/1.14.0/_sources/adding-connections-and-variables.rst.txt

@@ -0,0 +1,85 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+
+Adding Connections, Variables and Environment Variables
+=======================================================
+
+You can programmatically add Connections, Variables and arbitrary Environment Variables to your
+Airflow deployment using the Helm chart.
+
+
+Connections and Sensitive Environment Variables
+-----------------------------------------------
+Under the ``secret`` and ``extraSecret`` sections of the ``values.yaml`` you can pass connection strings and sensitive
+environment variables into Airflow using the Helm chart. To illustrate, lets create a yaml file called ``override.yaml``
+to override values under these sections of the ``values.yaml`` file.
+
+.. code-block:: yaml
+
+   # override.yaml
+
+   secret:
+     - envName: "AIRFLOW_CONN_GCP"
+       secretName: "my-airflow-connections"
+       secretKey: "AIRFLOW_CONN_GCP"
+     - envName: "my-env"
+       secretName: "my-secret-name"
+       secretKey: "my-secret-key"
+
+   extraSecrets:
+     my-airflow-connections:
+       data: |
+         AIRFLOW_CONN_GCP: 'base64_encoded_gcp_conn_string'
+     my-secret-name:
+       stringData: |
+         my-secret-key: my-secret
+
+
+Variables
+---------
+Airflow supports Variables which enable users to craft dynamic DAGs. You can set Variables in Airflow in three ways - UI,
+command line, and within your DAG file. See :doc:`apache-airflow:howto/variable` for more.
+
+With the Helm chart, you can also inject environment variables into Airflow. So in the example ``override.yaml`` file,
+we can override values of interest in the ``env`` section of the ``values.yaml`` file.
+
+.. code-block:: yaml
+
+   env:
+     - name: "AIRFLOW_VAR_KEY"
+       value: "value_1"
+     - name: "AIRFLOW_VAR_ANOTHER_KEY"
+       value: "value_2"
+
+
+You can also utilize ``extraEnv`` and ``extraEnvFrom`` if you need the name or value to be templated.
+
+.. code-block:: yaml
+
+   extraEnv: |
+     - name: AIRFLOW_VAR_HELM_RELEASE_NAME
+       value: '{{ .Release.Name }}'
+
+   extraEnvFrom: |
+     - configMapRef:
+         name: '{{ .Release.Name }}-airflow-variables'
+
+   extraConfigMaps:
+     '{{ .Release.Name }}-airflow-variables':
+       data: |
+         AIRFLOW_VAR_HELLO_MESSAGE: "Hi!"

docs-archive/helm-chart/1.14.0/_sources/airflow-configuration.rst.txt

@@ -0,0 +1,41 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Configuring Airflow
+-------------------
+
+The chart allows for setting arbitrary Airflow configuration in values under the ``config`` key.
+Some of the defaults in the chart differ from those of core Airflow and can be found in
+`values.yaml <https://github.com/apache/airflow/blob/main/chart/values.yaml>`__.
+
+As an example of setting arbitrary configuration, the following yaml demonstrates how one would
+allow webserver users to view the config from within the UI:
+
+.. code-block:: yaml
+
+   config:
+     webserver:
+       expose_config: 'True'  # by default this is 'False'
+
+Generally speaking, it is useful to familiarize oneself with the Airflow
+configuration prior to installing and deploying the service.
+
+.. note::
+
+  The recommended way to load example DAGs using the official Docker image and chart is to configure the ``AIRFLOW__CORE__LOAD_EXAMPLES`` environment variable
+  in ``extraEnv`` (see :doc:`Parameters reference <parameters-ref>`). The official Docker image has ``AIRFLOW__CORE__LOAD_EXAMPLES=False``
+  set within the image, so you need to override it with an environment variable when deploying the chart in order for the examples to be present.

docs-archive/helm-chart/1.14.0/_sources/customizing-workers.rst.txt

@@ -0,0 +1,68 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Customizing Workers
+===================
+
+Both ``CeleryExecutor`` and ``KubernetesExecutor`` workers can be highly customized with the :ref:`workers parameters <parameters:workers>`.
+For example, to set resources on workers:
+
+.. code-block:: yaml
+
+  workers:
+    resources:
+      requests:
+        cpu: 1
+      limits:
+        cpu: 1
+
+See :ref:`workers parameters <parameters:workers>` for a complete list.
+
+One notable exception for ``KubernetesExecutor`` is that the default anti-affinity applied to ``CeleryExecutor`` workers to spread them across nodes
+is not applied to ``KubernetesExecutor`` workers, as there is no reason to spread out per-task workers.
+
+Custom ``pod_template_file``
+----------------------------
+
+With ``KubernetesExecutor`` or ``CeleryKubernetesExecutor`` you can also provide a complete ``pod_template_file`` to configure Kubernetes workers.
+This may be useful if you need different configuration between worker types for ``CeleryKubernetesExecutor``
+or if you need to customize something not possible with :ref:`workers parameters <parameters:workers>` alone.
+
+As an example, let's say you want to set ``priorityClassName`` on your workers:
+
+.. note::
+
+  The following example is NOT functional, but meant to be illustrative of how you can provide a custom ``pod_template_file``.
+  You're better off starting with the `default pod_template_file`_ instead.
+
+.. _default pod_template_file: https://github.com/apache/airflow/blob/main/chart/files/pod-template-file.kubernetes-helm-yaml
+
+.. code-block:: yaml
+
+  podTemplate: |
+    apiVersion: v1
+    kind: Pod
+    metadata:
+      name: placeholder-name
+      labels:
+        tier: airflow
+        component: worker
+        release: {{ .Release.Name }}
+    spec:
+      priorityClassName: high-priority
+      containers:
+        - name: base

docs-archive/helm-chart/1.14.0/_sources/extending-the-chart.rst.txt

@@ -0,0 +1,124 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Extending the Chart
+-------------------
+
+In some cases, you will want to deploy your custom templates (e.g. maintenance CronJobs you want to add)
+together with the Airflow chart installation.
+However, sometimes those templates are not directly related to the Airflow chart,
+thus should not be added to the chart.
+
+Instead, you can easily extend the chart and create a custom chart with your custom templates that
+depends on the Airflow chart.
+When you'll install your custom chart, the Airflow chart will also be installed.
+
+You can extend the official Airflow chart by applying the following steps.
+
+Create your custom Chart
+'''''''''''''''''''''''''
+
+First, you will need to create you own chart directory. You can do it by running the following command:
+
+.. code-block:: bash
+
+    helm create my-custom-chart
+
+
+This command will create a directory called ``my-custom-chart`` with the following structure:
+
+.. code-block::
+
+    my-custom-chart/
+    ├── .helmignore
+    ├── Chart.yaml
+    ├── values.yaml
+    ├── charts/
+    └── templates/
+        └── tests/
+
+Add Airflow chart as dependency
+'''''''''''''''''''''''''''''''
+
+Second, you will need to add the Airflow chart as dependency to your chart.
+This will give you the ability to add your custom templates without the need to modify the Airflow chart itself.
+In order to add the Airflow chart as a dependency (often called ``subcharts``) to your chart,
+add the following lines to your ``Chart.yaml`` file:
+
+.. code-block::
+
+    dependencies:
+      - name: airflow
+        version: 1.11.0
+        repository: https://airflow.apache.org
+
+.. note::
+
+    Make sure you have already added the Airflow repo locally by running: ``helm repo add apache-airflow https://airflow.apache.org``.
+
+.. tip::
+
+    You can also use the name of the repo instead of the URL by replacing
+    ``https://airflow.apache.org`` with ``"@apache-airflow"``.
+
+Adding the Airflow chart as a dependency means that it will be deployed together with your custom chart.
+You can disable the installation of Airflow by adding the ``condition`` field to the ``dependencies`` section.
+For example:
+
+.. code-block::
+
+    dependencies:
+      - name: airflow
+        version: 1.11.0
+        repository: https://airflow.apache.org
+        condition: airflow.enabled
+
+This will check if the value of ``airflow.enabled`` inside your ``values.yaml`` is ``true``.
+If it is, the Airflow chart will be deployed together with your custom chart.
+Otherwise, only your templates will be deployed.
+
+Download the Airflow Chart
+''''''''''''''''''''''''''
+
+Third, after you have specified the Airflow chart inside the ``dependencies`` section in ``Chart.yaml`` file,
+you can download the Airflow chart by running the following command:
+
+.. code-block::
+
+    helm dependency build
+
+.. note::
+
+    Make sure you are inside the directory which contains the ``Chart.yaml`` file.
+
+The chart will be downloaded and saved inside the ``charts/`` directory.
+
+Overriding default values
+''''''''''''''''''''''''''
+
+When you add a chart as a subchart to your chart,
+you have the ability to override the default values of the subchart in your ``values.yaml``.
+This is useful when your chart needs a specific configuration for your custom chart.
+E.g. if you want that the Airflow chart be installed with the ``KubernetesExecutor``,
+you can do it by adding the following section to your ``values.yaml``:
+
+.. code-block::
+
+    airflow:
+      executor: KubernetesExecutor
+
+You can override as many values as you like.