[AIRFLOW-6025] Add label to uniquely identify creator of Pod

[kaxil]

Make sure you have checked all steps below.

Jira

• My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-6025

Description

• Here are some details about my PR, including screenshots of any UI changes:
  2 Components natively create Kubernetes Pods.

1. KubernetesPodOperator
2. KubernetesExecutor

It would be ideal to add a label that identifies which of the two launches pod.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
  • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

[turbaszek]

It seems that we have a flaky test:

   Traceback (most recent call last):
    tests/task/task_runner/test_standard_task_runner.py line 129 in test_on_kill
      with open(path, "r") as f:
   FileNotFoundError: [Errno 2] No such file or directory: '/tmp/airflow_on_kill'

   -------------------- >> begin captured stdout << ---------------------
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [2019-11-21 13:02:57,945] {test_task_view_type_check.py:49} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'>
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s
   [%(asctime)s] {{%(filename)s:%(lineno)d}} %(levelname)s - %(message)s

   --------------------- >> end captured stdout << ----------------------

[kaxil]

@nuclearpinguin Yeah, restarting the test worked :)

[ashb]

Does Kube recommend that we qualify our labels, like org.apache.airflow.version?

[ashb]

Or is that for annotations? When should we use one over another?

[kaxil]

https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/ recommend some but given we already use labels and don't follow such conventions, I made this change :

airflow/airflow/kubernetes/worker_configuration.py

Lines 372 to 378 in fab957e

	labels={
	'airflow-worker': worker_uuid,
	'dag_id': dag_id,
	'task_id': task_id,
	'execution_date': execution_date,
	'try_number': str(try_number),
	},

[kaxil]

Re. labels vs annotations:

You can use either labels or annotations to attach metadata to Kubernetes objects. Labels can be used to select objects and to find collections of objects that satisfy certain conditions. In contrast, annotations are not used to identify and select objects. The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels.
So labels make more sense for e.g to select all the pods that are created by Kube Pod Operator