Support k8s auth method in vault secrets provider

[reltuk]

---

Make sure to mark the boxes below before creating PR: [x]

• Description above provides context of the change
• Unit tests coverage for changes (not needed for documentation changes)
• Target Github ISSUE in description if exists
• Commits follow "How to write a good git commit message"
• Relevant documentation is updated including usage instructions.
• I will engage committers as explained in Contribution Workflow Example.

---

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (flake8, pylint and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://apache-airflow-slack.herokuapp.com/

[reltuk]

Not sure about the most recent lint failure...I moved pylint directive comment to the line above the class.

[potiuk]

Not sure about the most recent lint failure...I moved pylint directive comment to the line above the class.

You can see it in the output:

airflow/providers/hashicorp/secrets/vault.py:32:54: E261 at least two spaces before inline comment

[reltuk]

But in the most recent code there is no inline comment at airflow/providers/hashicorp/secrets/vault.py:32. The comment has been moved to airflow/providers/hashicorp/secrets/vault.py:31 instead, because of the previous pylint failure...

[reltuk]

Pushed an empty commit to try again. Thanks for feedback :).

[kaxil]

Have you tested this ?

[reltuk]

Yes, for the most part. See this gist for an example of an adhoc end-to-end test that works:

https://gist.github.com/reltuk/4ff3ee7edca0848d081a945abb5ad01b

[reltuk]

Small white lie from the above testing: in order to make it work, I had to have the following patch applied. Not sure what's going on there...maybe my pip install is wrong or vault.py is out of date or something.

diff --git a/airflow/providers/hashicorp/secrets/vault.py b/airflow/providers/hashicorp/secrets/vault.py
index c0fd23581..27058e9c4 100644
--- a/airflow/providers/hashicorp/secrets/vault.py
+++ b/airflow/providers/hashicorp/secrets/vault.py
@@ -24,7 +24,7 @@ import hvac
 from cached_property import cached_property
 from hvac.exceptions import InvalidPath, VaultError
 
-from airflow import AirflowException
+from airflow.exceptions import AirflowException
 from airflow.secrets import BaseSecretsBackend
 from airflow.utils.log.logging_mixin import LoggingMixin
 

Seems unrelated to the k8s support though.

[kaxil]

Small white lie from the above testing: in order to make it work, I had to have the following patch applied. Not sure what's going on there...maybe my pip install is wrong or vault.py is out of date or something.

diff --git a/airflow/providers/hashicorp/secrets/vault.py b/airflow/providers/hashicorp/secrets/vault.py
index c0fd23581..27058e9c4 100644
--- a/airflow/providers/hashicorp/secrets/vault.py
+++ b/airflow/providers/hashicorp/secrets/vault.py
@@ -24,7 +24,7 @@ import hvac
 from cached_property import cached_property
 from hvac.exceptions import InvalidPath, VaultError
 
-from airflow import AirflowException
+from airflow.exceptions import AirflowException
 from airflow.secrets import BaseSecretsBackend
 from airflow.utils.log.logging_mixin import LoggingMixin
 

Seems unrelated to the k8s support though.

Happy for you to make that change too. It will avoid cycles.

[reltuk]

Ok, added a commit to fix the AirflowException import in vault.py.

Also added a commit to make airflow variables get VAR_NAME print the value of the retrieved variable. Previously I had to use airflow variables get hello -d test, because the variable value was only printed if a default was also provided on the command line.

[boring-cyborg]

Awesome work, congrats on your first merged pull request!