Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix airflow-webserer startup errors (v1-10-test) #10047

Closed
wants to merge 24 commits into from
Closed

fix airflow-webserer startup errors (v1-10-test) #10047

wants to merge 24 commits into from

Conversation

Kurganov
Copy link

fix airflow-webserer startup errors

Jul 29 17:00:01 localhost sh: Traceback (most recent call last):
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/bin/airflow", line 37, in <module>
Jul 29 17:00:01 localhost sh: args.func(args)
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 76, in wrapper
Jul 29 17:00:01 localhost sh: return f(*args, **kwargs)
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 1076, in webserver
Jul 29 17:00:01 localhost sh: app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/lib/python3.6/site-packages/airflow/www/app.py", line 234, in cached_app
Jul 29 17:00:01 localhost sh: app = create_app(config, testing)
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/lib/python3.6/site-packages/airflow/www/app.py", line 89, in create_app
Jul 29 17:00:01 localhost sh: from airflow.www import views
Jul 29 17:00:01 localhost sh: File "/home/airflow/venv/lib/python3.6/site-packages/airflow/www/views.py", line 99, in <module>
Jul 29 17:00:01 localhost sh: login_required = airflow.login.login_required
Jul 29 17:00:01 localhost sh: AttributeError: module 'airflow.contrib.auth.backends.kerberos_auth' has no attribute 'login_required'
Jul 29 17:00:01 localhost systemd: airflow-webserver.service: main process exited, code=exited, status=1/FAILURE

if airflow.api.auth.backend.kerberos_auth is enabled

Hartorn and others added 23 commits July 22, 2020 14:46
(cherry picked from commit fd62b1c)
When quick-fixing Python 3.8.4 error #9820 PYTHON_BASE_IMAGE_VERSION
variable was added but it was initialized too early in Breeze and
it took the default version of Python rather than the one chosen
by --python switch. This caused the generated requirements
(locally by Breeze only) to generate wrong set of requirements
and images built locally for different python versions were
based on default Python version, not the one chosen by --python
switch.

(cherry picked from commit 7b9e8e0)
If a task is skipped by BranchPythonOperator, BaseBranchOperator or ShortCircuitOperator and the user then clears the skipped task later, it'll execute. This is probably not the right
behaviour.

This commit changes that so it will be skipped again. This can be ignored by running the task again with "Ignore Task Deps" override.

(cherry picked from commit 1cdab56)
)

So far tests were not triggered when only requirements changed,
but this is quite needed in fact.

(cherry picked from commit 72abf82)
Currently there is "unbound" variable error printed in CI logs
because of that.

(cherry picked from commit 69f82e6)
Reviewed the scripts and removed some of the old unused ones.

(cherry picked from commit faec41e)
* we come back to idea of having one CI workflow
* cancel and openapi are incorporated into that CI workflow
* cancel retrieves workflow id automatically (works for forks)
* static checks are now merged into one job
* less dependencies between jobs so that waiting is minimised
* better name for check if tests should be run
* separated out script for tests should be run check

(cherry picked from commit 496ed6f)
* Constraint files are now maintained automatically

* No need to generate requirements when setup.py changes
* requirements are kept in separate orphan branches not in main repo
* merges to master verify if latest requirements are working and
  push tested requirements to orphaned branches
* we keep history of requirement changes and can label them
  individually for each version (by constraint-1.10.n tag name)
* consistently changed all references to be 'constraints' not
  'requirements'

(cherry picked from commit de9eaeb)
Remove the checklist of always checked points.

(cherry picked from commit 7dd5c11)
Since we are mountign tmp dir now to inside container, some
of the remnants of what's going on inside remains after exit.
This is particularly bad if you are using tmux (some of the
directories remaining there prevent tmux from re-run)

This change cleans up /tmp directory on exit from Breeze command.
It does it from inside container so that we clean up all
root-owned files without sudo.

(cherry picked from commit a9c871b)
* Fix PodGenerator to handle Kubernetes resources

In Airflow 1.10.11, `namespaced['resources'] = resources` is missing.
This PR improves the definition of pod resources, `requests` and `limits` are optional.

* Make it working in 2.7

* Add limit_gpu and fix ephemeral-storage keys

* Fix flake8

Co-authored-by: Riccardo Bini <[email protected]>
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled
@boring-cyborg
Copy link

boring-cyborg bot commented Jul 29, 2020

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, pylint and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://apache-airflow-slack.herokuapp.com/

@mik-laj
Copy link
Member

mik-laj commented Aug 12, 2020

@Kurganov Can I ask for a review?

@Kurganov
Copy link
Author

done

kaxil pushed a commit that referenced this pull request Aug 14, 2020
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
@kaxil
Copy link
Member

kaxil commented Aug 14, 2020

Has been cherry-picked to v1-10-test: 8e11423

@kaxil kaxil closed this Aug 14, 2020
kaxil pushed a commit that referenced this pull request Aug 14, 2020
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
kaxil pushed a commit that referenced this pull request Aug 14, 2020
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
kaxil pushed a commit that referenced this pull request Aug 15, 2020
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
kaxil pushed a commit that referenced this pull request Aug 15, 2020
fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
…#10047)

fix airflow-webserer startup errors if airflow.api.auth.backend.kerberos_auth enabled

(cherry picked from commit 87b495b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.