Upgrade to stable constraints #12636

potiuk · 2020-11-26T00:03:55Z

This change upgrades setup.py to provide non-conflicting
pip check valid set of constraints for CI image.

PIP check is run once the images are available.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

potiuk · 2020-11-26T00:06:59Z

@turbaszek - I moved your changes to "Wait for images" step. As soon as #12635 gets merged, this PR (with "upgrade to latest dependencies" label added) should (I believe) cleanly build and successfully run pip-check (at least for the CI Images). Once the constraints are updated, we can also likely enable prod image check.

potiuk · 2020-11-26T11:10:29Z

I can split it further into two PRs, but I want to make it pass and see if indeed PIP check works @turbaszek !

github-actions · 2020-11-26T12:20:24Z

The PR needs to run all tests because it modifies core of Airflow! Please rebase it to latest master or ask committer to re-run it!

potiuk · 2020-11-26T12:22:14Z

I still disable the result of the pip check - they will be printed but won't fail the build. Working on making sure all those are solved

github-actions · 2020-11-26T12:35:25Z

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

github-actions · 2020-11-26T13:07:59Z

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

potiuk · 2020-11-26T13:38:45Z

@turbaszek -> final (????) test.

potiuk · 2020-11-26T13:44:12Z

ERROR    airflow.models.dagbag.DagBag:dagbag.py:297 Failed to import: /opt/airflow/airflow/providers/google/ads/example_dags/example_ads.py
Traceback (most recent call last):
  File "/opt/airflow/airflow/models/dagbag.py", line 294, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/airflow/airflow/providers/google/ads/example_dags/example_ads.py", line 24, in <module>
    from airflow.providers.google.ads.operators.ads import GoogleAdsListAccountsOperator
  File "/opt/airflow/airflow/providers/google/ads/operators/ads.py", line 24, in <module>
    from airflow.providers.google.ads.hooks.ads import GoogleAdsHook
  File "/opt/airflow/airflow/providers/google/ads/hooks/ads.py", line 25, in <module>
    from google.ads.google_ads.v2.types import GoogleAdsRow
ModuleNotFoundError: No module named 'google.ads.google_ads.v2'

Python 3.7 and 3.8 only !

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image. (cherry picked from commit 5370f3feb095e02704b0852fe630efdd118cb8f5)

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image. (cherry picked from commit 5370f3feb095e02704b0852fe630efdd118cb8f5)

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636 (cherry picked from commit fa8af2d)

- BugFix: Tasks with ``depends_on_past`` or ``task_concurrency`` are stuck (apache#12663) - Fix issue with empty Resources in executor_config (apache#12633) - Fix: Deprecated config ``force_log_out_after`` was not used (apache#12661) - Fix empty asctime field in JSON formatted logs (apache#10515) - [AIRFLOW-2809] Fix security issue regarding Flask SECRET_KEY (apache#3651) - [AIRFLOW-2884] Fix Flask SECRET_KEY security issue in www_rbac (apache#3729) - [AIRFLOW-2886] Generate random Flask SECRET_KEY in default config (apache#3738) - Add missing comma in setup.py (apache#12790) - Bugfix: Unable to import Airflow plugins on Python 3.8 (apache#12859) - Fix setup.py missing comma in ``setup_requires`` (apache#12880) - Don't emit first_task_scheduling_delay metric for only-once dags (apache#12835) - Update setup.py to get non-conflicting set of dependencies (apache#12636) - Rename ``[scheduler] max_threads`` to ``[scheduler] parsing_processes`` (apache#12605) - Add metric for scheduling delay between first run task & expected start time (apache#9544) - Add new-style 2.0 command names for Airflow 1.10.x (apache#12725) - Add Kubernetes cleanup-pods CLI command for Helm Chart (apache#11802) - Don't let webserver run with dangerous config (apache#12747) - Replace pkg_resources with importlib.metadata to avoid VersionConflict errors (apache#12694) - Clarified information about supported Databases

) This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

boring-cyborg bot added the area:dev-tools label Nov 26, 2020

potiuk requested review from turbaszek, kaxil and ashb November 26, 2020 00:04

potiuk added the upgrade to latest dependencies label Nov 26, 2020

potiuk force-pushed the upgrade-to-stable-constraints branch 3 times, most recently from 392acc1 to c4b0460 Compare November 26, 2020 09:13

turbaszek mentioned this pull request Nov 26, 2020

Old libraries in setup.py causing dependency resolution to pull old transitive constraints (3 years+) #12120

Closed

potiuk force-pushed the upgrade-to-stable-constraints branch 2 times, most recently from e0a84db to a1869bf Compare November 26, 2020 11:09

potiuk force-pushed the upgrade-to-stable-constraints branch 3 times, most recently from 0bccbd9 to e57f60f Compare November 26, 2020 12:12

turbaszek approved these changes Nov 26, 2020

View reviewed changes

github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Nov 26, 2020

potiuk removed the full tests needed We need to run full set of tests for this PR to merge label Nov 26, 2020

potiuk force-pushed the upgrade-to-stable-constraints branch from e57f60f to 9c6e941 Compare November 26, 2020 12:45

potiuk force-pushed the upgrade-to-stable-constraints branch from 9c6e941 to 1d7aa92 Compare November 26, 2020 13:10

potiuk added the full tests needed We need to run full set of tests for this PR to merge label Nov 26, 2020

potiuk force-pushed the upgrade-to-stable-constraints branch 2 times, most recently from 6f65d03 to ec7ee66 Compare November 26, 2020 21:31

potiuk added a commit that referenced this pull request Nov 27, 2020

Enable PIP check for both CI and PROD image (#12664)

fa8af2d

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636

potiuk added a commit that referenced this pull request Nov 30, 2020

Update setup.py to get non-conflicting set of dependencies (#12636)

8788acf

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

potiuk added this to the Airflow 1.10.14 milestone Nov 30, 2020

potiuk added the type:improvement Changelog: Improvements label Nov 30, 2020

kaxil pushed a commit that referenced this pull request Dec 3, 2020

Update setup.py to get non-conflicting set of dependencies (#12636)

d20f756

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

ashb pushed a commit that referenced this pull request Dec 3, 2020

Update setup.py to get non-conflicting set of dependencies (#12636)

3f43846

This change upgrades setup.py and setup.cfg to provide non-conflicting `pip check` valid set of constraints for CI image.

potiuk added a commit that referenced this pull request Dec 13, 2020

Enable PIP check for both CI and PROD image (#12664)

641a334

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636 (cherry picked from commit fa8af2d)

r-richmond mentioned this pull request Dec 28, 2020

Move snowflake limitations to snowflake #13333

Closed

kaxil pushed a commit that referenced this pull request Jan 21, 2021

Enable PIP check for both CI and PROD image (#12664)

52eb912

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636 (cherry picked from commit fa8af2d)

kaxil pushed a commit that referenced this pull request Jan 22, 2021

Enable PIP check for both CI and PROD image (#12664)

efca0c0

This PR enables PIP check after constraints have been updated to be stable and 'pip check' compliant in #12636 (cherry picked from commit fa8af2d)

dependabot bot mentioned this pull request Mar 18, 2021

chore(deps): bump apache-airflow from 1.10.6 to 1.10.14 in /packages/pipeline_workflows Open-Speech-EkStep/audio-to-speech-pipeline#4

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to stable constraints #12636

Upgrade to stable constraints #12636

potiuk commented Nov 26, 2020 •

edited

Loading

potiuk commented Nov 26, 2020

potiuk commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

potiuk commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

potiuk commented Nov 26, 2020

potiuk commented Nov 26, 2020 •

edited

Loading

Upgrade to stable constraints #12636

Upgrade to stable constraints #12636

Conversation

potiuk commented Nov 26, 2020 • edited Loading

potiuk commented Nov 26, 2020

potiuk commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

potiuk commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

github-actions bot commented Nov 26, 2020

potiuk commented Nov 26, 2020

potiuk commented Nov 26, 2020 • edited Loading

potiuk commented Nov 26, 2020 •

edited

Loading

potiuk commented Nov 26, 2020 •

edited

Loading