[AIRFLOW-5387] Fix show paused pagination bug

[alrolorojas]

Make sure you have checked all steps below.

Jira

• My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-5387
  • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.
  • In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal (AIP).
  • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Description

This PR addresses issues with pagination + showing/hiding paused DAGs. showPaused URL parameter was removed of the URL without taking into account the value of the airflow configuration hide_paused_dags_by_default.

Now it only removes showPaused if hide_paused_dags_by_default makes it redundant.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:
  • ./breeze --test-target tests.www.test_utils:TestUtils -- -s

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
  • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

[alrolorojas]

There is a test failure that seem unrelated to the changes:

ERROR [airflow.models.dagbag.DagBag] Failed to import: /opt/airflow/tests/dags/test_impersonation_custom.py
Traceback (most recent call last):
  File "/opt/airflow/airflow/models/dagbag.py", line 199, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/airflow/tests/dags/test_impersonation_custom.py", line 31, in <module>
    from tests.test_utils.fake_datetime import FakeDatetime
ModuleNotFoundError: No module named 'tests.test_utils'

Any idea of what could be happening?

[mik-laj]

Yesterday I prepared a fix. it is already available on masters branch. Can you do rebase onn latest apache/master?

[alrolorojas]

@mik-laj Done! Can you take a look, please?

[alrolorojas]

@feluelle @ashb I've addressed the feedback. Please take another look

[feluelle]

You could even use @conf_vars to annotate the test method to use the env var only in the whole test method. But either way it is better than before (try-finally...).

LGTM 👍

[alrolorojas]

You could even use @conf_vars to annotate the test method to use the env var only in the whole test method. But either way it is better than before (try-finally...).

@feluelle I did not find a way to pass arguments dynamically from @parameterized.expand to @conf_vars when used as a decorator. Is there a way I can do that annotating the whole method with conf_vars and use an argument from @parameterized.expand within it?

[feluelle]

Good point - haven't tried @parameterized.expand along with some other annotation which needs to retrieve @parameterized.expand args.

[alrolorojas]

@ashb @feluelle 👋 Is there anything else I should do as part of this PR to get it merged?

[ashb]

@alrolorojas Code looks good now. Could you resolve the conflicts then ping us when it's ready to merge?

[alrolorojas]

@ashb @feluelle Done!

[feluelle]

@alrolorojas

 import unittest
-from parameterized import parameterized
 from datetime import datetime
 from unittest import mock
 from urllib.parse import parse_qs

 from bs4 import BeautifulSoup
+from parameterized import parameterized

[alrolorojas]

@feluelle @ashb This is completed. I do not think CI failures are related to the changes made as part of this PR.

[ashb]

I've re-run the failed test to make sure. If we don't merge it once it's green please ping us.

[codecov-io]

Codecov Report

Merging #6100 into master will decrease coverage by 70.84%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master   #6100       +/-   ##
==========================================
- Coverage   80.32%   9.47%   -70.85%     
==========================================
  Files         612     607        -5     
  Lines       35395   35035      -360     
==========================================
- Hits        28432    3321    -25111     
- Misses       6963   31714    +24751

Impacted Files	Coverage Δ
airflow/www/utils.py	0% <0%> (-75.25%)	⬇️
...low/contrib/operators/wasb_delete_blob_operator.py	0% <0%> (-100%)	⬇️
airflow/example_dags/subdags/subdag.py	0% <0%> (-100%)	⬇️
airflow/gcp/sensors/bigquery_dts.py	0% <0%> (-100%)	⬇️
airflow/gcp/operators/text_to_speech.py	0% <0%> (-100%)	⬇️
airflow/contrib/sensors/emr_base_sensor.py	0% <0%> (-100%)	⬇️
airflow/gcp/hooks/discovery_api.py	0% <0%> (-100%)	⬇️
airflow/contrib/operators/gcs_list_operator.py	0% <0%> (-100%)	⬇️
airflow/example_dags/example_subdag_operator.py	0% <0%> (-100%)	⬇️
airflow/contrib/operators/file_to_gcs.py	0% <0%> (-100%)	⬇️
... and 573 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d71f33...e7ded8b. Read the comment docs.

[alrolorojas]

@ashb 🍏

[feluelle]

Thanks @alrolorojas