Update JS packages to latest versions (#9811) #9921

retornam · 2020-07-22T04:33:35Z

Signed-off-by: Raymond Etornam [email protected]

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

potiuk · 2020-07-22T05:52:25Z

You might want to add "Closes #9811 " in the commit message. Did you have a chance to test that the UI works after this upgrade @retornam ? Maybe some more people might test that one before we merge it (the UI is rather weakly tested) Summoning @apache/airflow-committers here.

potiuk · 2020-07-22T05:55:30Z

Also - even the limited tests here we have did not run because of "test-target" limitation. @retornam - can you please add "^airflow/www" to the patterns in script/ci/tools/ci_check_if_tests_should_be_run.sh to trigger them ?

Update: Ah no - no need to do that. It's another error I see.

kaxil · 2020-07-22T09:08:46Z

airflow/www/package.json

-    "imports-loader": "^0.8.0",
-    "jquery": ">=3.4.0",
+    "imports-loader": "^1.1.0",
+    "jquery": "^3.5.1",


jquery 3.5.0 previously broke the pause/unpause functionality. Have you tested the webserver after the changes in the PR @retornam ?

Details: dpgaspar/Flask-AppBuilder#1362 | #8613 | #8599

@kaxil I started the webserver, logged in but didn't test the example dags. I'll run more detailed testing and fix bugs I find.

@retornam while you are doing the tests, could you document and share with us on what UI functionalities you have tested? That way we can also double check on it with different data or see if any major feature is being missed.

Full details of what I did. I ended up upgrading each package separately. We cannot upgrade the following packages

d3

nvd3 (original package is no longer maintained and there are two active forks)

jquery due to a bug with CSRF support in Flask App Builder 2.3.3 (jQuery upgrade) breaks CSRF on some Safari, Chrome dpgaspar/Flask-AppBuilder#1362

moment ( upgrading to the new version broke moment support completely)

For UI testing, I checked the following pages

/home

made sure there were no new JS errors / CSS errors in the console.

turned the example dags on and off

triggered dags button

clicked on tree view button

clicked on graph view button

clicked on task view button

clicked on task tries button

clicked on landing times button

clicked on the code button

clicked on the refresh button

clicked on the delete dag button

performed a search for a dag using the search button

filtered dags using the string example

reset the filtered dags

clicked on last run for the example_bash operator which I enabled

clicked on the successful, failed and running dag links

clicked on the recent tasks icon

switched tabs between all, active and paused

/users/list/

added a new user

deleted a new user
-/tree for a dag

verified it displays the tree
-/graph for a dag

verified the graph renders
-/duration for a dag

verified the graph renders
-/tries for a dag

verified graph renders
-/landing_times for a graph

verified graph renders
-/gant for a dag

verified graph renders
-/dag_details for a dag

verified page logs
-/code for a dag

verified page loads

verified the trigger dag, refresh dag and delete dag buttons work on all pages

logged out

logged in

I tried to cover as many pages that include the scripts moment.*.js, base.*.js and d3.min.js

jquery due to a bug with CSRF support in Flask App Builder dpgaspar/Flask-AppBuilder#1362

@kaxil you answered that it was fixed in 2.3.4?! it isn't?

I think d3 and nvd3 is not worth the time we need to spend to upgrade the UI.

If we build a new UI from scratch then we should keep that in mind or use a different library then d3.

jquery due to a bug with CSRF support in Flask App Builder dpgaspar/Flask-AppBuilder#1362

@kaxil you answered that it was fixed in 2.3.4?! it isn't?

I believe "fixed" just means "FAB 2.3.4 reverted previous jQuery bump". It's not a fix but just a workaround.

Yeah that is correct, jQuery 3.5 still causes issue

Ah yeah I am remembering now.

- closes apache#9811 - upgrade webpack and webpack-cli Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - update css-loader to version 3.6.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - update mini-css-extract-plugin to v0.9.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - update lodash to v4.17.19 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - update url-loader to v4.1.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade style-loader to v1.2.1 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade stylelint to v13.6.1 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade handlebars to v4.7.6 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade clean-webpack-plugin to v3.0.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade copy-webpack-plugin to 6.0.3 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade js-yaml to 3.14.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade imports-loader to 1.1.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade file-loader to 6.0.0 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade redoc to 2.0.0-rc.30 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade datatables.net and datatables.net-bs 1.10.21 Signed-off-by: Raymond Etornam <[email protected]>

- closes apache#9811 - upgrade eslint-plugin-import to 2.22.0 - upgrade eslint to 7.5.0 - babel-eslint to 10.1.0 - babel-loader to 8.1.0 - eslint to 7.5.0 - eslint-config-airbnb-base to 14.2.0 - eslint-plugin-html to 6.0.2 - eslint-plugin-promise to 4.2.1 - eslint-plugin-node to 11.1.0 - eslint-plugin-standard to 4.0.1 Signed-off-by: Raymond Etornam <[email protected]>

retornam · 2020-08-04T07:25:35Z

@kaxil is anyone else looking at this? Its been a while now I dont want things to go too stale.

kaxil · 2020-08-04T23:08:17Z

I plan to include this in 1.10.12 (that would be released next week) so if no-one gets to it first, I will review it soon'ish

potiuk · 2020-08-05T15:20:53Z

I am also happy to do some testing during the weekend before we cut 1.10.12

fengsi · 2020-08-05T16:01:37Z

FYI: CSRF doesn't seem to work for Docker Desktop Kubernetes URL:
http://kubernetes.docker.internal/airflow/

It works on a "real" FQDN. One thing I noticed is the session cookie doesn't exist for kubernetes.docker.internal – could contribute to the CSRF issue. Not sure if it's a browser thing not issuing cookie or due to flask-wtf CSRF internals.

kaxil · 2020-08-06T21:38:04Z

@retornam I still see the following errors in Console on my deployment:

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b) (cherry picked from commit bce7d1c)

kaxil · 2020-09-17T13:34:11Z

Just want to thank you again @retornam for this PR as it eliminated some CVEs :)

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

boring-cyborg bot added the area:webserver Webserver related Issues label Jul 22, 2020

potiuk added this to the Airflow 1.10.12 milestone Jul 22, 2020

feluelle self-requested a review July 22, 2020 06:37

kaxil reviewed Jul 22, 2020

View reviewed changes

retornam force-pushed the retornam/fixjs branch 2 times, most recently from f903023 to 88555ab Compare July 23, 2020 21:18

retornam added 16 commits July 23, 2020 18:31

Update JS packages to latest versions

a9534b0

- closes apache#9811 - upgrade webpack and webpack-cli Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

2842a7e

- closes apache#9811 - update css-loader to version 3.6.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

c231c32

- closes apache#9811 - update mini-css-extract-plugin to v0.9.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

af76a59

- closes apache#9811 - update lodash to v4.17.19 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

35daac9

- closes apache#9811 - update url-loader to v4.1.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

b865361

- closes apache#9811 - upgrade style-loader to v1.2.1 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

07eec85

- closes apache#9811 - upgrade stylelint to v13.6.1 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

7d414d9

- closes apache#9811 - upgrade handlebars to v4.7.6 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

fc3689a

- closes apache#9811 - upgrade clean-webpack-plugin to v3.0.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

ba0a458

- closes apache#9811 - upgrade copy-webpack-plugin to 6.0.3 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

37ddd6e

- closes apache#9811 - upgrade js-yaml to 3.14.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

a44bb73

- closes apache#9811 - upgrade imports-loader to 1.1.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

1983f74

- closes apache#9811 - upgrade file-loader to 6.0.0 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

5ea55b2

- closes apache#9811 - upgrade redoc to 2.0.0-rc.30 Signed-off-by: Raymond Etornam <[email protected]>

Update JS packages to latest versions

904abcc

- closes apache#9811 - upgrade datatables.net and datatables.net-bs 1.10.21 Signed-off-by: Raymond Etornam <[email protected]>

retornam force-pushed the retornam/fixjs branch from 1532889 to 32f7e4e Compare July 24, 2020 01:31

fix formatting

7633d45

retornam force-pushed the retornam/fixjs branch from 32f7e4e to 7633d45 Compare July 24, 2020 02:44

kaxil requested a review from houqp July 29, 2020 12:51

This was referenced Jul 29, 2020

[Snyk] Upgrade datatables.net-bs from 1.10.20 to 1.10.21 astronomer/airflow#807

Closed

[Snyk] Upgrade datatables.net from 1.10.20 to 1.10.21 astronomer/airflow#806

Closed

kaxil mentioned this pull request Jul 29, 2020

[Snyk] Upgrade js-yaml from 3.13.1 to 3.14.0 astronomer/airflow#808

Closed

kaxil added the pinned Protect from Stalebot auto closing label Aug 4, 2020

potiuk assigned potiuk and kaxil Aug 5, 2020

kaxil approved these changes Aug 6, 2020

View reviewed changes

kaxil merged commit c920b1b into apache:master Aug 6, 2020

retornam deleted the retornam/fixjs branch August 14, 2020 05:25

potiuk pushed a commit that referenced this pull request Aug 14, 2020

Update JS packages to latest versions (#9811) (#9921)

4f7c991

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

kaxil pushed a commit that referenced this pull request Aug 15, 2020

Update JS packages to latest versions (#9811) (#9921)

177dad5

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

kaxil pushed a commit that referenced this pull request Aug 15, 2020

Update JS packages to latest versions (#9811) (#9921)

bce7d1c

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

kaxil mentioned this pull request Sep 3, 2020

Fix #10429 jquery dependency needs to be updated to 3.5.0 or newer #10684

Closed

kaxil pushed a commit to astronomer/airflow that referenced this pull request Sep 10, 2020

Update JS packages to latest versions (apache#9811) (apache#9921)

655c770

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b) (cherry picked from commit bce7d1c)

potiuk modified the milestones: Airflow 1.10.12, Airflow 1.10.13, Airflow 2.0.0 Sep 17, 2020

kaxil mentioned this pull request Nov 12, 2020

Package vulnerability scan failing on lodash CVE #12272

Closed

kaxil modified the milestones: Airflow 2.0.0 (rc1), Airflow 1.10.12 Dec 4, 2020

cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021

Update JS packages to latest versions (apache#9811) (apache#9921)

c4c2129

Signed-off-by: Raymond Etornam <[email protected]> (cherry picked from commit c920b1b)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update JS packages to latest versions (#9811) #9921

Update JS packages to latest versions (#9811) #9921

retornam commented Jul 22, 2020

potiuk commented Jul 22, 2020

potiuk commented Jul 22, 2020 •

edited

Loading

kaxil Jul 22, 2020

kaxil Jul 22, 2020

retornam Jul 22, 2020

houqp Jul 23, 2020

retornam Jul 23, 2020

feluelle Jul 27, 2020 •

edited

Loading

feluelle Jul 27, 2020

fengsi Jul 28, 2020

kaxil Jul 29, 2020

feluelle Jul 29, 2020

retornam commented Aug 4, 2020

kaxil commented Aug 4, 2020

potiuk commented Aug 5, 2020

fengsi commented Aug 5, 2020

kaxil commented Aug 6, 2020

kaxil commented Sep 17, 2020

Update JS packages to latest versions (#9811) #9921

Update JS packages to latest versions (#9811) #9921

Conversation

retornam commented Jul 22, 2020

potiuk commented Jul 22, 2020

potiuk commented Jul 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feluelle Jul 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

retornam commented Aug 4, 2020

kaxil commented Aug 4, 2020

potiuk commented Aug 5, 2020

fengsi commented Aug 5, 2020

kaxil commented Aug 6, 2020

kaxil commented Sep 17, 2020

potiuk commented Jul 22, 2020 •

edited

Loading

feluelle Jul 27, 2020 •

edited

Loading