Avoid color info in response of /dag_stats & /task_stats #8742
Conversation
| @@ -337,6 +337,9 @@ def get_int_arg(value, default=0): | |||
|
|
|||
| num_of_pages = int(math.ceil(num_of_all_dags / float(dags_per_page))) | |||
|
|
|||
| state_color_mapping = State.state_color.copy() | |||
| state_color_mapping["null"] = state_color_mapping.pop(None) | |||
There was a problem hiding this comment.
None is one of the keys in State.state_color, and it cannot be handled by Flask's tojson directly. So a special treatment is needed here.
A related issue I raised in Flask: pallets/flask#3599
There was a problem hiding this comment.
Keys in JSON must be strings, so { null: "x"} isn't valid JSON. So it sort of makes sense that it doesn't get added. Sort of.
There was a problem hiding this comment.
Python already handles it for us:
import json
print(json.dumps({None: 123}))gives us {"null": 123}.
The issue is the sort_keys argument when we run json.dumps().
XD-DENG
force-pushed
the
feature/avoid-repeated-color-data
branch
from
2969535
to
afd2136
Compare
| @@ -353,6 +356,7 @@ def get_int_arg(value, default=0): | |||
| status=arg_status_filter if arg_status_filter else None), | |||
| num_runs=num_runs, | |||
| tags=tags, | |||
| state_color=state_color_mapping, | |||
There was a problem hiding this comment.
Hi. Would you like to add a test that will check if the correct value was passed to the template? A method has been added today that makes it easy. I think it's worth gradually extending this idea to other tests to increase the confidence of the Web UI tests.
#8505
There was a problem hiding this comment.
Hi @mik-laj , thanks for the suggestion.
I have done an update by adding an assert in an existing test, which should suffice the purpose here.
Let me know if you have any other suggestion? Cheers.
There was a problem hiding this comment.
Hi @mik-laj , I have further updated the UI test code using the method you shared. Please take a look when you get time.
Two tests are failing, https://travis-ci.org/github/apache/airflow/jobs/684134650#L16198 and https://github.com/apache/airflow/pull/8742/checks?check_run_id=652054298 . But both seem interim and irrelevant to the changes of this PR
XD-DENG
force-pushed
the
feature/avoid-repeated-color-data
branch
from
6700ae4
to
071a742
Compare
|
Rebased to the latest master |
Currently the color for each state is repeatedly appearing in the response payload of endpoints /dag_stats and /task_stats. This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload. This significantly reduces the size of data to be transferred from server to client.
XD-DENG
force-pushed
the
feature/avoid-repeated-color-data
branch
from
071a742
to
390282b
Compare
* Avoid color info in response of /dag_stats & /task_stats Currently the color for each state is repeatedly appearing in the response payload of endpoints /dag_stats and /task_stats. This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload. This significantly reduces the size of data to be transferred from server to client. (cherry-picked from bed1995)
* Avoid color info in response of /dag_stats & /task_stats Currently the color for each state is repeatedly appearing in the response payload of endpoints /dag_stats and /task_stats. This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload. This significantly reduces the size of data to be transferred from server to client. (cherry-picked from bed1995)
* Avoid color info in response of /dag_stats & /task_stats Currently the color for each state is repeatedly appearing in the response payload of endpoints /dag_stats and /task_stats. This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload. This significantly reduces the size of data to be transferred from server to client. (cherry-picked from bed1995)
* Avoid color info in response of /dag_stats & /task_stats Currently the color for each state is repeatedly appearing in the response payload of endpoints /dag_stats and /task_stats. This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload. This significantly reduces the size of data to be transferred from server to client. (cherry-picked from bed1995)
Currently the color for each state is repeatedly appearing in the response payload of endpoints
/dag_statsand/task_stats.This can be avoided by having the mapping between state and color in the static file, and map each state into different color at client side after client receives the necessary info, instead of passing duplicated color information in the response payload.
This significantly reduces the size of data to be transferred from server to client.
Make sure to mark the boxes below before creating PR: [x]
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.