Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) #38446

jscheffl · 2024-03-25T01:20:04Z

This PR tries to continue the work from [ @aleksandr-shel @xgao1023 @uranusjr ] in PR #35320. It merges with current main. On top of this I fixed the bugs in pytest, some glitches in UI.

I'd LOVE to get this into 2.9.0 and as there were a couple of reviews already on the previous PR I hope it is not as compex as it looks. A lot of code is also due to the fact that I try to migrate all examples over to "nicer names". Sneak preview how it could look like after this PR:

What is open to be merged?

Review of course
Migrate the second half of example DAGs

closes: #22073
related: #32520, #28183

Co-authored-by: Tzu-ping Chung <[email protected]> Co-authored-by: Aleksandr Shelukheev <[email protected]>

potiuk

I reviewed the backend code and looked at the UI. Not sure if all the places in UI that should be updated, are but the backend implementation looks sound to me. I think some other pairs of eyes shold also look at that though.

potiuk · 2024-03-25T01:49:48Z

The failing MINSQL ALchemy test here is fixed in #38445

airflow/decorators/base.py

airflow/models/baseoperator.py

ephraimbuddy

I don't think we should modify a lot of example dags to add task_display_name on them

jscheffl · 2024-03-25T17:23:55Z

I don't think we should modify a lot of example dags to add task_display_name on them

Okay, then before going into a debate I'll revert the adjustments in the example DAGs for most cases and leave this for future cleanup.
I#l like to consolidate a lot of examples though. I believe we have many many and maybe (after release of 2.9) it is time to think about consolidating them.

uranusjr · 2024-03-26T22:23:48Z

We should add a couple of sections in the documentation about the fields. Not sure where tbh.

jscheffl · 2024-03-26T22:27:07Z

We should add a couple of sections in the documentation about the fields. Not sure where tbh.

How can I interpret your "Should"?
a) Must be before approval+merge
b) Okay to merge but do it before release
c) would be nice sooner or later

jscheffl · 2024-03-26T22:28:31Z

I hope (&pray) that I catched all errors now and pipeline turns green...

potiuk · 2024-03-26T22:46:58Z

Looks good

uranusjr · 2024-03-26T22:48:59Z

Docs can be added in a separate PR, but I would prefer to have it for the final release. (Don’t need to hit alphas though, docs don’t break things.)

airflow/api_connexion/schemas/dag_schema.py

airflow/models/mappedoperator.py

airflow/models/taskmixin.py

airflow/api_connexion/schemas/task_instance_schema.py

airflow/models/baseoperator.py

airflow/models/abstractoperator.py

… added display name for dag / task (v2) (apache#38446) * Add display name in DAGs and tasks Co-authored-by: Tzu-ping Chung <[email protected]> Co-authored-by: Aleksandr Shelukheev <[email protected]> * Fix frontend issues * Fix pytests * Add DAG display tool tips * Implement national language char display example oon two DAGs * Review nit from other PR * Fix WWW pytest on HTML code * Review feedback, make task_display_name a property, not a field in mapped operator * Review feedback, rename internal fields * Small nit, optimize sorting on DAG home page with display name * Review feedback * Fix pytests in API plus review feedback on API * Review feedback, extend schema, update apispecs and fix breadcrumb navigation * nit, update page titles as well * Fix pytests for extended API model * Ensure mapped operator also provides a display string * Ensure display field is only serialized if different from task id * fix pytest for API * Review feedback from TP * Move labels down to base and mapped operator * Move labels up to abstract operator * Only one decorator is needed --------- Co-authored-by: Vincent Gao <[email protected]> Co-authored-by: Tzu-ping Chung <[email protected]> Co-authored-by: Aleksandr Shelukheev <[email protected]>

xgao1023 and others added 5 commits February 5, 2024 16:31

Add display name in DAGs and tasks

b70c86a

Co-authored-by: Tzu-ping Chung <[email protected]> Co-authored-by: Aleksandr Shelukheev <[email protected]>

Fix frontend issues

2235815

Merge remote-tracking branch 'upstream/main' into fix-22073

5477408

Fix pytests

1df3726

Add DAG display tool tips

323451d

jscheffl requested review from uranusjr, ryanahamilton, ashb, bbovenzi, pierrejeambrun, potiuk, kaxil, bolkedebruin and XD-DENG as code owners March 25, 2024 01:20

boring-cyborg bot added the provider:cncf-kubernetes Kubernetes provider related issues label Mar 25, 2024

jscheffl changed the title ~~Fix 22073~~ Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) Mar 25, 2024

jscheffl added the area:UI Related to UI/UX. For Frontend Developers. label Mar 25, 2024

jscheffl mentioned this pull request Mar 25, 2024

allows users to write dag_id and task_id in their national characters, added display name for dag / task #32520

Closed

potiuk approved these changes Mar 25, 2024

View reviewed changes

uranusjr reviewed Mar 25, 2024

View reviewed changes

airflow/decorators/base.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 25, 2024

View reviewed changes

airflow/models/baseoperator.py Outdated Show resolved Hide resolved

ephraimbuddy added this to the Airflow 2.9.0 milestone Mar 25, 2024

ephraimbuddy reviewed Mar 25, 2024

View reviewed changes

ephraimbuddy mentioned this pull request Mar 25, 2024

Add executor field to the DB and parameter to the operators #38054

Merged

jscheffl added 4 commits March 25, 2024 18:26

Implement national language char display example oon two DAGs

22df132

Merge remote-tracking branch 'upstream/main' into fix-22073

9f34efd

Review nit from other PR

33ea087

Fix WWW pytest on HTML code

9355788

jscheffl added 3 commits March 26, 2024 22:28

Fix pytests for extended API model

95818f4

Ensure mapped operator also provides a display string

fa74ca8

Ensure display field is only serialized if different from task id

3445ce7

jscheffl added 2 commits March 26, 2024 23:24

fix pytest for API

d31f4b1

Merge remote-tracking branch 'upstream/main' into fix-22073

f2f8e10

uranusjr reviewed Mar 26, 2024

View reviewed changes

airflow/api_connexion/schemas/dag_schema.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 26, 2024

View reviewed changes

airflow/models/mappedoperator.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 26, 2024

View reviewed changes

airflow/models/taskmixin.py Outdated Show resolved Hide resolved

jscheffl added 2 commits March 27, 2024 00:22

Review feedback from TP

68accda

Move labels down to base and mapped operator

14a1f9f

jscheffl requested a review from uranusjr March 27, 2024 00:21

bbovenzi approved these changes Mar 27, 2024

View reviewed changes

uranusjr reviewed Mar 27, 2024

View reviewed changes

airflow/api_connexion/schemas/task_instance_schema.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 27, 2024

View reviewed changes

airflow/models/baseoperator.py Outdated Show resolved Hide resolved

Move labels up to abstract operator

74dc2a2

uranusjr reviewed Mar 27, 2024

View reviewed changes

airflow/models/abstractoperator.py Outdated Show resolved Hide resolved

Only one decorator is needed

ab7a813

uranusjr approved these changes Mar 27, 2024

View reviewed changes

ephraimbuddy merged commit 8c44bcb into apache:main Mar 27, 2024
47 checks passed

ephraimbuddy added the type:new-feature Changelog: New Features label Mar 27, 2024

cmarteepants mentioned this pull request Mar 28, 2024

Separate dag/task UI display name from dag / task id #22073

Closed

2 tasks

jscheffl mentioned this pull request Mar 29, 2024

Mark more fields on API as dump-only #38616

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) #38446

Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) #38446

jscheffl commented Mar 25, 2024 •

edited

Loading

potiuk left a comment

potiuk commented Mar 25, 2024

ephraimbuddy left a comment

jscheffl commented Mar 25, 2024

uranusjr commented Mar 26, 2024

jscheffl commented Mar 26, 2024

jscheffl commented Mar 26, 2024

potiuk commented Mar 26, 2024

uranusjr commented Mar 26, 2024

Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) #38446

Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) #38446

Conversation

jscheffl commented Mar 25, 2024 • edited Loading

potiuk left a comment

Choose a reason for hiding this comment

potiuk commented Mar 25, 2024

ephraimbuddy left a comment

Choose a reason for hiding this comment

jscheffl commented Mar 25, 2024

uranusjr commented Mar 26, 2024

jscheffl commented Mar 26, 2024

jscheffl commented Mar 26, 2024

potiuk commented Mar 26, 2024

uranusjr commented Mar 26, 2024

jscheffl commented Mar 25, 2024 •

edited

Loading