Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing attrs / correct bad attrs on TaskInstancePydantic #37854

Conversation

dstandish
Copy link
Contributor

@dstandish dstandish commented Mar 3, 2024

One was wrong type; others were missing; others needed a default.

@dstandish dstandish requested review from potiuk and mhenc March 3, 2024 04:40
@dstandish dstandish force-pushed the missing-or-wrong-attrs-on-task-instance-pydantic branch from af550aa to be054bd Compare March 3, 2024 04:41
Copy link
Collaborator

@dirrao dirrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Static checks are failing. Can you look into the static checks?

@dstandish dstandish force-pushed the missing-or-wrong-attrs-on-task-instance-pydantic branch 2 times, most recently from 4df2c2c to b48f44e Compare March 4, 2024 21:17
@dstandish dstandish force-pushed the missing-or-wrong-attrs-on-task-instance-pydantic branch 2 times, most recently from b8f5f2e to a8be0cd Compare March 15, 2024 06:07
@dstandish dstandish force-pushed the missing-or-wrong-attrs-on-task-instance-pydantic branch from a8be0cd to 588519b Compare March 17, 2024 20:00
@dstandish dstandish requested a review from XD-DENG as a code owner March 17, 2024 20:00
@dstandish
Copy link
Contributor Author

ok @uranusjr @potiuk @eladkal ...

In response to the question "why do we need to add defaults to pydantic"....

I have have added a default of None to attr task on TI

this isn't really a straightforward change because (1) mypy doesn't like it and, more importantly, (2) there is logic in airflow that looks at hasattr(ti, "task") to infer that both hasattr(ti, "task") and ti.task is not None. So all of this needs to be updated if we are going to avoid adding a default on the pydantic model. I have done so in this PR.

can you take a look and let me know what you think is the better approach? i.e. should we proceed with the changes i've just made, or go back and stick with just adding a default on the pydantic model.

@dstandish dstandish force-pushed the missing-or-wrong-attrs-on-task-instance-pydantic branch from 2c520d1 to b8d4b35 Compare March 19, 2024 07:00
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dstandish dstandish merged commit c6bc052 into apache:main Mar 19, 2024
47 checks passed
@dstandish dstandish deleted the missing-or-wrong-attrs-on-task-instance-pydantic branch March 19, 2024 18:49
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Mar 20, 2024
@ephraimbuddy ephraimbuddy added this to the Airflow 2.9.0 milestone Mar 20, 2024
utkarsharma2 pushed a commit to astronomer/airflow that referenced this pull request Apr 22, 2024
…antic (apache#37854)

This was motivated by the need to fix serialization of TaskInstance to TaskInstancePydantic.  In some cases the data type was wrong; in other cases the attr was missing.  In the case of `task` it was more complicated.  Historically the value is type-declared but not set at init.  So it's not alway there.  And in the code base there was a lot of hasattr... The problem here though was that when serializing to TaskInstancePydantic, if the value was not set, then serialization would fail.  So we had to make it optional on TaskInstance and provide a default of None.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants