-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Airflow tasks test is broken for xcom_pull based tasks #24232
Comments
So, this |
Hi everybody, Yeah... I've noticed the same since our upgrade to 2.3.2. This isn't the case anymore and makes testing and debugging projects a lot more cumbersome to do. Hopefully this can be resolved in a future upgrade as we rely a lot on this. Thank you. |
Yeah just returning an arbitrary value is probably is the best we can do. Since anything is equally arbitrary, None is also a possible value. I think @ashb mentioned a similar case yesterday in another issue where we can use None. |
This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author. |
Team: I tested it again with latest Airflow version 2.6.2 and this issue is still exists. Can't use Airflow task test to test any xcom_pull based jobs anymore. It's been open for so long, no one really bother to fix it. |
Suggestion @mmuru : Maybe a good time to attempt to fix it simply? Airflow has 2500 contributors who gave back to free software they use by fixing issues, documentation. And anyone can attempt to fix such an issue - anyone can submit a PR that can be reviewed and merged. This is really how open source works - if somethign bothers you, you roll your sleeves up, figure out how to fix it, submit a PR, and get it merged. Simply the fact of life is that no-one fixed it yet, because no-one found it important enough to be fixed. Likely few people have similar issue. We have now |
#29608 was on the right track but the author ran out of steam unfortunately. It’s probably easiest to build on that existing implementation. |
how one would go about running a single task that depends on an xcom, even if not using |
Apache Airflow version
2.3.2 (latest released)
What happened
Testing example_xcom_args via CLI throws following exception.
What you think should happen instead
It should have return the results without any exception.
However, test dags works without exceptions.
It seems that can't test individual tasks via CLI if the task has xcom. I think, since airflow 2.3.x, xcom stores the task results in-memory instead of DB, it couldn't retrieve previous task result from xcom.
How to reproduce
Run tasks from airflow example_dags/example_xcomargs.py or example_xcom.py
Operating System
Mac OS and Python 3.9.12
Versions of Apache Airflow Providers
No response
Deployment
Other
Deployment details
No response
Anything else
I believe it is a regression since airflow 2.3.x release xcom changes.
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: