Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DateTimeSensorAsync breaks if target_time is timezone-aware #29576

Closed
1 of 2 tasks
Gollum999 opened this issue Feb 16, 2023 · 0 comments · Fixed by #29606
Closed
1 of 2 tasks

DateTimeSensorAsync breaks if target_time is timezone-aware #29576

Gollum999 opened this issue Feb 16, 2023 · 0 comments · Fixed by #29606
Labels
affected_version:2.5 Issues Reported for 2.5 area:core-operators Operators, Sensors and hooks within Core Airflow kind:bug This is a clearly a bug

Comments

@Gollum999
Copy link
Contributor

Apache Airflow version

2.5.1

What happened

DateTimeSensorAsync fails with the following error if target_time is aware:

[2022-06-29, 05:09:11 CDT] {taskinstance.py:1889} ERROR - Task failed with exception
Traceback (most recent call last):a
  File "/opt/conda/envs/production/lib/python3.9/site-packages/airflow/sensors/time_sensor.py", line 60, in execute
    trigger=DateTimeTrigger(moment=self.target_datetime),
  File "/opt/conda/envs/production/lib/python3.9/site-packages/airflow/triggers/temporal.py", line 42, in __init__
    raise ValueError(f"The passed datetime must be using Pendulum's UTC, not {moment.tzinfo!r}")
ValueError: The passed datetime must be using Pendulum's UTC, not Timezone('America/Chicago')

What you think should happen instead

Given the fact that DateTimeSensor correctly handles timezones, this seems like a bug. DateTimeSensorAsync should be a drop-in replacement for DateTimeSensor, and therefore should have the same timezone behavior.

How to reproduce

#!/usr/bin/env python3
import datetime

from airflow.decorators import dag
from airflow.sensors.date_time import DateTimeSensor, DateTimeSensorAsync
import pendulum


@dag(
    start_date=datetime.datetime(2022, 6, 29),
    schedule='@daily',
)
def datetime_sensor_dag():
    naive_time1 = datetime.datetime(2023, 2, 16,  0,  1)
    aware_time1 = datetime.datetime(2023, 2, 16,  0,  1).replace(tzinfo=pendulum.local_timezone())
    naive_time2 = pendulum.datetime(2023, 2, 16, 23, 59)
    aware_time2 = pendulum.datetime(2023, 2, 16, 23, 59).replace(tzinfo=pendulum.local_timezone())

    DateTimeSensor(task_id='naive_time1', target_time=naive_time1, mode='reschedule')
    DateTimeSensor(task_id='naive_time2', target_time=naive_time2, mode='reschedule')
    DateTimeSensor(task_id='aware_time1', target_time=aware_time1, mode='reschedule')
    DateTimeSensor(task_id='aware_time2', target_time=aware_time2, mode='reschedule')

    DateTimeSensorAsync(task_id='async_naive_time1', target_time=naive_time1)
    DateTimeSensorAsync(task_id='async_naive_time2', target_time=naive_time2)
    DateTimeSensorAsync(task_id='async_aware_time1', target_time=aware_time1)  # fails
    DateTimeSensorAsync(task_id='async_aware_time2', target_time=aware_time2)  # fails

datetime_sensor_dag()

This can also happen if the target_time is naive and core.default_timezone = system.

Operating System

CentOS Stream 8

Versions of Apache Airflow Providers

N/A

Deployment

Other

Deployment details

Standalone

Anything else

This appears to be nearly identical to #24736. Probably worth checking other time-related sensors as well.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affected_version:2.5 Issues Reported for 2.5 area:core-operators Operators, Sensors and hooks within Core Airflow kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants