[AIRFLOW-5653] Log caught AirflowSkipException in task instance log #6330
Conversation
Logging any caught AirflowSkipException with message in models/taskinstance.py to remove confusion due to no task skipping information in the logs
airflow/models/taskinstance.py
Outdated
| except AirflowSkipException as e: | ||
| # log only if exception has any arguments to prevent log flooding | ||
| if e.args: | ||
| self.log.warning(e) |
There was a problem hiding this comment.
Maybe this should be at info level? It doesn't really seem like a "warning" to me.
WDYT?
There was a problem hiding this comment.
Agreed. Since task skipping is not so much of a warning in airflow. I have made the change. Thanks for the tip.
Codecov Report
@@ Coverage Diff @@
## master #6330 +/- ##
==========================================
+ Coverage 80.34% 80.34% +<.01%
==========================================
Files 616 616
Lines 35737 35739 +2
==========================================
+ Hits 28713 28715 +2
Misses 7024 7024
Continue to review full report at Codecov.
|
|
Have you tested this on a local instance? |
|
Yes. They ran fine on my local instance. In fact, the travis-ci test was fine in my first commit too and the only change in the second commit is the log level. |
|
Failing test looks flaky and unrelated, merging. Here is the failing test for context:
::::test_dag-test_task:[2019-10-14 14:53:08,975] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op /opt/airflow/airflow/models/dagbag.py:21: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses ::::test_dag-test_task:[2019-10-14 14:53:08,978] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op import imp ::::test_dag-test_task:[2019-10-14 14:53:09,074] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op [2019-10-14 14:53:09,073] {settings.py:175} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=8013 ::::test_dag-test_task:[2019-10-14 14:53:09,492] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op [2019-10-14 14:53:09,491] {init.py:53} INFO - Using executor LocalExecutor ::::test_dag-test_task:[2019-10-14 14:53:09,493] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op [2019-10-14 14:53:09,492] {dagbag.py:90} INFO - Filling up the DagBag from /opt/airflow/tests/dags/test_heartbeat_failed_fast.py ::::test_dag-test_task:[2019-10-14 14:53:09,518] {base_task_runner.py:111} INFO - Job None: Subtask test_heartbeat_failed_fast_op [2019-10-14 14:53:09,518] {cli.py:590} INFO - Running <TaskInstance: test_heartbeat_failed_fast.test_heartbeat_failed_fast_op 2016-01-01T00:00:00+00:00 [running]> on host 4a17e03fc9e0 FAILED |
aoen
changed the title
…6330) [AIRFLOW-5653] Log caught AirflowSkipException in task instance log Logging any caught AirflowSkipException with message in models/taskinstance.py to remove confusion due to no task skipping information in the logs (cherry picked from commit 1a4c164)
…pache#6330) [AIRFLOW-5653] Log caught AirflowSkipException in task instance log Logging any caught AirflowSkipException with message in models/taskinstance.py to remove confusion due to no task skipping information in the logs (cherry picked from commit 1a4c164) (cherry picked from commit a04cafd)
Jira
Description
to remove confusion due to no task skipping information being present in the logs
Tests
Commits
Documentation