You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order for a running Dagger task to wait on another task (such as when a task spawns a DAG dynamically and waits on its completion), we need to communicate to the scheduler that our task is going to temporarily sleep until a certain condition is met (namely that another task has completed). We currently hack around this by forcibly decreasing the current processor's pressure/occupancy by the amount that the yielding task consumes, which effectively allows another task to be run. This is generally the behavior that we want, but is not a very clean way to do it, and can introduce more bugs when the functionality needs to be tweaked.
A better approach would be to treat the yielding task as temporarily completed, and scheduling its continuation (the rest of its execution) as another task to be scheduled once a condition is reached. This would remove the task from view of the processor it was running on, which simplifies per-processor scheduling. This yielding mechanism would also be beneficial for increasing the number of low-occupancy tasks that can be scheduled at once, since we then have the ability to run another task while we wait for a waiting task to meet its condition.
The condition based-scheduling required for this feature would also be quite useful for other cases where we want to intentionally pause execution of a new or yielding task until same important constraint is loosened (such as waiting until more memory becomes available, waiting for data to become available on a Channel, etc.).
The text was updated successfully, but these errors were encountered:
In order for a running Dagger task to wait on another task (such as when a task spawns a DAG dynamically and waits on its completion), we need to communicate to the scheduler that our task is going to temporarily sleep until a certain condition is met (namely that another task has completed). We currently hack around this by forcibly decreasing the current processor's pressure/occupancy by the amount that the yielding task consumes, which effectively allows another task to be run. This is generally the behavior that we want, but is not a very clean way to do it, and can introduce more bugs when the functionality needs to be tweaked.
A better approach would be to treat the yielding task as temporarily completed, and scheduling its continuation (the rest of its execution) as another task to be scheduled once a condition is reached. This would remove the task from view of the processor it was running on, which simplifies per-processor scheduling. This yielding mechanism would also be beneficial for increasing the number of low-occupancy tasks that can be scheduled at once, since we then have the ability to run another task while we wait for a waiting task to meet its condition.
The condition based-scheduling required for this feature would also be quite useful for other cases where we want to intentionally pause execution of a new or yielding task until same important constraint is loosened (such as waiting until more memory becomes available, waiting for data to become available on a
Channel
, etc.).The text was updated successfully, but these errors were encountered: