Skip to content

Commit

Permalink
Refine comment.
Browse files Browse the repository at this point in the history
  • Loading branch information
jinxing committed Feb 15, 2017
1 parent ab8d13e commit e34cd85
Showing 1 changed file with 3 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1190,6 +1190,9 @@ class DAGScheduler(
// completed successfully from the perspective of the TaskSetManager, mark it as
// no longer pending (the TaskSetManager may consider the task complete even
// when the output needs to be ignored because the task's epoch is too small below).
// This can result in inconsistency between pending partitions and output locations
// of stage. When pending partitions is empty, the scheduler will check output
// locations, if there are missing, the stage will be resubmitted.
shuffleStage.pendingPartitions -= task.partitionId
}
if (failedEpoch.contains(execId) && smt.epoch <= failedEpoch(execId)) {
Expand Down

0 comments on commit e34cd85

Please sign in to comment.