Skip to content

Commit

Permalink
refine comments.
Browse files Browse the repository at this point in the history
  • Loading branch information
jinxing committed Feb 16, 2017
1 parent d225565 commit 6809d1f
Showing 1 changed file with 3 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1189,10 +1189,9 @@ class DAGScheduler(
// This task was for the currently running attempt of the stage. Since the task
// completed successfully from the perspective of the TaskSetManager, mark it as
// no longer pending (the TaskSetManager may consider the task complete even
// when the output needs to be ignored because the task's epoch is too small below,
// if so, this can result in inconsistency between pending partitions and output
// locations of stage. When pending partitions is empty, the scheduler will check
// output locations, if there is missing, the stage will be resubmitted.
// when the output needs to be ignored because the task's epoch is too small below.
// In this case, when pending partitions is empty, there will still be missing
// output locations, which will cause the DAGScheduler to resubmit the stage below.)
shuffleStage.pendingPartitions -= task.partitionId
}
if (failedEpoch.contains(execId) && smt.epoch <= failedEpoch(execId)) {
Expand Down

0 comments on commit 6809d1f

Please sign in to comment.