-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SPARK-3020: Print completed indices rather than tasks in web UI #1933
Conversation
QA tests have started for PR 1933. This patch merges cleanly. |
QA tests have started for PR 1933. This patch merges cleanly. |
@rxin I used OpenHashSet here. I'm hesitant to convert this into an integer when the stage completes, because we still haven't dealt correctly with stage re-submissionsa and as it stands I think it's not easy to reason about the last time |
@@ -38,6 +39,7 @@ private[jobs] object UIData { | |||
class StageUIData { | |||
var numActiveTasks: Int = _ | |||
var numCompleteTasks: Int = _ | |||
var completedIndices = new OpenHashSet[Int]() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
todo
lgtm other than the todo |
QA results for PR 1933: |
QA results for PR 1933: |
Thanks. Merging in master & branch-1.1. |
Author: Patrick Wendell <[email protected]> Closes #1933 from pwendell/speculation and squashes the following commits: 33a3473 [Patrick Wendell] Use OpenHashSet 8ce2ff0 [Patrick Wendell] SPARK-3020: Print completed indices rather than tasks in web UI (cherry picked from commit 0c7b452) Signed-off-by: Reynold Xin <[email protected]>
Author: Patrick Wendell <[email protected]> Closes apache#1933 from pwendell/speculation and squashes the following commits: 33a3473 [Patrick Wendell] Use OpenHashSet 8ce2ff0 [Patrick Wendell] SPARK-3020: Print completed indices rather than tasks in web UI
No description provided.