Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-4480] Avoid many small spills in external data structures (1.1) #3354

Closed
wants to merge 2 commits into from
Closed

[SPARK-4480] Avoid many small spills in external data structures (1.1) #3354

wants to merge 2 commits into from

Conversation

andrewor14
Copy link
Contributor

This is the branch-1.1 version of #3353. This requires a separate PR because the code in master has been refactored a little to eliminate duplicate code. I have tested this on a standalone cluster. The goal is to merge this into 1.1.1.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have started for PR 3354 at commit 7012595.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have finished for PR 3354 at commit 7012595.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23579/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have started for PR 3354 at commit 829d316.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have started for PR 3354 at commit f7e28bd.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have finished for PR 3354 at commit 829d316.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23592/
Test FAILed.

@andrewor14
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have started for PR 3354 at commit f2e552c.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have finished for PR 3354 at commit f7e28bd.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23593/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Nov 19, 2014

QA tests have finished for PR 3354 at commit f2e552c.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23596/
Test PASSed.

@mateiz
Copy link
Contributor

mateiz commented Nov 19, 2014

LGTM as well

@andrewor14
Copy link
Contributor Author

Ok pulling this in.

asfgit pushed a commit that referenced this pull request Nov 19, 2014
[SPARK-4480] Avoid many small spills in external data structures (1.1)
16bf5f3 
This is the branch-1.1 version of #3353. This requires a separate PR because the code in master has been refactored a little to eliminate duplicate code. I have tested this on a standalone cluster. The goal is to merge this into 1.1.1.

Author: Andrew Or <[email protected]>

Closes #3354 from andrewor14/avoid-small-spills-1.1 and squashes the following commits:

f2e552c [Andrew Or] Fix tests
7012595 [Andrew Or] Avoid many small spills
@andrewor14 andrewor14 closed this Nov 19, 2014
@andrewor14 andrewor14 deleted the avoid-small-spills-1.1 branch November 19, 2014 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewor14 @SparkQA @AmplabJenkins @mateiz