Skip to content

Commit

Permalink
Fixed test failures by setting spark.shuffle.compress to the same val…
Browse files Browse the repository at this point in the history
…ue as spark.shuffle.spill.compress.
  • Loading branch information
rxin committed Sep 4, 2014
1 parent 85165e6 commit 713d341
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ class ExternalAppendOnlyMapSuite extends FunSuite with LocalSparkContext {
conf.set("spark.serializer.objectStreamReset", "1")
conf.set("spark.serializer", "org.apache.spark.serializer.JavaSerializer")
conf.set("spark.shuffle.spill.compress", codec.isDefined.toString)
conf.set("spark.shuffle.compress", codec.isDefined.toString)
codec.foreach { c => conf.set("spark.io.compression.codec", c) }
// Ensure that we actually have multiple batches per spill file
conf.set("spark.shuffle.spill.batchSize", "10")
Expand Down

0 comments on commit 713d341

Please sign in to comment.