Skip to content

Commit

Permalink
[SPARK-29182][CORE][FOLLOWUP] Cache preferred locations of checkpoint…
Browse files Browse the repository at this point in the history
…ed RDD

### What changes were proposed in this pull request?

This is a followup to #25856. This fixes the document about the config value of spark.rdd.checkpoint.cachePreferredLocsExpireTime.

### Why are the changes needed?

The document is not correct. spark.rdd.checkpoint.cachePreferredLocsExpireTime can not be 0.

### Does this PR introduce any user-facing change?

No

### How was this patch tested?

This is document only change.

Closes #26251 from viirya/SPARK-29182-followup.

Authored-by: Liang-Chi Hsieh <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
viirya authored and dongjoon-hyun committed Oct 25, 2019
1 parent 68dca9a commit ae5b60d
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -804,7 +804,7 @@ package object config {
.doc("Expire time in minutes for caching preferred locations of checkpointed RDD." +
"Caching preferred locations can relieve query loading to DFS and save the query " +
"time. The drawback is that the cached locations can be possibly outdated and " +
"lose data locality. If this config is not specified or is 0, it will not cache.")
"lose data locality. If this config is not specified, it will not cache.")
.timeConf(TimeUnit.MINUTES)
.checkValue(_ > 0, "The expire time for caching preferred locations cannot be non-positive.")
.createOptional
Expand Down

0 comments on commit ae5b60d

Please sign in to comment.