Skip to content

Commit

Permalink
[CORE] MapOutputTracker.deserializeMapStatuses: close input streams
Browse files Browse the repository at this point in the history
GZIPInputStream allocates native memory that is not freed until close() or
when the finalizer runs. It is best to close() these streams explicitly.
  • Loading branch information
Evan Jones committed May 7, 2015
1 parent 4f87e95 commit 0d76e85
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion core/src/main/scala/org/apache/spark/MapOutputTracker.scala
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,11 @@ private[spark] object MapOutputTracker extends Logging {
// Opposite of serializeMapStatuses.
def deserializeMapStatuses(bytes: Array[Byte]): Array[MapStatus] = {
val objIn = new ObjectInputStream(new GZIPInputStream(new ByteArrayInputStream(bytes)))
objIn.readObject().asInstanceOf[Array[MapStatus]]
Utils.tryWithSafeFinally {
objIn.readObject().asInstanceOf[Array[MapStatus]]
} {
objIn.close()
}
}

// Convert an array of MapStatuses to locations and sizes for a given reduce ID. If
Expand Down

0 comments on commit 0d76e85

Please sign in to comment.