Skip to content

Commit

Permalink
[SPARK-4298][Core] - The spark-submit cannot read Main-Class from Man…
Browse files Browse the repository at this point in the history
…ifest.

Resolves a bug where the `Main-Class` from a .jar file wasn't being read in properly. This was caused by the fact that the `primaryResource` object was a URI and needed to be normalized through a call to `.getPath` before it could be passed into the `JarFile` object.

Author: Brennon York <[email protected]>

Closes apache#3561 from brennonyork/SPARK-4298 and squashes the following commits:

5e0fce1 [Brennon York] Use string interpolation for error messages, moved comment line from original code to above its necessary code segment
14daa20 [Brennon York] pushed mainClass assignment into match statement, removed spurious spaces, removed { } from case statements, removed return values
c6dad68 [Brennon York] Set case statement to support multiple jar URI's and enabled the 'file' URI to load the main-class
8d20936 [Brennon York] updated to reset the error message back to the default
a043039 [Brennon York] updated to split the uri and jar vals
8da7cbf [Brennon York] fixes SPARK-4298
  • Loading branch information
Brennon York authored and JoshRosen committed Dec 31, 2014
1 parent 06a9aa5 commit 8e14c5e
Showing 1 changed file with 18 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

package org.apache.spark.deploy

import java.net.URI
import java.util.jar.JarFile

import scala.collection.mutable.{ArrayBuffer, HashMap}
Expand Down Expand Up @@ -125,14 +126,23 @@ private[spark] class SparkSubmitArguments(args: Seq[String], env: Map[String, St

// Try to set main class from JAR if no --class argument is given
if (mainClass == null && !isPython && primaryResource != null) {
try {
val jar = new JarFile(primaryResource)
// Note that this might still return null if no main-class is set; we catch that later
mainClass = jar.getManifest.getMainAttributes.getValue("Main-Class")
} catch {
case e: Exception =>
SparkSubmit.printErrorAndExit("Cannot load main class from JAR: " + primaryResource)
return
val uri = new URI(primaryResource)
val uriScheme = uri.getScheme()

uriScheme match {
case "file" =>
try {
val jar = new JarFile(uri.getPath)
// Note that this might still return null if no main-class is set; we catch that later
mainClass = jar.getManifest.getMainAttributes.getValue("Main-Class")
} catch {
case e: Exception =>
SparkSubmit.printErrorAndExit(s"Cannot load main class from JAR $primaryResource")
}
case _ =>
SparkSubmit.printErrorAndExit(
s"Cannot load main class from JAR $primaryResource with URI $uriScheme. " +
"Please specify a class through --class.")
}
}

Expand Down

0 comments on commit 8e14c5e

Please sign in to comment.