From 1530e8dc44d7d963791ede03d54b1c2ad5e91c99 Mon Sep 17 00:00:00 2001 From: yangjie01 Date: Mon, 20 Mar 2023 16:24:10 +0800 Subject: [PATCH] Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to 4.8.1" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### What changes were proposed in this pull request? This pr aims to Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to 4.8.1". ### Why are the changes needed? As mentioned in https://github.com/apache/spark/pull/40442, there are some regression with the 4.8.1： - Run `./build/mvn -DskipTests clean package` with Java 17 will failed ``` [INFO] --- maven-compiler-plugin:3.11.0:compile (default-compile) spark-core_2.12 --- [INFO] Not compiling main sources [INFO] [INFO] --- scala-maven-plugin:4.8.1:compile (scala-compile-first) spark-core_2.12 --- [INFO] Compiler bridge file: /home/bjorn/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.8.0-bin_2.12.17__55.0-1.8.0_20221110T195421.jar [INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.17,1.7.10,null) [INFO] compiling 597 Scala sources and 103 Java sources to /home/bjorn/github/spark/core/target/scala-2.12/classes ... [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala:71: not found: value sun [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:26: not found: object sun [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:27: not found: object sun [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:206: not found: type DirectBuffer [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:210: not found: type Unsafe [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:212: not found: type Unsafe [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:213: not found: type DirectBuffer [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:216: not found: type DirectBuffer [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:236: not found: type DirectBuffer [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala:452: not found: value sun [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26: not found: object sun [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type SignalHandler [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type Signal [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:83: not found: type Signal [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: type SignalHandler [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: value Signal [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:114: not found: type Signal [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:116: not found: value Signal [ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:128: not found: value Signal [ERROR] 19 errors found [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Spark Project Parent POM 3.5.0-SNAPSHOT: [INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [ 3.848 s] [INFO] Spark Project Tags ................................. SUCCESS [ 12.106 s] [INFO] Spark Project Sketch ............................... SUCCESS [ 10.685 s] [INFO] Spark Project Local DB ............................. SUCCESS [ 8.743 s] [INFO] Spark Project Networking ........................... SUCCESS [ 9.362 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.828 s] [INFO] Spark Project Unsafe ............................... SUCCESS [ 9.071 s] [INFO] Spark Project Launcher ............................. SUCCESS [ 4.776 s] [INFO] Spark Project Core ................................. FAILURE [ 17.228 s] ``` - Run `build/mvn clean install -DskipTests -Pscala-2.13` with Java8 + Scala 2.13 There are compilation errors as `ERROR] -release is only supported on Java 9 and higher` although it does not cause compilation failures. More, I saw https://github.com/davidB/scala-maven-plugin/issues/686, So it seems that 4.8.1 and Java 8 are not compatible well. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GA Closes #40482 from LuciferYang/REVERT-SPARK-42809. Authored-by: yangjie01 Signed-off-by: Kent Yao --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 3fc4153f40292..86f6435ee861b 100644 --- a/pom.xml +++ b/pom.xml @@ -177,7 +177,7 @@ errors building different Hadoop versions. See: SPARK-36547, SPARK-38394. --> - 4.8.1 + 4.8.0 2.15.0 true