Skip to content

Commit

Permalink
Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to…
Browse files Browse the repository at this point in the history
… 4.8.1"

### What changes were proposed in this pull request?
This pr aims to  Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to 4.8.1".

### Why are the changes needed?
As mentioned in #40442,  there are some regression with the 4.8.1:

- Run `./build/mvn -DskipTests clean package` with Java 17 will failed
```
[INFO] --- maven-compiler-plugin:3.11.0:compile (default-compile)  spark-core_2.12 ---
[INFO] Not compiling main sources
[INFO]
[INFO] --- scala-maven-plugin:4.8.1:compile (scala-compile-first)  spark-core_2.12 ---
[INFO] Compiler bridge file: /home/bjorn/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.8.0-bin_2.12.17__55.0-1.8.0_20221110T195421.jar
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.17,1.7.10,null)
[INFO] compiling 597 Scala sources and 103 Java sources to /home/bjorn/github/spark/core/target/scala-2.12/classes ...
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala:71: not found: value sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:26: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:27: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:206: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:210: not found: type Unsafe
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:212: not found: type Unsafe
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:213: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:216: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:236: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala:452: not found: value sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type SignalHandler
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:83: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: type SignalHandler
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: value Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:114: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:116: not found: value Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:128: not found: value Signal
[ERROR] 19 errors found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.5.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 3.848 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 12.106 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 10.685 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 8.743 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 9.362 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.828 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 9.071 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 4.776 s]
[INFO] Spark Project Core ................................. FAILURE [ 17.228 s]
```
- Run `build/mvn clean install  -DskipTests -Pscala-2.13` with Java8 + Scala 2.13

There are compilation errors as `ERROR] -release is only supported on Java 9 and higher`  although it does not cause compilation failures. More, I saw davidB/scala-maven-plugin#686, So  it seems that 4.8.1 and Java 8 are not compatible well.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GA

Closes #40482 from LuciferYang/REVERT-SPARK-42809.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
  • Loading branch information
LuciferYang authored and yaooqinn committed Mar 20, 2023
1 parent 90f4d10 commit 1530e8d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@
errors building different Hadoop versions.
See: SPARK-36547, SPARK-38394.
-->
<scala-maven-plugin.version>4.8.1</scala-maven-plugin.version>
<scala-maven-plugin.version>4.8.0</scala-maven-plugin.version>
<versions-maven-plugin.version>2.15.0</versions-maven-plugin.version>
<!-- for now, not running scalafmt as part of default verify pipeline -->
<scalafmt.skip>true</scalafmt.skip>
Expand Down

0 comments on commit 1530e8d

Please sign in to comment.