This repository has been archived by the owner on Jan 9, 2020. It is now read-only.
driver pod not handle some executor pod fail #600
Comments
echarles
pushed a commit
to datalayer-externals/spark
that referenced
this issue
Jun 10, 2022
### What changes were proposed in this pull request? This pr aims upgrade scala-maven-plugin to 4.6.2 ### Why are the changes needed? This version brings some bug fix related to `Incremental compile`, although it seems that Spark has not encountered these issue: - [Fix incremental compiler not being able to find JDK classes when compiler macros with Java 11, close apache-spark-on-k8s#502](davidB/scala-maven-plugin#608) - [Fix incremental compilation on Java 11+, close apache-spark-on-k8s#600](davidB/scala-maven-plugin#609) all changes as follows: - davidB/scala-maven-plugin@4.6.1...4.6.2 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass Github Actions Closes apache#36800 from LuciferYang/scala-maven-plugin-462. Authored-by: yangjie01 <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
If an executor pod failed before register to the driver, the driver can't aware these executors and can't create a new executor.
e.g.
the pic below shows an exception before register to the driver, which is raised by CoarseGrainedExecutorBackend.askAsync
the driver expects the executor to register but not handle the failed executor, the log below shows that
the reason is that when watch executor, we have not handle the failed executor event which is Action.MODIFY and in Failed phase.
I will pull a request to fix this
The text was updated successfully, but these errors were encountered: