[SPARK-35347][SQL] Use MethodUtils for looking up methods in Invoke and StaticInvoke #32474
Conversation
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Test build #138280 has finished for PR 32474 at commit
|
| } else { | ||
| m.head | ||
| } | ||
| val method = MethodUtils.getMatchingAccessibleMethod(cls, functionName, argClasses: _*) |
There was a problem hiding this comment.
Technically, this has more code like setAccessibleWorkaround. Let's trigger JDK11 test.
dongjoon-hyun
changed the title
|
Retest this please |
|
Test build #138291 has finished for PR 32474 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
retest this please |
|
Test build #138297 has finished for PR 32474 at commit
|
Seems something is wrong with Jenkins? |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
Oh, it seems that UCB Amplab Jenkins lab still didn't finish the setup. In that case, let's ignore Jenkins.
After transition to run in yours' fork, only the author can re-trigger it. |
dongjoon-hyun
changed the title
|
Thanks @dongjoon-hyun. Re-triggered GA and the Java 11 build was passed. |
|
Thanks. And, did you run |
|
I tested it. |
There was a problem hiding this comment.
+1, LGTM. Merged to master for Apache Spark 3.2.0. Thank you, @viirya .
|
Thank you @dongjoon-hyun! |
| } | ||
| val method = MethodUtils.getMatchingAccessibleMethod(cls, functionName, argClasses: _*) | ||
| if (method == null) { | ||
| sys.error(s"Couldn't find $functionName on $cls") |
There was a problem hiding this comment.
shall we throw an exception to only fail the query, instead of sys.error which crashes the driver JVM?
There was a problem hiding this comment.
ok, will submit a follow-up.
…on type when cannot find the method instead of sys.error ### What changes were proposed in this pull request? A simple follow-up of #32474 to throw exception instead of sys.error. ### Why are the changes needed? An exception only fails the query, instead of sys.error. ### Does this PR introduce _any_ user-facing change? Yes, if `Invoke` or `StaticInvoke` cannot find the method, instead of original `sys.error` now we only throw an exception. ### How was this patch tested? Existing tests. Closes #32519 from viirya/SPARK-35347-followup. Authored-by: Liang-Chi Hsieh <[email protected]> Signed-off-by: Liang-Chi Hsieh <[email protected]>
What changes were proposed in this pull request?
This patch proposes to use
MethodUtilsfor looking up methodsInvokeandStaticInvokeexpressions.Why are the changes needed?
Currently we wrote our logic in
InvokeandStaticInvokeexpressions for looking up methods. It is tricky to consider all the cases and there is already existing utility package for this purpose. We should reuse the utility package.Does this PR introduce any user-facing change?
No, internal change only.
How was this patch tested?
Existing tests.