[SPARK-48056][CONNECT][PYTHON] Re-execute plan if a SESSION_NOT_FOUND error is raised and no partial response was received #46297
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nija-at
changed the title
nija-at
changed the title
nija-at
changed the title
grundprinzip
approved these changes
Apr 30, 2024
HyukjinKwon
approved these changes
Apr 30, 2024
nija-at
force-pushed
the
session-not-found
branch
from
8e5c921
to
15d332f
Compare
|
@juliuszsompolski - this will be followed up in a separate PR |
juliuszsompolski
approved these changes
Apr 30, 2024
grundprinzip
reviewed
May 2, 2024
HyukjinKwon
approved these changes
May 2, 2024
|
Merged to master. |
There was a problem hiding this comment.
Hi, @nija-at , @grundprinzip, @juliuszsompolski , @HyukjinKwon .
I made a follow-up to fix Python 3.12 issue. assertEquals is removed at Python 3.12.
|
This seems to be missed in another PR too. Maybe, we had better find a way to prevent this. |
dongjoon-hyun
added a commit
that referenced
this pull request
May 4, 2024
…f `assertEquals` for Python 3.12 ### What changes were proposed in this pull request? This is a follow-up of - #46297 This PR aims to use `assertEqual` instead of `assertEquals` for Python 3.12. ### Why are the changes needed? To recover Python CI, - https://github.com/apache/spark/actions/workflows/build_python.yml From Python 3.12, `assertEquals` doesn't exist. https://docs.python.org/3/library/unittest.html#assert-methods ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46377 from dongjoon-hyun/SPARK-48056. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
|
Thanks for fixing |
JacobZheng0927
pushed a commit
to JacobZheng0927/spark
that referenced
this pull request
May 11, 2024
… error is raised and no partial response was received ### What changes were proposed in this pull request? Similar to OPERATION_NOT_FOUND, re-attempt to execute the original spark connect plan when a SESSION_NOT_FOUND is received from the spark connect service and no partial responses were previously received. ### Why are the changes needed? This error has been noticed to occur during a cluster cold start and when a request arrives when the connect service is not fully initialized. ### Does this PR introduce _any_ user-facing change? Prevoiusly, connect-based pyspark APIs would fail with the error code "INVALID_HANDLE.SESSION_NOT_FOUND" in the very first request to the service. With this change, the client will now automatically retry. ### How was this patch tested? Attached unit tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46297 from nija-at/session-not-found. Authored-by: Niranjan Jayakar <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
JacobZheng0927
pushed a commit
to JacobZheng0927/spark
that referenced
this pull request
May 11, 2024
…f `assertEquals` for Python 3.12 ### What changes were proposed in this pull request? This is a follow-up of - apache#46297 This PR aims to use `assertEqual` instead of `assertEquals` for Python 3.12. ### Why are the changes needed? To recover Python CI, - https://github.com/apache/spark/actions/workflows/build_python.yml From Python 3.12, `assertEquals` doesn't exist. https://docs.python.org/3/library/unittest.html#assert-methods ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46377 from dongjoon-hyun/SPARK-48056. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
@juliuszsompolski I just uploaded a scala version of it: #46971, could you review this? |
zhengruifeng
pushed a commit
that referenced
this pull request
Jun 14, 2024
…ESSION_NOT_FOUND error is raised and no partial response was received ### What changes were proposed in this pull request? This change lets a Scala Spark Connect client reattempt execution of a plan when it receives a SESSION_NOT_FOUND error from the Spark Connect service if it has not received any partial responses. This is a Scala version of the previous fix of the same issue - #46297. ### Why are the changes needed? Spark Connect clients often get a spurious error from the Spark Connect service if the service is busy or the network is congested. This error leads to a situation where the client immediately attempts to reattach without the service being aware of the client; this leads to a query failure. ### Does this PR introduce _any_ user-facing change? Prevoiusly, a Scala Spark Connect client would fail with the error code "INVALID_HANDLE.SESSION_NOT_FOUND" in the very first attempt to make a request to the service, but with this change, the client will automatically retry. ### How was this patch tested? Attached unit test. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46971 from changgyoopark-db/SPARK-48056. Authored-by: Changgyoo Park <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Similar to OPERATION_NOT_FOUND, re-attempt to execute
the original spark connect plan when a SESSION_NOT_FOUND is
received from the spark connect service and no partial responses
were previously received.
Why are the changes needed?
This error has been noticed to occur during a cluster cold start
and when a request arrives when the connect service is not fully
initialized.
Does this PR introduce any user-facing change?
Prevoiusly, connect-based pyspark APIs would fail with the error code
"INVALID_HANDLE.SESSION_NOT_FOUND" in the very first request to
the service.
With this change, the client will now automatically retry.
How was this patch tested?
Attached unit tests.
Was this patch authored or co-authored using generative AI tooling?
No.