Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[runtime] Use CollectCoordinationResponse as a wrapper of CoordinationResponse #2744

Closed
2 tasks done
Tracked by #2600
PatrickRen opened this issue Nov 23, 2023 · 0 comments
Closed
2 tasks done
Tracked by #2600

[runtime] Use CollectCoordinationResponse as a wrapper of CoordinationResponse #2744

PatrickRen opened this issue Nov 23, 2023 · 0 comments
Assignees
@PatrickRen
Labels
enhancement New feature or request 【3.0】 task for Flink CDC 3.0

Comments

@PatrickRen
Copy link
Contributor

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse ca
n only be deserialized by AppClassloader instead of user code classloader, so we can't use customize Coordinat
ionResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined
in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for ho
lding the actual serialized custom CoordinationResponse.

Solution

No response

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@PatrickRen PatrickRen added enhancement New feature or request 【3.0】 task for Flink CDC 3.0 labels Nov 23, 2023
@PatrickRen PatrickRen self-assigned this Nov 23, 2023
PatrickRen added a commit to PatrickRen/flink-cdc that referenced this issue Nov 23, 2023
@PatrickRen
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of Coordin…
270ac63 
…ationResponse

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.

This closes apache#2744
PatrickRen added a commit to PatrickRen/flink-cdc that referenced this issue Nov 23, 2023
@PatrickRen
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of Coordin…
960fb54 
…ationResponse

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.

This closes apache#2744
PatrickRen added a commit to PatrickRen/flink-cdc that referenced this issue Nov 23, 2023
@PatrickRen
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of Coordin…
3d30522 
…ationResponse

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.

This closes apache#2744
PatrickRen added a commit to PatrickRen/flink-cdc that referenced this issue Nov 23, 2023
@PatrickRen
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of Coordin…
6c7dbe8 
…ationResponse

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.

This closes apache#2744
@PatrickRen PatrickRen mentioned this issue Nov 24, 2023
Closed
57 tasks
e-mhui pushed a commit to e-mhui/flink-cdc-connectors that referenced this issue Dec 2, 2023
@PatrickRen @e-mhui
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of Coordin…
892e8a2 
…ationResponse

The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.

This closes apache#2744
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PatrickRen PatrickRen

Labels
enhancement New feature or request 【3.0】 task for Flink CDC 3.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@PatrickRen