Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix][fn]make sure the classloader for ContextImpl is functionClassLoader in different runtimes #22501

Merged
merged 2 commits into from
May 1, 2024
Merged

[fix][fn]make sure the classloader for ContextImpl is functionClassLoader in different runtimes #22501

merged 2 commits into from
May 1, 2024

Conversation

freeznet
Copy link
Contributor

@freeznet freeznet commented Apr 15, 2024 •
edited
Loading

Fixes #xyz

Main Issue: #xyz

PIP: #xyz

Motivation

#20115 fixes the classloader in TopicSchema, but for Kubernetes Runtime, the classloader may be different from the other runtimes. Which caused the SerDe to fail to init in Kubernetes Runtime.

Modifications

  • make sure the classloader always be functionClassLoader when init the context

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: freeznet#11

@freeznet
make sure the classloader for ContextImpl is functionClassLoader
3dcf150 
# Conflicts:
#	pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java
@freeznet freeznet self-assigned this Apr 15, 2024
@freeznet freeznet closed this Apr 15, 2024
@freeznet freeznet reopened this Apr 15, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 15, 2024
@freeznet freeznet requested a review from nlu90 April 15, 2024 01:21
@lhotari
Copy link
Member

lhotari commented Apr 15, 2024

#20115 fixes the classloader in TopicSchema, but for Kubernetes Runtime, the classloader may be different from the other runtimes. Which caused the SerDe to fail to init in Kubernetes Runtime.

@freeznet A question about the description:
Regarding "but for Kubernetes Runtime, the classloader may be different from the other runtimes".

  • Is it "may be different" meaning that sometimes it succeeds and sometimes fails?
    • or do you mean to say that it is always different and always fails in Kubernetes Runtime?

@freeznet
Copy link
Contributor Author

#20115 fixes the classloader in TopicSchema, but for Kubernetes Runtime, the classloader may be different from the other runtimes. Which caused the SerDe to fail to init in Kubernetes Runtime.

@freeznet A question about the description: Regarding "but for Kubernetes Runtime, the classloader may be different from the other runtimes".

  • Is it "may be different" meaning that sometimes it succeeds and sometimes fails?

    • or do you mean to say that it is always different and always fails in Kubernetes Runtime?

The reason for "may" is that I am not sure why there is no related issue mentioned before. But from my testing with function-mesh, it always occurred, and we cannot observe the issue if we fallback to the previous version prior #20115.

@Awsmsniper
Copy link

Awsmsniper commented Apr 15, 2024 •
edited
Loading

#20115 fixes the classloader in TopicSchema, but for Kubernetes Runtime, the classloader may be different from the other runtimes. Which caused the SerDe to fail to init in Kubernetes Runtime.

@freeznet A question about the description: Regarding "but for Kubernetes Runtime, the classloader may be different from the other runtimes".

  • Is it "may be different" meaning that sometimes it succeeds and sometimes fails?

    • or do you mean to say that it is always different and always fails in Kubernetes Runtime?

I'm not sure if @freeznet he has this issue every time on Kubernetes, but I have encountered it when using the extension SDK to connect to Redis cluster. I am currently unable to use functions on Kubernetes to complete my requirements.

2024-04-15T06:18:32,716+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Sink open produced uncaught exception:
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
        at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
        at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at java.lang.Thread.run(Thread.java:840) ~[?:?]
2024-04-15T06:18:32,725+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/demo-function-las-auth:0] Uncaught exception in Java Instance
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
        at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
        at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
        at java.lang.Thread.run(Thread.java:840) ~[?:?]

@lhotari
Copy link
Member

lhotari commented Apr 15, 2024

2024-04-15T06:18:32,716+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Sink open produced uncaught exception:
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]

@Awsmsniper this might be a different issue. please report it separately.

@Awsmsniper
Copy link

#20115 fixes the classloader in TopicSchema, but for Kubernetes Runtime, the classloader may be different from the other runtimes. Which caused the SerDe to fail to init in Kubernetes Runtime.

@freeznet A question about the description: Regarding "but for Kubernetes Runtime, the classloader may be different from the other runtimes".

  • Is it "may be different" meaning that sometimes it succeeds and sometimes fails?

    • or do you mean to say that it is always different and always fails in Kubernetes Runtime?

2024-04-15T06:18:32,716+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Sink open produced uncaught exception:
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]

@Awsmsniper this might be a different issue. please report it separately.

@lhotari I have thought about it for a moment, and I believe the problem is caused by the same reason. Do I need another report？

@freeznet
Copy link
Contributor Author

freeznet commented Apr 16, 2024 •
edited
Loading

@Awsmsniper it should be another issue, could you please create a new issue first?

Also, https://hub.docker.com/layers/freeznet/pulsar-all/3.0.5-SNAPSHOT-1c46877/images/sha256-d881a766d57b77ec4b279688b96b714f34eb270496b052eb69362db5c62d8bd3?context=repo is the docker image I built from this PR based on branch-3.0, you may check if this fix solves your issue as well.

@freeznet freeznet requested a review from lhotari April 17, 2024 11:06
nlu90
nlu90 approved these changes Apr 17, 2024
View reviewed changes
Copy link
Member

@nlu90 nlu90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve based our internal tests have been passed.

@Technoboy- Technoboy- added this to the 3.3.0 milestone Apr 21, 2024
@Awsmsniper
Copy link

@Awsmsniper it should be another issue, could you please create a new issue first?

Also, https://hub.docker.com/layers/freeznet/pulsar-all/3.0.5-SNAPSHOT-1c46877/images/sha256-d881a766d57b77ec4b279688b96b714f34eb270496b052eb69362db5c62d8bd3?context=repo is the docker image I built from this PR based on branch-3.0, you may check if this fix solves your issue as well.

I did not use the version you provided. After using version 3.0.4, I found that it can be used normally. Thank you.

@freeznet
Copy link
Contributor Author

@lhotari could you please help to review this PR when you have time, thanks.

lhotari
lhotari approved these changes May 1, 2024
View reviewed changes
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lhotari lhotari merged commit d067efc into apache:master May 1, 2024
49 of 50 checks passed
Technoboy- pushed a commit that referenced this pull request May 8, 2024
@freeznet @Technoboy-
[fix][fn]make sure the classloader for ContextImpl is `functionClassL…
2c92ae3 
…oader` in different runtimes (#22501)
lhotari pushed a commit that referenced this pull request May 14, 2024
@freeznet @lhotari
[fix][fn]make sure the classloader for ContextImpl is `functionClassL…
d23c77b 
…oader` in different runtimes (#22501)

(cherry picked from commit d067efc)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request May 15, 2024
@freeznet @nikhil-ctds
[fix][fn]make sure the classloader for ContextImpl is `functionClassL…
8fda0a9 
…oader` in different runtimes (apache#22501)

(cherry picked from commit d067efc)
(cherry picked from commit d23c77b)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request May 16, 2024
@freeznet @srinath-ctds
[fix][fn]make sure the classloader for ContextImpl is `functionClassL…
87c4df3 
…oader` in different runtimes (apache#22501)

(cherry picked from commit d067efc)
(cherry picked from commit d23c77b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nlu90 nlu90 nlu90 approved these changes

@lhotari lhotari lhotari approved these changes

Assignees

@freeznet freeznet

Labels
cherry-picked/branch-3.0 cherry-picked/branch-3.2 doc-not-needed Your PR changes do not impact docs ready-to-test release/3.0.5 release/3.1.4 release/3.2.3
Projects
None yet
Milestone
3.3.0
Development

Successfully merging this pull request may close these issues.
5 participants
@freeznet @lhotari @Awsmsniper @nlu90 @Technoboy-