Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][client] Fix perf-producer get OOM with high publish latency #20914

Merged
merged 1 commit into from
Aug 3, 2023

Conversation

codelipenghui
Copy link
Contributor

Motivation

Fix the perf-producer to get direct memory OOM with high publish latency.
The root cause is the perf-producer without any memory limitation and pending
queue size limitation by default.

You can try to set the max pending queue size for a workaround.

Modifications

Use the default configuration of the PulsarClient instead of removing the memory
limitation in pulsar-perf.

We should also provide an option to specify the max memory for the perf tool.
But it will introduce new public API changes to the perf tool, so it should only be available
in the feature release. This PR is focused on the BUG fix to avoid confusing users
with OOM.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@codelipenghui codelipenghui self-assigned this Aug 2, 2023
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Aug 2, 2023
@codelipenghui codelipenghui added this to the 3.2.0 milestone Aug 2, 2023
@Technoboy- Technoboy- changed the title [fix][tool] fix perf-producer get OOM with high publish latency [fix][tool] Fix perf-producer get OOM with high publish latency Aug 2, 2023
@Technoboy- Technoboy- changed the title [fix][tool] Fix perf-producer get OOM with high publish latency [fix][client] Fix perf-producer get OOM with high publish latency Aug 2, 2023
@Technoboy- Technoboy- closed this Aug 2, 2023
@Technoboy- Technoboy- reopened this Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants