Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix asynch fetch polling without delay #490

Merged
merged 1 commit into from
Nov 21, 2022

Conversation

tvainika
Copy link
Contributor

About this change - What it does

Fix asynch fetch polling without delay in REST proxy

This is regression from #423 when REST consumer was ported from kafka-python to aiokafka.

Fixes #488

Why this way

First fetch min bytes negative values was misconfiguration even for kafka-python earlier, but worsened with aiokafka. Add extra configuration for aiokafka timeout settings tuning.

First fetch min bytes negative values was misconfiguration even for
kafka-python earlier, but worsened with aiokafka.  Add extra
configuration for aiokafka timeout settings tuning.

Fixes #488
@tvainika tvainika requested review from a team as code owners November 11, 2022 15:11
Copy link
Contributor

@jjaakola-aiven jjaakola-aiven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I testes manually and by setting the fetch.min.bytes to -1 the CPU issue can be reproduced.
With the fix proposed here I cannot reproduce the issue.

@@ -35,7 +35,7 @@
"consumer_request_timeout_ms": 11000,
"consumer_request_max_bytes": 67108864,
"consumer_idle_disconnect_timeout": 0,
"fetch_min_bytes": -1,
"fetch_min_bytes": 1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct, valid values are > 0 and default is 1.
https://kafka.apache.org/documentation/#consumerconfigs_fetch.min.bytes

@jjaakola-aiven jjaakola-aiven merged commit a47eb8d into main Nov 21, 2022
@jjaakola-aiven jjaakola-aiven deleted the fix-488-aiokafka-fetcher-polling branch November 21, 2022 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Karapace 3.4.x consuming all CPU
2 participants