Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] [broker] Enabling batch causes negative unackedMessages due to ack and delivery concurrency #22090

Merged
merged 3 commits into from
Feb 26, 2024

Conversation

poorbarcode
Copy link
Contributor

Motivation

Context:

  • After a consumer closes, it may leave an in-progress acknowledge task.

Issue:

  • There are two consumers: c1 and c2
  • c1 received a batch message: ledger: L1, entry: E1, batch-size: 100.
  • c1 was closed, after leaving an in-progress task to acknowledge {L1, E1, batch_index: 1}
  • Pulsar read {L1, E1} and tries to delivery to c2.
  • The acknowledgement {L1, E1, batch_index: 1} is finished.
  • c2 gets 100 messages, but unackedMessages of c2 is 98.

We can always get the warn logs like below:

07:01:11.037 [pulsar-io-4-17] WARN  org.apache.pulsar.broker.service.Consumer - unackedMsgs is : -1,

Modifications

Fix the bug.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

@poorbarcode poorbarcode added this to the 3.3.0 milestone Feb 21, 2024
@poorbarcode poorbarcode self-assigned this Feb 21, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Feb 21, 2024
@Technoboy-
Copy link
Contributor

Some tests are failed due to this fix

@poorbarcode
Copy link
Contributor Author

Some tests are failed due to this fix

passes

@poorbarcode poorbarcode merged commit 1b1cfb5 into apache:master Feb 26, 2024
50 checks passed
Technoboy- pushed a commit that referenced this pull request Feb 26, 2024
Technoboy- pushed a commit that referenced this pull request Feb 27, 2024
heesung-sn pushed a commit that referenced this pull request Feb 27, 2024
…ack and delivery concurrency (#22090)

(cherry picked from commit 1b1cfb5)
poorbarcode added a commit that referenced this pull request Feb 28, 2024
…ack and delivery concurrency (#22090)

(cherry picked from commit 1b1cfb5)
(cherry picked from commit b961c49)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
…ack and delivery concurrency (apache#22090)

(cherry picked from commit aa3f53b)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
…ack and delivery concurrency (apache#22090)

(cherry picked from commit aa3f53b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants