Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify quorum queue channel resend protocol #3671

Closed
Tracked by #3121
kjnilsson opened this issue Nov 8, 2021 · 1 comment
Closed
Tracked by #3121

Simplify quorum queue channel resend protocol #3671

kjnilsson opened this issue Nov 8, 2021 · 1 comment
Assignees
Milestone

Comments

@kjnilsson
Copy link
Contributor

Quorum queues have a resend and re-sequencing protocol in place to avoid messages sent to a QQ member that isn't leader anymore causing the channel -> queue publishing ordering guarantees to be violated.

This is implemented in the rabbit_fifo_client and rabbit_fifo modules but is probably more complicated than necessary.
Currently if a channels sends an enqueue command to a follower member the follower will reply with {rejected, not_leader, Leader} and the channel will resend this message to the leader member instead. When a QQ member becomes leader it will also send a {leader_change, Leader} message to all known channels (enqueuers and consumers). When they receive this message they will unconditionally resend all pending commands.

The rabbit_fifo state machine will handle the re-sequencing state needed to handle the case where only a single message is resent. This means holding messages in memory per channel until the right sequence of message arrives creating a potential risk of memory issues.

The suggestion here is to stop re-sending individual messages when the channel receives {rejected, not_leader, Leader} events and instead just resend all pending message when detecting the leader has changed and if detecting a gap in the committed correlation ids (increasing integer value that is used to line up message). If we do this we can stop keeping out of order messages in memory during re-sequencing and instead just return dropped in the correlation response term so that rabbit_fifo_client does send a publisher confirm for the message.

@kjnilsson
Copy link
Contributor Author

Done

@michaelklishin michaelklishin added this to the 3.10.0 milestone Mar 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants