Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race condition in IOUringEventLoop that prevents shutdown #227

Merged

Conversation

bryce-anderson
Copy link
Contributor

Motivation:

Short lived IOUringEventLoop's can fail to shutdown, being stuck in the read-completion phase. This is because under race conditions we can get a task that skips the submission of the eventFd and then exits.

Modifications:

  • After setting the initial eventfd read event we flush it. This is a one time cost per-loop and avoids the case where a wakeup was requested but never received because the loop enqueued but didn't submit the interest in receiving those event.

Result:

Less stalled threads on shutdown.

Motivation:

Short lived IOUringEventLoop's can fail to shutdown, being stuck
in the read-completion phase. This is because under race conditions
we can get a task that skips the submission of the eventFd and
then exits.

Modifications:

- After setting the initial eventfd read event we flush it. This
  is a one time cost per-loop and avoids the case where a wakeup was
  requested but never received because the loop enqueued but didn't
  submit the interest in receiving those event.

Result:

Less stalled threads on shutdown.
@bryce-anderson
Copy link
Contributor Author

For details about the debugging done to find this see #226

Copy link
Contributor

@franz1981 franz1981 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done!!!

@bryce-anderson
Copy link
Contributor Author

Discussed with @chrisvest offline, and presuming this is good to go, we should port the test to Netty5.

@normanmaurer normanmaurer added this to the 0.0.24.Final milestone Nov 16, 2023
Copy link
Member

@normanmaurer normanmaurer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @bryce-anderson

@normanmaurer normanmaurer merged commit 160dfde into netty:main Nov 16, 2023
8 checks passed
@idelpivnitskiy
Copy link
Member

Thanks a lot for fixing it!

@normanmaurer
Copy link
Member

@bryce-anderson can you also open a PR for netty 5 ?

@bryce-anderson bryce-anderson deleted the bl_anderson/io_uring_shutdown_stall_fix branch November 16, 2023 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants