-
-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Avoid assertion when hitting shutdown_timeout #323
Conversation
Codecov Report
@@ Coverage Diff @@
## master #323 +/- ##
==========================================
+ Coverage 87.18% 87.55% +0.37%
==========================================
Files 49 49
Lines 3971 3993 +22
==========================================
+ Hits 3462 3496 +34
+ Misses 509 497 -12
Continue to review full report at Codecov.
|
Notes from an offline discussion: When the crash happens close to startup, it is more likely that it interrupts an ongoing upload. In this case, it would be better to avoid dropping it. If we refcount the envelopes in the transport queue, we could have multiple threads read and clean the queue concurrently. |
dump send queue on unclean shutdown
2f50fa6
to
0073054
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good now. I'll to a final run through this and the tests tomorrow, to check if we're lacking coverage for anything vital.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good to go now. Thank you for taking a deep dive into this.
This reworks the task locking and background worker shutdown logic.
Tasks are now refcounted so they can be executed concurrently while another thread removes them from the queue. In case the background worker can’t be cleanly shut down (due to slow network), the task queue is being dumped, but the background worker itself is not freed from the main thread, to avoid potential use-after-free bugs.
Also, the background worker itself is refcounted and owns its state.
The transport state is now owned by the background worker, and passed explicitly to the task send function. The task state itself was removed in favor of providing the envelope directly.