time: eliminate timer wheel allocations #6779

tglane · 2024-08-15T17:39:13Z

Motivation

#6683 introduced additional allocations to keep the locks of the timer wheel shards alive. #6757 is a request to get rid of those additional allocations. This PR should address this issue.

Solution

I wrapped the timer wheel shards in an RwLock. With that we can get a write-lock in park_internal to prevent other threads from accessing the shards. In all other places the read-lock can be used for simultaneous access.

Closes #6757

RwLock

tokio/src/runtime/time/mod.rs

hawkw · 2024-08-16T22:48:26Z

tokio/src/runtime/time/mod.rs

-        self.wheels.len() as u32
+        self.wheels
+            .read()
+            .expect("Timer wheel shards poisened")


It's a bit unfortunate that this can panic. Not because the panic itself is bad, if the lock is poisoned, we should panic, but because panicking code will make this otherwise small and trivially inlined function generate way more code. Perhaps we should move the number of shards outside of the lock, instead? Since it will never change, it should be fine to do that...

also, a typo:

Suggested change

.expect("Timer wheel shards poisened")

.expect("Timer wheel shards poisoned")

Seems reasonable to put it as a member variable into the Inner struct to avoid the lock here. The increase in size of the struct would probably be ok since it's "only" a u32, so I would agree to do it like that if nobody objects.

I included the changes in e9c4351.

I also removed the method to get the shard size from the struct ShardedWheel to prevent duplications since we can get it from the struct Inner everywhere where It's needed.

tokio/src/runtime/time/mod.rs

Darksonn

Overall looks okay, but a few more details:

Darksonn · 2024-08-21T09:40:36Z

tokio/src/runtime/time/mod.rs

+            let expiration_time = (0..rt_handle.time().inner.get_shard_size())
+                .filter_map(|id| {
+                    let wheel = wheels_lock.get_sharded_wheel_mut(id);
+                    wheel.next_expiration_time()
+                })


Could something like this make more sense?

wheels_lock.0.iter_mut() .filter_map(|wheel| wheel.get_mut().next_expiration_time()) .min();

This way, we don't need to touch indexes at all.

I like this. At least when calculating the min expiration time we indeed don't need the indexes anymore and we can get rid of one function.
Implemented it in 1610c12

Darksonn · 2024-08-21T09:41:59Z

tokio/src/runtime/time/mod.rs

@@ -334,10 +352,16 @@ impl Handle {
                if !waker_list.can_push() {
                    // Wake a batch of wakers. To avoid deadlock, we must do this with the lock temporarily dropped.
                    drop(lock);
+                    drop(wheels_lock);


You also need to drop wheels_lock after the loop, since there's also a call to wake_all there.

Sure! Missed that... Implemented in 1610c12.

Darksonn

Thanks, looks good to me.

Bumps tokio from 1.39.3 to 1.40.0. Release notes Sourced from tokio's releases. Tokio v1.40.0 1.40.0 (August 30th, 2024) Added io: add util::SimplexStream (#6589) process: stabilize Command::process_group (#6731) sync: add {TrySendError,SendTimeoutError}::into_inner (#6755) task: add JoinSet::join_all (#6784) Added (unstable) runtime: add Builder::{on_task_spawn, on_task_terminate} (#6742) Changed io: use vectored io for write_all_buf when possible (#6724) runtime: prevent niche-optimization to avoid triggering miri (#6744) sync: mark mpsc types as UnwindSafe (#6783) sync,time: make Sleep and BatchSemaphore instrumentation explicit roots (#6727) task: use NonZeroU64 for task::Id (#6733) task: include panic message when printing JoinError (#6753) task: add #[must_use] to JoinHandle::abort_handle (#6762) time: eliminate timer wheel allocations (#6779) Documented docs: clarify that [build] section doesn't go in Cargo.toml (#6728) io: clarify zero remaining capacity case (#6790) macros: improve documentation for select! (#6774) sync: document mpsc channel allocation behavior (#6773) #6589: tokio-rs/tokio#6589 #6724: tokio-rs/tokio#6724 #6727: tokio-rs/tokio#6727 #6728: tokio-rs/tokio#6728 #6731: tokio-rs/tokio#6731 #6733: tokio-rs/tokio#6733 #6742: tokio-rs/tokio#6742 #6744: tokio-rs/tokio#6744 #6753: tokio-rs/tokio#6753 #6755: tokio-rs/tokio#6755 #6762: tokio-rs/tokio#6762 #6773: tokio-rs/tokio#6773 #6774: tokio-rs/tokio#6774 #6779: tokio-rs/tokio#6779 #6783: tokio-rs/tokio#6783 #6784: tokio-rs/tokio#6784 #6790: tokio-rs/tokio#6790 Commits ea6d652 chore: prepare Tokio v1.40.0 (#6806) 11f66f4 chore: replace ready! with std::task::ready! (#6804) 479a56a time: eliminate timer wheel allocations (#6779) b37f0de runtime: implement initial set of task hooks (#6742) c9fad08 codec: fix typo in the docs for Encoder::Error (#6800) cc70a21 task: add join_all method to JoinSet (#6784) 1ac8dff task: add AbortOnDropHandle type (#6786) ff3f2a8 io: add SimplexStream (#6589) 5b9a290 io: clarify zero remaining capacity case (#6790) 70569bd task: fix typo in TaskTracker docs (#6792) Additional commits viewable in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

With tokio-rs#6779 we removed unnecessary allocations from the timerwheel by wrapping it in an `std::sync::RwLock`. Since the `Mutex` used in this part of the project uses an abstraction in `loom::sync::Mutex` to get rid of the poisoning aspects of `std::sync::Mutex` the same should probably be done for the used read-write lock struct. This commit introduces an abstraction to get rid of the poisoning aspects of `std::sync::RwLock` by introducing a wrapper to the `loom::sync` module similar to `loom::sync::Mutex`. Refs: tokio-rs#6779

With #6779 we removed unnecessary allocations from the timerwheel by wrapping it in an `std::sync::RwLock`. Since the `Mutex` used in this part of the project uses an abstraction in `loom::sync::Mutex` to get rid of the poisoning aspects of `std::sync::Mutex` the same should probably be done for the used read-write lock struct. This commit introduces an abstraction to get rid of the poisoning aspects of `std::sync::RwLock` by introducing a wrapper to the `loom::sync` module similar to `loom::sync::Mutex`. Refs: #6779

github-actions bot added the R-loom-time-driver Run loom time driver tests on this PR label Aug 15, 2024

tglane changed the title ~~time: Eliminate timer wheel allocations~~ time: eliminate timer wheel allocations Aug 15, 2024

tglane force-pushed the 6757-timer-wheel-allocations branch from 3e12f33 to 083cf36 Compare August 15, 2024 17:53

Eliminate timer wheel allocations by wrapping the wheel shards in an

302a757

RwLock

tglane force-pushed the 6757-timer-wheel-allocations branch from 083cf36 to 302a757 Compare August 15, 2024 18:04

mox692 added A-tokio Area: The main tokio crate M-time Module: tokio/time labels Aug 16, 2024

mox692 requested a review from wathenjiang August 16, 2024 06:00

Darksonn reviewed Aug 16, 2024

View reviewed changes

tokio/src/runtime/time/mod.rs Outdated Show resolved Hide resolved

Unlock complete timer wheel before call to wake_all

68e5cf1

hawkw reviewed Aug 16, 2024

View reviewed changes

Review recommendations

e9c4351

Darksonn reviewed Aug 21, 2024

View reviewed changes

Drop lock and simplify calculation of expiration time

1610c12

tglane requested review from Darksonn and hawkw August 27, 2024 18:06

Darksonn approved these changes Aug 27, 2024

View reviewed changes

Darksonn merged commit 479a56a into tokio-rs:master Aug 27, 2024
81 checks passed

mox692 mentioned this pull request Aug 29, 2024

chore: prepare Tokio v1.40.0 #6806

Merged

tglane mentioned this pull request Aug 30, 2024

loom: add abstraction for RwLock to remove poisoning aspect #6807

Merged

andrewdavidmackenzie mentioned this pull request Sep 2, 2024

build(deps): bump tokio from 1.39.3 to 1.40.0 andrewdavidmackenzie/flow#2299

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

time: eliminate timer wheel allocations #6779

time: eliminate timer wheel allocations #6779

tglane commented Aug 15, 2024

hawkw Aug 16, 2024

hawkw Aug 16, 2024

tglane Aug 18, 2024

tglane Aug 18, 2024 •

edited

Loading

Darksonn left a comment

Darksonn Aug 21, 2024

tglane Aug 21, 2024 •

edited

Loading

Darksonn Aug 21, 2024

tglane Aug 21, 2024 •

edited

Loading

Darksonn left a comment

	.expect("Timer wheel shards poisened")
	.expect("Timer wheel shards poisoned")

time: eliminate timer wheel allocations #6779

time: eliminate timer wheel allocations #6779

Conversation

tglane commented Aug 15, 2024

Motivation

Solution

hawkw Aug 16, 2024

Choose a reason for hiding this comment

hawkw Aug 16, 2024

Choose a reason for hiding this comment

tglane Aug 18, 2024

Choose a reason for hiding this comment

tglane Aug 18, 2024 • edited Loading

Choose a reason for hiding this comment

Darksonn left a comment

Choose a reason for hiding this comment

Darksonn Aug 21, 2024

Choose a reason for hiding this comment

tglane Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Darksonn Aug 21, 2024

Choose a reason for hiding this comment

tglane Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Darksonn left a comment

Choose a reason for hiding this comment

tglane Aug 18, 2024 •

edited

Loading

tglane Aug 21, 2024 •

edited

Loading

tglane Aug 21, 2024 •

edited

Loading