Performance issues

[tingfeng-key]

A Performance Evaluation on Rust Asynchronous Frameworks

I saw this analysis. Does Tokio have an optimization plan?

[Darksonn]

Looking over the code, the following thoughts come to mind:

1. The version of the code available when the article was posted ran all of their code inside block_on rather than inside spawned task. It's known that this hurts performance of IO in Tokio. The ping tests appear to have fixed this since the article was posted, but it's still the case in the pong tests, and the article hasn't been updated since the ping tests were fixed.
2. The tests appear to make significant use of sleeping to time the messages. However, Tokio's timer only has a resolution of one millisecond. This can cause the Tokio tests to spend a lot longer sleeping than the other tests. For example, the benchmarks uses sleeps all the way down to one microsecond, which Tokio will handle by not sleeping at all 999 times, then sleeping for one full millisecond the 1000th time. (This is not a completely precise description, but the behavior is similar to this.)

I suspect that the timer issue is probably the bigger problem in this case.

All that said, there are areas where we know that better performance is possible. The mpsc channel is the main example. Additionally, there has been some discussion about whether we can improve the performance of code running in block_on by letting block_on calls drive the IO driver, which is a common pitfall in benchmarks.

[spastorino]

@gabrik, @carllerche asked me to mention you in this issue. I'd guess you may want to bring Zenoh issues here.

[Mallets]

Thanks @spastorino and @carllerche for bringing this discussion to the Zenoh team attention.

@Darksonn thanks for taking the time to go through our tests and provide feedback.
See a couple of comments from our side here below:

1 - Ping/Pong test

The ping tests appear to have fixed this since the article was posted, but it's still the case in the pong tests.
We will update the pong test and verify if there is any impact on performance.

2 - Sleep resolution

For example, the benchmarks uses sleeps all the way down to one microsecond, which Tokio will handle by not sleeping at all 999 times, then sleeping for one full millisecond the 1000th time.

Very interesting insight on how sleep work. A bit more of context on Zenoh and our use case: what we mainly deal with is machine-type of communication which requires lower latency and higher guarantees in terms of latency and jitter compared to e.g. web applications. In that context, we need to pace and prioritise different traffic flows with a resolution of few microseconds.

Do you hence have any idea on how to achieve that pattern in Tokio or, in a Tokio-friendly manner?
We would like avoiding busy loops and alike that may interfere with the Tokio scheduler.

[Darksonn]

If Tokio's built-in timer has a too low resolution for your needs, then there are some other options you can try. One of them is to create a timer file descriptor and put it in Tokio's AsyncFd. If you have multiple timers, then you would need to handle them yourself (e.g. put them in a BTreeMap or BinaryHeap and use that to find the minimum timer), but it should allow high resolution timers. The tokio-timerfd crate should be usable for this purpose.

Another option is to try out other crates that provide timers. For example, there's the futures-time crate. It works by spawning a separate thread dedicated for sending out timer wakeups. However, I don't know what the resolution for its timers is. Using the crate involves some cross-thread communication costs that you don't get with Tokio's timer, but if using block_on doesn't affect your performance, then this probably wont either since the costs from using block_on are also just cross-thread communication overhead.

[Mallets]

Thanks for the hints and the pointers, we will be looking into it.

[carllerche]

Tokio aims to provide compatibility across many platforms. Unfortunately, across supported platforms, we can only guarantee the OS provides us w/ ms level timer resolution. This is why we have 1ms as a hard limit. We also decided to not support lower resolution when available as that would lead to applications running differently based on the platform.

As @Darksonn mentioned, the solution if you need higher resolution is to pick a custom timer API that meets your portability needs and resolution.

[carllerche]

The reason we round up is, if you try to set sub ms granularity timers on platforms that don't support it, you end up w/ a "yield" (round down to zero) which results in a tight loop and burning CPU. We had bug reports a while back from that, so now opt to round up.