Pre-RFC Tokio/Futures Cookbook

[rrichardson]

Intent

The deliverable of this ticket will be a fairly complete list of topics for v1 of a Tokio and Futures cookbook, as well as a document structure and design guideline for ongoing additions. The finished product is intended to address the cookbook part of #13.

In addition to fielding and refining topics, there is a question of the actual structure of the topics as well. This is because many of the topics have valid solutions as compositions of combinators, or as implementations as futures traits (or both).

Also, at the high level, I'd like to come up with heuristics for the size and scope of what is considered a 'recipe' as many of these might exceed those, and could be broken down into sub-recipes.

Format

The plan is to plagiarize the structure and format of the existing Rust Cookbook

Hosting

This effort does not propose an actual hosting location. It is hoped that it will live either as a subcategory of the existing tokio documentation site, or under a Futures topic of the Rust Cookbook.

Topics

Common Tasks

1. Starting and stopping the reactor core.
2. Interacting with async interfaces that don't have an FD
   1. Polling resources (e.g. hardware pins, etc)
   2. Callback APIs
   3. Non-blocking APIs (try_*, etc)
3. Running multiple reactors/execs in multiple threads.
4. Running 1 reactor/exec in multiple threads.
5. Sending to futures from blocking/sync threads.
6. Receiving from futures in blocking/sync threads.
7. Aggregating from multiple streams into a single stream.

Managing Monomorphization (need a better name)

1. Returning multiple error types from a function
2. Returning multiple futures/streams types but with the same Item
   1. (from match/if let)
3. A workflow that might produce one of several futures based on some conditional

Io Tasks

1. Read/Write a file asynchronously, complete future when read/write completes. (futures_cpupool)
2. Stream stdin, Sink stdout while not blocking (similar to Io Tasks Fix spelling mistake #1)

Standard Protocols

1. http 1 and 2.0 client basic request response
2. http 1.1 and 2.0 server - basic RESTful style
3. Database read/write
4. TLS / APN
5. ASIO? (not common, but interesting)

Advanced networking

1. (client or server) streams of data that reconnect semi-transparently.
2. Managing connections with out of band data (separating ping/heartbeats)

[dbrgn]

The issue I struggled with for a few weeks was having an open bidirectional WebSocket connection, but at the same time allowing the user to enqueue messages to be sent. So the sink of the WebSocket must be available for both user messages and signaling messages.

I finally solved the issue by using two streams and one sink, and async channels: https://github.com/saltyrtc/saltyrtc-client-rs/blob/0c44e669941b62a6cb8a2fed4dbc2399f40703b6/src/lib.rs#L581-L726 Maybe something like that (simplified) could be included. I struggled with the fact that most examples only do request-response, which is easy. (My code is still quite ugly with many ignored errors, should get cleaned up over the next few days.)

[rrichardson]

Thanks, @dbrgn - so maybe your example could be described as "converging multiple event sources (streams or otherwise) into a sink" ?

[dbrgn]

Yes, I think that would work!

Maybe another one could be "communicating between futures using channels".

[eminence]

Another common example in the "multiple event sources" category (this is something I've struggled with multiple times) is: a single event loop that can wait on a TCP socket, a UDP socket, and a timer

[timClicks]

One thing that would be useful would be listening to STDIN and writing to STDOUT/STDERR in a cross-portable way. I spent hours trying to figure out how things work in Windows, only to discover that they don't really.

[rrichardson]

@eminence - Yea, that should be one of the core "design patterns". That should be right in the wheelhouse of a good futures/streams library.

@timClicks - Hmm. I definitely agree. Really Stdout isn't async on Linux either, because stdout buffers no matter what, and it might block. This should be documented.. If we make a Recipe for it, it would be something like "stdout as a Sink" and "stdin as a source" which would have an explanation and some example code for how to use a thread and blocking code to achive your goal.

[rrichardson]

I was just pointed to rust-lang-nursery/rust-cookbook#267
There was a request in that ticket for a demonstration how to load a file asynchronously.
That's a fun one, since there are multiple levels to that on multiple platforms ASIO vs simply loading a file out of band in a thread, and signalling the completion of the future. I am fairly sure if you poll for READ on a file descriptor that points to a file with contents, it will always return read-able. That doesn't mean it won't block while actually doing the xfer, though. If someone wants to load a large file in an async way, they probably mean background-thread.

[carllerche]

Sorry for the delay, we should definitely put together a cookbook. Docs should become a focus very soon as the final bits of the release come together.

[Lokathor]

Might I request a tutorial on the best way to run something like a chat server? Could just be line based telnet, doesn't have to be a full IRC stack or anything. A lot of the proposed examples look like a "connect, response, close" style. Something where connections have to be maintained and the server tracks some state over time would be a good way to show a "serious" use case for the lib.

[kpp]

Might I request a tutorial on the best way to run something like a chat server?

@Lokathor tutorial: https://tokio.rs/docs/getting-started/chat/ example: https://github.com/tokio-rs/tokio/blob/master/examples/chat.rs

[carllerche]

A good example would be to write a stream combinator that takes up to M messages or process for up to N seconds.

[dbischof90]

I think it's also important to have a few throughout futures examples that do not necessarily rely on network I/O. In that way, less experienced users can get a feeling about how that crate is structured, how Futures work and how that principle integrates then into tokio.

[carllerche]

For anyone still interested in this, I would like to up prioritize getting a cookbook setup.

Discussion is moved here, please help out 👍

I will close this issue now.