From b59a493d427e616f95156303481d73a71e12497f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Christian=20L=C3=BCck?= <christian@clue.engineering>
Date: Fri, 3 Jul 2020 11:56:17 +0200
Subject: [PATCH] Add concurrency documentation

---
 README.md | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 53 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 9e8e4fe..31907c8 100644
--- a/README.md
+++ b/README.md
@@ -43,6 +43,7 @@ mess with most of the low-level details.
     * [Authentication](#authentication)
     * [Redirects](#redirects)
     * [Blocking](#blocking)
+    * [Concurrency](#concurrency)
     * [Streaming response](#streaming-response)
     * [Streaming request](#streaming-request)
     * [HTTP proxy](#http-proxy)
@@ -385,9 +386,60 @@ $responses = Block\awaitAll($promises, $loop);
 Please refer to [clue/reactphp-block](https://github.com/clue/reactphp-block#readme) for more details.
 
 Keep in mind the above remark about buffering the whole response message in memory.
-As an alternative, you may also see the following chapter for the
+As an alternative, you may also see one of the following chapters for the
 [streaming API](#streaming-response).
 
+### Concurrency
+
+As stated above, this library provides you a powerful, async API. Being able to
+send a large number of requests at once is one of the core features of this
+project. For instance, you can easily send 100 requests concurrently while
+processing SQL queries at the same time.
+
+Remember, with great power comes great responsibility. Sending an excessive
+number of requests may either take up all resources on your side or it may even
+get you banned by the remote side if it sees an unreasonable number of requests
+from your side.
+
+```php
+// watch out if array contains many elements
+foreach ($urls as $url) {
+    $browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
+        var_dump($response->getHeaders());
+    });
+}
+```
+
+As a consequence, it's usually recommended to limit concurrency on the sending
+side to a reasonable value. It's common to use a rather small limit, as doing
+more than a dozen of things at once may easily overwhelm the receiving side. You
+can use [clue/reactphp-mq](https://github.com/clue/reactphp-mq) as a lightweight
+in-memory queue to concurrently do many (but not too many) things at once:
+
+```php
+// wraps Browser in a Queue object that executes no more than 10 operations at once
+$q = new Clue\React\Mq\Queue(10, null, function ($url) use ($browser) {
+    return $browser->get($url);
+});
+
+foreach ($urls as $url) {
+    $q($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
+        var_dump($response->getHeaders());
+    });
+}
+```
+
+Additional requests that exceed the concurrency limit will automatically be
+enqueued until one of the pending requests completes. This integrates nicely
+with the existing [Promise-based API](#promises). Please refer to
+[clue/reactphp-mq](https://github.com/clue/reactphp-mq) for more details.
+
+This in-memory approach works reasonably well for some thousand outstanding
+requests. If you're processing a very large input list (think millions of rows
+in a CSV or NDJSON file), you may want to look into using a streaming approach
+instead. See [clue/reactphp-flux](https://github.com/clue/reactphp-flux) for
+more details.
+
 ### Streaming response
 
 <a id="streaming"></a><!-- legacy fragment id -->