document streaming

README.md

@@ -341,8 +341,56 @@ LIT_SERVER_API_KEY=supersecretkey python main.py
 Clients are expected to auth with the same API key set in the `X-API-Key` HTTP header.
 
 </details>
+
+
+<details>
+  <summary>Stream long responses</summary>
+
+&nbsp;
+
+`LitServer` can stream longer responses, such as LLM generated text.
+
+To enable streaming, you need to implement `LitAPI.predict` and `LitAPI.encode_response` as a generator (a Python
+function that yields output) and set `LitServer(..., stream=True)`.
+
+For example, streaming long responses generated over time:
+
+```python
+from typing import Generator
+
+import json
+from litserve import Request
+from litserve import LitServer
+from litserve import LitAPI
+
+
+class SimpleStreamAPI(LitAPI):
+    def setup(self, device) -> None:
+        self.model = lambda x, y: x*y
+
+    def decode_request(self, request: Request) -> float:
+        return request["input"]
+
+    def predict(self, x) -> Generator:
+        for i in range(10):
+            yield self.model(x, i)
+
+    def encode_response(self, output: Generator) -> Generator:
+        for out in output:
+            yield json.dumps({"output": out})
+
+
+if __name__ == "__main__":
+    api = SimpleStreamAPI()
+    server = LitServer(api, stream=True)
+    server.run(port=8000)
+```
+
 &nbsp;
 
+</details>
+
+
 # Contribute
 LitServe is a community project accepting contributions. Let's make the world's most advanced AI inference engine.
 
@@ -366,4 +414,3 @@ pytest tests
 
 litserve is released under the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) license.
 See LICENSE file for details.
-Then, run pytest in your terminal as follows: