http parser bottleneck for server

[goyalyashpal]

Hi!

in this blogpost about uvloop; it is said/shown alongwith benchmarks that:

• the http parser used by aiohttp is a huge overpowering bottleneck for HTTP communicatiens;
• any improvement at that space shows up directly in overall aiohttp performance.

but the httptools doesn't seem to be a very maintained project (so much so that it still uses the http-parser for urls which is now officially unmaintained for >3years).

so i wanted to discuss; is it known well in here?

• what are ur thoughts about it?
• how much possibility do u see of an optional simpler parser being viable as a plugin (similar to say aiodns or those aiohttp[speedups]) * ... even of it means restricting some features (like this http-parser cut down on routing

10KiB message size:

	Requests/sec (higher better)	Latency (msec) (lower better)
asyncio+aiohttp	3076 Requests/sec	96.82
asyncio+httptools	21346 Requests/sec	13.90

---

Source: blog post on uvloop

About aiohttp

[1] HTTP (requests / sec)
... the performance bottleneck in aiohttp turned out to be its HTTP parser, which is so slow, that it matters very little how fast the underlying I/O library is.

[3] Quite surprisingly, pure-Python asyncio, with the help of high-performance HTTP parser is faster than nodejs, which uses the same HTTP parser!

About http-parser

[2] To make things more interesting, we created a Python binding for http-parser (nodejs HTTP parser C library, originally developed for Nginx).
The library is called httptools, and is available on Github and PyPI.

[4] Admittedly, httptools-based server is very minimal and does not include any routing logic, unlike other implementations.

[Dreamsorcerer]

Hmm, we use llhttp from nodejs, which they claim is significantly faster than httptools, so I'm not sure how these comparisons line up...
https://github.com/nodejs/llhttp/#performance

[Dreamsorcerer]

Actually, that post is 7 years ago and from an aiohttp contributor. My guess is they were probably involved in adding to llhttp to aiohttp. So, consider that post outdated and not relevant anymore.

[goyalyashpal]

Actually, that post is 7 years ago

ahwkayhhh... wow, i didnt notice that.

👍