Which is the fastest?

This project aims to be a load benchmarking suite, no more, no less

Measuring response times (routing times) for each framework (middleware).

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Results are not production-ready yet

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Additional purposes :

• Helping decide beetween languages, depending on use case
• Learning languages, best practices, devops culture ...
• Having fun ❤️

Requirements

• Crystal as built-in tools are made in this language
• Docker as frameworks are isolated into containers

⚠️ docker is used for development purpose, production results will be computed on DigitalOcean ⚠️

Usage

• Install all dependencies

shards install

• Build internal tools

shards build

• Build containers

job is either a language (example : crystal) or a framework (example : router.cr)

bin/neph [job]

• Start the benchmark ....

tools is a list of language / framework to challenge (example : ruby kemal amber go python)

bin/benchmarker [tools]

Results

Last update: 2018-10-30

OS: Linux (version: 4.18.16-200.fc28.x86_64, arch: x86_64)
CPU Cores: 8
threads: 9, connections: 1000

Latency

Ranking (top 5)

1️⃣ nickel (rust)

2️⃣ laravel (php)

3️⃣ iron (rust)

4️⃣ rack-routing (ruby)

5️⃣ roda (ruby)

Full table

Language (Runtime)	Framework (Middleware)	Average	50th percentile	90th percentile	99th percentile	99.9th percentile	Standard deviation
rust (1.30)	nickel (0.10)	0.09 ms	0.08 ms	0.13 ms	0.17 ms	1.25 ms	32.00
php (7.2)	laravel (5.7)	126.64 ms	0.44 ms	336.76 ms	2211.19 ms	6959.76 ms	427179.67
rust (1.30)	iron (0.6)	0.50 ms	0.49 ms	0.84 ms	1.17 ms	37.11 ms	362.00
ruby (2.5)	rack-routing (0.0)	5.61 ms	1.19 ms	16.49 ms	47.61 ms	159.68 ms	9924.33
ruby (2.5)	roda (3.13)	4.54 ms	1.48 ms	12.77 ms	35.75 ms	114.91 ms	7436.00
ruby (2.5)	hanami (1.3)	9.31 ms	1.71 ms	28.22 ms	75.36 ms	195.18 ms	15930.00
rust (nightly)	rocket (0.3)	70.72 ms	1.96 ms	4.91 ms	2247.67 ms	4951.82 ms	383861.00
python (3.6)	japronto (1.1)	3.73 ms	2.47 ms	8.53 ms	17.66 ms	39.13 ms	3880.67
rust (1.30)	actix-web (0.7)	3.54 ms	2.88 ms	6.49 ms	12.91 ms	91.07 ms	3002.33
php (7.2)	slim (3.11)	150.43 ms	3.34 ms	258.65 ms	3240.83 ms	6394.02 ms	561535.00
php (7.2)	symfony (4.1)	162.43 ms	3.49 ms	310.11 ms	3411.09 ms	6183.14 ms	586094.67
python (3.6)	vibora (0.0)	4.76 ms	3.62 ms	9.71 ms	18.26 ms	37.03 ms	3908.67
ruby (2.5)	sinatra (2.0)	10.63 ms	3.72 ms	30.39 ms	74.09 ms	191.04 ms	15880.33
ruby (2.5)	flame (4.18)	8.89 ms	4.15 ms	24.10 ms	55.12 ms	147.67 ms	11969.67
cpp (11.0)	evhtp (1.2)	3.85 ms	4.19 ms	5.74 ms	9.92 ms	62.19 ms	2073.00
go (1.11)	fasthttprouter (0.1)	4.67 ms	4.44 ms	7.20 ms	15.05 ms	101.53 ms	2869.33
nim (0.19)	jester (0.4)	4.92 ms	4.64 ms	7.80 ms	14.07 ms	52.04 ms	2776.67
rust (1.30)	gotham (0.3)	5.61 ms	5.10 ms	10.65 ms	18.02 ms	158.62 ms	4161.00
crystal (0.26.1)	spider-gazelle (1.1)	5.71 ms	5.14 ms	9.97 ms	18.01 ms	38.01 ms	3400.67
c (99)	kore (3.1)	25.52 ms	5.49 ms	9.61 ms	853.58 ms	1606.15 ms	126354.33
csharp (7.3)	aspnetcore (2.1)	7.57 ms	6.49 ms	10.33 ms	20.37 ms	447.70 ms	9898.33
go (1.11)	muxie (1.0)	8.25 ms	7.27 ms	12.65 ms	25.67 ms	115.30 ms	4907.33
java (8)	act (1.8)	8.13 ms	7.49 ms	13.03 ms	24.36 ms	268.55 ms	7870.67
go (1.11)	chi (3.3)	8.61 ms	7.54 ms	13.33 ms	28.13 ms	174.45 ms	6358.33
scala (2.12)	akkahttp (10.1)	223.18 ms	7.89 ms	104.44 ms	5221.79 ms	7937.38 ms	889400.33
go (1.11)	iris (11.0)	9.12 ms	8.40 ms	13.65 ms	27.77 ms	180.07 ms	5703.67
go (1.11)	echo (3.3)	9.62 ms	8.46 ms	14.70 ms	31.18 ms	248.64 ms	8931.33
go (1.11)	gin (1.3)	9.56 ms	8.52 ms	14.57 ms	30.31 ms	253.51 ms	8092.67
go (1.11)	beego (1.10)	9.80 ms	8.57 ms	14.76 ms	32.46 ms	359.40 ms	9584.00
go (1.11)	gorilla-mux (1.6)	9.58 ms	8.74 ms	14.90 ms	30.00 ms	120.74 ms	5670.67
nim (0.19)	mofuw (2.0)	29.77 ms	8.90 ms	86.77 ms	306.05 ms	468.82 ms	63476.33
python (3.7)	bottle (0.12)	12.73 ms	10.12 ms	22.92 ms	43.34 ms	291.11 ms	9467.33
ruby (2.5)	rails (5.2)	40.16 ms	10.90 ms	126.81 ms	307.83 ms	763.71 ms	67253.33
node (10.12)	polka (0.5)	14.99 ms	11.07 ms	22.82 ms	70.64 ms	537.32 ms	22326.00
node (10.12)	rayo (1.2)	19.34 ms	11.48 ms	23.63 ms	249.78 ms	931.39 ms	51950.00
swift (4.2)	perfect (3.0)	12.10 ms	12.09 ms	14.60 ms	17.09 ms	95.27 ms	2529.00
swift (4.2)	vapor (3.0)	25.79 ms	13.19 ms	23.93 ms	475.43 ms	1935.65 ms	98299.33
node (10.12)	fastify (1.12)	27.50 ms	16.67 ms	34.27 ms	354.71 ms	1142.36 ms	65568.67
scala (2.12)	http4s (0.18)	18.23 ms	16.75 ms	33.07 ms	54.89 ms	227.80 ms	11995.00
node (10.12)	koa (2.6)	23.43 ms	17.52 ms	34.19 ms	126.85 ms	795.77 ms	35722.67
node (10.12)	foxify (0.10)	25.34 ms	17.84 ms	37.24 ms	172.73 ms	916.28 ms	43980.67
node (10.12)	restify (7.2)	22.64 ms	18.66 ms	34.98 ms	69.67 ms	465.42 ms	18420.33
python (3.7)	aiohttp (3.4)	25.87 ms	24.75 ms	42.23 ms	63.95 ms	143.77 ms	12590.00
swift (4.2)	kitura (2.5)	27.86 ms	25.98 ms	40.55 ms	55.23 ms	199.85 ms	9151.00
node (10.12)	express (4.16)	39.97 ms	27.30 ms	50.19 ms	456.88 ms	1316.83 ms	79434.00
crystal (0.26.1)	raze (0.3)	29.86 ms	27.43 ms	38.11 ms	47.78 ms	172.79 ms	6542.67
crystal (0.26.1)	router.cr (0.2)	29.47 ms	28.58 ms	37.06 ms	43.22 ms	241.80 ms	6698.33
python (3.7)	flask (1.0)	36.92 ms	29.87 ms	65.91 ms	95.28 ms	320.13 ms	19101.00
crystal (0.26.1)	lucky (0.11)	33.84 ms	31.41 ms	46.12 ms	54.84 ms	258.97 ms	10354.00
crystal (0.26.1)	kemal (0.24)	35.50 ms	33.30 ms	46.28 ms	58.62 ms	192.76 ms	8276.33
node (10.12)	hapi (17.6)	73.62 ms	34.28 ms	65.39 ms	1255.26 ms	2396.78 ms	201888.33
crystal (0.26.1)	amber (0.9)	38.25 ms	38.47 ms	46.35 ms	89.21 ms	266.29 ms	12933.33
crystal (0.26.1)	orion (1.6)	42.16 ms	39.78 ms	54.01 ms	63.58 ms	235.60 ms	8701.33
python (3.7)	django (2.1)	62.48 ms	47.97 ms	124.61 ms	163.79 ms	494.06 ms	35608.33
python (3.7)	sanic (0.8)	60.67 ms	49.19 ms	119.43 ms	222.73 ms	429.56 ms	45586.00
python (3.7)	tornado (5.1)	94.91 ms	90.36 ms	128.73 ms	163.13 ms	625.22 ms	36766.67
python (3.7)	quart (0.6)	105.14 ms	97.73 ms	196.24 ms	299.63 ms	468.95 ms	66268.00

Requests per seconds

Ranking (top 5)

1️⃣ (japronto) (python)

2️⃣ (actix-web) (rust)

3️⃣ (vibora) (python)

4️⃣ (evhtp) (cpp)

5️⃣ (jester) (nim)

Full table

Language (Runtime)	Framework (Middleware)	Requests / s	Throughput
python (3.6)	japronto (1.1)	299285.00	358.36 MB
rust (1.30)	actix-web (0.7)	261557.00	297.27 MB
python (3.6)	vibora (0.0)	230848.67	262.11 MB
cpp (11.0)	evhtp (1.2)	229509.00	222.56 MB
nim (0.19)	jester (0.4)	213867.33	430.09 MB
go (1.11)	fasthttprouter (0.1)	201064.67	323.36 MB
crystal (0.26.1)	spider-gazelle (1.1)	170595.00	182.33 MB
rust (1.30)	gotham (0.3)	169342.33	347.24 MB
java (8)	act (1.8)	148957.33	254.63 MB
csharp (7.3)	aspnetcore (2.1)	130776.67	213.12 MB
rust (1.30)	iron (0.6)	119720.67	151.14 MB
go (1.11)	muxie (1.0)	119573.67	160.55 MB
go (1.11)	chi (3.3)	116012.67	155.32 MB
go (1.11)	iris (11.0)	108704.67	145.56 MB
go (1.11)	echo (3.3)	105331.00	184.85 MB
go (1.11)	gin (1.3)	104924.00	184.23 MB
go (1.11)	beego (1.10)	104702.67	140.65 MB
go (1.11)	gorilla-mux (1.6)	103582.00	138.43 MB
rust (1.30)	nickel (0.10)	95483.33	189.19 MB
nim (0.19)	mofuw (2.0)	94089.67	165.21 MB
python (3.7)	bottle (0.12)	81387.00	200.64 MB
swift (4.2)	perfect (3.0)	81197.00	76.29 MB
node (10.12)	polka (0.5)	76242.67	114.14 MB
node (10.12)	rayo (1.2)	74473.33	111.55 MB
scala (2.12)	akkahttp (10.1)	66883.67	143.51 MB
swift (4.2)	vapor (3.0)	66420.33	110.88 MB
php (7.2)	symfony (4.1)	63266.00	314.55 MB
php (7.2)	slim (3.11)	61407.67	305.13 MB
scala (2.12)	http4s (0.18)	59833.00	104.81 MB
c (99)	kore (3.1)	57530.33	155.94 MB
node (10.12)	fastify (1.12)	54307.00	129.79 MB
node (10.12)	koa (2.6)	49256.67	104.26 MB
node (10.12)	foxify (0.10)	47822.00	100.33 MB
node (10.12)	restify (7.2)	46208.33	80.87 MB
php (7.2)	laravel (5.7)	41383.33	215.65 MB
rust (nightly)	rocket (0.3)	40708.67	64.69 MB
python (3.7)	aiohttp (3.4)	39394.33	89.35 MB
swift (4.2)	kitura (2.5)	36255.33	67.28 MB
crystal (0.26.1)	router.cr (0.2)	33634.33	31.54 MB
crystal (0.26.1)	raze (0.3)	33294.00	31.20 MB
node (10.12)	express (4.16)	32692.33	79.83 MB
crystal (0.26.1)	lucky (0.11)	29134.00	35.78 MB
ruby (2.5)	roda (3.13)	28137.67	26.83 MB
crystal (0.26.1)	kemal (0.24)	27810.33	45.32 MB
python (3.7)	flask (1.0)	27547.33	67.84 MB
node (10.12)	hapi (17.6)	26420.67	68.38 MB
crystal (0.26.1)	amber (0.9)	26408.33	38.28 MB
crystal (0.26.1)	orion (1.6)	23329.33	38.01 MB
ruby (2.5)	rack-routing (0.0)	22904.67	13.21 MB
python (3.7)	sanic (0.8)	17927.33	31.96 MB
python (3.7)	django (2.1)	16340.33	47.45 MB
ruby (2.5)	flame (4.18)	14389.00	8.30 MB
ruby (2.5)	hanami (1.3)	13768.67	104.14 MB
ruby (2.5)	sinatra (2.0)	12049.67	31.24 MB
python (3.7)	tornado (5.1)	10334.67	27.72 MB
python (3.7)	quart (0.6)	9859.67	19.63 MB
ruby (2.5)	rails (5.2)	3191.33	9.74 MB

How to contribute ?

In any way you want ...

• Request a framework addition
• Report a bug (on any implementation)
• Suggest an idea
• ...

Any kind of idea is ❤️

Contributors

• Taichiro Suzuki - Author, maintainer
• OvermindDL1 - Maintainer
• Marwan Rabbâa - Mainainer