benchmark.md

CPU benchmarks

Run on Intel (R) Xeon (R) CPU @ 2.30 GHz with 2 cores (Google Colab session)

ONNX	batch	encode_image	encode_text	total
ViT-B/32	2	0.234	0.162	0.396
ViT-B/32	8	0.923	0.656	1.579
ViT-B/32	16	2.079	1.288	3.367
ViT-B/32	32	3.937	2.658	6.595
ViT-B/32	64	7.944	5.567	13.511

TORCH	batch	encode_image	encode_text	total
ViT-B/32	2	0.343	0.243	0.586
ViT-B/32	8	1.093	0.831	1.924
ViT-B/32	16	1.952	1.523	3.475
ViT-B/32	32	4.079	3.015	7.094
ViT-B/32	64	8.07	6.212	14.282

GPU benchmarks

Run on NVIDIA Tesla K80 (Google Colab session)

ONNX	batch	encode_image	encode_text	total
ViT-B/32	2	0.136	0.021	0.157
ViT-B/32	8	0.054	0.04	0.094
ViT-B/32	16	0.089	0.071	0.16
ViT-B/32	32	0.158	0.134	0.292
ViT-B/32	64	0.325	0.258	0.583

TORCH	batch	encode_image	encode_text	total
ViT-B/32	2	0.02	0.035	0.055
ViT-B/32	8	0.081	0.098	0.179
ViT-B/32	16	0.207	0.196	0.403
ViT-B/32	32	0.44	0.374	0.814
ViT-B/32	64	0.919	0.719	1.638

Run on NVIDIA Tesla T4 (Google Colab session)

ONNX	batch	encode_image	encode_text	total
ViT-B/32	2	0.155	0.01	0.165
ViT-B/32	8	0.032	0.014	0.046
ViT-B/32	16	0.037	0.029	0.066
ViT-B/32	32	0.076	0.059	0.135
ViT-B/32	64	0.169	0.117	0.286

TORCH	batch	encode_image	encode_text	total
ViT-B/32	2	0.017	0.009	0.026
ViT-B/32	8	0.008	0.008	0.016
ViT-B/32	16	0.009	0.012	0.021
ViT-B/32	32	0.008	0.025	0.033
ViT-B/32	64	0.009	0.049	0.058