Skip to content

Latest commit

 

History

History
46 lines (33 loc) · 1.91 KB

BENCHMARKS.md

File metadata and controls

46 lines (33 loc) · 1.91 KB

Konverter Benchmarks

Snapdragon 821 (LeEco Le Pro3) - 10,000 random single predictions

Comparison of a model converted with SNPE 1.19 (Snapdragon Neural Processing Engine) and the same model converted with Konverter.

SNPE model Konverted model
Total time 16.150222 sec. 10.021809 sec.
Average time 0.0016150 sec. 0.0010022 sec.
Model rate 619.18654 Hz 997.82385 Hz

The model:

model = Sequential()
model.add(Dense(204, activation='relu', input_shape=(103,)))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))

Ryzen 5 3600 (Desktop) - 10,000 random predictions

(see exact model in build_test_model.py):

Batch prediction:

Keras model Konverted model
Total time 0.403091 sec. 0.088019 sec.

Single prediction:

Keras model Konverted model
Total time 135.074061 sec. 1.848414 sec.
Average time 0.01350741 sec. 0.000185 sec.
Model rate 74.0334593 Hz 5410.043 Hz

Benchmark info:

The batch predictions are simply that, 10,000 random samples are fed into each model to be predicted on all at once. This is usually the fastest method of executing a prediction for a lot of unrelated samples.

With the single predictions, we are predicting on the same samples as before, however we are using a loop and predicting on each sample one by one. This is usually how you will be executing predictions in production. You won't know future data, so this is a good way to benchmark inference times for both model formats.