ONNX runtime quantization is under active development. please use 1.6.0+ to get more quantization support.
This example load an object detection model converted from ONNX Model Zoo and confirm its accuracy and speed based on MS COCO 2017 dataset. You need to download this dataset yourself.
onnx: 1.9.0 onnxruntime: 1.10.0
Download model from ONNX Model Zoo
wget https://github.com/onnx/models/raw/main/vision/object_detection_segmentation/fcn/model/fcn-resnet50-12.onnx
Quantize model with QLinearOps:
bash run_tuning.sh --input_model=path/to/model \ # model path as *.onnx
--config=fcn_rn50.yaml \
--data_path=path/to/coco/val2017 \
--label_path=path/to/coco/annotations/instances_val2017.json \
--output_model=path/to/save
Quantize model with QDQ mode:
bash run_tuning.sh --input_model=path/to/model \ # model path as *.onnx
--config=fcn_rn50_qdq.yaml \
--data_path=path/to/coco/val2017 \
--label_path=path/to/coco/annotations/instances_val2017.json \
--output_model=path/to/save
bash run_benchmark.sh --input_model=path/to/model \ # model path as *.onnx
--config=fcn_rn50.yaml \
--data_path=path/to/coco/val2017 \
--label_path=path/to/coco/annotations/instances_val2017.json \
--mode=performance # or accuracy