demo

Getting Started with D2Go

This document provides a brief intro of the usage of builtin command-line tools in d2go.

For a tutorial that involves coding with the API, see our Jupyter Notebook which covers 1). how to run inference with an existing model, 2). how to train a builtin model on a custom dataset, and 3). how to apply quantization to the model for int8 deployment.

Inference Demo with Pre-trained Models

• Choose a model from model_zoo, e.g. faster_rcnn_fbnetv3a_C4.yaml.
• Use the provided demo.py to try demo on an input image:

cd demo/
python demo.py --config-file faster_rcnn_fbnetv3a_C4.yaml --input input1.jpg --output output1.jpg

• To run on a video, replace the --input files with --video-input video.mp4

Training & Evaluation

D2Go is built on top of detectron2 toolkit, please follow the instructions on detectron2 to setup the builtin datasets before training.

• To train a model:

d2go.train_net --config-file ./configs/faster_rcnn_fbnetv3a_C4.yaml

• To evaluate a model checkpoint:

d2go.train_net --config-file ./configs/faster_rcnn_fbnetv3a_C4.yaml --eval-only \
MODEL.WEIGHTS https://mobile-cv.s3-us-west-2.amazonaws.com/d2go/models/246823121/model_0479999.pth

(change the URL to a local path, if evaluating local models)

Export to Torchscript & Int8 Model

• Export to Torchscript model:

d2go.exporter --config-file configs/faster_rcnn_fbnetv3a_C4.yaml \
--predictor-types torchscript --output-dir ./ \
MODEL.WEIGHTS https://mobile-cv.s3-us-west-2.amazonaws.com/d2go/models/246823121/model_0479999.pth

• Export to Int8 model (using post-training quantization):

d2go.exporter --config-file configs/faster_rcnn_fbnetv3a_C4.yaml \
--output-dir ./ --predictor-type torchscript_int8 \
MODEL.WEIGHTS https://mobile-cv.s3-us-west-2.amazonaws.com/d2go/models/246823121/model_0479999.pth

Quantization-aware Training

The previous method exports int8 models using post-training quantization, which is easy-to-use but may suffers accuracy drop. Quantization aware training emulates inference-time quantization during the training, so that the resulting lower-precision model can benefit during deployment.

To apply quantization-aware training, we need to resume from a pretrained checkpoint:

d2go.train_net --config-file configs/qat_faster_rcnn_fbnetv3a_C4.yaml \
MODEL.WEIGHTS https://mobile-cv.s3-us-west-2.amazonaws.com/d2go/models/246823121/model_0479999.pth

Please see the config file for relevant hyper-params.