Introduce yolort.runtime interface

[zhiqwang]

This release improves support of ONNXRuntime by introducing the yolort.runtime interface and adding a C++ interface example. It also improves support for loading the checkpoint trained with ultralytics/yolov5.

Highlights

Intruduce the yolort.runtime Interface

We introduce the yolort.runtime interface to accelerate the application on the Python frontend. For example, you can define the runtime and do inferencing with ONNXRuntime as follows

from yolort.runtime import PredictorORT

detector = PredictorORT("best.onnx")
img_path = "bus.jpg"
scores, class_ids, boxes = detector.run_on_image(img_path)

ONNXRuntime C++ Inference Example

We adopt the dynamic shape mechanism when exporting the ONNX model, and within this, we can embed both pre-processing (letterbox) and post-processing (nms) into the model graph, which simplifies the deployment strategy. And @itsnine have provided an example on how to use it in here.

Load Checkpoint from Official YOLOv5

We now provide an interface for loading the checkpoint weights trained from the official yolov5 directly.

from yolort.models import YOLOv5

# 'yolov5s.pt' is downloaded from https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt
ckpt_path_from_ultralytics = "yolov5s.pt"
model = YOLOv5.load_from_yolov5(ckpt_path_from_ultralytics, score_thresh=0.25)

model.eval()
img_path = "test/assets/bus.jpg"
predictions = model.predict(img_path)

Backwards Incompatible Changes

• Move common utils to yolort.v5 (Support loading model weight from ultralytics #167)
• Rename the YOLOModule to YOLOv5 (Refactor load_from_yolov5 in YOLOv5 #179)
• Move graph visualization tools to yolort.relaying (Refactor graph visualization #165)

New Features

• [PROTOTYPE] Add ncnn deployment examples #145
• [PROTOTYPE] Train models from scratch (Fixing data augmentation #115, Add anchor / target assignment visualization notebook #116, Fixing normalization in dataloader #124, Refactorring skeleton of SetCriterion #125, Fixing trainer with lightning #128, Fixing build_targets in SetCriterion #143)
• Support loading checkpoint from official yolov5 (Support loading model weight from ultralytics #167, Add ultralytics/yolov5 to yolort #168, Update statement for loading checkpoint from ultralytics #169, Refactor load_from_yolov5 in YOLOv5 #179)
• Add ONNXRuntime C++ inference example (Add ONNXRuntime C++ inference code #171, Fix dynamic shape when exporting ONNX #172, Add ONNXRuntime support statement #174, Add notes about the exported ONNX model #186), thanks @itsnine and @xiguadong
• Introduce yolort.runtime interface (Refactor ONNXRuntime Python interface #176, Add ONNXRuntime python inference. #178)
• Add pre-commit hooks #182
• Add CLI tool for exporting ONNX models #175, thanks @runrunrun1994
• Add CLI tool for evaluating mAP metrics (Add coco metrics evaluation CLI #148, Add time logging in eval_metric.py #150)

Improvement

• Add alignment with ultralytics yolov5 tutorial (Add updating weights example from ultralytics/yolov5 #117, Cleanup scripts and notebooks #118, Update alignment with ultralytics docs #121), thanks @xiguadong
• Refactor graph visualization tools (Update jit graph visualization #126, Update model graph visualization notebook [skip ci] #127, Refactor graph visualization #165), thanks @xiguadong
• Update C++ interfaces of TorchVision to 0.9.0+ (Update C++ interfaces of TorchVision to 0.9.0+ #136, Attach libtorch inference results #138), thanks @xiguadong
• Replace cxxopts with cmdline #137
• Update dependencies (Upgrade PT to 1.9.0 in unit-test #161, Create dependabot.yml #162, Bump codecov/codecov-action from 1 to 2.1.0 #163, Bump cirrus-actions/rebase from 1.4 to 1.5 #164, Update PyTorch to 1.9.1 in CI #181)
• Simplify the unit-test (Use test images from repo rather than internet #149, Add output names in ONNX and simplify the unit-test #151, Simplify torchscript test and port unit-test to pytest #152)
• Support autoenvolved anchors from ultralytics #120, thanks @Tomakko
• Workaround on YOLOTransform to support fixed size input #109
• Unify onnx and JIT resize implementations #105

Bugfixes

• Fix exception #183
• Fix dtype in AnchorGenerator (AnchorGenerator will use dtype of input feature_maps #111, AnchorGenerator forces torch.arange in fp32 to allow export to onnx #113, Change dtype from torch.float32 to torch.int32 in anchor_utils.py #166), thanks @dkloving
• Made targets argument in YOLOTransform.forward() truly optional. #108, thanks @dkloving

Documents

• Update CONTRIBUTING.md #185
• Update badges and ONNXRuntime usage #184
• New issue templates (New issue templates #173, Minor fixes in bug-report and README #180)
• Fix docstring (Add missing copyright, cleanup and fix docs #110, Fix docstring #158)
• Added more descriptive error logging #140, thanks @mattpopovich
• Add yolort logo #106

---

This discussion was created from the release Introduce yolort.runtime interface.