Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add notes about the exported ONNX model #186

Merged
merged 2 commits into from
Oct 3, 2021
Merged

Add notes about the exported ONNX model #186

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
813bfc3
Add notes about the exported ONNX model
zhiqwang Oct 3, 2021
99f2071
Minor fixes
zhiqwang Oct 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion deployment/onnxruntime/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ The ONNXRuntime inference for `yolort`, both GPU and CPU are supported.

*We didn't impose too strong restrictions on the versions of dependencies.*

## Features

The `ONNX` model exported with `yolort` differs from the official one in the following three ways.

- The exported `ONNX` graph now supports dynamic shapes, and we use `(3, H, W)` as the input shape (for example `(3, 640, 640)`).
- We embed the pre-processing ([`letterbox`](https://github.com/ultralytics/yolov5/blob/9ef94940aa5e9618e7e804f0758f9a6cebfc63a9/utils/augmentations.py#L88-L118)) into the graph as well. We only require the input image to be in the `RGB` channel, and to be rescaled to `float32 [0-1]` from general `uint [0-255]`. The main logic we use to implement this mechanism is below. (And [this](https://github.com/zhiqwang/yolov5-rt-stack/blob/b9c67205a61fa0e9d7e6696372c133ea0d36d9db/yolort/models/transform.py#L210-L234) plays the same role of the official `letterbox`, but there will be a little difference in accuracy now.)
- We embed the post-processing (`nms`) into the model graph, which performs the same task as [`non_max_suppression`](https://github.com/ultralytics/yolov5/blob/fad57c29cd27c0fcbc0038b7b7312b9b6ef922a8/utils/general.py#L532-L623) except for the format of the inputs. (And here the `ONNX` graph is required to be dynamic.)

## Usage

1. First, Setup the environment variable.
Expand All @@ -35,7 +43,7 @@ The ONNXRuntime inference for `yolort`, both GPU and CPU are supported.
[--simplify]
```

Afterwards, you can see that a new pair of ONNX models ("best.onnx" and "best.sim.onnx") has been generated in the directory of "best.pt".
And then, you can find that a new pair of ONNX models ("best.onnx" and "best.sim.onnx") has been generated in the directory of "best.pt".

1. \[Optional\] Quick test with the ONNXRuntime Python interface.

Expand Down