Implement yolo_TVM

[angry-crab]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

Implement yolo with TVM

Purpose

Implement yolo with TVM

Possible approaches

Implement cuda kernels using cpp/tvm

Definition of done

• verify accuracy of compiled model
• Create PR under modelzoo
• implement cuda kernel using cpp/tvm
• port yolo peripheral functions

[raminterpl]

@angry-crab : Could you pls provide more technical details
documents for this task @Deepak3994 : will look into it and we will plan any discussion if required

[angry-crab]

@raminterpl @Deepak3994
Thank you so much for your interest.
For information about tvm, please see the link above.
AS for the current tvm pipeline of Universe and steps to port TensorRT packages to tvm, please check the following link:
tvm_overview
FYI, we have already ported some packages from TensorRT, ie lidar_centerpoint_tvm and lidar_apollo_segmentation_tvm. Maybe you can take these as examples.

To put simply,

1. we need to first verify the accuracy of compiled model for llvm and opencl backend. Please see
   tvm_test for an example.
2. Create a PR under modelzoo repo to upload the model.
3. Implement cuda kernel functions with cpp/tvmscript
4. Port other functions to the tvm package.

Please let me know if there is any question/problem.

[raminterpl]

@Deepak3994 : Deepak is looking into it.

[Deepak3994]

@angry-crab

I see the initial step for verifying accuracy of the compiled model is to run the test from "https://github.com/angry-crab/tvm_example/blob/main/python/tvm_test.py". But in that script i see that we will be importing the pre-compiled models, backbone etc. From where can we find these models for the current yolo part?

[angry-crab]

From where can we find these models for the current yolo part?

Hi, maybe you can check the one currently being used by tensorrt.
https://github.com/autowarefoundation/autoware.universe/blob/main/perception/tensorrt_yolox/CMakeLists.txt

[Deepak3994]

Hi, I went through the camke file. It mentioned that models will be downloaded. For now i tried "https://autowarefoundation.github.io/autoware-documentation/main/installation/autoware/docker-installation-devel/"
this link for setting up the enviornment. But after the last step of running "colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release", i am not able to see any model files getting downloaded.

Is it the ryt way or there are some other commands/ methods to get the model files downloaded?

[angry-crab]

Is it the ryt way or there are some other commands/ methods to get the model files downloaded?

Hi, sry for the confusion. We need to do colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release -DDOWNLOAD_ARTIFACTS=ON to download the package.
Further information could be found here.
https://github.com/autowarefoundation/autoware.universe/blob/main/common/tvm_utility/design/tvm-utility-design.md

[Deepak3994]

Hi @angry-crab , I have downloaded the models for yolo-tinyx. Currently I am working on the test script which you have suggested to look into for getting accuracy of compiled model for llvm and opencl backend.

Script: https://github.com/angry-crab/tvm_example/blob/main/python/tvm_test.py

I cloned the code and ryt now working on running this script for the "yolo-tinyx.onnx" model file. Few libraries its requesting to be installed, currently doing the same. I will post the test results here once its generated.

[Deepak3994]

Hi I have installed tvm and other libraries. But, when i execute the "test" script with updated .onnx file, I get the following issue..

Any suggestion?

[angry-crab]

@Deepak3994 It seems that the operation is available. Which version of tvm are you using?

[Deepak3994]

Actually, import tvm itself is throwing error. Do you happen to have any doc which has installations for tvm?

[angry-crab]

Do you happen to have any doc which has installations for tvm?

I'm simply following the instructions from official tvm doc.
https://tvm.apache.org/docs/install/index.html

btw, maybe you can use the docker image from model_zoo. but make sure to change the entry point and rebuild the image.
https://github.com/autowarefoundation/modelzoo

[sujithvemi]

Hi,

The issue we are facing is registered as a bug on TVM Github issues page here.

apache/tvm#13507

Please let us know what call to take because I am facing this issue even after a thorough source installation and not just through package installation.

Regards,
Sujith.

[angry-crab]

@sujithvemi
Thanks for the info. I've never met this issue before. Have you tried to use the docker image of modelzoo?
https://github.com/autowarefoundation/modelzoo

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[raminterpl]

@xmfcx @mitsudome-r : Deepak is leaving Interplai this week. could you please assign this task to any other team who can take up this .sorry for any inconvenience

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[amadeuszsz]

@kminoda
I see we slowly move towards TensoRT based inference only. Since we keep removing TVM based package, should we close this issue?

[xmfcx]

Yeah, this task is long overdue.