Hi, any suggestion on how to serve yolov5 on torchserve ?

[yuanyuangoo]

Is your feature request related to a problem? Please describe.

I'd like to serve yolov5 model, but there is no template in the example.

Describe the solution

serve model from https://github.com/ultralytics/yolov5/

Describe alternatives solution

[dhaniram-kshirsagar]

@yuanyuangoo You can use the following hints to deploy this model -

1. create torchscripted model using following from yolo repo. -
   TFLite, ONNX, CoreML, TensorRT Export ultralytics/yolov5#251

2. Create a custom handler -
   Load YOLOv5 from PyTorch Hub ⭐ ultralytics/yolov5#36
   https://github.com/pytorch/serve/blob/v0.2.0/docs/custom_service.md

3. Create torchscripted model MAR file -
   https://github.com/pytorch/serve/tree/master/examples#creating-mar-file-for-torchscript-mode-model

4. Follow this guide to deploy the MAR file -
   https://github.com/pytorch/serve/tree/v0.2.0#serve-a-model

[dhaniram-kshirsagar]

Closing as there is no response on this.

[abhinavanurag-garudaire]

HI . I have been facing the same error :500 worker died issue when deploying yolov8 using torchserve and sending requests using torchserve.Can anyone please help?the log files are just so large and showing too many error. It'd be better to have a full tutorial on yolo deployment using torchserve

[edworId]

could help to write a handle file to run yolo?? A tutorial will be usefull , only what is needed to run

[agunapal]

Hi @abhinavanurag-garudaire @edworId I tried running yolov8 in TorchServe,

The issue is happening here https://github.com/ultralytics/ultralytics/blob/main/ultralytics/utils/__init__.py#L325
when trying to read settings.yaml

Its reading None . Not sure about the reason yet. Still looking into it.

It works fine with standalone yolov8

[agunapal]

@abhinavanurag-garudaire @edworId I have included the PR for YOLOv8. Please give it a try and let me know if this works.