Unexpected output shape of onnx model

[Fschoeller]

Hi! i'm on the paper branch and have exported a yolor-p6 model to onnx using models/export.py.
The onnx model has the output shape output: batch, 3, 96, 160, nc though I expected the shape output: batch, 3, max_boxes, nc
Am I doing anything wrong?
Thanks!

[Fschoeller]

Solved by commenting out line 44 in export model.model[-1].export = True # set Detect() layer export=True
and using inplace operation during export:

if self.inplace:
    y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i]) * self.stride[i]  # xy
    y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh

[jnskplr]

Hi,

I'm using paper branch too and exported my model to onnx. Having the same issue with output shape and tried to follow your steps.

I don't really get what you're meaning with the inplace operation. Where did you insert it into your code, I guess in forward() function of IDetect() class in yolo.py? And what else did you adjust there? Maybe you can provide your yolo.py file.

Thanks and best regards.

[jnskplr]

Solved it for me, the way is almost the same just a little bit different. #207