Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix RoiAlign CPU EP issues #7354

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Fix RoiAlign CPU EP issues #7354

wants to merge 27 commits into from

Conversation

fdwr
Copy link
Contributor

@fdwr fdwr commented Apr 16, 2021

Description:
(1) max mode should take the entire bilinear sample, not a fragment of it. This line should be output value = max(sum of all weighted values, previous maximum), not output value = max(max(max(max(value1, value2), value3) value4), previous maximum).
(2) 0 size regions are valid and should be treated as sampled points, not forced to 1. This leads to test failures when sampling tiny points or slivers in our ONNX conformance tests. bin_size_w and bin_size_h are only multiplied anyway, not divided, and so an attempt to prevent division by zero (or whatever the original intention) is unnecessary.
(3) The ONNX compat attribute was already added in onnx/onnx#3625 input pixel alignment is 0.5 off - the correct equation is (x + 0.5) * scale - 0.5, but the final 0.5 was missed (x + 0.5) * scale (see https://arxiv.org/abs/1703.06870).

Motivation and Context

…e valid and should be treated as points (3) max mode should take the entire bilinear sample, not a fragment of it
@jiafatom
Copy link
Contributor

jiafatom commented Jul 31, 2021

Thanks for referring my PR to this PR. I think we agree that the ultimate solution is to update onnx spec for ROIAlign - we need add an attribute aligned. And based on that, we change onnxruntime logic accordingly -- refer to the PR for aligned=True case. The logic in my PR is in sync with torch ROIAlign implementation (I tested the result matches). So I would prefer to use logic there.

@garymm garymm removed the request for review from a team April 11, 2022 19:44
@fdwr
Copy link
Contributor Author

fdwr commented Jun 21, 2022

@hariharans29 - do you have instructions on updating this C# test baseline? I believe these values were recorded with the bug baked into them:

System.Exception : Opset opset10, Model mask_rcnn: ModelFile = D:\a\_work\1\b\models\opset10\mask_rcnn\mask_rcnn_R_50_FPN_1x.onnx error = Assert.Equal() Failure
Expected: DenseTensor<Single> [475.39844, 286.16537, 803.5265, 553.59546, 873.8627, ...]
Actual:   DenseTensor<Single> [475.39844, 286.16534, 803.5265, 553.5954, 873.8627, ...]

https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=674145&view=logs&jobId=ed9f9512-dfe4-5f72-a421-4225d265fdd2&j=ed9f9512-dfe4-5f72-a421-4225d265fdd2&t=d93860a7-6075-51af-35c2-8a3248009377

@hariharans29
Copy link
Member

@hariharans29 - do you have instructions on updating this C# test baseline? I believe these values were recorded with the bug baked into them:

System.Exception : Opset opset10, Model mask_rcnn: ModelFile = D:\a\_work\1\b\models\opset10\mask_rcnn\mask_rcnn_R_50_FPN_1x.onnx error = Assert.Equal() Failure
Expected: DenseTensor<Single> [475.39844, 286.16537, 803.5265, 553.59546, 873.8627, ...]
Actual:   DenseTensor<Single> [475.39844, 286.16534, 803.5265, 553.5954, 873.8627, ...]

https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=674145&view=logs&jobId=ed9f9512-dfe4-5f72-a421-4225d265fdd2&j=ed9f9512-dfe4-5f72-a421-4225d265fdd2&t=d93860a7-6075-51af-35c2-8a3248009377

Maybe we could change the test referencing the PR and stating that the expected values were generated with the buggy implementation to begin with ?

@fdwr fdwr requested a review from a team as a code owner July 5, 2022 19:15
@fdwr fdwr force-pushed the user/dwayner/FixRoiAlign branch from 7cbe5ec to aa5461d Compare July 5, 2022 19:25
@yuslepukhin
Copy link
Member

Is there an issue filed with ONNX? We need to reference it here.

@fdwr
Copy link
Contributor Author

fdwr commented Oct 28, 2022

yuslepukhin onnx/onnx#3428

Is there an issue filed with ONNX? We need to reference it here.

Well here was the original ONNX issue onnx/onnx#3428, which RoiAlign opset 16 addressed by adding coordinate_transformation_mode, but did you mean one specific to model generation, for the ONNX model zoo?

linnealovespie added a commit that referenced this pull request May 10, 2023
### Description
<!-- Describe your changes. -->
Add registration for DML RoiAlign-16 and tests for new
coordinate_transform_mode attribute. PR
[7354](#7354) is still open
to fix the CPU EP version, which is why there are skipped tests right
now. That will be completed separately so that, for now, we can
officially support opset16 with the next release.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Linnea May <[email protected]>
Co-authored-by: Dwayne Robinson <[email protected]>
linnealovespie added a commit that referenced this pull request May 10, 2023
### Description
<!-- Describe your changes. -->
Add registration for DML RoiAlign-16 and tests for new
coordinate_transform_mode attribute. PR
[7354](#7354) is still open
to fix the CPU EP version, which is why there are skipped tests right
now. That will be completed separately so that, for now, we can
officially support opset16 with the next release.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Linnea May <[email protected]>
Co-authored-by: Dwayne Robinson <[email protected]>
snnn pushed a commit that referenced this pull request May 11, 2023
### Description
<!-- Describe your changes. -->
Add registration for DML RoiAlign-16 and tests for new
coordinate_transform_mode attribute. PR
[7354](#7354) is still open
to fix the CPU EP version, which is why there are skipped tests right
now. That will be completed separately so that, for now, we can
officially support opset16 with the next release.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

### Description
<!-- Describe your changes. -->



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Co-authored-by: Linnea May <[email protected]>
Co-authored-by: Dwayne Robinson <[email protected]>
fs-eire pushed a commit that referenced this pull request May 12, 2023
### Description
<!-- Describe your changes. -->
Add registration for DML RoiAlign-16 and tests for new
coordinate_transform_mode attribute. PR
[7354](#7354) is still open
to fix the CPU EP version, which is why there are skipped tests right
now. That will be completed separately so that, for now, we can
officially support opset16 with the next release.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Linnea May <[email protected]>
Co-authored-by: Dwayne Robinson <[email protected]>
preetha-intel pushed a commit to intel/onnxruntime that referenced this pull request Jun 7, 2023
### Description
<!-- Describe your changes. -->
Add registration for DML RoiAlign-16 and tests for new
coordinate_transform_mode attribute. PR
[7354](microsoft#7354) is still open
to fix the CPU EP version, which is why there are skipped tests right
now. That will be completed separately so that, for now, we can
officially support opset16 with the next release.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

### Description
<!-- Describe your changes. -->



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Co-authored-by: Linnea May <[email protected]>
Co-authored-by: Dwayne Robinson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants