[ConvertLayout] Support QNN ops. #5066
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
anijain2305
force-pushed
the
qnn_layout
branch
4 times, most recently
from
235c079
to
a4c5092
Compare
yzhliu
reviewed
Mar 16, 2020
src/relay/qnn/op/convolution.cc
Outdated
|
|
||
| // Fill the layouts of remaining input tensors - scales and zero points. The layouts of these | ||
| // tensors can be ignored as they dont go through any transformation. | ||
| Layout ignore_layout = Layout("I"); |
There was a problem hiding this comment.
They can be scalar, or output channel. I initially thought of putting them as "C". But, chose "I" to be more specific. I am open to discuss.
There was a problem hiding this comment.
maybe "C" is better. I don't have strong opinion though
yzhliu
approved these changes
Mar 18, 2020
trevor-m
pushed a commit
to trevor-m/tvm
that referenced
this pull request
Apr 16, 2020
* [ConvertLayout] Support QNN ops. * Changing layouts to C. * Fixing dilation. * Empty commit. Co-authored-by: Ubuntu <[email protected]>
zhiics
pushed a commit
to neo-ai/tvm
that referenced
this pull request
Apr 17, 2020
* [ConvertLayout] Support QNN ops. * Changing layouts to C. * Fixing dilation. * Empty commit. Co-authored-by: Ubuntu <[email protected]>
shoubhik
added a commit
to shoubhik/incubator-tvm
that referenced
this pull request
May 12, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Recently introduced Op strategy has disabled conversion from NHWC to NCHW in AlterOpLayout (which is correct thing to do). We can solve this problem by calling ConvertLayout in the parser if needed. However, this only works for FP32.
For quantized models, parsers give a QNN graph. And this QNN graph goes to relay.build. Relay build internally calls QNN Legalize passes to convert it to Relay-only ops. The problem is ConvertLayout does not work on QNN ops. Therefore, even if we call ConvertLayout after parser, the layouts will not change.
This PR implements ConvertLayout for QNN ops. In addition, I have changed the interface of FInferCorrectLayout to ingest an array of Relay Types instead of shapes. This is helpful in operators like Concatenate where we need to know the number of input data tensors.
@icemelon9 @zhiics @yzhliu