[QNN] [BYOC] MergeComposite pass breaks on QNN graph

[corehalt]

I'm running the MergeComposite pass on a QNN graph.
Since this PR #6704 the type inference process somehow breaks TVM with the following error:

======================================================================
ERROR: qnn_test.test_quantized_imagenet
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/corehalt/pyenv/tvm_pt16/lib/python3.7/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/corehalt/src/private-tvm/tests/python/frontend/pytorch/qnn_test.py", line 437, in test_quantized_imagenet
    runtime = get_tvm_runtime(script_module, input_name, (1, 3, 224, 224))
  File "/home/corehalt/src/private-tvm/tests/python/frontend/pytorch/qnn_test.py", line 88, in get_tvm_runtime
    partitioned = composite_partition(mod)
  File "/home/corehalt/src/private-tvm/python/tvm/ir/transform.py", line 127, in __call__
    return _ffi_transform_api.RunPass(self, mod)
  File "/home/corehalt/src/private-tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (8) /home/corehalt/src/private-tvm/build/libtvm.so(void tvm::relay::ExpandDataflow<tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}>(tvm::RelayExpr, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2})+0xc85) [0x7fe03242ae75]
  [bt] (7) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)#5}::_FUN(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)+0x2c) [0x7fe03241bffc]
  [bt] (6) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeInferencer::VisitExpr_(tvm::relay::FunctionNode const*)+0x260) [0x7fe03242c2c0]
  [bt] (5) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeInferencer::GetType(tvm::RelayExpr const&)+0xee) [0x7fe03242b6ee]
  [bt] (4) /home/corehalt/src/private-tvm/build/libtvm.so(void tvm::relay::ExpandDataflow<tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}>(tvm::RelayExpr, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2})+0xc85) [0x7fe03242ae75]
  [bt] (3) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)#5}::_FUN(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)+0x2c) [0x7fe03241bffc]
  [bt] (2) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeInferencer::VisitExpr_(tvm::relay::FunctionNode const*)+0x46) [0x7fe03242c0a6]
  [bt] (1) /home/corehalt/src/private-tvm/build/libtvm.so(+0x4c71ff) [0x7fe0317011ff]
  [bt] (0) /home/corehalt/src/private-tvm/build/libtvm.so(+0x103e668) [0x7fe032278668]
  [bt] (8) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeInferencer::GetType(tvm::RelayExpr const&)+0xee) [0x7fe03242b6ee]
  [bt] (7) /home/corehalt/src/private-tvm/build/libtvm.so(void tvm::relay::ExpandDataflow<tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}>(tvm::RelayExpr, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::TypeInferencer::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2})+0xc85) [0x7fe03242ae75]
  [bt] (6) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)#5}::_FUN(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Type (tvm::RelayExpr const&)>*)+0x2c) [0x7fe03241bffc]
  [bt] (5) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeInferencer::VisitExpr_(tvm::relay::FunctionNode const*)+0x46) [0x7fe03242c0a6]
  [bt] (4) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::relay::TypeSolver::Solve()+0x449) [0x7fe032279fa9]
  [bt] (3) /home/corehalt/src/private-tvm/build/libtvm.so(tvm::runtime::TypedPackedFunc<bool (tvm::runtime::Array<tvm::Type, void> const&, int, tvm::Attrs const&, tvm::TypeReporter const&)>::AssignTypedLambda<bool (*)(tvm::runtime::Array<tvm::Type, void> const&, int, tvm::Attrs const&, tvm::TypeReporter const&)>(bool (*)(tvm::runtime::Array<tvm::Type, void> const&, int, tvm::Attrs const&, tvm::TypeReporter const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const+0x55f) [0x7fe031ab97ef]
  [bt] (2) /home/corehalt/src/private-tvm/build/libtvm.so(+0x12fc77b) [0x7fe03253677b]
  [bt] (1) /home/corehalt/src/private-tvm/build/libtvm.so(+0x12fc4fd) [0x7fe0325364fd]
  [bt] (0) /home/corehalt/src/private-tvm/build/libtvm.so(+0x12fc288) [0x7fe032536288]
  File "/home/corehalt/src/private-tvm/src/relay/analysis/type_solver.cc", line 621
TVMError: 
---------------------------------------------------------------
An internal invariant was violated during the execution of TVM.
Please read TVM's error reporting guidelines.
More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
---------------------------------------------------------------
  Check failed: false == false: [18:43:21] /home/corehalt/src/private-tvm/src/relay/qnn/op/../utils.h:175: 
---------------------------------------------------------------
An internal invariant was violated during the execution of TVM.
Please read TVM's error reporting guidelines.
More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
---------------------------------------------------------------
  Check failed: tensor_type == false: Only tensor type can be checked for scalar values. But got#[version = "0.0.5"]
meta[IncompleteType][0]
/* For debugging purposes the metadata section has been omitted.
 * If you would like to see the full metadata section you can set the 
 * option to `True` when invoking `astext`. 
 */

Steps to reproduce
Please note that currently there is no tests in TVM for checking both QNN and BYOC passes at the same time so we need to patch current TVM main branch.

• TVM branch: current main branch
• Pytorch version: 1.6
• This patch: https://gist.github.com/fed9c25b9e93d2f7838b3778b00498c5.git

Then:

cd <TVM root>
git clone https://gist.github.com/fed9c25b9e93d2f7838b3778b00498c5.git patch
git apply patch/type_inf_error.patch
TVM_FFI=ctypes python -m nose -v tests/python/frontend/pytorch/qnn_test.py

[corehalt]

@mbrookhart thank you for fixing the issue so fast, appreciate it.