[FIX] Clear relay cache after every build & Clear warning message cache after autotvm task extraction

[tkonolige]

Clear the compile cache between module builds so that schedule changes will have an effect. Also, clear the warning cache so that schedule changes properly list untuned ops.

Note: this makes CompileEngine a mutable global variable.

@merrymercy @jwfromm

[jwfromm]

What's the reason for dropping the const here?

[tkonolige]

Clearing the cache mutates the CompileEngine, so in order to clear the cache after each build we need to modify the global state.

[jwfromm]

@tkonolige, looks like the CI got aborted, can you rebase and push to trigger a new run?

[tkonolige]

CI is getting errors like this:

>           raise get_last_ffi_error()
E           tvm._ffi.base.TVMError: Traceback (most recent call last):
E             [bt] (6) /workspace/build/libtvm.so(TVMFuncCall+0x65) [0x7f57e94d4785]
E             [bt] (5) /workspace/build/libtvm.so(+0xcc2606) [0x7f57e9567606]
E             [bt] (4) /workspace/build/libtvm.so(+0xcc24c3) [0x7f57e95674c3]
E             [bt] (3) /workspace/build/libtvm.so(tvm::runtime::GraphRuntime::Init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::Module, std::vector<DLContext, std::allocator<DLContext> > const&)+0x241) [0x7f57e9567141]
E             [bt] (2) /workspace/build/libtvm.so(tvm::runtime::GraphRuntime::SetupOpExecs()+0x591) [0x7f57e9566d01]
E             [bt] (1) /workspace/build/libtvm.so(tvm::runtime::GraphRuntime::CreateTVMOp(tvm::runtime::TVMOpParam const&, std::vector<DLTensor, std::allocator<DLTensor> > const&, unsigned long)+0x6e1) [0x7f57e9564dd1]
E             [bt] (0) /workspace/build/libtvm.so(+0xcbce12) [0x7f57e9561e12]
E             File "/workspace/src/runtime/graph/graph_runtime.cc", line 387
E           TVMError: Check failed: pf != nullptr: no such function in module: fused_add_subtract_concatenate

python/tvm/_ffi/_ctypes/packed_func.py:225: TVMError

I have no clue how this PR is triggering those errors.

[jwfromm]

@merrymercy, any idea why this change is triggering CI errors?

[merrymercy]

@tkonolige Can you reproduce the error locally? If so, I guess CompileEngine::Global()->Clear() might not clear everything correctly, resulting in missing functions in the built module.
cc @tqchen

[tkonolige]

@tkonolige I can reproduce the errors locally. Will look into it.

[tkonolige]

@tqchen @merrymercy I can't figure this error out. Could one of you take a look at it?

[tkonolige]

One of the failing tests is tests/python/relay/test_external_runtime.py. You can play around with it to try and debug.

[jroesch]

I pinged @zhiics since he is the owner afaik of this code, he said he would look at it tomorrow.

[zhiics]

Yeah, one quick thing I can do is to retire the example runtime since we have a real json runtime now. This example is not really use any more. I can send a PR later. cc @comaniac

[tkonolige]

@merrymercy @tqchen We can merge this now.