[Refactor] Introduce target generic dispatch system

[tqchen]

This is a major refactor to TVM's python build and TOPI's build system. The major goal is to provide a target invariant way of calling TOPI API and enable easy patch in of new schedules under specific target.

In the future, we should always encourage user and NNVM to use the topi.generic functions and specify dispatch using the tvm.target

Target Generic Function

All schedules under topi.generic are target generic functions. When a target generic function is called, it will lookup the specific registered implementations under its own registry, call the specific implementation when matched.

Target Special Implementation

The target special schedule function can call generic_func.register to register a target special implementation of the function.

Target Keys

Each target has an ordered list of keys, for example, [ "cuda", "gpu"]. When a function is called under a target context, it will match the registered function by the key. So if an implementation is registered under key "cuda", it will be called, if not the function under "gpu" will be called.

Example

# Default schedule injective
@tvm.target.generic_func
def schedule_injective(outs):
    return tvm.create_schedule([x.op for x in outs])

# Register special schedule function for cuda
@schedule_injective.register(["cuda", "gpu"])
def schedule_injective_cuda(outs):
    ...

with tvm.target.cuda(): 
   # calls  schedule_injective_cuda
    s = schedule_injective(outs)

with tvm.target.rocm(): 
   # calls  schedule_injective_cuda, because it matches the "gpu key"
    s = schedule_injective(outs)

with tvm.target.rasp(): 
   # calls the default schedule_injective
    s = schedule_injective(outs)

[tqchen]

cc @Huyuwei @Laurawly @ZihengJiang @adityaatluri

[tqchen]

Will also need refactor of nnvm to directly always use the topi.generic function after this is merged.

[masahi]

num_thread = target.max_num_threads is nice. It solves AMD opencl problem I reported.

[masahi]

@tqchen if you set num_thread = target.max_num_threads, num_thread will be something like 1024 for cuda and rocm(hip). Is that ok?

Shouldn't it be like min(512, target.max_num_threads)?

Ah, I see this block in your commit, so no problem.

 elif target_name in ("cuda", "nvptx"):
             self.keys += ("cuda", "gpu")
             self.max_num_threads = 512