[Relay][AutoTVM] Relay op strategy #4644

icemelon · 2020-01-07T18:57:41Z

RFC & discussion: https://discuss.tvm.ai/t/relay-op-strategy/5247/18
Update AutoTVM log format: https://discuss.tvm.ai/t/autotvm-log-format/5630

Todo list

Add op strategy for conv2d & conv2d related ops (e.g., conv2d_nchwc, conv2d_winograd)
Add & update op strategy for conv2d_transpose, dense, bitpack, bitserial_conv2d, bitserial_dense
Clean up autotvm (after updating all old autotvm templates)
Rewrite TOPI test
Update AutoTVM related tutorials

Thanks @kevinthesun @comaniac for the help to this PR!

MarisaKirisame · 2020-01-23T23:38:51Z

When is conv2d gonna be here? Need it for training.

icemelon · 2020-01-23T23:39:43Z

x86 conv2d is already in the pr. working on cuda and more targets now

icemelon · 2020-01-29T20:28:31Z

I've added the strategy for all ops. We can start to review this PR since it's huge. Could you help review it?
@tqchen @kevinthesun @comaniac @masahi @MarisaKirisame @hlu1 @yzhliu @zhiics @ZihengJiang @merrymercy @vinx13 @FrozenGene @jroesch @jwfromm

comaniac

Leave some comments related to AutoTVM changes.

python/tvm/relay/backend/compile_engine.py

python/tvm/build_module.py

python/tvm/relay/backend/compile_engine.py

comaniac · 2020-01-29T23:47:38Z

python/tvm/autotvm/task/dispatcher.py

            self._counter += 1
-            self.update(target, workload, cfg)
+            self.update(target, wkl, cfg)
+            cfg.workload = wkl


Where is cfg.workload initialized? I didn't find a definition in ConfigSpace. Also what's the purpose to have a workload field in a config space?

This is only specific to ApplyGraphBest. The reason is complicated. Because ApplyGraphBest relies on the order of query, we cannot use relay.backend.compile_engine.select_implement to collect the autotvm workload as it may query more than once. Therefore, this is a temporary work around that we sneak in the workload in the return cfg. We can remove this part of logic after we make ApplyGraphBest no longer relies on the query order.
@kevinthesun

I see. Could we define self.workload in ConfigSpace and add your comment on it? So that we will remember to remove it in the future.

python/tvm/autotvm/task/task.py

python/tvm/relay/quantize/_annotate.py

Laurawly · 2020-01-30T00:27:52Z

topi/python/topi/intel_graphics/conv2d.py

    kernel_L = s.cache_read(kernel, "local", [conv_L])

+    if temp.name == "pad_temp":
+        data = temp.op.input_tensors[0]
+        # TODO(@Laurawly): Do we need to schedule pad op here?


Yeah we need to schedule temp.

tqchen · 2020-01-31T23:30:33Z

cc @merrymercy @vinx13 @ZihengJiang @jwfromm please help review if you have time

python/tvm/relay/backend/compile_engine.py

include/tvm/relay/op_attr_types.h

python/tvm/relay/op/strategy/hls.py

python/tvm/relay/op/strategy/x86.py

include/tvm/te/schedule.h

include/tvm/relay/op_attr_types.h

include/tvm/te/schedule.h

python/tvm/relay/backend/compile_engine.py

python/tvm/schedule.py

python/tvm/autotvm/task/space.py

…phub

icemelon · 2020-02-24T19:50:36Z

@merrymercy Now the tophub version number has been updated and VTA TSIM test time is also back to normal after the fix. Could you review the PR again?

* relay op strategy fix lint bitpack strategy bitserial_dense (neo-ai#6) * update strategy * address comments fix a few topi test Dense strategy (neo-ai#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (neo-ai#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (neo-ai#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (neo-ai#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (neo-ai#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (neo-ai#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <[email protected]>

* relay op strategy fix lint bitpack strategy bitserial_dense (#6) * update strategy * address comments fix a few topi test Dense strategy (#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <[email protected]>

* relay op strategy fix lint bitpack strategy bitserial_dense (apache#6) * update strategy * address comments fix a few topi test Dense strategy (apache#5) * dense * add biforst; remove comments * address comment Refactor x86 conv2d_NCHWc (#4) * Refactor x86 conv2d * Add x86 depthwise_conv2d_NCHWc * Add back topi x86 conv2d_nchw * Merge x86 conv2d_nchw and conv2d_NCHWc * Minor fix for x86 conv2d fix more strategy Add x86 conv2d_NCHWc_int8 strategy (apache#8) * Add x86 conv2d_NCHWc_int8 strategy * Remove contrib_conv2d_nchwc_int8 * Fix generic conv2d_NCHWc for int8 * Fix topi arm_cpu conv2d_NCHWc_int8 update x86 conv2d enable specify relay ops to be tuned for autotvm add cuda conv2d strategy add conv2d strategy for rocm add conv2d strategy for hls add conv2d strategy for arm cpu add conv2d strategy for mali add conv2d strategy for bifrost add conv2d strategy for intel graphics clean up and fix lint remove template keys from autotvm remove 2 in the func name address comments fix * fix bugs * lint * address comments * add name to op implement * Modify topi tests (apache#9) * Add pooling, reorg, softmax and vision * Add lrn * fix topi test * fix more topi test * lint * address comments * x * fix more tests & bugs * Modify more tests (apache#10) * Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn * Minor fix * More minor fix * fix more test * try to update vta using strategy * fix cpptest * x * fix rebase err * Fix two tests (apache#11) * change autotvm log format * lint * minor fix * try fix vta test * fix rebase err * tweak * tmp hack for vta pass * fix tutorial * fix * fix more tutorials * fix vta tutorial * minor * address comments * fix * address comments * fix cpptest * fix docs * change data structure name and api * address comments * lint * fix rebase err * updates * fix winograd test * fix doc * rebase * upgrade tophub version number * fix bug * re-enable vta tsim test after tophub is upgraded * fix vta test to use the correct args so the config can be found in tophub Co-authored-by: Yao Wang <[email protected]>

apivovarov · 2021-11-08T22:37:41Z

There are couple TODOs related to PR4644 - TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged

tvm/python/tvm/relay/op/nn/nn.py

Line 220 in d11bdcd

# TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged
tvm/python/tvm/relay/qnn/op/qnn.py

Line 348 in 4ffbdcd

# TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged

icemelon force-pushed the op-dispatch branch from 3d3984d to ff0ca78 Compare January 8, 2020 22:41

icemelon mentioned this pull request Jan 9, 2020

[CONV3D] remove incorrect schedule for conv3d_ndhwc #4659

Closed

yzhliu mentioned this pull request Jan 11, 2020

[Arith] add SizeVar representing non-neg valued variable in a tensor shape #4684

Merged

icemelon force-pushed the op-dispatch branch 2 times, most recently from 6b4cb34 to d42c343 Compare January 22, 2020 21:50

icemelon mentioned this pull request Jan 23, 2020

[TOPI] Remove cpp upsampling and resize op #4769

Merged

icemelon force-pushed the op-dispatch branch 2 times, most recently from a90ee23 to 02cd271 Compare January 27, 2020 19:49

icemelon marked this pull request as ready for review January 29, 2020 20:20

icemelon requested review from Huyuwei and Laurawly as code owners January 29, 2020 20:20

icemelon changed the title ~~[Draft] Relay op strategy~~ [WIP] Relay op strategy Jan 29, 2020

comaniac reviewed Jan 29, 2020

View reviewed changes

Laurawly reviewed Jan 30, 2020

View reviewed changes

icemelon mentioned this pull request Jan 30, 2020

[Relay] Conv2D padding representation #4787

Merged

tqchen added the status: need review label Jan 31, 2020

icemelon force-pushed the op-dispatch branch from 8dd4a6b to 6c2d508 Compare February 3, 2020 19:30

MarisaKirisame reviewed Feb 4, 2020

View reviewed changes

python/tvm/relay/backend/compile_engine.py Show resolved Hide resolved

icemelon force-pushed the op-dispatch branch 2 times, most recently from 897eb64 to b233005 Compare February 5, 2020 23:06

junrushao reviewed Feb 5, 2020

View reviewed changes

include/tvm/relay/op_attr_types.h Outdated Show resolved Hide resolved

icemelon force-pushed the op-dispatch branch from efcddbb to 6ad83fe Compare February 6, 2020 00:04

jwfromm reviewed Feb 6, 2020

View reviewed changes

python/tvm/relay/op/strategy/hls.py Outdated Show resolved Hide resolved

python/tvm/relay/op/strategy/x86.py Outdated Show resolved Hide resolved

icemelon force-pushed the op-dispatch branch from 6ad83fe to d832dd6 Compare February 6, 2020 20:13

zhiics reviewed Feb 7, 2020

View reviewed changes

ZihengJiang reviewed Feb 7, 2020

View reviewed changes

python/tvm/autotvm/task/space.py Show resolved Hide resolved

icemelon added 13 commits February 23, 2020 16:55

fix cpptest

2f3c719

fix docs

0445e5a

change data structure name and api

e496ae0

address comments

ea3dfaa

lint

eb630ef

fix rebase err

6cf45b5

updates

8b0081a

fix winograd test

3510177

fix doc

cf43e16

rebase

f07d92a

upgrade tophub version number

59bd399

fix bug

5338a57

re-enable vta tsim test after tophub is upgraded

24ae797

icemelon force-pushed the op-dispatch branch from 630cbec to 24ae797 Compare February 24, 2020 00:56

fix vta test to use the correct args so the config can be found in to…

4f1806a

…phub

yzhliu approved these changes Feb 24, 2020

View reviewed changes

merrymercy approved these changes Feb 24, 2020

View reviewed changes

merrymercy merged commit 623dd20 into apache:master Feb 24, 2020

icemelon deleted the op-dispatch branch February 24, 2020 21:14

icemelon added status: accepted and removed status: need review labels Feb 25, 2020

w-zr mentioned this pull request Feb 29, 2020

Performance regression of quantization on CUDA after [Relay][AutoTVM] Relay op strategy (#4644) #4972

Closed

FrozenGene mentioned this pull request Mar 11, 2020

[QNN] Support 4D padding. #5036

Merged

ZihengJiang mentioned this pull request Sep 17, 2020

TVM v0.7 Release Note Candidate #6486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Relay][AutoTVM] Relay op strategy #4644

[Relay][AutoTVM] Relay op strategy #4644

icemelon commented Jan 7, 2020 •

edited

Loading

MarisaKirisame commented Jan 23, 2020

icemelon commented Jan 23, 2020

icemelon commented Jan 29, 2020 •

edited

Loading

comaniac left a comment

comaniac Jan 29, 2020

icemelon Jan 30, 2020 •

edited

Loading

comaniac Jan 30, 2020

Laurawly Jan 30, 2020

tqchen commented Jan 31, 2020

icemelon commented Feb 24, 2020

apivovarov commented Nov 8, 2021

[Relay][AutoTVM] Relay op strategy #4644

[Relay][AutoTVM] Relay op strategy #4644

Conversation

icemelon commented Jan 7, 2020 • edited Loading

MarisaKirisame commented Jan 23, 2020

icemelon commented Jan 23, 2020

icemelon commented Jan 29, 2020 • edited Loading

comaniac left a comment

Choose a reason for hiding this comment

comaniac Jan 29, 2020

Choose a reason for hiding this comment

icemelon Jan 30, 2020 • edited Loading

Choose a reason for hiding this comment

comaniac Jan 30, 2020

Choose a reason for hiding this comment

Laurawly Jan 30, 2020

Choose a reason for hiding this comment

tqchen commented Jan 31, 2020

icemelon commented Feb 24, 2020

apivovarov commented Nov 8, 2021

icemelon commented Jan 7, 2020 •

edited

Loading

icemelon commented Jan 29, 2020 •

edited

Loading

icemelon Jan 30, 2020 •

edited

Loading