Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Topi] Tensorcore support for Conv3D #5284

Merged
merged 8 commits into from
Apr 13, 2020
Merged

Conversation

jwfromm
Copy link
Contributor

@jwfromm jwfromm commented Apr 8, 2020

This PR is a pretty direct port of the conv2d tensorcore schedules introduced in PR #5099 to conv3d. In my early testing I've found this new schedule to be up to 10X faster than the default conv3d schedule. I also snuck one little adjustment into the conv3d winograd schedule that helps for smaller workloads. Given that tensorcore support is currently only for NHWC and NDHWC layouts, I've also added support for converting to these layouts using the ConvertLayout pass.

@jwfromm
Copy link
Contributor Author

jwfromm commented Apr 8, 2020

@Shawn-Inspur, @Laurawly could you take a look at this PR?

@tqchen
Copy link
Member

tqchen commented Apr 8, 2020

also cc @icemelon9 since it is related to stragey

@jwfromm
Copy link
Contributor Author

jwfromm commented Apr 9, 2020

Anyone know what the deal with the CI failure is? Seems to work fine on my local branch and has nothing to do with this PR.

Running './tests/scripts/task_sphinx_precheck.sh' inside tvmai/ci-gpu:v0.61...
mesg: ttyname failed: Inappropriate ioctl for device
Adding group `tvm' (GID 1000) ...
Done.
PreCheck sphinx doc generation WARNINGS..
rm -rf _build/*
rm -rf gen_modules
rm -rf tutorials
rm -rf vta/tutorials
python3 -m sphinx -b html -d _build/doctrees   . _build/html
Running Sphinx v2.4.1
Makefile:70: recipe for target 'html' failed
script returned exit code 2

@tqchen
Copy link
Member

tqchen commented Apr 9, 2020

The sphinx error is a known flaky case that we should look into, please push to retrigger

@FrozenGene
Copy link
Member

@yangjunpro @minminsun Please also have a look, because you have done tensorcore support ever.

Copy link
Member

@icemelon icemelon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. just some minor comments

topi/tests/python/test_topi_conv3d_ndhwc_tensorcore.py Outdated Show resolved Hide resolved
python/tvm/relay/op/nn/_nn.py Outdated Show resolved Hide resolved
Copy link

@yangjunpro yangjunpro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work.
Just two small cents.
The schedule looks a little bit complicated.
Have your considered enriching the Auto TensorCore codegen pass to automate the TensorCore schedule generation process?
The previous work of Auto TensorCore codegen only covers GEMM and it does require additional work to add conv support. However with the addition of conv support into Auto TensorCore codegen pass, it may make the convolution tensorcore optimization more generic.

python/tvm/relay/op/nn/_nn.py Outdated Show resolved Hide resolved
@jwfromm
Copy link
Contributor Author

jwfromm commented Apr 12, 2020

@yangjunpro, funnily enough I asked nearly the same question to the authors of the conv2d tensorcore schedule. You can read their answer here. The quick take is that it does not support conv (like you mentioned) but more importantly that it causes significant performance regression compared to a bespoke approach like this one.

Copy link
Contributor

@Laurawly Laurawly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tqchen tqchen merged commit 3df8d56 into apache:master Apr 13, 2020
@tqchen
Copy link
Member

tqchen commented Apr 13, 2020

Thanks @jwfromm @Laurawly @Shawn-Inspur @yangjunpro @icemelon9 !

trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Apr 16, 2020
* one weird trick.

* Added schedule knob for different workloads.

* Initial conv3d tensorcore working.

* Added conv3d tensorcore strategy.

* Added layout conversion to tensorcore friendly format for conv2d and conv3d.

* Add target name check.

* Fixed bad names and depthwise check.

* Removed duplicated attribute assignment.
zhiics pushed a commit to neo-ai/tvm that referenced this pull request Apr 17, 2020
* one weird trick.

* Added schedule knob for different workloads.

* Initial conv3d tensorcore working.

* Added conv3d tensorcore strategy.

* Added layout conversion to tensorcore friendly format for conv2d and conv3d.

* Add target name check.

* Fixed bad names and depthwise check.

* Removed duplicated attribute assignment.
dpankratz pushed a commit to dpankratz/incubator-tvm that referenced this pull request Apr 24, 2020
* one weird trick.

* Added schedule knob for different workloads.

* Initial conv3d tensorcore working.

* Added conv3d tensorcore strategy.

* Added layout conversion to tensorcore friendly format for conv2d and conv3d.

* Add target name check.

* Fixed bad names and depthwise check.

* Removed duplicated attribute assignment.
@jwfromm jwfromm deleted the winograd3d branch April 12, 2023 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants