Accurate op count for convolutions?

[jonathanasdf]

I'm not sure how cudnn does the convolutions, but I think fbcunn uses some FFT based method that probably requires much fewer ops than naive convolution. Any ideas?

[apaszke]

Oh yes, definitely. I created it to get a rough estimates of operation counts, so I assumed a naive algorithm there, but it can be definitely improved. I have no idea how many operations are required for FFT based convolutions, and I would have to read up more first. If you know this, you can either send me a formula, or I'll be more than happy to merge a PR.

As for cudnn, it depends. You can set it to use a particular algorithm, but by default it uses some heuristics to choose the best one, depending on kernel and input sizes, as well as available memory.