Add reduction ops

[sbrunk]

https://pytorch.org/docs/stable/torch.html#reduction-ops

• ops implementation
• tests

[sbrunk]

Hey @davoclavo could you have a look when you find the time? I've added most reduction ops and also tests for most of them (made much easier thanks to your helpers).

[davoclavo]

LGTM! Amazing work! And awesome docs, I will try to follow that style from now on.

I just added a quick comment regarding type promotion.

And also another quick question: I have been changing snake_case to camelCase to match scala style, however there is some benefit of sticking with snake_case to match pytorch nomenclature. Should pick
one over the other to have consistency?

[davoclavo]

Maybe you could use FloatPromoted[D]

[sbrunk]

Right, but trying that I think we might need to add cases for float16/bfloat16 to FloatPromoted:

I.e. sin in PyTorch:

torch.sin(torch.tensor(1., dtype=torch.bfloat16)).dtype # torch.bfloat16

scala> torch.sin(torch.Tensor(1f).to(dtype=torch.bfloat16)).dtype
java.lang.ClassCastException: class torch.DType$bfloat16$ cannot be cast to class torch.Float32 (torch.DType$bfloat16$ and torch.Float32 are in unnamed module of loader sbt.internal.inc.classpath.ClasspathUtil$$anon$2 @586fb5d6)

[sbrunk]

I think I got it now, although the type is getting somewhat verbose 😆

Tensor[Promoted[FloatPromoted[ComplexToReal[D]], FloatPromoted[ComplexToReal[D2]]]]

[sbrunk]

LGTM! Amazing work! And awesome docs, I will try to follow that style from now on.

Thanks! I wanted to document how I convert the docs, but didn't get to it yet.

I'm using the original PyTorch rst doc sources, i.e. for ops in torch:
https://raw.githubusercontent.com/pytorch/pytorch/master/torch/_torch_docs.py
or for torch.tensor methods:
https://github.com/pytorch/pytorch/blob/main/torch/_tensor_docs.py
Sometimes they are also regular docstrings:
https://github.com/pytorch/pytorch/blob/main/torch/functional.py

1. Search for the function you're about to port. I usually have then generated doc open in the browser as well to help with search and to verify I get everything I want from the source (sometimes things are factored out into variables and reused etc.).
2. Copy and convert the rst to markdown. I'm just using the online interface of pandoc: https://pandoc.org/try/
3. Copy the markdown into Scaladoc and manually clean it up/fix things
4. Run scalafmt

The conversion is not perfect but it often gets you 80% there reducing a lot of tedious manual work.

Also please don't feel pressured to add all the docs immediately if you don't feel like it when porting things over. They can always be added/improved later on as well. :)

And also another quick question: I have been changing snake_case to camelCase to match scala style, however there is some benefit of sticking with snake_case to match pytorch nomenclature. Should pick one over the other to have consistency?

Yeah that's something I'm not satisfied with either way. I started with camelCase for idiomatic style but realized it makes porting and searching for docs a bit more inconvenient. Since most names are now camelCase already, I'm currently inclined to stick with it, but I'm absolutely not religious about it. If you feel it makes more sense to move to snake_case, now or later, let's discuss it.