[WIP] Fix .view().detach() not handled correctly by AOT autograd

[sangongs]

This is to fix issue pytorch/pytorch#93677

Depending on pytorch/pytorch#86838

[Chillee]

@sangongs I don't think this is the right solution. Inductor should not be responsible for handling things like autograd.

[sangongs]

@Chillee Thanks for having an eye on this.

Inductor should not be responsible for handling things like autograd.

Agreed.

This is just a tentative workaround for an AOT autograd issue. AOT autograd currently does not handle detach() correctly. According to @bdhirsh:

it's because we're compiling the whole thing (including the detach() call) into an autograd.Function, and autograd.Function will unconditionally mark all of its forward outputs as requiring gradients.

@bdhirsh made a fix:

We can use autograd.function's mark-nondifferentiable API, to (statically) mark those outputs as not requiring gradients

But the fix does not work as expected if the output is a view, because mark_non_differentiable takes no effect in such a case:
https://github.com/pytorch/pytorch/blob/ae45dab57e22e3d04516e7dd81ef8dbefd51bfe3/torch/csrc/autograd/custom_function.cpp#L290-L299

See the AOT autograd issue for more details: pytorch/functorch#1052

In my view, there are four options to fix this:

1. Leave autograd.Function as it is. Work around the is_view() problem in Inductor like this PR.
2. Work around the problem in AOT autograd. We are discussing this in the AOT autograd issue. But apparently @bdhirsh has concerns.
3. Work around the problem in Dynamo. This may take a lot of efforts.
4. Update autograd.Function such that mark_non_differentiable() takes effect when an output is a view. But I am not sure if this is viable.

@Chillee Do you have any suggestions?

[bdhirsh]

Oh tbc, my vote is probably still for (2) - manually add in some extra .detach() calls in AOTAutograd, so inductor isn't forced to worry about requires-grad-ness (thread)

[jansel]

We have migrated torchdynamo to torch._dynamo and will use the pytorch/pytorch repo for future development. Please resubmit this PR to https://github.com/pytorch/pytorch/

More details and instructions to port this PR over can be found in #1588