Enable autograd graph to propagate after multi-device syncing for loss functions in ddp
#4580
| Job | Run time |
|---|---|
| 7s | |
| 47m 0s | |
| 46m 48s | |
| 46m 47s | |
| 1h 8m 39s | |
| 1h 7m 49s | |
| 1h 10m 54s | |
| 1h 9m 56s | |
| 1h 8m 11s | |
| 1h 8m 57s | |
| 48m 6s | |
| 1h 6m 44s | |
| 1h 4m 56s | |
| 1h 5m 1s | |
| 1h 0m 49s | |
| 1h 23m 33s | |
| 52m 1s | |
| 20m 31s | |
| 38m 49s | |
| 44m 52s | |
| 43m 8s | |
| 1s | |
| 19h 3m 39s |