Loss problem when training the conditional denoising U-net for diffusion

[ssj9596]

I am currently training a conditional denoising U-net for a diffusion model, but I am encountering an issue with the loss function. When the batch size is set to 1, the loss value remains at 1 and does not decrease. However, when I set the batch size to 4, the loss value becomes NaN. I would like to know which parameters I should adjust to resolve this issue. Thank you very much!

[projectersMan]

Hi , I have the same issue , do you solve the problem yet?
would you like to share the method to solve it with me?