Fix [core/GLIGEN]: TypeError when iterating over 0-d tensor with In-painting mode when EulerAncestralDiscreteScheduler is used

[rchuzh99]

What does this PR do?

Fixes #5216
Re-opens PR #5214

This PR fixes the TypeError caused by trying to directly iterate over a 0-dimension tensor in the denoising stage of GLIGEN In-painting operation.

The error occurs when using diffusion noise schedulers that iterate over timesteps
(e.g. EulerAnchestralDiscreteScheduler, KDPM2AncestralDiscreteScheduler), during in-painting operation with the StableDiffusionGLIGENPipeline and StableDiffusionGLIGENTextImagePipeline .

For further clarification, this operation of the add_noise function 🔽

diffusers/src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

Lines 387 to 388 in ae2fc01

	step_indices = [(schedule_timesteps == t).nonzero().item() for t in timesteps]

in the affected noise schedulers expects the timesteps to be a non-0 dim torch Tensor. However, in the affected pipelines, timesteps is 0-dimension.

This PR references the approach found in the StableDiffusionInpaintingPipeline

diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py

Lines 1029 to 1034 in ae2fc01

	if i < len(timesteps) - 1:
	noise_timestep = timesteps[i + 1]
	init_latents_proper = self.scheduler.add_noise(
	init_latents_proper, noise, torch.tensor([noise_timestep])
	)

which is to wrap the timestep(t) 0-d tensor in a list to convert to 1-d tensor as follow 🔽

  if gligen_inpaint_image is not None:
      gligen_inpaint_latent_with_noise = (
          self.scheduler.add_noise(
              gligen_inpaint_latent, torch.randn_like(gligen_inpaint_latent), torch.tensor([t])
          )
          .expand(latents.shape[0], -1, -1, -1)
          .clone()
      )

https://github.com/rchuzh99/diffusers/blob/fb82fc4bdcead457e24a780cfb193070227f3e31/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_gligen.py#L799-L806 and https://github.com/rchuzh99/diffusers/blob/fb82fc4bdcead457e24a780cfb193070227f3e31/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_gligen_text_image.py#L960-L967

Affected pipelines

1. StableDiffusionGLIGENPipeline
2. StableDiffusionGLIGENTextImagePipeline

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

cc: @sayakpaul , @nikhil-masterful, @tuanh123789

References

1. StableDiffusionGLIGENPipeline: Add GLIGEN implementation #4441
2. StableDiffusionGLIGENTextImagePipeline: Add GLIGEN Text Image implementation #4777
3. StableDiffusionInpaintingPipeline: https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py

[sayakpaul]

Clean.

[sayakpaul]

Thanks so much!

Will merge once the CI is green

[rchuzh99]

Thanks so much!

Will merge once the CI is green

Thanks for the review @sayakpaul 👍🏻

[WuyangLuo]

Very useful ! Thx