allow pndm scheduler to be used with ldm pipeline #165
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
| @@ -113,6 +113,7 @@ def step( | |||
| model_output: Union[torch.FloatTensor, np.ndarray], | |||
| timestep: int, | |||
| sample: Union[torch.FloatTensor, np.ndarray], | |||
| **kwargs, | |||
There was a problem hiding this comment.
@patil-suraj @anton-l - I'd prefer a bit to keep the schedulers as clean as possible and instead make the pipeline more hacky.
Could we maybe delete the unused kwargs here and instead make use of inspect.signature in the pipeline?
Analogs to https://github.com/huggingface/transformers/blob/05d3a43c59dadf815fe46df8fed8cc8e816c8588/src/transformers/generation_utils.py#L1171
Note that the schedulers should eventually work for all frameworks and it's IMO components where we strongly favor readability and intuitive design over hacky/easy-to-use.
What do you think?
There was a problem hiding this comment.
I see. I didn't think of using inspect.signature. I am also not in full favor of adding unused arguments, but didn't find a better solution so went with it as I wanted to get it rolling today.
Will remove this and adapt the pipeline.
rewrite some tflite code from SharkImporter to tflite_utils.py
The Stable diffusion model uses PNDM as the main scheduler. Right now it's not possible to use PNDM scheduler in
LDMText2ImagePipelineas theetafor DDIM is always passed tostepmethod. ThePNDMSchedulerdoes not have this argument.This PR adds
**kwargsinPNDMScheduler.stepand passesetaas kwarg inLDMText2ImagePipeline, so that the pipeline can be used with both schedulers. Since both LDM and stable diffusion model can be used with both these schedulers it's important to allow this interoperability.