Diffusion model benchmark

[zzpmiracle]

We have support diffusers in #867 . This issue tracks performance of all the diffuser pipelines. For the concern of performance, we use BlaDNN to tuning models during runtime.
The following pipelines would be tested:

• StableDiffusionPipeline
  • runwayml/stable-diffusion-v1-5
  • stabilityai/stable-diffusion-2-1-base
• StableDiffusionImg2ImgPipeline
  • runwayml/stable-diffusion-v1-5
• StableDiffusionDepth2ImgPipeline
  • stabilityai/stable-diffusion-2-depth
• StableDiffusionInpaintPipeline
  • runwayml/stable-diffusion-inpainting
• AltDiffusionPipeline
  • BAAI/AltDiffusion

[zzpmiracle]

pipeine	model	img size	unet-pytorch	unet-disc	e2e-pytorch	e2e-disc
stableDiffusionPipeline	stabilityai/stable-diffusion-2-1	768*768	119.81ms	60.08ms	6.21s	3.14s
StableDiffusionImg2ImgPipeline	runwayml/stable-diffusion-v1-5	768*512	66.22ms	29.49ms	3.25s	1.30s
StableDiffusionDepth2ImgPipeline	stabilityai/stable-diffusion-2-depth	480*640	44.33ms	19.89ms	2.08s	0.89s
StableDiffusionInpaintPipeline	runwayml/stable-diffusion-inpainting	512*512	37.52ms	19.17ms	2.63s	1.10s
AltDiffusionPipeline	BAAI/AltDiffusion	512*512	37.93ms	18.74ms	2.65s	1.08s

[tanyokwok]

How about the e2e latency run with TensorRT/AIT?

[vivienfanghuagood]

pipeine model img size unet-pytorch unet-disc e2e-pytorch e2e-disc
stableDiffusionPipeline stabilityai/stable-diffusion-2-1 768768 119.81ms 60.08ms 6.21s 3.14s
StableDiffusionImg2ImgPipeline runwayml/stable-diffusion-v1-5 768512 66.22ms 29.49ms 3.25s 1.30s
StableDiffusionDepth2ImgPipeline stabilityai/stable-diffusion-2-depth 480640 44.33ms 19.89ms 2.08s 0.89s
StableDiffusionInpaintPipeline runwayml/stable-diffusion-inpainting 512512 37.52ms 19.17ms 2.63s 1.10s
AltDiffusionPipeline BAAI/AltDiffusion 512*512 37.93ms 18.74ms 2.65s 1.08s

Can you share the scripts to run these benchmarks?

[bonlime]

@zzpmiracle Where I can find scripts to run this benchmarks? i've tried using it on the latest diffusers version, and it can't trace the components...

[zzpmiracle]

@zzpmiracle Where I can find scripts to run this benchmarks? i've tried using it on the latest diffusers version, and it can't trace the components...

maybe we can use docker images in this zhihu article https://zhuanlan.zhihu.com/p/631461489

[zwj536]

@zzpmiracle Where I can find scripts to run this benchmarks? i've tried using it on the latest diffusers version, and it can't trace the components...

maybe we can use docker images in this zhihu article https://zhuanlan.zhihu.com/p/631461489

@zzpmiracle hi, do you have an example of Hires.fix ?