Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[transformer v2] Load + evaluate FID for OpenAI guided-diffusion models #84

Open
wants to merge 7 commits into
base: transformer-model-v2
Choose a base branch
from

Conversation

Birch-san
Copy link

@Birch-san Birch-san commented Oct 17, 2023

  • loads OpenAI guided-diffusion UNet
    • assigns to inner_model_ema only
    • constructs 256x256 guided diffusion UNet, with config.json overrides
    • prepares model_ema with accelerate
    • wraps it with k-diffusion wrapper
    • grabs sigma_min, sigma_max from wrapper
    • loads weights, requires_grad(False) + eval()s it
  • --evaluate-only no longer requires you to specify a (redundant) --evaluate-every option
  • --evaluate-only now turns off a bit more stuff
    • avoids constructing trainable model (so you don't pay RAM price of a copy you don't use, and so you don't pay VRAM to have accelerator prepare it)
    • avoids constructing + accelerator-preparing optimizer and schedules
    • doesn't force you to define a config for optimizer/schedules (well, there are defaults for those but still)
  • switched demo/evaluate from no_grad to inference_mode, which is even lower-overhead

as for the config.json overrides for constructing the guided diffusion UNet…
compared to what was in your clip_guided_diffusion/main.py, I:

  • changed image_size 512->256
  • removed "rescale_timesteps": True, (making it default to False)
  • removed "timestep_respacing": "1000", (making it default to "")
  • removed "use_checkpoint": False, (which was already the default)

I did this to closer match the MODEL_FLAGS they documented in their README:
https://github.com/openai/guided-diffusion#classifier-guidance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant