Use UNet precision, add --upcast-sampling support

[brkirch]

This will fix Macs needing --no-half (by adding dtype casting for --upcast-sampling) but also has a significant change - currently the ControlNet models appear to be loaded with full precision by default, this PR changes the precision to match the UNet precision of the loaded model in web UI. Unless there is something I'm missing, this change should significantly reduce VRAM usage for all users that aren't using --no-half.

Fixes #87, fixes #68.

[Mikubill]

Looks good. Will do some test later

[Mikubill]

Confirm works on M1 Pro and M1. But the result is weird: Looks like something was upscaled inproperly

[jwooldridge234]

Just tested on MacOs M1 Pro, can't see any issues and it runs super smooth now. Thanks for the hard work on this!

[Mikubill]

Hmm.. any screenshots? Maybe I misconfigured something on my MacBook.

[jwooldridge234]

Sure!

I'm only using Scribble at the moment and I always use the preprocessor bc I've found it steers the final image better.

Here's without the preprocessor:

[jwooldridge234]

Haven't tested the other models yet, maybe there's issues with those.

[Mikubill]

Looks nice. Note that MPS support is experimental, feel free to open issue if you have any questions.

[brkirch]

Confirm works on M1 Pro and M1. But the result is weird: Looks like something was upscaled inproperly

I’m going to guess that it has something to do with the PyTorch version; I’m using nightly builds (from February 10 currently; see AUTOMATIC1111/stable-diffusion-webui#7453 for the webui-user.sh file with the exact configuration). In the near future I plan to have PyTorch 2.0 set as the default version for Macs running web UI anyway, so any PRs I submit here for MPS will be intended to be used with that webui-user.sh configuration.

[baigalibao]

Error
Expecting value: line 1 column 1 (char 0)

recall this problem ,i dont know why