Underflow when saving LoRa as fp16

[IdiotSandwichTheThird]

--save_precision=fp16 will cause a lot of 0's in lora modules.
--save_precision=float seems to work fine.

[kohya-ss]

Thank you for this!
I found an issue in the check_lora_weights.py. The script showed 0.0 for some layers even if the weights are trained, because of the overflow in torch.mean. Sorry for that.

I've fixed the issue, and the script now shows a proper result.

However, I think there is the underflow issue yet, described in kohya-ss/sd-webui-additional-networks#49. I am working on the issue, and will fix sooner.

[IdiotSandwichTheThird]

Thank you for the hard work!