Full bf16 support

[KohakuBlueleaf]

full fp16 will cause some unstable problem.
And if the GPU support bf16, use full bf16 will be better (bf16 weight + bf16 amp + bf16 grad)
The only problem of this thing is, if user want to use optimizer in the bitsandbytes, they will need the newest version to utilize full bf16 training.
Which is bad nes for windows user. (or we need to compile the newest bitsandbytes for windows, which could be tricky)

so just a tiny start for a useful feature, need more improvement. (like check if the gpu support bf16, if the bitsandbytes support bf16 grad ...)

[kohya-ss]

Thank you for this!

There are some concerns as you wrote, I don't see a problem to implement this as an experimental feature.

It is unfortunate that bitsandbytes does not work on Windows, but it would be possible to use another optimizer.