Textual Inversion's dlr0 diverges to inf

[and1011]

I didn't test the algorithm before implementation of d-adaptation, but now, when I try to train a textual inversion, dlr0 starts low, which I suppose is correct anyway, but quickly starts increasing (exponentially) with each step until it reaches the float limit.
Also, after the value diverges to inf the embedding becomes unusable.
If I understood correctly, what could be wrong, but in theory shouldn't the dlr converge to some value?
Anyone else having this problem or who could help?

[bmaltais]

This is usually because the LR is too high... You could try to lower it a bit.

[and1011]

I saw here that you fixed it in this commit 34ab844

Now that I've updated this bug has been fixed, I can now use AdamW normally.
Thank you.