Is there an explanation for training new adapters?

[Njasa2k]

How much VRAM and time does training take?

[scarbain]

According to their paper, they trained each adapter for 10 epochs with a batch size of 8 on at least 120K images.
I've managed to run the training with a batch size of only 1 with 12GB VRAM.

[scarbain]

I'll fork the project and create a PR then :)

My training is currently at 90K steps but still no sign of it converging... Have you seen my question in the issues @xinntao ? #3

[xinntao]

For the training, it is better to finetune from a pretrained adapter. Otherwise, it will consume a lot of computation to converge.

Later, we will try to figure out a better strategy for customized training.

[scarbain]

I was wondering if I should have finetuned one of your pretrained adapter but since it was quite different I didn't think it was worth it.

I have 2 questions then :

1. When you trained your adapters, did you train one first and then finetuned it for the others ?
2. And while training, did you notice that it slowly started to follow the guide or did it show "sudden convergence phenomenon" (as the authors of ControlNet noticed in their paper https://github.com/lllyasviel/ControlNet) ?

[Saquib764]

I am curious about your two question @scarbain . I am trying to train an adapter from scratch, it would be nice if you can share your learning. @xinntao

[zhanghh-031102]

How to retrain an adapter and adjust the number of channels of the extracted four-layer features from [320, 640, 1280, 1280] to [320, 320, 640, 1280]? It would be great if you could share it