[Bug] Multiple IP-Adapters apply order

[huchenlei]

#2245 requests multiple images for single IP-Adapter unit because multiple IP-Adapter units do not seem to work together.

Reproduction:

Following 2 setups use exact IP-Adapter units but in different orders, and give totally different results.

Setup1

Unit 0:

Unit 1:

Result:

Despite unit 1 having very low weight, the result still mostly look like unit1 (A tiger)

Setup2

Unit 1:

Unit 2:

Result:

Result mostly look like unit2, and obviously the weight is too much.

Using only unit 0 (girl) weight = 1

Using only unit 1 (tiger) weight = 0.3

There is definitely something going wrong with how multiple IP-Adapters are applied.

[huchenlei]

After some debug, it seems like only the last unit of IPAdapter get applied, and it get applied twice when 2 units are enabled. This is reasonable as tiger (weight = 0.3) applied twice gives a tiger result, but girl (weight = 1) applied twice gives a over saturated girl result.

[huchenlei]

Fix:

    @staticmethod
    def load_control_model(p, unet, model):
        # ip-adapter model contains embedding data, so each model is unique.
        if 'ip-adapter' not in model and model in Script.model_cache:
            logger.info(f"Loading model from cache: {model}")
            return Script.model_cache[model]

Previously, it was assumed that model only contains state_dict loaded from model file. However, ip-adapter model also contains input image embedding, weight, and many values, so should not be cached.