[Bug]: img2img-img2img controlnet-inpaint #1766

xiaojiemeidu · 2023-07-05T06:34:27Z

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

if I don't upload an image, controlnet-inpaint reports an error.

Steps to reproduce the problem

Go to ....
Press ....
...

What should have happened?

In the previous version, the whole picture would be drawn according to Denoising strength.

Commit where the problem happens

webui: 1.3.2and1.4
controlnet: 1.1.231

What browsers do you use to access the UI ?

Edge

Command Line Arguments

NO

List of enabled extensions

ONLY controlnet 1.1.231

Console logs

venv "D:\StabilityAI\stable-diffusion-webui-1.3.2\venv\Scripts\Python.exe"
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: <none>
Commit hash: <none>
Installing requirements


Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
[-] ADetailer initialized. version: 23.7.3, num models: 9
2023-07-06 08:57:11,601 - ControlNet - INFO - ControlNet v1.1.231
ControlNet preprocessor location: D:\StabilityAI\stable-diffusion-webui-1.3.2\extensions\sd-webui-controlnet\annotator\downloads
2023-07-06 08:57:11,783 - ControlNet - INFO - ControlNet v1.1.231
Loading weights [e4a30e4607] from D:\StabilityAI\stable-diffusion-webui-1.3.2\models\Stable-diffusion\majicmixRealistic_v6.safetensors
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.2s (import torch: 1.6s, import gradio: 1.0s, import ldm: 0.5s, other imports: 0.8s, list SD models: 0.5s, load scripts: 4.2s, create ui: 0.3s, gradio launch: 0.2s).
Creating model from config: D:\StabilityAI\stable-diffusion-webui-1.3.2\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\StabilityAI\stable-diffusion-webui-1.3.2\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying optimization: sdp-no-mem... done.
Textual inversion embeddings loaded(0):
Model loaded in 3.1s (load weights from disk: 0.7s, create model: 0.3s, apply weights to model: 0.4s, apply half(): 0.5s, move model to device: 0.4s, load textual inversion embeddings: 0.8s).
2023-07-06 08:58:09,137 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2023-07-06 08:58:09,409 - ControlNet - INFO - Loaded state_dict from [D:\StabilityAI\stable-diffusion-webui-1.3.2\extensions\sd-webui-controlnet\models\control_v11p_sd15_inpaint.pth]
2023-07-06 08:58:09,409 - ControlNet - INFO - Loading config: D:\StabilityAI\stable-diffusion-webui-1.3.2\extensions\sd-webui-controlnet\models\control_v11p_sd15_inpaint.yaml
2023-07-06 08:58:11,351 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138] loaded.
2023-07-06 08:58:11,396 - ControlNet - INFO - Loading preprocessor: inpaint_only
2023-07-06 08:58:11,397 - ControlNet - INFO - preprocessor resolution = -1
Error running process: D:\StabilityAI\stable-diffusion-webui-1.3.2\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "D:\StabilityAI\stable-diffusion-webui-1.3.2\modules\scripts.py", line 451, in process
    script.process(p, *script_args)
  File "D:\StabilityAI\stable-diffusion-webui-1.3.2\extensions\sd-webui-controlnet\scripts\controlnet.py", line 913, in process
    final_inpaint_mask = final_inpaint_feed[0, 3, :, :].astype(np.float32)
IndexError: index 3 is out of bounds for axis 1 with size 3

100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:01<00:00, 12.54it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 16.18it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 18.85it/s]

Additional information

No response

huchenlei · 2023-07-05T13:59:09Z

Can you link the full context as you provided in #1763 (comment)?

For example, what setting are you using in A detailer extension?

Running ControlNet alone with the setting in the screenshot does not seem to have any problem.

leporel · 2023-07-05T16:48:02Z

I don't know if my problem is similar, but i have same trouble with controlnet-inpaint, trying repeat this Video
Result: image in output is fully new generated
Console:

*** Error running process: C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\MyPrograms\stable-diffusion-webui\modules\scripts.py", line 474, in process
        script.process(p, *script_args)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 838, in process
        detected_map, is_image = preprocessor(
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 79, in decorated_func
        return cached_func(*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 67, in cached_func
        return func(*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 35, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 489, in lama_inpaint
        prd_color = model_lama(img_res)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\lama\__init__.py", line 51, in __call__
        color = color * (1 - mask)
    RuntimeError: The size of tensor a (3) must match the size of tensor b (0) at non-singleton dimension 2

huchenlei · 2023-07-05T17:25:15Z

I don't know if my problem is similar, but i have same trouble with controlnet-inpaint, trying repeat this Video Result: image in output is fully new generated Console:

*** Error running process: C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\MyPrograms\stable-diffusion-webui\modules\scripts.py", line 474, in process
        script.process(p, *script_args)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 838, in process
        detected_map, is_image = preprocessor(
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 79, in decorated_func
        return cached_func(*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 67, in cached_func
        return func(*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 35, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 489, in lama_inpaint
        prd_color = model_lama(img_res)
      File "C:\MyPrograms\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\lama\__init__.py", line 51, in __call__
        color = color * (1 - mask)
    RuntimeError: The size of tensor a (3) must match the size of tensor b (0) at non-singleton dimension 2

This seems like a different issue. The code fails in the lama preprocessor in your case. Can you file a new issue on how to reproduce the crash?

xiaojiemeidu · 2023-07-06T01:07:32Z

Reuploaded

huchenlei · 2023-07-06T01:12:14Z

@xiaojiemeidu I think the issue is because you select inpaint model but no mask is provided by A1111. I am going to address this issue more gracefully instead of an unhandled exception.

xiaojiemeidu · 2023-07-06T01:17:05Z

在之前使用的版本中并不会报错
在之前的版本中，如果按照图中设置点击生成图像会根据重绘幅度的大小重绘整张照片而不是报错。
生成窗口可看到目前只有一张照片，我我理解为即使不手动操作（不在controlnet上传图像栏上传任何照片，或者上传照片不进行任何涂抹遮罩），那么controlnet应该默认遮罩整张照片。

huchenlei · 2023-07-06T01:21:34Z

Can you provide the first known good version? I am going to perform a bisect.

xiaojiemeidu · 2023-07-06T01:45:46Z

Can you provide the first known good version? I am going to perform a bisect.

Are you saying that the same settings run on your webui without any errors?

huchenlei · 2023-07-06T01:55:04Z

I am asking which previous ControlNet version has the behaviour you described with no error, global inpaint.

xiaojiemeidu · 2023-07-06T01:58:20Z

I remember that all versions before webui1.4 will not report errors.
But I am trying the previous version now, all versions will report an error.

BobBoxArt · 2023-07-19T09:43:08Z

I have found a workaround for this.
If you go into the settings and change to only have 1 controlnet the issue is gone.
So it seems that multi controlnet is breaking this feature.

huchenlei closed this as completed Nov 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: img2img-img2img controlnet-inpaint #1766

[Bug]: img2img-img2img controlnet-inpaint #1766

xiaojiemeidu commented Jul 5, 2023 •

edited

Loading

huchenlei commented Jul 5, 2023 •

edited

Loading

leporel commented Jul 5, 2023

huchenlei commented Jul 5, 2023 •

edited

Loading

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

BobBoxArt commented Jul 19, 2023

[Bug]: img2img-img2img controlnet-inpaint #1766

[Bug]: img2img-img2img controlnet-inpaint #1766

Comments

xiaojiemeidu commented Jul 5, 2023 • edited Loading

Is there an existing issue for this?

What happened?

Steps to reproduce the problem

What should have happened?

Commit where the problem happens

What browsers do you use to access the UI ?

Command Line Arguments

List of enabled extensions

Console logs

Additional information

huchenlei commented Jul 5, 2023 • edited Loading

leporel commented Jul 5, 2023

huchenlei commented Jul 5, 2023 • edited Loading

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

huchenlei commented Jul 6, 2023

xiaojiemeidu commented Jul 6, 2023

BobBoxArt commented Jul 19, 2023

xiaojiemeidu commented Jul 5, 2023 •

edited

Loading

huchenlei commented Jul 5, 2023 •

edited

Loading

huchenlei commented Jul 5, 2023 •

edited

Loading