PASD-SDXL RuntimeError: Boolean value of Tensor with more than one value is ambiguous

[AIisCool]

python test_pasd_sdxl.py
The config attributes {'use_sd3_vae': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
The config attributes {'use_sd3_vae': False} were passed to ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
INFO:root:Loaded coca_ViT-L-14 model config.
INFO:root:Loading pretrained coca_ViT-L-14 weights (mscoco_finetuned_laion2B-s13B-b90k).
Traceback (most recent call last):
  File "C:\PASD\test_pasd_sdxl.py", line 316, in <module>
    main(args)
  File "C:\PASD\test_pasd_sdxl.py", line 226, in main
    validation_prompt = get_validation_prompt(args, validation_image, model, preprocess, category)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PASD\test_pasd_sdxl.py", line 176, in get_validation_prompt
    generated = model.generate(image)
                ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Roaming\Python\Python311\site-packages\open_clip\coca_model.py", line 233, in generate
    output = self._generate_beamsearch(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Roaming\Python\Python311\site-packages\open_clip\coca_model.py", line 442, in _generate_beamsearch
    if beam_scorer.is_done or stopping_criteria(input_ids, None):
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

I'm not sure what is wrong or how to fix it?

Was attempting to test with this image:

[zelenooki87]

@AIisCool
For me code working but results are far from expected. Osediff does better job.(and lot fasster)

[yangxy]

python test_pasd_sdxl.py
The config attributes {'use_sd3_vae': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
The config attributes {'use_sd3_vae': False} were passed to ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
INFO:root:Loaded coca_ViT-L-14 model config.
INFO:root:Loading pretrained coca_ViT-L-14 weights (mscoco_finetuned_laion2B-s13B-b90k).
Traceback (most recent call last):
  File "C:\PASD\test_pasd_sdxl.py", line 316, in <module>
    main(args)
  File "C:\PASD\test_pasd_sdxl.py", line 226, in main
    validation_prompt = get_validation_prompt(args, validation_image, model, preprocess, category)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PASD\test_pasd_sdxl.py", line 176, in get_validation_prompt
    generated = model.generate(image)
                ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Roaming\Python\Python311\site-packages\open_clip\coca_model.py", line 233, in generate
    output = self._generate_beamsearch(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Roaming\Python\Python311\site-packages\open_clip\coca_model.py", line 442, in _generate_beamsearch
    if beam_scorer.is_done or stopping_criteria(input_ids, None):
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

I'm not sure what is wrong or how to fix it?

Was attempting to test with this image:

My fault. I have updated the huggingface repo. Please install the latest open_clip_torch.

[yangxy]

@AIisCool For me code working but results are far from expected. Osediff does better job.(and lot fasster)

Thanks for your interest. I would recommend to increase the values of --upscale or --process_size for better results. You can also try to tune a proprer --guidance_scale and --conditioning_scale.

[yangxy]

P.S., I had concentrated on making the generated results of PASD-SDXL more photorealistic (in other words, nature) with less SR artifacts. Unfortunately, the results may not that clean or sharp in some cases as that is not my primary purpose.

Feedback and failure cases are welcome to be reported.

[AIisCool]

My fault. I have updated the huggingface repo. Please install the latest open_clip_torch.

Thanks that fixed it.

[AIisCool]

@yangxy Is there any way that I can get it to better recognize these as circles and not squares?

[yangxy]

@yangxy Is there any way that I can get it to better recognize these as circles and not squares?

You can try to reduce --guidance_scale, increase --conditioning_scale, and adjust --process_size . You can also try to add "circles" to the text prompt. The negative prompt (that I used during training) is also adjustable.

[AIisCool]

You can try to reduce --guidance_scale, increase --conditioning_scale, and adjust --process_size . You can also try to add "circles" to the text prompt. The negative prompt (that I used during training) is also adjustable.

Hm I have tried those, but no matter what, it will not generate circles.

[AIisCool]

@yangxy unrelated to the previous question, I wonder as someone who created this restoration for images, do you think stable diffusion will ever be able to process a video and not change the content with each frame? That it will be able to process/"understand" the data as a sequence rather than individual images? Also perhaps one day it will better understand/identify the low-resolution objects on the input image/video and better replicate them?

[yangxy]

@yangxy unrelated to the previous question, I wonder as someone who created this restoration for images, do you think stable diffusion will ever be able to process a video and not change the content with each frame? That it will be able to process/"understand" the data as a sequence rather than individual images? Also perhaps one day it will better understand/identify the low-resolution objects on the input image/video and better replicate them?

You mean VSR? I am working on it, but still far from perfect. Maybe we need a good text2video pretrain.

[AIisCool]

You mean VSR? I am working on it, but still far from perfect.

Yes! The restoration results I've seen from PASD and SeeSR far surpass any commercial product like Topaz Photo AI or Topaz Video AI. If it was configured for video, it would be truly amazing to see the results be temporally stable.

[zelenooki87]

You mean VSR? I am working on it, but still far from perfect.

Yes! The restoration results I've seen from PASD and SeeSR far surpass any commercial product like Topaz Photo AI or Topaz Video AI. If it was configured for video, it would be truly amazing to see the results be temporally stable.

My friend, try this project, it is based on Osediff but much more realistic results. I was not able to run it on Windows. At Ubuntu everyithing goes well.
https://github.com/ArcticHare105/S3Diff
Greetz