[Bug]: after 1.7.0 update -> Extras tab / batch from directory/ upscale stops randomly

[Rojinski]

Checklist

• The issue exists after disabling all extensions
• The issue exists on a clean installation of webui
• The issue is caused by an extension, but I believe it is caused by a bug in the webui
• The issue exists in the current version of the webui
• The issue has not been reported before recently
• The issue has been reported before but has not been fixed yet

What happened?

upscaling images by using "Batch From Directory" stops in the process at a random moments. It doesn't show the pictures that have been done. And i get this messaage belowon the image window -> "OSError: unrecognized data stream contents when reading image file". Never had that issue.

Steps to reproduce the problem

1. go to "Extra" tab
2. go to "Batch from directory"
3. enter the location of the directory where your images are in "Input directory" (try with more than 6 pictures). If needed, enter the location of your "output directory"
4. check the "Show result images" box
5. choose 4x for the resize option
6. Choose your upscaler 1 (i use 8x_NMKD-Superscale_150000_G)
7. No upscaler 2
8. click on "generate"

What should have happened?

Normally, before the last update (1.7.0), there was no problem at all and the upscaling worked perfectly until the end of its process. I hope you will find what's wrong. :) For now, I'm using SDNext to do that job.

What browsers do you use to access the UI ?

Mozilla Firefox

Sysinfo

sysinfo-2023-12-20-12-20.json

Console logs

Already up to date.
venv "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)]
Version: v1.7.0
Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e
ReActor preheating... Device: CUDA
Launching Web UI with arguments: --xformers --medvram --no-half-vae
Additional Network extension not installed, Only hijack built-in lora
LoCon Extension hijack built-in lora successfully
[-] ADetailer initialized. version: 23.11.1, num models: 20
CivitAI Browser+: Aria2 RPC started
ControlNet preprocessor location: D:\Programmes 2\Logiciels\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-12-20 12:42:47,153 - ControlNet - INFO - ControlNet v1.1.423
2023-12-20 12:42:47,265 - ControlNet - INFO - ControlNet v1.1.423
12:42:47 - ReActor - STATUS - Running v0.6.0-a3
Loading weights [9a7477f085] from D:\Programmes 2\Logiciels\stable-diffusion-webui\models\Stable-diffusion\SDXL\batchCOREALISMXL_v30.safetensors
Creating model from config: D:\Programmes 2\Logiciels\stable-diffusion-webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 24.0s (prepare environment: 6.3s, import torch: 3.5s, import gradio: 1.4s, setup paths: 1.2s, initialize shared: 0.2s, other imports: 1.0s, setup codeformer: 0.2s, setup gfpgan: 0.1s, list SD models: 0.5s, load scripts: 3.3s, create ui: 5.9s, gradio launch: 0.3s).
Loading VAE weights specified in settings: D:\Programmes 2\Logiciels\stable-diffusion-webui\models\VAE\sdxl_vae.safetensors
Applying attention optimization: xformers... done.
Model loaded in 57.2s (load weights from disk: 1.3s, create model: 0.9s, apply weights to model: 50.8s, apply half(): 1.5s, load VAE: 0.5s, load textual inversion embeddings: 1.1s, calculate empty prompt: 1.0s).
*** Error completing request
*** Arguments: ('task(8lhsvhd8qxuonl7)', 2, None, None, 'E:\\Images\\AI\\Small\\2023-12-19', '', True, 0, 4, 512, 512, True, '8x_NMKD-Superscale_150000_G', 'None', 0, False, 1, False, 1, 0, False, 0.9, 0.15, 0.5, False, False, 384, 768, 4096, 409600, 'Maximize area', 0.1, False, ['Deepbooru'], False, ['Horizontal'], False, 0.5, 0.2, None, False, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, 1, 0, 0, 0.5, 'CUDA', False, 0, 'None', '', None) {}
    Traceback (most recent call last):
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\modules\postprocessing.py", line 132, in run_postprocessing_webui
        return run_postprocessing(*args, **kwargs)
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\modules\postprocessing.py", line 71, in run_postprocessing
        initial_pp = scripts_postprocessing.PostprocessedImage(image_data.convert("RGB"))
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 933, in convert
        self.load()
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 288, in load
        raise_oserror(err_code)
      File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 72, in raise_oserror
        raise OSError(msg)
    OSError: unrecognized data stream contents when reading image file

---
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 165, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\modules\progress.py", line 118, in progressapi
    image.save(buffered, format=opts.live_previews_image_format, **save_kwargs)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 2395, in save
    self._ensure_mutable()
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 610, in _ensure_mutable
    self._copy()
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 603, in _copy
    self.load()
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 249, in load
    s = read(self.decodermaxblock)
  File "D:\Programmes 2\Logiciels\stable-diffusion-webui\venv\lib\site-packages\PIL\PngImagePlugin.py", line 939, in load_read
    self.fp.read(4)  # CRC
AttributeError: 'NoneType' object has no attribute 'read'

Additional information

If I use SDNext, i have no problem upscaling images from A1111.

Here's my screen

[slonce70]

Me and my friends (3 people) have the same problem.... Can't improve my photo folder - never had a problem before. Reinstalled from 0 stable, the problem remains. Then 300 does and error hits, then 30 and error... Never had the problem before.

[mprz1024]

I'm impacted by this; I noticed it in both 1.6 and 1.7. The errors happen entirely at random on images that work the next time through, whether they come first or not. It seems to be a bug in PIL, perhaps Automatic1111 could just retry?

Here's another example exception trace:

*** Error completing request                                                                                                                                                 
*** Arguments: ('task(vg8m56zy4mcrowv)', 2, None, None, '/datassd/Image/SD/tochoose/class3hd/upscale', '/datassd/Image/SD/tochoose/class3hd', True, 1, 4, 512, 3840, False, 'Remacri_ESRGAN_4x__landscapes', 'None', 0, False, 1, False, 1, 0, False, 0.9, 0.15, 0.5, False, False, 384, 768, 4096, 409600, 'Maximize area', 0.1, False, ['Deepbooru'], Fal
se, ['Horizontal'], False, 0.5, 0.2) {}
    Traceback (most recent call last):                                                                                                                                       
      File "/opt/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
                   ^^^^^^^^^^^^^^^^^^^^^                                          
      File "/opt/stable-diffusion-webui/modules/call_queue.py", line 36, in f  
        res = func(*args, **kwargs)                                                                                                                                          
              ^^^^^^^^^^^^^^^^^^^^^
      File "/opt/stable-diffusion-webui/modules/postprocessing.py", line 132, in run_postprocessing_webui        
        return run_postprocessing(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                           
      File "/opt/stable-diffusion-webui/modules/postprocessing.py", line 71, in run_postprocessing
        initial_pp = scripts_postprocessing.PostprocessedImage(image_data.convert("RGB"))                 
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/stable-diffusion-webui/venv/lib/python3.11/site-packages/PIL/Image.py", line 933, in convert
        self.load()                                                                                                                                                          
      File "/opt/stable-diffusion-webui/venv/lib/python3.11/site-packages/PIL/ImageFile.py", line 288, in load
        raise_oserror(err_code)
      File "/opt/stable-diffusion-webui/venv/lib/python3.11/site-packages/PIL/ImageFile.py", line 72, in raise_oserror
        raise OSError(msg)
    OSError: unrecognized data stream contents when reading image file

[angelahahaa]

same issue

[p80ric]

Same

Here my full Trace:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 165, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\modules\progress.py", line 118, in progressapi
    image.save(buffered, format=opts.live_previews_image_format, **save_kwargs)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 2395, in save
    self._ensure_mutable()
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 610, in _ensure_mutable
    self._copy()
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 603, in _copy
    self.load()
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 249, in load
    s = read(self.decodermaxblock)
  File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\PngImagePlugin.py", line 939, in load_read
    self.fp.read(4)  # CRC
AttributeError: 'NoneType' object has no attribute 'read'

    Traceback (most recent call last):
      File "E:\StableDiffusion_ex\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "E:\StableDiffusion_ex\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "E:\StableDiffusion_ex\stable-diffusion-webui\modules\postprocessing.py", line 132, in run_postprocessing_webui
        return run_postprocessing(*args, **kwargs)
      File "E:\StableDiffusion_ex\stable-diffusion-webui\modules\postprocessing.py", line 71, in run_postprocessing
        initial_pp = scripts_postprocessing.PostprocessedImage(image_data.convert("RGB"))
      File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 933, in convert
        self.load()
      File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 288, in load
        raise_oserror(err_code)
      File "E:\StableDiffusion_ex\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageFile.py", line 72, in raise_oserror
        raise OSError(msg)
    OSError: unrecognized data stream contents when reading image file

---

[p80ric]

I think i found a workaround: Just close down the browser window after you hit generate. (Was able to generate over 150 images without interruption)

I think its a race condition introduced with the new image preview and Progress bar feature.

[Rojinski]

I think i found a workaround: Just close down the browser window after you hit generate. (Was able to generate over 150 images without interruption)

I think its a race condition introduced with the new image preview and Progress bar feature.

:) My solution, for the moment, is using SDNext to do the batch upscales... :/

[LaughterOnWater]

Issue persists. Others are also logging the issue.
https://www.reddit.com/r/StableDiffusion/comments/18l4zvw/oserror_unrecognized_data_stream_contents_when/

[Rojinski]

Issue persists. Others are also logging the issue. https://www.reddit.com/r/StableDiffusion/comments/18l4zvw/oserror_unrecognized_data_stream_contents_when/

Yes, it seems it's a real bug... I've closed my browser window to test the idea of @p80ric but it doesn't work for me... The bug is still there today. :(

[Exozze]

Same issue

[Rojinski]

Same issue

And no answer, no reaction since 3 weeks... It's just annoying to use another repo to make upscales (directory batch)...

[p80ric]

I can confirm it also happens with all addons disabled!

[slonce70]

I know the solution - but you won't see the image generation. The problem is LIVE image display - maybe someone will find how to disable just for upscale. I haven't looked. Here's what I disabled and now upscales fine

[mprz1024]

@slonce70 and others in this thread: indeed this is a bug and the workaround is to keep previews disabled when you use the extras batch tab. How do we make it happen quickly? Simply go to Settings, User interface/User interface, and add live_previews_enable to quick settings, then apply and reload the UI.

You'll gain a checkbox on top so you can one-click disable/reenable this without going to settings and finding the right one again every time. (I also use this for sd_vae, CLIP_stop_at_last_layers (CLIP skip) and show_progress_every_n_steps which are settings you may want to change very often but are buried deep within settings.)

[VL4DST3R]

Still an issue. Another annoying change is that ever since the old "preprocess images" got replaced with batch from directory, the resulting filenames are no longer kept the same (original filenames with a numerical prefix). Is there no way to get the old preprocess back in the latest A1111?

[Rojinski]

I wonder why nobody is answering something about that "bug". Even a "we don't know what is the cause of the pb" could be great :)

[light-and-ray]

Fixed

• [Bug] Fix extras big batch crashes #14699

[Rojinski]

It seems the problem has been fixed with the last update to version 1.8 !