Image generation hangs on windows 10

[ammaar8]

Hey, I'm trying to convert figures into static images but it refuses to work on windows 10. I am using this script for testing -

import plotly.graph_objects as go
import numpy as np
import plotly.io as pio
np.random.seed(1)

N = 100
x = np.random.rand(N)
y = np.random.rand(N)
colors = np.random.rand(N)
sz = np.random.rand(N) * 30

fig = go.Figure()
fig.add_trace(go.Scatter(
    x=x,
    y=y,
    mode="markers",
    marker=go.scatter.Marker(
        size=sz,
        color=colors,
        opacity=0.6,
        colorscale="Viridis"
    )
))

fig.write_image('test.png')
fig.show()

It just hangs forever. I tried with both kaleido and orca, same thing. I tried to find out where it was hanging using pycharm and it seems to be this line -

Kaleido/repos/kaleido/py/kaleido/scopes/base.py

Line 192 in 6a46eca

startup_response_string = self._proc.stdout.readline().decode('utf-8')

If you require anymore info let me know.

[ghost]

Having the same issue 😢.

[ammaar8]

Having the same issue 😢.

If it's any help, I am now using wsl2 with ubuntu 20 for this. If it doesn't work on localhost, you can host it on 0.0.0.0 and view it on the wsl's ip. Hoping this gets fixed soon since running it on wsl is a lot slower but have no other option.

[jonmmease]

If possible, it would be helpful to know if the instructions in this comment produce any additional logging info: #36 (comment)

[ammaar8]

If possible, it would be helpful to know if the instructions in this comment produce any additional logging info: #36 (comment)

I tried doing it in cmd and jupyter notebook but it won't interrupt using ctrl+c/interrupt kernel. Any other ideas?

[jonmmease]

Ok, thanks for giving that a try. The next idea is to try to run the kaleido executable CMD, without python.

From python, run this to print out the (long) kaleido launch command

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

Then run this from the CMD prompt. If successful, it would print out

{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}

along with some benign warnings. But if it hangs here, it would be great to record what is logged.

[ammaar8]

[0922/185509.685:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale
[0922/185509.743:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}
[0922/185510.510:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

Sorry about the late reply, this is what I get.

[leeTimothy]

+1 for similar issue with similar output as above

[jonmmease]

Ok, thanks @ammaar8 and @leeTimothy. That's helpful.

It's possible that the key error is

[0922/185510.510:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

Although the {"code: 0,...} line indicates that kaleido did launch successfully.

After running that first command, could you try pasting the following content into the console and pressing Enter:

{"data":{"layout": {}}, "format":"svg"}

This is an actual image export request, and if successful would print out an SVG string. Again, it would be helpful to know if this prints out any warnings or errors, or if it hangs.

Thanks!

[leeTimothy]

this might be where the behaviour im experiencing might deviate from @ammaar8

although running the output of

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

provides a very similar trace like ammaar

[0924/131936.214:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale 
[0924/131936.234:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}
[0924/131936.291:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

the command itself never runs to completion and just hangs there.

if the behaviour deviates - please let me know if you want me to raise this separately.

cheers

[jonmmease]

@leeTimothy, the first command is intended to stay running and wait for image export requests. Did you try pasting the JSON snippet I described above and hitting enter? It would be good to know if this hangs, or if any additional info is logged.
Thanks!

[leeTimothy]

Ah apologies!

Please see below.

{"data":{"layout": {}}, "format":"svg"}
{"code": 1, "message": "Failed to serialize document: Uncaught", "result": null, "version": "0.2.1"}

[jonmmease]

Ok, thanks @leeTimothy.

Here are a couple of experiments to try. The long kaleido command has a bunch of chromium flags. e.g. one of them is --disable-gpu.

1. Try deleting the --disable-gpu flag, and performing the test process again.
2. Try adding the --disable-webgl flag.

If you, or anyone else, is interested in experimenting more with chromium options. Here's a giant list of the available chromium flags: https://peter.sh/experiments/chromium-command-line-switches/.

[leeTimothy]

Hey Jon,

Apols for the late response - believe I may have narrowed it down slightly more.
Issue appears to be specific to Windows machines as haven't seen any other reports on the issuse for anything else.

I've got access to 3 different W10 machines, 2 Running OS Build 19043+ both of which Kaleido doesn't currently work.
The W10 Machine running on OS Build 19041 is still able to properly export static images.

@ammaar8 @guilhermebarrett can you confirm what OS build of W10 you're currently running?

[ammaar8]

Hey @leeTimothy,

Sorry for the late reply, I'm on 19043.1237 like you are!

[jonmmease]

That version info helps a lot, thanks for investigating @leeTimothy and @ammaar8

[ammaar8]

@jonmmease I wasn't able to paste the string in the console.

[RyanChapman2x]

Just to add: I have three Windows 10 machines. Two work, these are builds 18363 and 19041. One machine freezes the exact way as described by everyone in this issue, build 19042.

[tomdod]

My machine with Windows build 19043.1237 does not work, but it works fine on a machine running 19042.1266.

[amarhod]

My machine with Windows build 19043.1237 also freezes when writing figures. Running the CMD script yields:

[1014/172325.599:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale
[1014/172325.636:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir. 
{"code": 1, "message": "No Scope Specified", "result": null, "version": "0.2.1"}

[stroitzsch]

I'm seemingly having the same issue on Windows 10 Build 19043.1320.

Has anyone figured out a workaround for this?

[lordee]

I changed back to the other engine (orca?)

[amarhod]

I'm seemingly having the same issue on Windows 10 Build 19043.1320.

Has anyone figured out a workaround for this?

It works with WSL for me

[manujosephv]

I have the exact same issue. downgrading kaleido to 0.1.0 solved it for me.

[michaeloliverx]

I am also having this issue on Windows 10 Build: 19043.1348.

write_image will hang forever and cannot be killed with Ctrl+C.

[StSchulze]

Same issue here, with Windows 11 Version 10.0.22000

I was able to get it to work by downgrading kaleido to 0.1.0post1 (0.1.0 didn't work for me).

[mshakib]

I hope you find this helpful
I was able to resolve the issue by downgrade Kaleido to 0.1.0 and Configuring the Kaleido library to see the path for the Kaleido executable.
here is the change that I made to the Kaleido library:

file: Kaleido\scope\base.py 
Line:70

  @classmethod
  def executable_path(cls):
      vendored_executable_path = os.path.join(
          os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
          'executable',
          'kaleido.cmd'
  
      )
    

[jglascock]

I had the same result at @StSchulze ...downgrading to kaleido to 0.1.0post1 worked without any issues.

[stanos4]

any other work-arounds besides downgrading to kaleido 0.1.0post1? I'm on w11 machine, 22000.1455, have the same issue. Problem with downgrade is that you're unable to export images of large plots - i'm particularly speaking about ~30 million points heatmap, currently with many NaN points.

#19

[AmedBrook]

same issue , downgrading to kaleido=0.1.0 works for me on windows 11

[TheNoneMan]

Same issue, Windows 10 19044.2846
While investigating and following this thread discovered that CMD immediately closes for my particular user, meanwhile I was working from powershell just fine.
Managed to resolve this by deleting registry entry

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Command Processor

Afterward, CMD started launching fine as well as Kaleido subprocess. I assume it is because Kaleido is being launched via Subprocess library with shell=True and its readiness is assumed based on console output, specifically json line

{"code: 0,...}

And since CMD wasn't launching normally, there wasn't any output, any json to assume Kaleido readiness.
Edit: By the end I stuck to using plotly=5.14.1 and kaledio=0.2.1

[hhaensel]

Retried to reproduce the error today but the issue seems to have disappeared on my updated Windows 11 and Windows 10 machines.

[amks1]

I got the same issue in Windows 11.

[MaartenBW]

I still have this issue on Windows 11 with kaleido 0.2.1.

0.2.1.post1 is for some reason niet available for my platform (windows 11, 64-bit processor, python 3.11)

[szmcdull]

Sad for such a long time no one is fixing this

[neuropathbasel]

After about 2 years of encountering this issue (plotly stalls or gets an error when writing plots to PNG on some computers I use, but not all) I found a solution. This does not seem to be linked to particular kaleido/plotly version combinations but essentially boils down to headless (or sort of headless) systems.

kaleido (based on chromium, as I understood) can be launched with command line arguments that make it ignore GPU support, physical or xsever screens etc. However, these command line options need to be passed to kaleido from plotly.
Here is how (e.g., WLS, Windows 10, Ubuntu 20.04, python 3.7, kaleido==0.2.1, plotly==5.3.1):

I got the idea here: https://discuss.streamlit.io/t/unable-to-run-plotly-kaleido-on-streamlit-cloud-to-generate-image-exports-from-graphs/23425/6

You can test this in Jupyer notebook:

import kaleido
import plotly.express as px
import plotly.io as pio

# set explicit headless parameters for chromium (not sure if all are needed)
pio.kaleido.scope.chromium_args = (
    "--headless",
#    "--no-sandbox",
    "--single-process",
    "--disable-gpu"
)  # tuple with chromium args

fig = px.scatter(x=[0, 1, 2, 3, 4], y=[0, 1, 4, 9, 16])
fig.show()
fig.write_html('first_figure.html', auto_open=False)
fig.write_image('first_figure.png') # fails with kaleido 0.1.0, 0.2.1

# https://github.com/plotly/Kaleido/issues/134

Here's what happens behind the scenes:

1. plotly calls kaleido
2. kaleido binary starts, headless paremeters not set
3. kaleido crashes
4. plotly waits indefinitely for kaleido to respond, which will never happen (like here write_image hangs #134)

If you launch the kaleido binary (e.g. from ... /lib/python/site-packages.... /kaleido on a headless system (like WSL, which is formally headless) , you will see a crash:

3f)
  r8: 0000000000000000  r9: 00007fffec0c87c0 r10: 0000000000000008 r11: 0000000000000008
 r12: 00007fffec0c9260 r13: 00007fffec0c9ad0 r14: 00007fffec0c9ae0 r15: 00007fffec0c9ae8
  di: 0000000000000002  si: 00007fffec0c87c0  bp: 00007fffec0c8a10  bx: 00007f0f7eee0e00
  dx: 0000000000000000  ax: 0000000000000000  cx: 0000000000000008  sp: 00007fffec0c87c0
  ip: 00007f0f7f36300b efl: 0000000000000246 cgf: 00000053002b0033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

If you launch kaleido on the same headless system with above-mentioned command line arguments, it runs perfectly fine:

 ./kaleido --headless --no-sandbox --single-process --disable-gpu
[0515/220312.534508:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
[0515/220312.616704:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
[0515/220312.652037:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
{"code": 1, "message": "No Scope Specified", "result": null, "version": "0.2.1"}

I'm not sure that there is a simple explanation why some version combinations of plotly and kaleido fixed the issue. I tried all of these suggestions (without success) before I encountered above-mentioned fix. And there is a logical explanation to it.

[neuropathbasel]

I outcommented the --no-sandbox option. It can lead up to orphaned kaleido processes as I just found out. These keep running after completing their task. And the option is not required to make it work.

[ant1j]

@neuropathbasel Do you confirm it works for WSL but not Windows 11 natively?
Just tried on W11, Python 3.11.7, kaleido==0.2.1, but I got an error message (better than hanging forever though)

[neuropathbasel]

I have no Windows machine where I could try. Its an institutional Windows system with admin rights only within WSL. I have never used Python in Windows before.

…

-- Dr. med. Jürgen Hench Oberarzt, Neuropathologie Universitätsspital Basel | Institut für Medizinische Genetik und Pathologie Schönbeinstrasse 40 | CH-4031 Basel Tel +41 61 328 6891 | Fax +41 61 265 3194 E-Mail: ***@***.*** http://pathologie.unispital-basel.ch ü Please consider our environment before printing out this e-mail. Bitte denken Sie an unsere Umwelt, bevor Sie dieses E-Mail ausdrucken!

________________________________ Von: Antoine Jouanjean ***@***.***> Gesendet: Dienstag, 21. Mai 2024 11:17:58 An: plotly/Kaleido Cc: Hench Juergen, Dr.; Mention Betreff: Re: [plotly/Kaleido] Image generation hangs on windows 10 (#110) @neuropathbasel Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3. 11. 7, kaleido==0. 2. 1, but I got an error message (better than hanging forever though) — Reply to this email directly, view it on ZjQcmQRYFpfptBannerStart Diese Nachricht stammt von einem externen Absender Diese Nachricht kam von ausserhalb Ihrer Organisation. ZjQcmQRYFpfptBannerEnd @neuropathbasel<https://urldefense.com/v3/__https://github.com/neuropathbasel__;!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXxxOFcAI$> Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3.11.7, kaleido==0.2.1, but I got an error message (better than hanging forever though) — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/plotly/Kaleido/issues/110*issuecomment-2122162630__;Iw!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXULKsNho$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AOIQ23LB6MPNM7GTXEHSFOLZDMGMNAVCNFSM5DMFOE62U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJSGIYTMMRWGMYA__;!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXgjyvtdU$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sgbaird]

@neuropathbasel Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3.11.7, kaleido==0.2.1, but I got an error message (better than hanging forever though)

Exception has occurred: ValueError
Failed to start Kaleido subprocess. Error stream:

[0529/145805.700:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale 
[0529/145805.724:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
[0529/145805.730:ERROR:dxva_video_decode_accelerator_win.cc(1405)] DXVAVDA fatal error: Could not load mf.dll: The specified module could not be found. (0x7E)
  File "C:\Users\sterg\Documents\GitHub\ACC-HelloWorld\6-connecting-the-pieces-sgbaird\orchestrator.py", line 337, in <module>
    fig.write_image(image_name)
ValueError: Failed to start Kaleido subprocess. Error stream:

[0529/145805.700:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale 
[0529/145805.724:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
[0529/145805.730:ERROR:dxva_video_decode_accelerator_win.cc(1405)] DXVAVDA fatal error: Could not load mf.dll: The specified module could not be found. (0x7E)

[gvwilson]

Thanks for your interest in Kaleido. We are currently working on an overhaul that might address your issue - we hope to have news in a few weeks and will post an update then. Thanks - @gvwilson

[Chuck321123]

Any updates @gvwilson ?