Error: Failed to write flamegraph: I/O error: No stack counts found

[zmitchell]

I'm running a program that has a concurrent.futures.ProcessPoolExecutor and I'm trying to diagnose what I think is a deadlock. I ran my program and then used ps -x to find the processes that belong to this process pool. I let it run and then hit Ctrl-C once I felt like it had run long enough (a few seconds), and it generated the same error regardless of whether I pointed it at the main process or one of the subprocesses.

❯ sudo py-spy record -o profile.svg --pid 52864
Password:
py-spy> Sampling process 100 times a second. Press Control-C to exit.

^C
py-spy> Stopped sampling because Control-C pressed
[2020-11-22T20:27:40.116852000Z ERROR inferno::flamegraph] No stack counts found
Error: Failed to write flamegraph: I/O error: No stack counts found

If I just write the raw profile data there's no error:

❯ sudo py-spy record -o procs_profile -f raw --pid 52864
Password:
py-spy> Sampling process 100 times a second. Press Control-C to exit.

^C
py-spy> Stopped sampling because Control-C pressed
py-spy> Wrote raw flamegraph data to 'procs_profile'. Samples: 1999 Errors: 0
py-spy> You can use the flamegraph.pl script from https://github.com/brendangregg/flamegraph to generate a SVG

I went and downloaded Brendan Gregg's flamegraph program to see if there was an issue with inferno or with the profile data, and it also finds no stack counts:

❯ ./flamegraph.pl procs_profile >procs_profile.svg
ERROR: No stack counts found

System:

• macOS 10.15.6
• Python 3.8.5 installed via pyenv
• py-spy 0.3.3 installed via pip

Any idea what's going on here? Is there something weird with my Python program, or is something preventing py-spy from reading the program's stack?

[benfred]

That error usually means all the stack traces are filtered out - py-spy ignores stack traces that it thinks are idle , and if your program is deadlocked all the threads are probably idle.

Can you try with the 'py-spy dump' command instead ? That's probably the most useful way of diagnosing deadlocks. Alternatively you can pass the --idle flag to include idle traces with the record command

[zmitchell]

Sure enough, py-spy dump shows a bunch of idle threads. Thanks! Would a different error message make sense in this case?