Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack traces can be interleaved when running multithreaded #51056

Open
kpamnany opened this issue Aug 25, 2023 · 3 comments
Open

Stack traces can be interleaved when running multithreaded #51056

kpamnany opened this issue Aug 25, 2023 · 3 comments
Labels
display and printing Aesthetics and correctness of printed representations of objects. multithreading Base.Threads and related functionality

Comments

@kpamnany
Copy link
Contributor

If exceptions occur in tasks running on 2 (or more) threads and we try to output backtraces concurrently, the output can have interleaved lines, making it difficult to figure out the actual stack for either exception.

Probably the best (only?) solution is to replace the use of jl_safe_printf() wherever it is used, and instead accumulate the entire backtrace into a buffer, then use a single jl_safe_printf to output the buffer.

Any other solutions?

@jpsamaroo jpsamaroo added multithreading Base.Threads and related functionality display and printing Aesthetics and correctness of printed representations of objects. labels Aug 27, 2023
@NHDaly
Copy link
Member

NHDaly commented Aug 28, 2023

Another solution (and the patch we're going with for now in our own builds, just to get something going here) is to prepend the thread id, as in thread (4) before every stacktrace line. This way the user can disentangle them in post-processing, or just with their eyes.

I can see either one being sufficient. I would be a bit afraid of buffering, in case one of the threads hits some other fatal error and kills the whole process before we managed to output any of the buffers? 😬

@gbaraldi
Copy link
Member

I agree with Kiran, prepending the thread isn't the way to go. The distributed tests do that by printing worker before and it's unreadable. Maybe there is a way to make it a bit safer, but you are always at risk of dying in the middle of printing anyway.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Aug 30, 2023

printing the Task id (the address of the Task object) seems like a good idea to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
display and printing Aesthetics and correctness of printed representations of objects. multithreading Base.Threads and related functionality
Projects
None yet
Development

No branches or pull requests

5 participants