Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compiled program SIGSEGV after thread panic!()s #23850

Closed
mikedilger opened this issue Mar 30, 2015 · 5 comments
Closed

compiled program SIGSEGV after thread panic!()s #23850

mikedilger opened this issue Mar 30, 2015 · 5 comments
Labels
I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.

Comments

@mikedilger
Copy link
Contributor

A panic!() in one thread is killing entire program. Program is too large to post (website, hyper w/ 5 threads) and condition may be hard for others to replicate, but I will try to create something to demonstrate. Tail end of output from gdb:

Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7fffdf5fc700 (LWP 22406)]
0x00007ffff7b1b337 in send () from /lib64/libc.so.6
(gdb) 
(gdb) c
Continuing.
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os(32) }', /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/result.rs:775

Program received signal SIGPIPE, Broken pipe.
0x00007ffff7b1b337 in send () from /lib64/libc.so.6
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x0000555555a96dc8 in je_arena_dalloc_bin_locked ()
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x0000555555a96dc8 in je_arena_dalloc_bin_locked ()
(gdb) c
Continuing.
[Thread 0x7fffef1fc700 (LWP 22651) exited]
[Thread 0x7fffeeffb700 (LWP 22652) exited]
[Thread 0x7fffef3fd700 (LWP 22650) exited]
[Thread 0x7fffef5fe700 (LWP 22649) exited]
[Thread 0x7ffff43ff700 (LWP 22647) exited]
[Thread 0x7ffff4bff700 (LWP 22646) exited]
[Thread 0x7ffff5bf1700 (LWP 22640) exited]
[Thread 0x7fffeffff700 (LWP 22639) exited]
[Thread 0x7ffff5df2700 (LWP 22638) exited]
[Thread 0x7ffff7fcf840 (LWP 22632) exited]
[Thread 0x7fffef7ff700 (LWP 22648) exited]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb) q

What caused the panic was calling hyper::server::response::Response .end().unwrap() after SIGPIPE, so the response can't be sent. But the panic!() is expected, not a bug. The bug is the SEGV afterwards.

mike@maze ~/local/joist $ rustc --version
rustc 1.0.0-nightly (27901849e 2015-03-25) (built 2015-03-26)
@sfackler sfackler added the I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. label Mar 30, 2015
@nagisa
Copy link
Member

nagisa commented Mar 30, 2015

We install handler for SIGPIPE, so you should do handle SIGPIPE nostop noprint in your gdb. Otherwise it sounds like something messes up somewhere in unsafe code. Backtrace for SIGSEGV would be nice.

@mikedilger
Copy link
Contributor Author

Thanks for the handle SIGPIPE nostop noprint advice.

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os(32) }', /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/result.rs:775
stack backtrace:
   1:     0x5555559058f9 - sys::backtrace::write::h36825a208b9947f11fD
   2:     0x55555590d1f2 - panicking::on_panic::h138c9fe8d2c9cb42KvJ
   3:     0x5555558e9c53 - rt::unwind::begin_unwind_inner::h8e4b45288b41856f1bJ
   4:     0x5555558ea011 - rt::unwind::begin_unwind_fmt::h5c0935071a473332CaJ
   5:     0x55555590cc37 - rust_begin_unwind
   6:     0x55555593a3a4 - panicking::panic_fmt::h730a3fc241d0fea6N0A
   7:     0x55555567ac3a - result::Result<T, E>::unwrap::h16844723464354635204
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/macros.rs:27
   8:     0x55555566e125 - resp::respond::h17579872f79925afOAa
                        at /home/mike/.cargo/git/checkouts/joist_rr-e2971a27a9232c5b/master/src/resp.rs:49
   9:     0x55555567b42d - resp::respond_basic::h63c67eef97e61e04YFa
                        at /home/mike/.cargo/git/checkouts/joist_rr-e2971a27a9232c5b/master/src/resp.rs:98
  10:     0x5555555d3851 - pages::home::hb71384c0c9d556c8eqb
                        at src/pages/mod.rs:77
  11:     0x5555555cf1f6 - dispatch::dispatch::h933c39cf4a86b785t1a
                        at src/dispatch.rs:51
  12:     0x5555555b9c3f - handler::JoistHandler::handle_inner::h463e17a7bbf2f736nVa
                        at src/handler.rs:175
  13:     0x5555555b4d2b - handler::JoistHandler.Handler::handle::hc46b7a6b93351cedcQa
                        at src/handler.rs:62
  14:     0x55555562542e - server::handle_connection::h2378902348759844952
                        at /home/mike/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.3.5/src/server/mod.rs:171
  15:     0x55555562350b - server::Server<'a, H, L>::with_listener::closure.11787
                        at /home/mike/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.3.5/src/server/mod.rs:105
  16:     0x55555563cf2e - server::listener::spawn_with::closure.12450
                        at /home/mike/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.3.5/src/server/listener.rs:49
  17:     0x55555563cd11 - thunk::Thunk<'a, (), R>::new::closure.12445
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thunk.rs:27
  18:     0x55555563cc43 - thunk::F.Invoke<A, R>::invoke::h18175814957297039224
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thunk.rs:54
  19:     0x55555561cd4c - thunk::Thunk<'a, A, R>::invoke::h8298915138473532031
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thunk.rs:41
  20:     0x55555561c418 - thread::Builder::spawn_inner::closure.11593
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thread/mod.rs:354
  21:     0x55555561c3ae - rt::unwind::try::try_fn::__rust_abi::h12818367128688919522
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/rt/unwind.rs:139
  22:     0x55555561c349 - rt::unwind::try::try_fn::h12818367128688919522
  23:     0x555555911498 - rust_try_inner
  24:     0x555555911485 - rust_try
  25:     0x55555561b803 - rt::unwind::try::h11208916803606304307
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/rt/unwind.rs:125
  26:     0x55555561b456 - thread::Builder::spawn_inner::closure.11488
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thread/mod.rs:354
  27:     0x55555561d269 - thunk::Thunk<'a, (), R>::new::closure.11609
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thunk.rs:27
  28:     0x55555561d1ad - thunk::F.Invoke<A, R>::invoke::h12196283279245811297
                        at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/thunk.rs:54
  29:     0x55555590bfd8 - sys::thread::create::thread_start::hdf801f731196670aM5H
  30:     0x7ffff6c8c223 - start_thread
  31:     0x7ffff7b2b5ec - __clone
  32:                0x0 - <unknown>
[New Thread 0x7fffdfbff700 (LWP 13199)]
[New Thread 0x7fffdc2d2700 (LWP 13204)]
[New Thread 0x7fffdf5fc700 (LWP 13203)]
[New Thread 0x7fffdf7fd700 (LWP 13202)]
[New Thread 0x7fffdf9fe700 (LWP 13201)]
[New Thread 0x7fffd79fe700 (LWP 13200)]
[New Thread 0x7ffff43ff700 (LWP 13198)]
[New Thread 0x7ffff57f0700 (LWP 13197)]
[New Thread 0x7ffff59f1700 (LWP 13191)]
[New Thread 0x7ffff5bf2700 (LWP 13190)]
[New Thread 0x7ffff5df3700 (LWP 13189)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffdfbff700 (LWP 13199)]
0x0000555555930f58 in je_arena_dalloc_bin_locked ()
(gdb) bt
#0  0x0000555555930f58 in je_arena_dalloc_bin_locked ()
#1  0x00005555559285d5 in tcache_destroy ()
#2  0x0000555555928ed2 in je_tcache_cleanup ()
#3  0x000055555591d50d in je_tsd_cleanup ()
#4  0x00007ffff6c8c072 in __nptl_deallocate_tsd () from /lib64/libpthread.so.0
#5  0x00007ffff6c8c237 in start_thread () from /lib64/libpthread.so.0
#6  0x00007ffff7b2b5ed in clone () from /lib64/libc.so.6
(gdb)

@mikedilger
Copy link
Contributor Author 

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffdfbff700 (LWP 14047)]
0x0000555555930f58 in je_arena_dalloc_bin_locked ()
(gdb) n
Single stepping until exit from function je_arena_dalloc_bin_locked,
which has no line number information.
0x000055555590bbe0 in sys::stack_overflow::imp::signal_handler::h9a97eac2e1f2a604brH ()

So it's a stack overflow, which makes sense because it was triggered by an infinite loop issue hyperium/hyper#406.

edit: I stand corrected.

@nagisa
Copy link
Member

nagisa commented Mar 30, 2015

It cannot be a stack overflow if a message that the stack overflowed isn’t printed. The sys::stack_overflow::imp::signal_handler is called on any SIGSEGV and will then check whether the fault is inside the guard page. If fault is not inside the guard page, signal handler will unregister itself and raise a SIGSEGV again, which is why there is double SIGSEGV in your first snippet.

@steveklabnik
Copy link
Member

Given that this issue was fixed in hyper, and we don't have a reproduction to see if there was a broader issue, I'm going to give this a close. @mikedilger if you're still seeing something like this, please let me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@steveklabnik @nagisa @sfackler @mikedilger