Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash with TLS Enabled #41

Closed
garethgreenaway opened this issue Mar 28, 2016 · 25 comments
Closed

Crash with TLS Enabled #41

garethgreenaway opened this issue Mar 28, 2016 · 25 comments

Comments

@garethgreenaway
Copy link

Excited about the the TLS support! Unfortunately when I attempted to use it crashed. I was able to connect fine but as soon as I attempted to login I saw this:

2016-03-28 02:02:22.147 DEBUG <- PRIVMSG mattermost :login [email protected] xxxx
2016/03/28 02:02:22 retrying login croscon [email protected] localhost:82
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4b408f]

goroutine 10 [running]:
github.com/42wim/mm-go-irckit.(_User).addUsersToChannels(0xc8200b6460)
/go/src/github.com/42wim/mm-go-irckit/mmuser.go:72 +0x74f
github.com/42wim/mm-go-irckit.(_User).handleMMServiceBot(0xc8200b6460, 0xc8200b6540, 0xc8201c6f54, 0x29)
/go/src/github.com/42wim/mm-go-irckit/mmuser.go:458 +0xf48
created by github.com/42wim/mm-go-irckit.(*server).handle
/go/src/github.com/42wim/mm-go-irckit/server.go:786 +0x511b

goroutine 1 [IO wait]:
net.runtime_pollWait(0x7f465accfb28, 0x72, 0xc820012160)
/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(_pollDesc).Wait(0xc8200e9090, 0x72, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(_pollDesc).WaitRead(0xc8200e9090, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(_netFD).accept(0xc8200e9030, 0x0, 0x7f465accfbe8, 0xc8200119a0)
/usr/lib/go/src/net/fd_unix.go:408 +0x27c
net.(_TCPListener).AcceptTCP(0xc8200240f8, 0xc8ffffffff, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:254 +0x4d
net.(*TCPListener).Accept(0xc8200240f8, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:264 +0x3d
main.start(0x7f465acceb80, 0xc8200240f8)
/go/src/github.com/42wim/matterircd/main.go:83 +0x4e
main.main()
/go/src/github.com/42wim/matterircd/main.go:62 +0xe4e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 5 [chan receive]:
github.com/alecthomas/log4go.(*ConsoleLogWriter).run(0xc820010520, 0x7f465acce358, 0xc820024010)
/go/src/github.com/alecthomas/log4go/termlog.go:33 +0x69
created by github.com/alecthomas/log4go.NewConsoleLogWriter
/go/src/github.com/alecthomas/log4go/termlog.go:26 +0xb5

goroutine 6 [IO wait]:
net.runtime_pollWait(0x7f465accfa68, 0x72, 0xc820012160)
/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(_pollDesc).Wait(0xc820118fb0, 0x72, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(_pollDesc).WaitRead(0xc820118fb0, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(_netFD).accept(0xc820118f50, 0x0, 0x7f465accfbe8, 0xc820129140)
/usr/lib/go/src/net/fd_unix.go:408 +0x27c
net.(_TCPListener).AcceptTCP(0xc820024110, 0x7f465af51000, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:254 +0x4d
net.(_TCPListener).Accept(0xc820024110, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:264 +0x3d
crypto/tls.(_listener).Accept(0xc8201290e0, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/crypto/tls/tls.go:46 +0x60
main.start(0x7f465accfc88, 0xc8201290e0)
/go/src/github.com/42wim/matterircd/main.go:83 +0x4e
main.main.func1()
/go/src/github.com/42wim/matterircd/main.go:49 +0x68
created by main.main
/go/src/github.com/42wim/matterircd/main.go:50 +0x8c7

goroutine 9 [IO wait]:
net.runtime_pollWait(0x7f465accf9a8, 0x72, 0xc820012160)
/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(_pollDesc).Wait(0xc820119020, 0x72, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(_pollDesc).WaitRead(0xc820119020, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(_netFD).Read(0xc820118fc0, 0xc820136400, 0x400, 0x400, 0x0, 0x7f465acca168, 0xc820012160)
/usr/lib/go/src/net/fd_unix.go:232 +0x23a
net.(_conn).Read(0xc820024118, 0xc820136400, 0x400, 0x400, 0x0, 0x0, 0x0)
/usr/lib/go/src/net/net.go:172 +0xe4
crypto/tls.(_block).readFromUntil(0xc8201bfe00, 0x7f465acd01f0, 0xc820024118, 0x5, 0x0, 0x0)
/usr/lib/go/src/crypto/tls/conn.go:455 +0xcc
crypto/tls.(_Conn).readRecord(0xc8200c02c0, 0x9fee17, 0x0, 0x0)
/usr/lib/go/src/crypto/tls/conn.go:540 +0x2d1
crypto/tls.(_Conn).Read(0xc8200c02c0, 0xc820132000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/go/src/crypto/tls/conn.go:901 +0x167
bufio.(_Reader).fill(0xc82011e300)
/usr/lib/go/src/bufio/bufio.go:97 +0x1e9
bufio.(_Reader).ReadSlice(0xc82011e300, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/bufio/bufio.go:328 +0x21a
bufio.(_Reader).ReadBytes(0xc82011e300, 0xc82012910a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/bufio/bufio.go:406 +0xa9
bufio.(_Reader).ReadString(0xc82011e300, 0x7e130a, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/bufio/bufio.go:446 +0x4d
github.com/sorcix/irc.(_Decoder).Decode(0xc8201291c0, 0x0, 0x0, 0x0)
/go/src/github.com/sorcix/irc/stream.go:80 +0x59
github.com/42wim/mm-go-irckit.(_User).Decode(0xc8200b6460, 0x9fe1b8, 0x0, 0x0)
/go/src/github.com/42wim/mm-go-irckit/user.go:137 +0xab
github.com/42wim/mm-go-irckit.(_server).handle(0xc8200f2340, 0xc8200b6460)
/go/src/github.com/42wim/mm-go-irckit/server.go:582 +0xb1
created by github.com/42wim/mm-go-irckit.(*server).Connect
/go/src/github.com/42wim/mm-go-irckit/server.go:283 +0xb5

I used an existing key/cert pay and I used the go certificate generator to get a self signed cert. Both had the same results.
@garethgreenaway
Copy link
Author

Actually seems unrelated to TLS, seeing the same thing without TLS involved. Using the Docker images, both v0.4 and latest.

@garethgreenaway
Copy link
Author

See this is a duplicate of #39. Closing.

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

No, this was because of incorrect docker images. Needed to be build with go 1.6 (for vendoring).
Seems like the automated build process on dockerhub isn't that automated. I just triggered a new build.

Can you do a docker pull and try again?

@garethgreenaway
Copy link
Author

Pulled the latest docker image and saw the same results.

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

Ok, just saw the docker build logs, it built with the old Dockerfile ofcourse because it builds the tag (v0.4) and in that commit the dockerfile wasn't correct. -sigh-

So I pinned v0.4 on dockerhub to master. If you do docker pull again of v0.4 or latest, it should work now.

@garethgreenaway
Copy link
Author

Different crash this time:
2016-03-28 16:30:37.748 INFO New connection: 127.0.0.1:52695
2016/03/28 16:30:42 retrying login croscon [email protected] localhost:82
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4b7924]

goroutine 8 [running]:
panic(0x9099e0, 0xc820012070)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_User).addUsersToChannels(0xc8200dc380)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/mmuser.go:72 +0x754
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_User).handleMMServiceBot(0xc8200dc380, 0xc8200dc460, 0xc820045514, 0x29)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/mmuser.go:458 +0xf48
created by github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(*server).handle
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:786 +0x512a

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

And this worked fine in the non-TLS setup ?
Can you run with -debug ? And paste output (watch your password, it could be in the debug log)

@garethgreenaway
Copy link
Author

This is actually without TLS enabled, with debug enabled and watching the console. I'm running Mattermost 2.1 and it wasn't clear if 0.4 should work with that version.

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

Hm, in the other ticket (#40) you told me it worked fine if you were not using ELB, but now it doesn't work anymore ? Is this correct or are you testing somewhere else now?

If you're using a stable mattermost 2.1 release you can use matterircd stable (master branch), if your using development mattermost you have to use matterircd develop branch.

@garethgreenaway
Copy link
Author

Apologies for the confusion. I was working when I was using the docker images from xyproto/matterircd without the ELB in place. Following your comment that TLS support on in there now I switched over to your docker images. Currently I'm using a custom mattermost 2.1 docker image and your 0.4 matterircd docker image, and seeing the errors I described.

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

Ok, I can reproduce it on a new setup.

By refactoring into the new library I dropped http support, only https (with valid certificates) is allowed.
I guess you're not using ssl on port 82 ?

@garethgreenaway
Copy link
Author

Correct! I can try switching back to HTTPS though, I started using port 82 and HTTP when I was having issues with the AWS ELB.

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

Ok, bug fixed and new docker images triggered.
It should work now (famous last words) on insecure connections too.

@garethgreenaway
Copy link
Author

Back in business! Latest docker hub image for 0.4 is working. Thanks!

@garethgreenaway
Copy link
Author

Spoke too soon. It works but after a few minutes crashes:
016-03-28 22:42:51.771 INFO New connection: 98.116.12.109:43620
2016-03-28 22:42:52.406 DEBUG <- CAP LS
2016-03-28 22:42:52.610 DEBUG <- NICK gareth
2016-03-28 22:42:52.610 DEBUG <- USER gareth gareth mattermost.croscon.com :Gareth J. Greenaway
2016-03-28 22:42:52.610 DEBUG -> :matterircd 001 gareth :Welcome! gareth![email protected]
2016-03-28 22:42:52.610 DEBUG -> :matterircd 002 gareth :Your host is matterircd, running version 0.4
2016-03-28 22:42:52.610 DEBUG -> :matterircd 003 gareth :This server was created Mon Mar 28 22:42:51 UTC 2016
2016-03-28 22:42:52.610 DEBUG -> :matterircd 004 gareth :matterircd 0.4 o o
2016-03-28 22:42:52.610 DEBUG -> :matterircd 251 gareth :There are 2 users and 0 services on 1 servers
2016-03-28 22:42:52.610 DEBUG -> :matterircd 375 gareth :- matterircd Message of the Day -
2016-03-28 22:42:52.610 DEBUG -> :matterircd 376 gareth :End of /MOTD command.
2016-03-28 22:42:52.779 DEBUG <- MODE gareth +i
2016-03-28 22:42:52.940 INFO New connection: 98.116.12.109:43622
2016-03-28 22:42:53.028 DEBUG <- JOIN #off-topic,#town-square
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x28 pc=0x4c8553]

goroutine 47 [running]:
panic(0x9099e0, 0xc820012070)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_server).handle(0xc8202ca8f0, 0xc820287960)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:651 +0x10b3
created by github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_server).Connect
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:283 +0xb5

@42wim
Copy link
Owner

42wim commented Mar 28, 2016

Seems like you tried to join a channel with a client without that client being logged in into mattermost.

If so, don't do that for now ;)

Always login via /msg mattermost login before doing anything else.

@garethgreenaway
Copy link
Author

First attempt I used irssi and was able to login, it joined the channels automatically. Second time I attempted to use Hexchat, when I logged in, it crashed:

2016-03-28 23:14:51.823 DEBUG <- PING LAG1459156886972
2016-03-28 23:14:51.823 DEBUG -> :matterircd PONG matterircd
2016-03-28 23:15:03.453 DEBUG <- PRIVMSG mattermost :login [email protected] xxxxxx
2016/03/28 23:15:03 retrying login croscon [email protected] localhost:82
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4b7974]

goroutine 10 [running]:
panic(0x9099e0, 0xc820012070)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_User).addUsersToChannels(0xc8200de540)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/mmuser.go:75 +0x754
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_User).handleMMServiceBot(0xc8200de540, 0xc8200de620, 0xc8202725d4, 0x28)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/mmuser.go:461 +0xf48
created by github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(*server).handle
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:786 +0x512a

@42wim
Copy link
Owner

42wim commented Mar 29, 2016

Ok, I fixed the "join a channel without logging in" bug, and added much more debug for the hexchat problem. It also shouldn't panic anymore.

The dockerhub images are updated, if you could try docker pull 42wim/matterircd:latest and try again?

@garethgreenaway
Copy link
Author

Awesome! I shall give it a try and report back.

@garethgreenaway
Copy link
Author

Looking good so far! Was able to connect & log in successfully using Hexchat and irssi.

@42wim
Copy link
Owner

42wim commented Mar 29, 2016

Great to hear, thanks for the testing and feedback.

@garethgreenaway
Copy link
Author

Happy to help!

@garethgreenaway
Copy link
Author

Just an fyi, something a coworker found when testing out matterircd using Hexchat 2.9.6, when he connected and hadn't logged in yet, he issued a /LIST and the daemon crashed with the following dump:
2016-03-29 22:43:03.991 DEBUG <- LIST
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x28 pc=0x4c59e1]

goroutine 13 [running]:
panic(0x915080, 0xc820012070)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_server).list(0xc8202e5ee0, 0xc8202df260, 0x0, 0x0, 0x0)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:490 +0x291
github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(_server).handle(0xc8202e5ee0, 0xc8202df260)
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:696 +0x1e70
created by github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit.(*server).Connect
/go/src/github.com/42wim/matterircd/vendor/github.com/42wim/mm-go-irckit/server.go:283 +0xb5

@42wim
Copy link
Owner

42wim commented Mar 30, 2016

Ok, fixed and also proactive the /TOPIC command. ;-)
Any other issues otherwise this can be closed I guess ?

@garethgreenaway
Copy link
Author

Awesome! Will update. i think this one can be considered resolved. Will close. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@garethgreenaway @42wim