Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

xrdp-chansrv coredump upon second connection attempt #1502

Closed
Smit-tay opened this issue Feb 21, 2020 · 5 comments
Closed

xrdp-chansrv coredump upon second connection attempt #1502

Smit-tay opened this issue Feb 21, 2020 · 5 comments

Comments

@Smit-tay
Copy link

Smit-tay commented Feb 21, 2020 •
edited
Loading

Consistant core dump when initializing remote connection via rdp - a session already exists for this user.

Using remmina 1.3.10 RDP client - default RDP protocol settings.

Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[INFO ] A connection received from 127.0.0.1 port 57402
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: pam_sss(xrdp-sesman:auth): authentication success; logname= uid=0 euid=0 tty=xrdp-sesman ruser= rhost= user=MYCOMPANY\ME
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[INFO ] ++ created session (access granted): username MYCOMPANY\ME, ip XXX.XXX.XXX.XXX:42558 - socket: 12
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[INFO ] starting Xvnc session...
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[DEBUG] Closed socket 13 (AF_INET 0.0.0.0:5911)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[DEBUG] Closed socket 13 (AF_INET 0.0.0.0:6011)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[DEBUG] Closed socket 13 (AF_INET 0.0.0.0:6211)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[DEBUG] Closed socket 10 (AF_INET 127.0.0.1:3350)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[INFO ] calling auth_start_session from pid 7079
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: pam_unix(xrdp-sesman:session): session opened for user MYCOMPANY\ME by (uid=0)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] Closed socket 9 (AF_INET 127.0.0.1:3350)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] Closed socket 10 (AF_INET 127.0.0.1:3350)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Xvnc TigerVNC 1.9.0 - built Nov 8 2019 23:18:11
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Copyright (C) 1999-2018 TigerVNC Team and many others (see README.rst)
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: See http://www.tigervnc.org for information on TigerVNC.
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Underlying X server release 12003000, The X.Org Foundation
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Fri Feb 21 10:59:19 2020
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: vncext: VNC extension running!
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: vncext: Listening for VNC connections on local interface(s), port 5911
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: vncext: created VNC server for screen 0
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Connections: accepted: 127.0.0.1::37970
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[CORE ] waiting for window manager (pid 7082) to exit
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: SConnection: Client needs protocol version 3.3
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: VNCSConnST: Client pixel format depth 24 (32bpp) little-endian rgb888
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: mountpoint is not empty
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: if you are sure this is safe, use the 'nonempty' mount option
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: mountpoint is not empty
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: if you are sure this is safe, use the 'nonempty' mount option
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: mountpoint is not empty
Feb 21 10:59:19 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: fuse: if you are sure this is safe, use the 'nonempty' mount option
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[CORE ] window manager (pid 7082) did exit, cleaning up session
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[INFO ] calling auth_stop_session and auth_end from pid 7079
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: pam_unix(xrdp-sesman:session): session closed for user MYCOMPANY\ME
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] cleanup_sockets:
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Fri Feb 21 10:59:20 2020
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: Connections: closed: 127.0.0.1::37970 (Server shutdown)
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: Framebuffer updates: 1
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: Raw:
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: Solid: 5 rects, 286.308 kpixels
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: 1.09224 MiB (1:1 ratio)
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: Total: 5 rects, 286.308 kpixels
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: EncodeManager: 1.09224 MiB (1:1 ratio)
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: ComparingUpdateTracker: 0 pixels in / 0 pixels out
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: ComparingUpdateTracker: (1:-nan ratio)
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] cleanup_sockets: deleting /tmp/.xrdp/xrdp_chansrv_audio_out_socket_11
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] cleanup_sockets: deleting /tmp/.xrdp/xrdp_chansrv_audio_in_socket_11
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[7079]: (7079)(140636279781184)[DEBUG] cleanup_sockets: deleting /tmp/.xrdp/xrdpapi_11
Feb 21 10:59:20 MY_MACHINE.MYCOMPANY.com xrdp-sesman[4963]: (4963)(140636279781184)[INFO ] ++ terminated session: username MYCOMPANY\ME, display :11.0, session_pid 7079, ip XXX.XXX.XXX.XXX:42558 - socket:>
~

systemd-coredump[7277]: Process 7112 (xrdp-chansrv) of user XXXXXX dumped core.                                                                                                                                Stack trace of thread 7123:                                                                #0 0x000055c65e37353d xfs_delete_xfs_fs (xrdp-chansrv)                                                                #1 0x000055c65e3721f7 xfuse_deinit (xrdp-chansrv)                                                                #2 0x000055c65e36e7a2 segfault_signal_handler (xrdp-chansrv)                                                                #3 0x00007f15a6aaea20 __restore_rt (libc.so.6)                                                                #4 0x000055c65e37353d xfs_delete_xfs_fs (xrdp-chansrv)                                                                #5 0x000055c65e3721f7 xfuse_deinit (xrdp-chansrv)                                                                #6 0x000055c65e36e7d2 x_server_fatal_handler (xrdp-chansrv)                                                                #7 0x000055c65e384156 xcommon_fatal_handler (xrdp-chansrv)                                                                #8 0x00007f15a70c1a42 _XIOError (libX11.so.6)                                                                #9 0x00007f15a70bf245 _XEventsQueued (libX11.so.6)                                                                #10 0x00007f15a70b0cc7 XPending (libX11.so.6)                                                                #11 0x000055c65e384462 xcommon_check_wait_objs (xrdp-chansrv)                                                                #12 0x000055c65e36f8d5 channel_thread_loop (xrdp-chansrv)                                                                #13 0x00007f15a814b2de start_thread (libpthread.so.0)                                                                #14 0x00007f15a6b734b3 __clone (libc.so.6)                                                                                                                                Stack trace of thread 7112:                                                                #0 0x00007f15a6b6ab1f __select (libc.so.6)                                                                #1 0x00007f15a836f5bb g_obj_wait (libcommon.so.0)                                                                #2 0x000055c65e36df7d main (xrdp-chansrv)                                                                #3 0x00007f15a6a9a873 __libc_start_main (libc.so.6)                                                                #4 0x000055c65e36e2be _start (xrdp-chansrv)
@Smit-tay
Copy link
Author

Smit-tay commented Feb 23, 2020 •
edited
Loading

Should be fixed with pull request:
Check before free #1504

@Smit-tay
Copy link
Author

#1504

@matt335672 matt335672 mentioned this issue Feb 25, 2020
Closed
@Smit-tay Smit-tay reopened this Feb 26, 2020
@matt335672
Copy link
Member

Thanks for that.

I'm pretty sure from the stack trace above that you're running on CentOS/RHEL 8 with xrdp-0.9.12-4 installed from EPEL. Am I right?

If I'm wrong, please tell me what you are running and we can go from there.

This version of the OS uses ASLR, so the program counter in the stack trace is difficult to relate directly to the code segment. However, in your stack trace above, subtracting 0x55c65e369000 from the addresses which are not in shared libraries gives offsets which are all consistent with the 0.9.12-4 EPEL xrdp-chansrv executable:-

#0  0x000000000000a53d xfs_delete_xfs_fs (xrdp-chansrv)
#1  0x00000000000091f7 xfuse_deinit (xrdp-chansrv)
#2  0x00000000000057a2 segfault_signal_handler (xrdp-chansrv)
. . .
#4  0x000000000000a53d xfs_delete_xfs_fs (xrdp-chansrv)
#5  0x00000000000091f7 xfuse_deinit (xrdp-chansrv)
#6  0x00000000000057d2 x_server_fatal_handler (xrdp-chansrv)
#7  0x000000000001b156 xcommon_fatal_handler (xrdp-chansrv)
. . .
#11 0x000000000001b462 xcommon_check_wait_objs (xrdp-chansrv)
#12 0x00000000000068d5 channel_thread_loop (xrdp-chansrv)

The initial SEGV is taken at frame 4, and the signal handler is entered. This causes another call to xfuse_deinit() and xfs_delete_xfs_fs (). At this point a second SEGV is taken which is fatal.

The code which is being executed can be picked up from gdb with the debuginfo package for xrdp installed:-

$ gdb `which xrdp-chansrv`
. . .
(gdb) disassemble/s xfs_delete_xfs_fs
Dump of assembler code for function xfs_delete_xfs_fs:
chansrv_xfs.c:
389	{
   0x000000000000a530 <+0>:	endbr64 

390	    if (xfs != NULL && xfs->inode_table != NULL)
   0x000000000000a534 <+4>:	push   %rbp
   0x000000000000a535 <+5>:	mov    %rdi,%rbp
   0x000000000000a538 <+8>:	push   %rbx
   0x000000000000a539 <+9>:	sub    $0x8,%rsp
   0x000000000000a53d <+13>:	mov    (%rdi),%rdi
   0x000000000000a540 <+16>:	test   %rdi,%rdi
   0x000000000000a543 <+19>:	je     0xa569 <xfs_delete_xfs_fs+57>
   0x000000000000a545 <+21>:	mov    0x10(%rbp),%eax
   0x000000000000a548 <+24>:	test   %eax,%eax
   0x000000000000a54a <+26>:	je     0xa569 <xfs_delete_xfs_fs+57>
   0x000000000000a54c <+28>:	xor    %ebx,%ebx
   0x000000000000a54e <+30>:	xchg   %ax,%ax

391	    {
392	        size_t i;
393	        for (i = 0 ; i < xfs->inode_count; ++i)
394	        {
395	            free(xfs->inode_table[i]);
   0x000000000000a550 <+32>:	mov    (%rdi,%rbx,8),%rdi
   0x000000000000a554 <+36>:	add    $0x1,%rbx
   0x000000000000a558 <+40>:	callq  0x4480 <free@plt>
   0x000000000000a55d <+45>:	mov    0x10(%rbp),%eax
   0x000000000000a560 <+48>:	mov    0x0(%rbp),%rdi
   0x000000000000a564 <+52>:	cmp    %rbx,%rax
   0x000000000000a567 <+55>:	ja     0xa550 <xfs_delete_xfs_fs+32>

396	        }
397	    }
398	    free(xfs->inode_table);
   0x000000000000a569 <+57>:	callq  0x4480 <free@plt>

399	    free(xfs->free_list);
   0x000000000000a56e <+62>:	mov    0x8(%rbp),%rdi
   0x000000000000a572 <+66>:	callq  0x4480 <free@plt>

400	    free(xfs);
   0x000000000000a577 <+71>:	add    $0x8,%rsp
   0x000000000000a57b <+75>:	mov    %rbp,%rdi
   0x000000000000a57e <+78>:	pop    %rbx
   0x000000000000a57f <+79>:	pop    %rbp
   0x000000000000a580 <+80>:	jmpq   0x4480 <free@plt>
End of assembler dump.

On entry to xfs_delete_xfs_fs (), %rdi contains the first function parameter (i.e. xfs). This is dereferenced at 0xa53d, causing the segfault.

The source lines are a little confusing here, as a lot of optimisation has been applied. I'm pretty sure however, that the dereference at 0xa53d is simply to get the address of xfs->inode_table in to %rdi in preparation for the call to free(xfs->inode_table) at source line 398. inode_table is the first member of the xfs structure, and so dereferencing this pointer without an offset yields the address of xfs->inode_table.

The above analysis is simply saying that this is a duplicate of the problem fixed by PR #1487

Until this PR makes an official build, remedies include building from source, or rolling back to V0.9.11 for now.

Using EPEL makes this harder, as EPEL does not keep old packages as it moves forward. The older packages can however be obtained from the EPEL build system koji at this link:-

https://koji.fedoraproject.org/koji/buildinfo?buildID=1388259

I hope this is useful. Feel free to come back to me if you can pick out any problems in the above analysis.

@Smit-tay
Copy link
Author

Smit-tay commented Feb 27, 2020 •
edited
Loading 

$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"

[js ~]$ uname -a
Linux MY_MACHINE 4.18.0-147.5.1.el8_1.x86_64 #1 SMP Wed Feb 5 02:00:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[js~]$ sudo dnf list installed 'xrdp*'
Installed Packages
xrdp.x86_64                                                                        1:0.9.12-4.el8                                                                @epel
xrdp-selinux.x86_64                                                                1:0.9.12-4.el8                                                                @epel
[js ~]$

@Smit-tay
Copy link
Author

Thanks for the very excellent description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@metalefty @Smit-tay @matt335672