X server could not be started

[paul-hansen]

xrdp version

0.9.80

Detailed xrdp version, build options

xrdp 0.9.80
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --prefix=/usr
      --sysconfdir=/etc
      --localstatedir=/var
      --sbindir=/usr/bin
      --with-systemdsystemunitdir=/usr/lib/systemd/system
      --enable-jpeg
      --enable-tjpeg
      --enable-fuse
      --enable-fdkaac
      --enable-opus
      --enable-rfxcodec
      --enable-mp3lame
      --enable-pixman
      --enable-painter
      --enable-vsock
      --enable-ipv6
      --enable-pam-config=arch
      --enable-rdpsndaudin
      --with-imlib2
      CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection
      LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
      CXXFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS

  Compiled with OpenSSL 3.1.2 1 Aug 2023

Operating system & version

EndeavorOS

Installation method

other

Which backend do you use?

xorgxrdp

What desktop environment do you use?

KDE

Environment xrdp running on

Physical machine. Nvidia GeForce RTX 3080

What's your client?

Windows 11 default rdp client (mstsc.exe)

Area(s) with issue?

No response

Steps to reproduce

I followed these instructions: https://wiki.archlinux.org/title/xrdp

Specifically I used these steps:

yay -S xrdp
yay -S xorgxrdp
systemctl enable --now xrdp.service

# This failed giving a "One or more files did not pass the validity check!" error
# Should work without it I thought?
yay -S xorgxrdp-nvidia

cp /etc/X11/xinit/xinitrc ~/.xinitrc

Next I opened ~/.xinitrc and commented out the last 5 lines starting with twm & and added these lines:

# twm &
# xclock -geometry 50x50-1+1 &
# xterm -geometry 80x50+494+51 &
# xterm -geometry 80x20+494-0 &
# exec xterm -geometry 80x66+0+0 -name login
export DESKTOP_SESSION=plasma
exec startplasma-x11

Finally I attempted to connect with a Windows RDP client.

✔️ Expected Behavior

I was expecting the windows rdp client to connect

❌ Actual Behavior

On the windows client after entering credentials it waits for ~5-10 seconds on a blue screen then shows the connection log with this log:

connecting to sesman on sesman.socket
sesman connect ok
Performing login request for paul
sending login info to session manager, please wait...
login was successful - creating session
sending create session request to session manager. Please
it...
Can't create session for user paul - X server could not be started

See the attached xrdp-sesman.log for an additional message

waitforx: Unable to find any RandR outputs

It looks like I previously got the error:

waitforx: Unable to open display :10

I'm not sure what I did to change that error message, I wasn't watching those logs so I didn't notice the change.

Anything else?

/var/log/xrdp-sesman.log
/var/log/xrdp.log
/etc/X11/xrdp/xorg.conf

[metalefty]

You might need to edit sesman.ini as hinted in comments.

[Xorg]
; Specify the path of non-suid Xorg executable. It might differ depending
; on your distribution and version. Find out the appropreate path for your
; environment. The typical path is known as follows:
;
; Fedora 26 or later    :  param=/usr/libexec/Xorg
; Debian 9 or later     :  param=/usr/lib/xorg/Xorg
; Ubuntu 16.04 or later :  param=/usr/lib/xorg/Xorg
; Arch Linux            :  param=/usr/lib/Xorg
; CentOS 7              :  param=/usr/bin/Xorg or param=Xorg
; CentOS 8              :  param=/usr/libexec/Xorg
;

[paul-hansen]

Thanks for looking at this!

I tried /usr/lib/Xorg as suggested for Arch based distros. It didn't seem to change anything, I still get the same result with the same errors in the xrdp-sesman.log:

[ERROR] waitforx: Unable to find any RandR outputs
[ERROR] An error occurred waiting for the X server

I verified /usr/lib/Xorg does exist:

Here's my full /etc/xrdp/sesman.ini

[metalefty]

Then, what will you get when executing following command in the terminal (via SSH or something)? Outputs will help to investivate what's happening. Try both /usr/lib/Xorg and `Xorg.

$ /usr/lib/Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log 

$ Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log 

[paul-hansen]

I was able to run them in my local terminal. Results are below:

first command results

❯ /usr/lib/Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log

X.Org X Server 1.21.1.8
X Protocol Version 11, Revision 0
Current Operating System: Linux paul-desktop 6.4.12-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 24 Aug 2023 00:38:14 +0000 x86_64
Kernel command line: BOOT_IMAGE=/@/boot/vmlinuz-linux root=UUID=d36a4549-6ea1-4f73-881e-21b4c9c8ec90 rw rootflags=subvol=@ nowatchdog nvme_load=YES resume=UUID=f52b941f-3151-4209-8b15-b034a9ab1b92 nvidia-drm.modeset=1 loglevel=3

Current version of pixman: 0.42.2
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: ".xorgxrdp.10.log", Time: Thu Sep  7 00:16:33 2023
(++) Using config file: "/etc/X11/xrdp/xorg.conf"
(==) Using config directory: "/etc/X11/xorg.conf.d"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
xorgxrdpSetup:
xrdpdevSetup:
rdpmousePlug:
rdpkeybPlug:
rdpIdentify:
rdpDriverFunc: op 10
rdpProbe:
rdpPreInit:
rdpScreenInit: virtualX 800 virtualY 600 rgbBits 8 depth 24
rdpScreenInit: pfbMemory bytes 1920000
rdpScreenInit: pfbMemory 0x7f6a2125f010
rdpSimdInit: assigning yuv functions
rdpSimdInit: cpuid ax 1 cx 0 return ax 0x00a60f12 bx 0x1f200800 cx 0x7ef8320b dx 0x178bfbff
rdpSimdInit: sse2 amd64 yuv functions assigned
rdpClientConInit: disconnect idle session after [0] sec
rdpClientConInit: kill disconnected [0] timeout [0] sec
rdpClientConInit: do_touchpad_scroll_hack [0]
rdpXvInit: depth 24
rdpScreenInit: out
rdpCreateScreenResources:
rdpmousePreInit: drv 0x559d4cdea6d0 info 0x559d4d08ebc0, flags 0x0
rdpmouseControl: what 0
rdpmouseDeviceInit:
rdpmouseCtrl:
rdpRegisterInputCallback: type 1 proc 0x7f6a2175f2c0
rdpmouseControl: what 1
rdpmouseDeviceOn:
rdpkeybPreInit: drv 0x559d4cdeb510 info 0x559d4d091cd0, flags 0x0
rdpkeybControl: what 0
rdpkeybDeviceInit:
rdpkeybChangeKeyboardControl:
rdpkeybChangeKeyboardControl: autoRepeat on
rdpRegisterInputCallback: type 0 proc 0x7f6a2175a920
rdpkeybControl: what 1
rdpkeybDeviceOn:
rdpDeferredRandR:
rdpResizeSession: width 1024 height 768
  calling RRScreenSizeSet
rdpRRScreenSetSize: width 1024 height 768 mmWidth 271 mmHeight 203
rdpRRGetInfo:
  screen resized to 1024x768
  RRScreenSizeSet ok 1
rdpInDeferredRepeatCallback:
rdpkeybChangeKeyboardControl:
rdpkeybChangeKeyboardControl: autoRepeat off

second command results

❯ Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log

X.Org X Server 1.21.1.8
X Protocol Version 11, Revision 0
Current Operating System: Linux paul-desktop 6.4.12-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 24 Aug 2023 00:38:14 +0000 x86_64
Kernel command line: BOOT_IMAGE=/@/boot/vmlinuz-linux root=UUID=d36a4549-6ea1-4f73-881e-21b4c9c8ec90 rw rootflags=subvol=@ nowatchdog nvme_load=YES resume=UUID=f52b941f-3151-4209-8b15-b034a9ab1b92 nvidia-drm.modeset=1 loglevel=3

Current version of pixman: 0.42.2
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: ".xorgxrdp.10.log", Time: Thu Sep  7 00:18:06 2023
(++) Using config file: "/etc/X11/xrdp/xorg.conf"
(==) Using config directory: "/etc/X11/xorg.conf.d"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
xorgxrdpSetup:
xrdpdevSetup:
rdpmousePlug:
rdpkeybPlug:
rdpIdentify:
rdpDriverFunc: op 10
rdpProbe:
rdpPreInit:
rdpScreenInit: virtualX 800 virtualY 600 rgbBits 8 depth 24
rdpScreenInit: pfbMemory bytes 1920000
rdpScreenInit: pfbMemory 0x7f18e9fcb010
rdpSimdInit: assigning yuv functions
rdpSimdInit: cpuid ax 1 cx 0 return ax 0x00a60f12 bx 0x12200800 cx 0x7ef8320b dx 0x178bfbff
rdpSimdInit: sse2 amd64 yuv functions assigned
rdpClientConInit: disconnect idle session after [0] sec
rdpClientConInit: kill disconnected [0] timeout [0] sec
rdpClientConInit: do_touchpad_scroll_hack [0]
rdpXvInit: depth 24
rdpScreenInit: out
rdpCreateScreenResources:
rdpmousePreInit: drv 0x563d184c46d0 info 0x563d18768bc0, flags 0x0
rdpmouseControl: what 0
rdpmouseDeviceInit:
rdpmouseCtrl:
rdpRegisterInputCallback: type 1 proc 0x7f18ea4cb2c0
rdpmouseControl: what 1
rdpmouseDeviceOn:
rdpkeybPreInit: drv 0x563d184c5510 info 0x563d1876bcd0, flags 0x0
rdpkeybControl: what 0
rdpkeybDeviceInit:
rdpkeybChangeKeyboardControl:
rdpkeybChangeKeyboardControl: autoRepeat on
rdpRegisterInputCallback: type 0 proc 0x7f18ea4c6920
rdpkeybControl: what 1
rdpkeybDeviceOn:
rdpDeferredRandR:
rdpResizeSession: width 1024 height 768
  calling RRScreenSizeSet
rdpRRScreenSetSize: width 1024 height 768 mmWidth 271 mmHeight 203
rdpRRGetInfo:
  screen resized to 1024x768
  RRScreenSizeSet ok 1
rdpInDeferredRepeatCallback:
rdpkeybChangeKeyboardControl:
rdpkeybChangeKeyboardControl: autoRepeat off

[metalefty]

Thanks, both look working fine. So it is an issue of waitforx recently added to the development version. 0.9.80 is a tentative version of the development snapshot.

[ERROR] waitforx: Unable to find any RandR outputs
[ERROR] An error occurred waiting for the X server

Quick solution is to use 0.9.23 instead, it is stable version.

However, I would be glad if you would be willing to help us debug the development version.

[paul-hansen]

Oh wow, I didn't realize I ended up on the development version actually. Looks like trying to install the xorgxrdp-nvidia AUR pulled in xrdp-git, I should have noticed that.

Installed stable version and it seems to be working now! (after a reboot and making sure I wasn't logged into a local session)

However, I would be glad if you would be willing to help us debug the development version.

Happy to help any way I can!

[metalefty]

@matt335672 @derekschrock thoughts on waitforx?

[matt335672]

Yes - I have.

@paul-hansen - here's a bit of background for you.

The versions of xrdp before the development version start the X server and the session at the same time. That's normally fine, although if there's a problem with the X server it's only discovered when it's to late to report it on the login screen.

The development version starts sub-processes in this order:-

1. X server
2. waitforx
3. session

waitforx not only checks the X server is available - it checks it works properly. If a problem is found this can then be reported back before the session is started.

When we put this architecture together we found a problem with xorgxrdp - see #2580 and neutrinolabs/xorgxrdp#254. The error exactly matches what you are reporting.

This is fixed in the non-NVidia xorgxrdp development version. However, because you're using NVidia, you aren't running quite the same code paths. Rather than the randr module you're using a local replacement called lrandr. I don't believe this currently creates an output at startup under all situations - see this comment.

@Nexarian - does my analysis above sound plausible to you? You know more about this area than I do.

There's currently no option to not check for RandR outputs in waitforx. The reason it was added in the first place is that some window managers need these to be present when starting up.

A possible workaround would be:-

1. Rename /usr/libexec/xrdp/waitforx to /usr/libexec/xrdp/watforx.orig
2. Create this script in /usr/libexec/xrdp/waitforx:-

   #!/bin/sh
   
   exit 0

3. Make sure the script has permissions 755.

This effectively makes waitforx a NOP. It may give you other problems down the line however.

[metalefty]

The waitforx issue was reproduced for me. In my case, the cause is I'm not using xorgxrdp with neutrinolabs/xorgxrdp#254 applied.

[2023-09-21T12:50:23.728+0000] [INFO ] Received sys login status for almalinux : logged in
[2023-09-21T12:50:23.733+0000] [INFO ] Received request from xrdp to create a session for user almalinux
[2023-09-21T12:50:23.784+0000] [INFO ] Starting X server on display 10: /usr/libexec/Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log
[2023-09-21T12:50:34.848+0000] [ERROR] waitforx: Unable to find any RandR outputs
[2023-09-21T12:50:34.885+0000] [ERROR] An error occurred waiting for the X server
[2023-09-21T12:50:34.902+0000] [INFO ] Session on display 10 has finished.

[metalefty]

@paul-hansen When using xrdp 0.9.80 development snapshot, xorgxrdp also must be latest development snapshot. Do you remember which version did you use?

[paul-hansen]

That was likely the cause. As I mentioned, I hadn't realized that the failed attempt to install xorgxrdp-nvidia pulled in a dev version of xrpd after I had installed the non-dev version so they would have been mis-matched. (See my repro steps in OP)

Sounds like this is solved so I'll go ahead and close this. Feel free to reopen if there's something outstanding though. Thanks all!

[alexleach]

HI @paul-hansen, please can you clarify exactly what packages you have installed? I'm on Arch Linux, and getting the same errors as in your original post, with almost identical hardware (RTX 3070 though) and package choices as you earlier.

$ pacman -Qs xrdp
local/xorgxrdp-nvidia 0.2.18.r55.g3a4d465-1
    Xorg drivers for xrdp, with NVIDIA GPU support.
local/xrdp-git 0.9.18.r597.g04db78f6-1
    An open source remote desktop protocol (RDP) server. Git version, devel branch.

As you mentioned, xorgxrdp-nvidia depends on xrdp-git. Are you using different packages / versions from the AUR? Did you decide to just use xorgxrdp in the end? i.e. the non-nvidia version?

Thanks

[paul-hansen]

I believe I just used the non Nvidia from what I remember. I've done a system wipe since then and I didn't end up using this project since it seems to require always being logged in to be able to connect meaning I'd have to leave my PC unlocked and restarting would have to auto login which was too insecure for me.

Ended up just installing Linux on both machines and using ssh with x11 forwarding. Not a long term solution as x11 will eventually be retired for Wayland.

[justmeee]

same Problem here, but...
install and start/enable sddm

the waitforx nop from above is also needed

then the login works for me

[caot]

It's an issue related to the version. And the version 0.9.25 works.

xrdp-0.9.25-2.el9.x86_64