using xrdp with NVidia drivers

[jsorg71]

Latest xrdp can use glamor to accelerate X drawing and make use of hardware 3D rendering but it only works well with Intel or AMD hardware.
Nouveau works with glamor but does not compare to NVidia drivers and does not allow access to CUDA, nvenc, etc.

[Costor]

Hi,
is there any chance to make the glamor option / nvidia_hack available for arm64 with nvidia tegra GPU (i.e. jetson nano etc.)?
I have tested the nvidia_hack on this device with promising results (see below). The nvidia jetson series has become quite popular and maybe a good reason to extend the nvidia support to arm64.
Or provide SIMD-support for arm64 in glamor compile as arm64 supports SIMD instructions.

First I can report that standard xrdp runs great on jetson nano (arm64 / aarch64 cpu, lightdm with LXDE resp LXQt).
I run the jetson nano headless. Obviously I want to make use of the nvidia tegra GPU on the jetson nano using xrdp.
So I found myself precisely in the situation of #1690 with the difference: arm64 instead of intel.

I first build a new xorgxrdp with --enable-glamor option with no problems and tested it. However it didn't make use of the GPU, so there was no notable difference to the standard build.

I checked with #1029 and then build the nvidia_hack with -enable-glamour etc. options, also with no problems (installed libgdm-dev as additional prerequisite). I removed the PCI-bus attribute from /etc/X11/xrdp/xorg_nvidia.conf because that is also not present in /etc/X11/xorg.conf.

The test had a quite promising result as you can see from the xorgxrdp.log file below: nvidia driver was found and initialised.

However xordxrdp crashed but only when loading the rdpmouse driver, so its got farther than one could expect!

Any settings I could add here, either in /etc/X11/xrdp/xorg_nvidia.conf or elsewhere?
Any chance nvidia_hack could be developed further?
And making arm64 SIMD support available in xorgxrdp (that would probably improve the sw-rendering in glamour-option?)

Thanks for the good work!

xorgxrdp.log with nvidia_hack:

xorgxrdp.10.nvidia_hack.log

[Nexarian]

I've run into some interesting integration issues between this and #1422 -- It works with avc420, but there are graphical glitches. It looks like the color channel and the B&W channel are somehow not always aligned, maybe an even/odd thing, or a divisible by 4 thing?

Anyway, to integrate this fully in the future, we should also think about using nvenc encoder and nvfbc where possible as well! Though x264 does do a pretty darn good job!

[Costor]

Until we make further progress on using nvidia opengl support through xrdp, I can confirm that virtualgl (www.virtualgl.org, v 2.6.5) works with xrdp/xorgxrdp (0.9.15) on arm8 (nvidia jetson with aarch64 architecture, ubuntu 20.04, nvidia tegra gpu). Tested with the streaming and broadasting software OBS 26.1. All self build.
In more detail: running "vglrun obs" in an xrdp/xorgxrdp remote session terminal makes OBS report "OpenGL version 3.3.0 NVIDIA 32.4.4 on a nvidia Tegra" and utilise the GPU resulting in a massive reduction of CPU usage and a massive increase in fps compared to the situation without virtualGL (where the MESA openGL sw implementation would be used).
(An off-topic note on OBS here: running OBS from the linux console screen still shows higher performance and should be preferred as the OBS preview window and internal rendering suffer from remoting overhead, by design of OBS)

[svargh]

Using here xrdp 0.9.12, Ubuntu 20.04, NVidia RTX 5000, NVidia RTX 3090.
xrdp works fine, with Intel and over virtualGl -
But VirtualGL is not a solution for us, as it needs an active VT and I feel uncomfortable sharing a single X11 3D server among many users.
We use another "temporary" solution to make WebGL work until nvidia works directly.
Instead, we use VMWare 16.0 for an headless Ubuntu 20.04:
Its SVA3D driver works nicely in xrdp.
VMWare or its SVGA3D leverages the powerful NVIDIA GPU, also my notebook'S RTX 5000 works fine
E.g. so each user can do WebGl at 60fps in the Webbrowser.

Is there some alternative similar like VMWare's SVGA3D which can be used without using a VM?

[Nexarian]

This works: https://github.com/jsorg71/xorgxrdp/tree/nvidia_hack

And make sure you download and install the Nvidia drivers from Nvidia's website directly and install them using the *.run file (DO NOT use the drivers that are bundled with your OS)

Then do this:

Find what PCI address your Nvidia card is on:
lspci | grep NVIDIA
It’ll look something like this:
65:00.0 VGA compatible controller: NVIDIA Corporation GP104GL [Quadro P5000] (rev a1)
That 65 value is the PCI bus address of your card in hexadecimal. Convert it to decimal (65 -> 101)

4. Now, edit the new configuration file for the Nvidia specific setup: sudo vim /etc/X11/xrdp/xorg_nvidia.conf
5. Change Line 38 to replace the ‘2’ with whatever value you obtained above
6. Now, open up /etc/xrdp/sesman.ini and change the line that has param=xrdp/xorg.conf to param=xrdp/xorg_nvidia.conf — On my system this line is 91 but YMMV

[svargh]

/etc/X11/xrdp/xorg_nvidia.conf

I managed to get it work. Nice to see that Xrdp can leverage nvidia directly.

Still, I haven't understood fully how things are working:
RDP Login as root works, and uses the NVidia card.Rendering seems to be slower, but tty access gives me more headache:

• Is there a possibility to login as non-root?
  In /var/log/xrdp-sesman.log I get this:
  [20210408-20:55:56] [INFO ] Xorg :10 -auth .Xauthority -config xrdp/xorg_nvidia.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log [20210408-20:55:56] [INFO ] waiting for window manager (pid 2930) to exit [20210408-20:55:57] [CORE ] window manager exited quickly (1 secs). Window manager config problem?

Xorg needs access to tty, and I am not sure what security implications this will have:
vmuser2@jupiter-host:~$ Xorg :10 -auth .Xauthority -config xrdp/xorg_nvidia.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log /usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server

• RDP Login is not working, if someone is logged in on console.
  So when access to VT is needed, then what is the advantage over using VirtualGL which has similar restrictions. (no one should be logged in, needs an active VT).

• Is this a limitation "imposed" by NVidia? Do you see any possibility to use nvidia over a single proxy like /dev/dri file like VMware/Intel, that can be easily exposed in non root lxc containers, and even in nested lxc containers (works with VMWare dri device and Host is RTX 3090). So I can easily fire up many remote desktop servers quickly.

Build information:
xrdp-0.9.15.tar.gz
root@jupiter-host:/home/vmuser/xrdp# ./configure --enable-fuse --enable-mp3lame --enable-pixman --enable-rdpsndaudin --enable-rfxcodec --enable-painter --enable-opus --enable-jpeg --enable-vsock

xorgxrdp 0.2.13
root@jupiter-host:/home/vmuser/xorgxrdp# git show --oneline -s 3246c17 (HEAD -> nvidia_hack, origin/nvidia_hack) fix for multiple nvidia gpus

[svargh]

/etc/X11/xrdp/xorg_nvidia.conf

I managed to get it work. Nice to see that Xrdp can leverage nvidia directly.

Still, I haven't understood fully how things are working:
RDP Login as root works, and uses the NVidia card.Rendering seems to be slower, but tty access gives me more headache:

* Is there a possibility to login as non-root?
  In /var/log/xrdp-sesman.log I get this:
  `[20210408-20:55:56] [INFO ] Xorg :10 -auth .Xauthority -config xrdp/xorg_nvidia.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log [20210408-20:55:56] [INFO ] waiting for window manager (pid 2930) to exit [20210408-20:55:57] [CORE ] window manager exited quickly (1 secs). Window manager config problem?`

Xorg needs access to tty, and I am not sure what security implications this will have:
vmuser2@jupiter-host:~$ Xorg :10 -auth .Xauthority -config xrdp/xorg_nvidia.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log /usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server

* RDP Login is not working, if someone is logged in on console.
  So when access to VT is needed, then what is the advantage over using VirtualGL which has similar restrictions. (no one should be logged in, needs an active VT).

* Is this a limitation "imposed" by NVidia? Do you see any possibility to use nvidia over a single proxy like  /dev/dri file like VMware/Intel, that can be easily exposed in non root lxc containers, and even in nested lxc containers (works with VMWare dri device and Host is RTX 3090). So I can easily fire up many remote desktop servers quickly.

Build information:
xrdp-0.9.15.tar.gz
root@jupiter-host:/home/vmuser/xrdp# ./configure --enable-fuse --enable-mp3lame --enable-pixman --enable-rdpsndaudin --enable-rfxcodec --enable-painter --enable-opus --enable-jpeg --enable-vsock

xorgxrdp 0.2.13
root@jupiter-host:/home/vmuser/xorgxrdp# git show --oneline -s 3246c17 (HEAD -> nvidia_hack, origin/nvidia_hack) fix for multiple nvidia gpus

I think, I can answer my own question now,
VirtualGL pre 3.0 (dev branch) provides exactly what I want:

From the dev branch I built the pre 3.0 Version (virtualgl_2.6.80_amd64.deb),
Configured VirtualGL to use only EGL backend only.

Then I perform
vglrun -d /dev/dri/card1 konsole or glmark2 across tens of xrdp sessions.
Each vglrun showing RTX 3090 in different firefox-webgl/glmark2 instances.

No Active VT is needed, or even another console non-root user can be logged in -
this is like in Windows Terminal what our users want!

And passing /dev/dri/* to LXC instances or even nested LXC containers is possible. Really nice!!

[cur1ou5soul]

@Nexarian could you maybe share your xorg_nvidia.conf? This would be awesome!

[joehays]

Any update here beyond @Nexarian's nvidia_hack branch? I'm looking for a remote solution that supports Vulkan with nvidia GPUs. @Nexarian's hack seems to be the closest solution...

[Nexarian]

Try this? #1422 (comment)

[Nexarian]

Note: @jsorg71 deserves the credit here. I largely took his code, repackaged it, rebased it off of recent devel branches, and then sprinkled in my dynamic resolution knowledge, but the core innovation belongs to Jay :)

[Nexarian]

@cur1ou5soul The xorg_nvidia.conf file is modified with this step:

echo "Configuring Nvidia..."
BUS_ID=$(lspci | grep -e 'VGA.*NVIDIA' | cut -d: -f1 | xargs -I{} printf "%d\n" 0x{} | sed -n 1p)
sudo sed -i -E 's/(BusID "PCI:)[[:digit:]]+(:0:0")/\1'$BUS_ID'\2/' /etc/X11/xrdp/xorg_nvidia.conf

[moetayuko]

Hello, I rebased jsorg71's nvidia_hack on top of xorgxrdp v0.2.18 and configured the rest like #1697 (comment). Now glxgears indeed shows nvidia driver is in use. However, dynamic resolution is not working and the virtual rdp0 monitor is not shown in KDE settings. Is this expected? If not, how can I get working dynamic resolution with nvidia driver?

---

Edit: I'm experiencing exactly the same issue with @Nexarian's mainline_merge branch. According to xrandr output, the OS is aware of resolution change when resizing RDP window, but KDE is not due to the absence of the virtual monitor.

It turns out that nvidia proprietary driver does not support virtual monitor according to https://wiki.archlinux.org/title/NVIDIA/Tips_and_tricks#Headless_(no_monitor)_resolution, is there any kind of workaround?