automatic backend driver loading does not work in xorgxrdp

[akarl10]

xrdp can spawn a Xorg server that uses drm (/dev/dri/renderD*) as backend. As such opengl and vulkan work out of the box.
video acceleration seems to work to with also with at least radeonsi and iHD if forced with the appropriate environment variable (LIBVA_DRIVER_NAME=...)

Automatic selection of the driver however does not work.

I found that in va/x11/dri2_util.c there is a call to drmGetMagic that fails since there is no drm master in this case (Xorg runs on top of a render node, not card)

If I modify the code to check if dri_state->base.fd is a render node if drmGetMagic fails, skip VA_DRI2Authenticate it seems to do the trick.

Since I'm not a Xorg expert I don't know if this does not break something, but as far as I understand the whole drmGetMagic and subsequent authentication is only a /dev/dri/card thing (like in drm/va_drm.c in function va_DisplayContextGetNumCandidates, where it specifically checks if it is not a render node and if so it skips the magic/authentication steps)