[showstopper] [fhs-like mode] Qt platform plugin "xcb" missing

[probonopd]

Running QtCreator-5.8.0-x86_64.AppImage on ubuntu-16.04-desktop-amd64.iso and then doing

git clone --recursive https://github.com/LibrePCB/LibrePCB.git && cd LibrePCB
export PATH=/tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64/bin/:$PATH
qmake -r ../librepcb.pro
sudo apt-get install libglu1-mesa-dev openssl zlib1g zlib1g-dev
make -j 8
mkdir -p LibrePCB.AppDir/usr
cp -r generated/unix  LibrePCB.AppDir/usr/bin
cp -r generated/{lib,share}  LibrePCB.AppDir/usr/
wget -c https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage
chmod a+x linuxdeployqt-continuous-x86_64.AppImage 
./linuxdeployqt-continuous-x86_64.AppImage ./LibrePCB.AppDir/usr/bin/librepcb 

results in:

me@host:~/LibrePCB/build$ ./linuxdeployqt-continuous-x86_64.AppImage /home/me/LibrePCB/build/LibrePCB.AppDir/usr/bin/librepcb 
"/tmp/.mount_W7rCgi/usr/bin:/tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64/bin/:/tmp/.mount_QtCreator-5.8.0-x86_64/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
app-binary: "/home/me/LibrePCB/build/LibrePCB.AppDir/usr/bin/librepcb"
FHS-like mode with PREFIX, fhsPrefix: "/home/me/LibrePCB/build/LibrePCB.AppDir/usr"
appDirPath: "/home/me/LibrePCB/build/LibrePCB.AppDir"
relativeBinPath: "usr/bin/librepcb"
ERROR: ldd outputLine: "\tlibmysqlclient.so.18 => not found"
ERROR: ldd outputLine: "\tlibssl.so.10 => not found"
ERROR: ldd outputLine: "\tlibcrypto.so.10 => not found"
ERROR: ldd outputLine: "\tlibpq.so.5 => not found"

me@host:~/LibrePCB/build$ ./LibrePCB.AppDir/AppRun 
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Reinstalling the application may fix this problem.
Aborted

This is because the plugins/ directory has been generated in the top-level directory of the AppImage but qt.conf does not point there. A workaround is to move it to usr/bin where qt.conf resides.

[probonopd]

So where should linuxdeployqt put the plugins directory in fhs-like mode? QtCreator-5.8.0-x86_64.AppImage has it in Tools/QtCreator/lib/Qt/plugins/, so I think usr/lib/Qt/plugins might be appropriate?

Which would probably mean that in FHS-like mode, we should set prefix = ../lib/Qt and leave the rest of qt.conf as-is, and move the plugins/ directory to usr/lib/Qt/plugins.

@ubruhin does this make sense to you?

[ubruhin]

Which would probably mean that in FHS-like mode, we should set prefix = ../lib/Qt and leave the rest of qt.conf as-is, and move the plugins/ directory to usr/lib/Qt/plugins.

Sounds good, the only thing I would change is to make Qt lowercase, i.e. /usr/lib/qt/plugins ;)

[probonopd]

If we move stuff to usr/lib/qt/plugins then this has to be accounted for in many other places.
So I left the plugins where they are and tried to add prefix = ../../ to qt.conf.

Argh. Moving around stuff tends to break other stuff. E.g.,

me@host:~/LibrePCB/build/LibrePCB.AppDir$ patchelf --print-rpath  plugins/platforms/libqxcb.so 
$ORIGIN/../../lib

$ORIGIN/../../lib is wrong for FHS-like mode...

also resulting in the dreaded

me@host:~/LibrePCB/build$ ./LibrePCB.AppDir/AppRun 
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Reinstalling the application may fix this problem.
Aborted

As a quick and dirty(!) workaround, we could symlink usr/lib to lib/... but this needs to be done properly, PRs welcome.

[ubruhin]

I wonder if a non-FHS-like mode is even required? Always using FHS would make the application simpler and thus less error-prone...

[probonopd]

Not all Qt applications can be installed to a FHS-like structure; in fact, that's the default case with Qt Creator.

[probonopd]

So actually Qt Creator uses a slightly different "FHS-like" structure; there is no dedicated ´lib/qtdirectory, but the subdirectories of the qt directory are right inside theqt_prfxpath, where bin/andlib/` also are.

In other words, it looks like we should set qt_prfxpath to the directory above bin/ and put the Qt subdirectories like plugins/ etc. into that directory (habitually usr/) too.

me@host:~/foo$ strings /tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64/lib/libQt5Core.so.5 | grep qt_
qt_prfxpath=/tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64

me@host:~/foo$ ls /tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64
bin  include  libexec  phrasebooks  qml        translations
doc  lib      mkspecs  plugins      resources

[probonopd]

As per the tests, the tests/QtWidgetsApplication/QtWidgetsApplication.pro example application gets bundled correctly in FHS-like mode, hence closing here. Please open individual tickets for remaining issues with FHS-like mode.

[probonopd]

Succeeded in making a LibrePCB AppImage like this:

# Compile LibrePCB as usual
git clone --recursive https://github.com/LibrePCB/LibrePCB.git && cd LibrePCB
export PATH=/tmp/.mount_QtCreator-5.8.0-x86_64/5.8/gcc_64/bin/:$PATH
qmake -r ../librepcb.pro
sudo apt-get install libglu1-mesa-dev openssl zlib1g zlib1g-dev
make -j 8

# Copy into AppDir FHS-like stucture
mkdir -p LibrePCB.AppDir/usr
cp -r generated/unix  LibrePCB.AppDir/usr/bin
cp -r generated/{lib,share}  LibrePCB.AppDir/usr/

# Get and run linuxdeployqt
wget -c https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage
chmod a+x linuxdeployqt-continuous-x86_64.AppImage 

# Create AppImage using linuxdeployqt
./linuxdeployqt-continuous-x86_64.AppImage ./LibrePCB.AppDir/usr/bin/librepcb
( cd LibrePCB.AppDir/ ; cp ./usr/share/applications/librepcb.desktop . )
( cd LibrePCB.AppDir/ ; cp ./usr/share/pixmaps/librepcb.svg . )
export VERSION=$(git rev-list --count HEAD).$(git describe --tags | cut -d - -f 3)
./linuxdeployqt-continuous-x86_64.AppImage ./LibrePCB.AppDir/usr/bin/librepcb -appimage

# Success!
ls -lh LibrePCB*AppImage
# -rwxr-xr-x 1 me me 31M Feb 19 15:34 LibrePCB-864.gf0d32a2-x86_64.AppImage

Note that no wrapper script, no LD_LIBRARY_PATH, and no qt.conf file are used.

[ubruhin]

Very nice!

Note that no wrapper script, no LD_LIBRARY_PATH, and no qt.conf file are used.

Does this also solve the problem with sourcing /opt/qt57/bin/qt57-env.sh?

[probonopd]

No, but I am not sure that this is a "problem" actually. Just unset the variables set by /opt/qt57/bin/qt57-env.sh prior to running linuxdeployqt. I think these variables are just doing their job, as they would with any other Qt application. Or am I wrong here?

[ubruhin]

With the release linuxdeployqt-3-x86_64.AppImage I still get this error when using Qt5.2 on Travis:

This application failed to start because it could not find or load the Qt platform plugin "xcb".

Here is my AppImage: https://bintray.com/librepcb/LibrePCB-Nightly/download_file?file_path=LibrePCB-Nightly-Linux-x86_64.AppImage

And here is the build output: https://travis-ci.org/LibrePCB/LibrePCB/jobs/203230997#L2705

When using Qt5.7 from PPA, it works properly.

[probonopd]

For some weird reason that I don't understand yet, the following workaround is needed:

( cd opt/bin/ ;  ln -s ../plugins/platforms/ . ; cd - ) # Has been fixed now, see below
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/platforms/libqxcb.so
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/bearer/libqnmbearer.so 
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/bearer/libqconnmanbearer.so 
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/sqldrivers/libqsqlite.so

It may have to do with the fact that

strings opt/lib/libQt5Core.so.5 | grep qt_p
# qt_plugpath=/usr/lib/x86_64-linux-gnu/qt5/plugins

Maybe we need to patch this away, too. Can you verify the last command with a working one? I'd bet qt_plugpath= is empty there... In that case, it's not a matter of the Qt version but whether the Qt came from the distribution (has qt_plugpath=/usr/lib/x86_64-linux-gnu/qt5/plugins populated) or the ppa (doesn't have qt_plugpath poplulated).

[ubruhin]

With Qt5.2 from Ubuntu repository: qt_plugpath=/usr/lib/x86_64-linux-gnu/qt5/plugins

With Qt5.7 from PPA: qt_prfxpath=..

Wouldn't a qt.conf solve that problem?

[probonopd]

Thanks for verifying @ubruhin

Let's see if 3aca9d5 in the latest https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage changes this.

It should unset qt_plugpath although I couldn't test it so far.

[ubruhin]

Yes, the plugpath is now cleared (qt_plugpath=), but the AppImage still fails to start with the same error message (xcb missing) :( Shouldn't the path be set to ".."?

[probonopd]

Do you have qt_prfxpath=.. in libQt5Core.so.5?

I was assuming that the combination of qt_prfxpath=.. and qt_plugpath= would mean that plugins would be loaded from ../plugins, relative to the location of libQt5Core.so.5. Is this assumption wrong?

[probonopd]

Please run with export SET QT_DEBUG_PLUGINS=1

[ubruhin]

$ export SET QT_DEBUG_PLUGINS=1
$ ./LibrePCB-Nightly-Linux-x86_64.AppImage 
QFactoryLoader::QFactoryLoader() checking directory path "/tmp/.mount_V3eHXv/opt/bin/platforms" ... 
This application failed to start because it could not find or load the Qt platform plugin "xcb".

Obviously is the plugins path wrong, it isn't located inside the bin directory as shown here.

[probonopd]

Thanks, can you confirm you have qt_prfxpath=.. in libQt5Core.so.5?

[ubruhin]

As I wrote above, qt_prfxpath isn't set so .. but cleared completely. This is also what the implementation looks like to do.

[probonopd]

qt_plugpath and qt_prfxpath are two separate things...

[probonopd]

Trying b32682b, the next continuous build should have it. Please test

[ubruhin]

Oops, I didn't realize that we discuss about two different paths 😮

Qt5.2 from Ubuntu repository: qt_prfxpath isn't set(?)
Qt5.7 from PPA: qt_prfxpath=..

[probonopd]

Are you really sure? Please run on the Qt5.2 from Ubuntu repository, after it is bundled by appimagetool into the AppDir:

strings opt/lib/libQt5Core.so.5 | grep qt_p | grep =
# qt_prfxpath=..
# qt_plugpath=

[probonopd]

New https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage is available

[ubruhin]

Are you really sure?

Here the strings command: https://travis-ci.org/LibrePCB/LibrePCB/jobs/203268328#L2454

And here its output: https://travis-ci.org/LibrePCB/LibrePCB/jobs/203268328#L2829

Bot now I need some sleep... ;)

[probonopd]

Omg, many more hardcoded paths to be patched in the distribution-provided one.

I am opening a new ticket for those, let's stay on the topic in this ticket.

[probonopd]

Just patching away qt_plugpath entirely does not help:
sed -i -e 's|qt_plugpath|xx_plugpath|g' opt/lib/libQt5Core.so.5

So we need to set it to a sensible value, namely, qt_plugpath=plugins. This gets it working.

[probonopd]

I think the original issue of this ticket should be solved now (no more Qt platform plugin "xcb" missing with the latest continous build, even using the distro-provided Qt).

I think for LibrePCB compiled with the distro-provided Ubuntu trusty Qt 5.2 you still need to run

patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/platforms/libqxcb.so
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/bearer/libqnmbearer.so 
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/bearer/libqconnmanbearer.so 
patchelf --set-rpath '$ORIGIN/../../lib:$ORIGIN' ./opt/plugins/sqldrivers/libqsqlite.so

which is a separate issue. Hence closing here. Feel free to comment if you think otherwise.