kdb fail to open backed (dlopen) libelektra-resolver.so, but is installed in /usr/lib/elektra instead of /usr/lib

[sl1pkn07]

open kdb without arguments

└───╼  kdb
Usage: kdb <command> [args]

kdb is a program to manage elektra's key database.
Please run a command with -H or --help as args to show a help text for
a specific command.

Known commands are:
Sorry, I have a severe problem, it seems like I am not installed correctly!
kdbOpen() failed with the info:
 Sorry, 1 warning was issued ;(
 Warning (#1):
        Description: could not load module, dlopen failed
        Ingroup: modules
        Module: dl
        At: /tmp/makepkg/elektra-git/src/elektra/src/libs/loader/dl.c:88
        Reason: of module: libelektra-resolver.so, because: libelektra-resolver.so: cannot open shared object file: No such file or directory
        Mountpoint: 
        Configfile: 
Sorry, the error (#40) occurred ;(
Description: failed to open default backend (see warnings for more information)
Ingroup: kdb
Module: 
At: /tmp/makepkg/elektra-git/src/elektra/src/libs/elektra/kdb.c:287
Reason: could not open default backend
Mountpoint: 
Configfile: 

Please report an issue on http://git.libelektra.org/issues

but the file exist in /usr/lib/elecktra

└───╼  locate libelektra-resolver.so
/usr/lib/elektra/libelektra-resolver.so

builded with:

  cd build
  cmake ../elektra \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DENABLE_TESTING=ON \
    -DBUILD_STATIC=OFF \
    -DTOOLS=ALL \
    -DPLUGINS='ALL;-experimental' \
    -DBINDINGS=ALL \
    -DLUA_EXECUTABLE=/usr/bin/lua \
    -DLUA_LIBRARY=/usr/lib/liblua.so \
    -DLUA_INCLUDE_DIR=/usr/include \
    -DTARGET_LUA_CMOD_FOLDER=lib/lua/5.3 \
    -DTARGET_LUA_LMOD_FOLDER=share/lua/5.3 \
    -DJAVA_INCLUDE_PATH="${JAVA_HOME}/include" \
    -DCMAKE_SKIP_INSTALL_RPATH=ON

  LC_ALL=C make

[markus2330]

Thank you for reporting the issue!

Did you execute ldconfig after installation?

Is libelektra-resolver.so a valid symlink?

Can you please check the RPATH (using readelf -a /usr/lib/libelektra-core.so | grep RPATH), it should include the folder where libelektra-resolver.so is.

Any specific reason why you added -DCMAKE_SKIP_INSTALL_RPATH=ON? It likely is the problem. We should check for it and issue a warning when it is set.

Btw. if you really want to have no RPATH, you can use Elektra without RPATH, but then you need to set TARGET_PLUGIN_FOLDER to empty string or something which is found by ldconfig. This is currently used for openwrt, because musl does not support RPATH in libraries.

[sl1pkn07]

because RPATH is a possible security hole, and point to wrong path if exist the original build path

like

┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~]|
└───╼  ldd /usr/bin/elektra-qt-editor
        linux-vdso.so.1 (0x00007ffe587dd000)
        libmarkdown.so.2 => /usr/lib/libmarkdown.so.2 (0x00007fa1458a6000)
        libQt5Quick.so.5 => /usr/lib/libQt5Quick.so.5 (0x00007fa14526a000)
        libQt5Qml.so.5 => /usr/lib/libQt5Qml.so.5 (0x00007fa144c66000)
        libelektratools.so.2 => /tmp/makepkg/elektra-git/src/build/lib/libelektratools.so.2 (0x00007fa1449e1000)
        libQt5Widgets.so.5 => /usr/lib/libQt5Widgets.so.5 (0x00007fa144187000)
        libQt5Gui.so.5 => /usr/lib/libQt5Gui.so.5 (0x00007fa143a4c000)
        libQt5Core.so.5 => /usr/lib/libQt5Core.so.5 (0x00007fa143370000)
        libelektra-kdb.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-kdb.so.4 (0x00007fa143155000)
        libelektra-ease.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-ease.so.4 (0x00007fa142f52000)
        libelektra-core.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-core.so.4 (0x00007fa142d42000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fa1429ba000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fa1427a3000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fa142405000)
        libQt5Network.so.5 => /usr/lib/libQt5Network.so.5 (0x00007fa14207a000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007fa141d76000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fa141b59000)
        libelektra-plugin.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-plugin.so.4 (0x00007fa141957000)
        libelektra-meta.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-meta.so.4 (0x00007fa141752000)
        libGL.so.1 => /usr/lib/libGL.so.1 (0x00007fa1414c2000)
        libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007fa14128c000)
        libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0x00007fa141029000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00007fa140e13000)
        libicui18n.so.58 => /usr/lib/libicui18n.so.58 (0x00007fa140999000)
        libicuuc.so.58 => /usr/lib/libicuuc.so.58 (0x00007fa1405ed000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fa1403e9000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007fa1401e1000)
        libpcre16.so.0 => /usr/lib/libpcre16.so.0 (0x00007fa13ff78000)
        libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007fa13fc65000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa145ab8000)
        libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007fa13f9f4000)
        libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0x00007fa13f57c000)
        libelektra-proposal.so.4 => /tmp/makepkg/elektra-git/src/build/lib/libelektra-proposal.so.4 (0x00007fa13f379000)
        libGLX.so.0 => /usr/lib/libGLX.so.0 (0x00007fa13f149000)
        libGLdispatch.so.0 => /usr/lib/libGLdispatch.so.0 (0x00007fa13ee60000)
        libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007fa13eba3000)
        libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0x00007fa13e977000)
        libicudata.so.58 => /usr/lib/libicudata.so.58 (0x00007fa13ce77000)
        libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007fa13cc04000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007fa13c8c5000)
        libXext.so.6 => /usr/lib/libXext.so.6 (0x00007fa13c6b3000)
        libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x00007fa13c4a3000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007fa13c27a000)
        libXau.so.6 => /usr/lib/libXau.so.6 (0x00007fa13c076000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007fa13be70000)

removing /tmp/makepkg/elektra-git directory (created by the package manager):

└───╼  ldd /usr/bin/elektra-qt-editor
        linux-vdso.so.1 (0x00007fff651f8000)
        libmarkdown.so.2 => /usr/lib/libmarkdown.so.2 (0x00007ff7b7f91000)
        libQt5Quick.so.5 => /usr/lib/libQt5Quick.so.5 (0x00007ff7b7955000)
        libQt5Qml.so.5 => /usr/lib/libQt5Qml.so.5 (0x00007ff7b7351000)
        libQt5DBus.so.5 => /usr/lib/libQt5DBus.so.5 (0x00007ff7b70c5000)
        libelektratools.so.2 => /usr/lib/libelektratools.so.2 (0x00007ff7b6e40000)
        libQt5Widgets.so.5 => /usr/lib/libQt5Widgets.so.5 (0x00007ff7b65e6000)
        libQt5Gui.so.5 => /usr/lib/libQt5Gui.so.5 (0x00007ff7b5eab000)
        libQt5Core.so.5 => /usr/lib/libQt5Core.so.5 (0x00007ff7b57cf000)
        libelektra-kdb.so.4 => /usr/lib/libelektra-kdb.so.4 (0x00007ff7b55b4000)
        libelektra-ease.so.4 => /usr/lib/libelektra-ease.so.4 (0x00007ff7b53b1000)
        libelektra-core.so.4 => /usr/lib/libelektra-core.so.4 (0x00007ff7b51a1000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ff7b4e19000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ff7b4c02000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007ff7b4864000)
        libQt5Network.so.5 => /usr/lib/libQt5Network.so.5 (0x00007ff7b44d9000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007ff7b41d5000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007ff7b3fb8000)
        libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0x00007ff7b3d68000)
        libelektra-plugin.so.4 => /usr/lib/libelektra-plugin.so.4 (0x00007ff7b3b66000)
        libelektra-meta.so.4 => /usr/lib/libelektra-meta.so.4 (0x00007ff7b3961000)
        libGL.so.1 => /usr/lib/libGL.so.1 (0x00007ff7b36d1000)
        libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007ff7b349b000)
        libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0x00007ff7b3238000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00007ff7b3022000)
        libicui18n.so.58 => /usr/lib/libicui18n.so.58 (0x00007ff7b2ba8000)
        libicuuc.so.58 => /usr/lib/libicuuc.so.58 (0x00007ff7b27fc000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007ff7b25f8000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007ff7b23f0000)
        libpcre16.so.0 => /usr/lib/libpcre16.so.0 (0x00007ff7b2187000)
        libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007ff7b1e74000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff7b81a3000)
        libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007ff7b1c03000)
        libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0x00007ff7b178b000)
        libsystemd.so.0 => /usr/lib/libsystemd.so.0 (0x00007ff7b82bb000)
        libelektra-proposal.so.4 => /usr/lib/libelektra-proposal.so.4 (0x00007ff7b1588000)
        libGLX.so.0 => /usr/lib/libGLX.so.0 (0x00007ff7b1358000)
        libGLdispatch.so.0 => /usr/lib/libGLdispatch.so.0 (0x00007ff7b106f000)
        libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007ff7b0db2000)
        libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0x00007ff7b0b86000)
        libicudata.so.58 => /usr/lib/libicudata.so.58 (0x00007ff7af086000)
        libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007ff7aee13000)
        libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007ff7aebfc000)
        libcap.so.2 => /usr/lib/libcap.so.2 (0x00007ff7ae9f8000)
        liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007ff7ae7d2000)
        liblz4.so.1 => /usr/lib/liblz4.so.1 (0x00007ff7ae5c0000)
        libgcrypt.so.20 => /usr/lib/libgcrypt.so.20 (0x00007ff7ae2b1000)
        libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x00007ff7ae09d000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007ff7add5e000)
        libXext.so.6 => /usr/lib/libXext.so.6 (0x00007ff7adb4c000)
        libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x00007ff7ad93c000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007ff7ad713000)
        libXau.so.6 => /usr/lib/libXau.so.6 (0x00007ff7ad50f000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007ff7ad309000)

libelektra-resolver.so is a valid symlink.

┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~]|
└───╼  ls /usr/lib/elektra/libelektra-resolver.so
lrwxrwxrwx 1 root root 31 dic  4 17:03 /usr/lib/elektra/libelektra-resolver.so -> libelektra-resolver_fm_hpu_b.so
┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~]|
└───╼  ls /usr/lib/elektra/libelektra-resolver_fm_hpu_b.so
-rwxr-xr-x 1 root root 80488 dic  4 17:03 /usr/lib/elektra/libelektra-resolver_fm_hpu_b.so

ldconfig is run automagic after update/install package (elecktra is builded through distro package manager)

EDIT:
oks, with -DTARGET_PLUGIN_FOLDER='' seems works again. sorry for the noise and thanks for the tip

merry chrismas!

greetings

[markus2330]

because RPATH is a possible security hole

Is it prohibited to use RPATH in your distribution?

sorry for the noise

np, thanks for opening the issue! Now we can improve and make better warnings if CMAKE_SKIP_INSTALL_RPATH is set and in the COMPILING docu.

Merry christmas!

[sl1pkn07]

not prohibited, but yes recomended

[markus2330]

In 5b3cfb1 I added some notes about RPATH in the docu, does it cover everything you need?

not prohibited, but yes recomended

Is this also the case for private libraries or plugins? Usually exactly the use case of Elektra is an exception. I added the pro/cons of RPATH in doc/COMPILE.md. The major advantage of not using RPATH (relocatable binaries) unfortunately is currently not supported by Elektra for another reason (TARGET_TOOL_EXEC_FOLDER is hardcoded at some places). Are relocatable binaries important for you?

ldd /usr/bin/elektra-qt-editor

Thanks, this is a good catch! But this is an unrelated problem which is not fixed by CMAKE_SKIP_INSTALL_RPATH=ON, elektra-qt-editor was installed in a wrong way. I fixed it in c46f93d (now installed as shell wrapper instead of duplicated binary).

[sl1pkn07]

the problem is the RPATH points to build folder instead of prefix path.

the RPATH should be point to CMAKE_INSTALL_PREFIX

[markus2330]

the problem is the RPATH points to build folder instead of prefix path.

Yes, there should be no RPATH in elektra-qt-editor at all, the installation routine was wrong.

the RPATH should be point to CMAKE_INSTALL_PREFIX

Yes, in installed Elektra by default there is only one RPATH which points to the plugin folder, which is a subdirectory of CMAKE_INSTALL_PREFIX.

Is now everything documented properly in doc/COMPILE.md?
Are relocatable binaries important for your distribution?

Or simply close the issue if you think no further discussion is needed.

[sl1pkn07]

yes. all is ok
i think can be close this issue (for now)

grettings (and sorry again)

[markus2330]

We have to thank you, you found an important issue.

Please never hesitate to open an issue, we love feedback ❤️

[beku]

RPATH is a serious problem in openSUSE. The current rpm's do not work :-P The oS RPM manual states RPATH is forbidden in Fedora (not checked).

Please consider a solution for searching the plugins. Easy would be to scan all LD_LIBRARY_PATH contained paths and add the TARGET_PLUGIN_FOLDER sub directory to the full path name. Additional the CMAKE_INSTALL_LIBDIR/TARGET_PLUGIN_FOLDER would be good I guess.

In Oyranos, we use the runtime LD_LIBRARY_PATH + OY_MODULE_PATH environment variables in order to allow relocatable applications like OS X app bundles. Additionally and first the static (compile time) OY_LIBDIR/OY_CMMSUBPATH OY_USER_PATH/OY_CMMSUBPATH paths are always scanned. And Oyranos modules have a version inside, which is checked after dlopen() time for avoiding conflicts caused by outdated module API's.

[beku]

For "ICC Examin.app" we use a load script and need to specify
export DYLD_FALLBACK_LIBRARY_PATH="$dyld_path:$RESOURCESPATH/elektra"

The unfortunate situation is, all applications, which link against elektra and oyranos need to set the according variables. That's not intuitive ATM.

[markus2330]

@beku The solution is simple and supported since some releases: Simply use -DTARGET_PLUGIN_FOLDER="" It installs plugins side by-side to the libraries. Nothing else is needed. It is already used in production on openwrt (musl does not support RPATH in libraries).

Alternatively, you could add the plugin folder in /etc/ld.so.conf.d/elektra.

The tradeoffs are discussed here.

In Oyranos, we use [..] environment variables
The unfortunate situation is, all applications, which link against elektra and oyranos need to set the according variables. That's not intuitive ATM.

Exactly! Thus we want to avoid to use any environment variables, they bring inconsistent KDB behavior depending on how they are set, which is against Elektra's goal to unify configuration. We discuss the tradeoff usability vs. unified access here: #1249

I recommend you to use KDB to get the variables/paths and avoid wrapper scripts/environment variables.

[beku]

Thanks for pointing out. Reducing dependence on environment variables is well explained and understandable for elektra.

As a consequence of installing plugins into the main ldconfig paths, could elektra link it's modules into bigger libraries sorted by their dependencies? That way they can still be separately packaged without cluttering the lib paths. E.g.:
libelektra-modules.so
libelektra-modules-augeas.so
libelektra-modules-crypto.so
libelektra-modules-dbus.so
...

[markus2330]

As a consequence of installing plugins into the main ldconfig paths, could elektra link it's modules into bigger libraries sorted by their dependencies? That way they can still be separately packaged without cluttering the lib paths

Thanks for the suggestion, we will discuss it! (it would be a solution between the full build and being completely modular: faster, but still good package-able)

But if your concern is only the cluttering of the libs directory, why not using the solution with /etc/ld.so.conf.d/elektra? It gives you exactly that.

What is your opinion about using RPATH by default? Thus it does not work on more and more distros ... (though nobody could explain to me why they are turning it off, it is quite clear that RPATH has its use cases, e.g., for private libs)

[beku]

Can elektra provide /etc/ld.so.conf.d/elektra by default in case of -DTARGET_PLUGIN_FOLDER="" ?

[markus2330]

Currently not, we can discuss it. So you mean, when -DTARGET_PLUGIN_FOLDER is not "" and -DCMAKE_SKIP_INSTALL_RPATH=ON is set and INSTALL_SYSTEM_FILES is not set, we install /etc/ld.so.conf.d/elektra?

Alternatively we could set RUNPATH?

/etc/ld.so.conf.d/elektra can always be installed without harm, but with TARGET_PLUGIN_FOLDER="" it is not needed.

[beku]

/etc/ld.so.conf.d/elektra can always be installed without harm, but with TARGET_PLUGIN_FOLDER="" it is not needed.

Oops. The other way round. With TARGET_PLUGIN_FOLDER != "", create a /etc/ld.so.conf.d/elektra.conf file with the system module path.

[markus2330]

Ok, thanks for your feedback! We will definitely do something till next release, I started the discussion in #1275. We would love to hear what is your favorite way!