Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[21.05] pkg-config-wrapper: mangle PKG_CONFIG_PATH{,_FOR_BUILD} correctly #124149

Merged

Conversation

sternenseemann
Copy link
Member

Previously, mangleVarList would be used which would concatenate the
variables using a space as a separator. Paths are however separated by
: in PKG_CONFIG_PATH leading to entries being broken.

This is fixed by introducing mangleVarListGeneric which allows us to
specify the desired separator.

Reproducer for the issue prior to this change:

$ nix-shell -A pkgsLLVM.wayland
[nix-shell] $ pkg-config --libs expat
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
$ printf 'Host: %s\nBuild: %s' $PKG_CONFIG_PATH $PKG_CONFIG_PATH_FOR_BUILD
Host: /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/z3y9ska2h4l1map25m195iq577g7g3gz-libxml2-x86_64-unknown-linux-gnu-2.9.12-dev/lib/pkgconfig:/nix/store/lbz5m1s0r7zn0cxvl21czfspli6ribzb-zlib-1.2.11-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/rfhvp8r8n3ygpzh8j0l34lk8hwwi3z0h-libffi-3.3-x86_64-unknown-linux-gnu-dev/lib/pkgconfig
Build: /nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig

strace reveals the issue:

stat("/nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig/expat-uninstalled.pc", 0x7fff49829fa0) = -1 ENOENT (No such file or directory)

In the pkg-config wrapper $PKG_CONFIG_PATH_FOR_BUILD and
$PKG_CONFIG_PATH are concatenated with a space which leads to two paths
being messed up. This issue likely only affects native cross
compilation.

(cherry picked from commit b11d65c)

Backport of #123415.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Added a release notes entry if the change is major or breaking
  • Fits CONTRIBUTING.md.

Previously, mangleVarList would be used which would concatenate the
variables using a space as a separator. Paths are however separated by
`:` in PKG_CONFIG_PATH leading to entries being broken.

This is fixed by introducing mangleVarListGeneric which allows us to
specify the desired separator.

Reproducer for the issue prior to this change:

    $ nix-shell -A pkgsLLVM.wayland
    [nix-shell] $ pkg-config --libs expat
    Package expat was not found in the pkg-config search path.
    Perhaps you should add the directory containing `expat.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'expat' found
    $ printf 'Host: %s\nBuild: %s' $PKG_CONFIG_PATH $PKG_CONFIG_PATH_FOR_BUILD
    Host: /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/z3y9ska2h4l1map25m195iq577g7g3gz-libxml2-x86_64-unknown-linux-gnu-2.9.12-dev/lib/pkgconfig:/nix/store/lbz5m1s0r7zn0cxvl21czfspli6ribzb-zlib-1.2.11-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/rfhvp8r8n3ygpzh8j0l34lk8hwwi3z0h-libffi-3.3-x86_64-unknown-linux-gnu-dev/lib/pkgconfig
    Build: /nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig

strace reveals the issue:

    stat("/nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig/expat-uninstalled.pc", 0x7fff49829fa0) = -1 ENOENT (No such file or directory)

In the pkg-config wrapper $PKG_CONFIG_PATH_FOR_BUILD and
$PKG_CONFIG_PATH are concatenated with a space which leads to two paths
being messed up. This issue likely only affects native cross
compilation.

(cherry picked from commit b11d65c)
@r-rmcgibbo
Copy link

Result of nixpkgs-review pr 124149 at 771d699 run on x86_64-linux 1

1529 packages marked as broken and skipped:
  • SDL_sixel
  • adobe-reader
  • adoptopenjdk-hotspot-bin-13
  • adoptopenjdk-hotspot-bin-14
  • adoptopenjdk-jre-hotspot-bin-13
  • adoptopenjdk-jre-hotspot-bin-14
  • adoptopenjdk-jre-openj9-bin-13
  • adoptopenjdk-jre-openj9-bin-14
  • adoptopenjdk-openj9-bin-13
  • adoptopenjdk-openj9-bin-14
  • ...
35669 packages skipped due to time constraints:
  • AMB-plugins
  • AusweisApp2
  • CHOWTapeModel
  • CoinMP
  • DisnixWebService
  • EBTKS
  • EmptyEpsilon
  • FIL-plugins
  • Fabric (python38Packages.Fabric)
  • LAStools
  • ...
389 packages built successfully:
  • acl
  • alegreya-sans
  • amoeba-data
  • asciidoctor
  • asdf
  • atk (gnome.atk ,gnome2.atk)
  • bash (runtimeShellPackage)
  • bfetch
  • bintoolsNoLibc (binutilsNoLibc)
  • cairo (gnome2.cairo)
  • ccloud-cli
  • commonsFileUpload
  • copyDesktopItems
  • copyright-update
  • dav1d
  • deer
  • dehydrated
  • discocss
  • disfetch
  • docbook-xsl-ns (docbook_xsl_ns)
  • docbook_sgml_dtd_41
  • dwmblocks
  • edusong
  • eigen
  • enableGCOVInstrumentation
  • essentia-extractor
  • fet-sh
  • fixup_yarn_lock
  • fontconfig
  • fontforge (fontforge-fonttools ,python38Packages.fontforge)
  • fpp
  • gd
  • gdk-pixbuf
  • ghc_filesystem
  • gi-docgen
  • gobject-introspection (gnome.gobject-introspection)
  • gsettings-desktop-schemas (gnome.gsettings-desktop-schemas)
  • pango (gnome2.pango)
  • gnomeExtensions.archlinux-updates-indicator
  • gnomeExtensions.better-osd-gnome-40
  • gnomeExtensions.colortint
  • gnomeExtensions.colosseum
  • gnomeExtensions.draw-on-your-screen
  • gnomeExtensions.espresso
  • gnomeExtensions.jiggle
  • gnomeExtensions.kimpanel
  • gnomeExtensions.lunar-calendar
  • gnomeExtensions.mpris-indicator-button
  • gnomeExtensions.nasa-apod
  • gnomeExtensions.panel-scroll
  • gnomeExtensions.right-click-down
  • gnomeExtensions.sermon
  • gnomeExtensions.worksets
  • gnomeExtensions.workspace-switch-wraparound
  • gnu-config
  • python3 (gnuradioPackages.python ,python38 ,python38Packages.python ,sourcehut.python)
  • gwtdragdrop
  • hacpack
  • harfbuzz
  • hicolor-icon-theme
  • hostname (unixtools.hostname)
  • hunspellDicts.it-it (hunspellDicts.it_IT)
  • installShellFiles
  • iodash
  • ipcalc
  • isocodes
  • jpylyzer (python38Packages.jpylyzer)
  • json-glib
  • kakounePlugins.connect-kak
  • kakounePlugins.kakoune-rainbow
  • keepBuildTree
  • kexectools
  • ladspaH
  • lasem
  • lcms2
  • ld-is-cc-hook
  • libbtbb
  • libevdev
  • libimagequant
  • libjpeg (libjpeg_turbo)
  • libksba
  • libmng
  • libqrtr-glib
  • libtiff
  • libwebp
  • linuxHeaders
  • llvmPackages_11.libcxx (llvmPackages_latest.libcxx)
  • llvmPackages_11.libcxxabi (llvmPackages_latest.libcxxabi)
  • llvmPackages_11.libunwind (llvmPackages_latest.libunwind)
  • llvmPackages_5.libcxxabi
  • llvmPackages_6.libcxx
  • llvmPackages_6.libcxxabi
  • lsb-release
  • lz4
  • mobile-broadband-provider-info
  • monoDLLFixer
  • mpvScripts.simple-mpv-webui
  • mtdev
  • multipart-parser-c
  • netsurf.buildsystem
  • nix-zsh-completions
  • npth
  • octavePackages.writeRequiredOctavePackagesHook
  • opencl-headers
  • passExtensions.pass-genphrase
  • perl532Packages.AlgorithmDiff
  • perl532Packages.BFlags
  • perl532Packages.BHooksEndOfScope
  • perl532Packages.CanaryStability
  • perl532Packages.ClassDataInheritable
  • perl532Packages.ClassInspector
  • perl532Packages.CwdGuard
  • perl532Packages.DataOptList
  • perl532Packages.DevelCheckBin
  • perl532Packages.DevelCheckCompiler
  • perl532Packages.DevelGlobalDestruction
  • perl532Packages.DevelStackTrace
  • perl532Packages.DistCheckConflicts
  • perl532Packages.EncodeLocale
  • perl532Packages.EvalClosure
  • perl532Packages.ExporterTiny
  • perl532Packages.ExtUtilsConfig
  • perl532Packages.ExtUtilsHelpers
  • perl532Packages.ExtUtilsInstallPaths
  • perl532Packages.FCGIProcManager
  • perl532Packages.FileCopyRecursiveReduced
  • perl532Packages.FileShareDir
  • perl532Packages.FileShareDirInstall
  • perl532Packages.Filepushd
  • perl532Packages.FontTTF
  • perl532Packages.HTTPServerSimple
  • perl532Packages.MROCompat
  • perl532Packages.ModuleBuildTiny
  • perl532Packages.ModuleImplementation
  • perl532Packages.ModuleRuntime
  • perl532Packages.ModuleRuntimeConflicts
  • perl532Packages.Opcodes
  • perl532Packages.ParamsUtil
  • perl532Packages.Readonly
  • perl532Packages.RoleTiny
  • perl532Packages.StreamBuffered
  • perl532Packages.SubExporterProgressive
  • perl532Packages.SubIdentify
  • perl532Packages.SubInstall
  • perl532Packages.TestException
  • perl532Packages.TestLeakTrace
  • perl532Packages.TestRequires
  • perl532Packages.TestWithoutModule
  • perl532Packages.TextDiff
  • perl532Packages.TimeDate
  • perl532Packages.XMLSAXExpat
  • pikchr
  • pixman (xorg.pixman)
  • polytopes_db
  • poppler_data
  • profont
  • python38Packages.Babel
  • python38Packages.Mako
  • python38Packages.apipkg
  • python38Packages.async_generator
  • python38Packages.automat
  • python38Packages.backcall
  • python38Packages.backports-entry-points-selectable
  • python38Packages.blinker
  • python38Packages.brotli
  • python38Packages.certifi
  • python38Packages.click
  • python38Packages.cloudpickle
  • python38Packages.constantly
  • python38Packages.cookies
  • python38Packages.cppy
  • python38Packages.cycler
  • python38Packages.dateutil (python38Packages.python-dateutil)
  • python38Packages.decorator
  • python38Packages.django (python38Packages.django_2)
  • python38Packages.execnet
  • python38Packages.extras
  • python38Packages.flake8
  • python38Packages.flaky
  • python38Packages.flit-core
  • python38Packages.freezegun
  • python38Packages.gunicorn
  • python38Packages.hyperlink
  • python38Packages.idna
  • python38Packages.imagesize
  • python38Packages.incremental
  • python38Packages.intreehooks
  • python38Packages.ipython_genutils
  • python38Packages.iso8601
  • python38Packages.itsdangerous
  • python38Packages.jinja2
  • python38Packages.jupyterlab-pygments
  • python38Packages.libevdev
  • python38Packages.m2r
  • python38Packages.mccabe
  • python38Packages.mistune (python38Packages.mistune_0_8)
  • python38Packages.multidict
  • python38Packages.nest-asyncio
  • python38Packages.nose
  • python38Packages.olefile
  • python38Packages.ordereddict
  • python38Packages.outcome
  • python38Packages.packaging
  • python38Packages.pandocfilters
  • python38Packages.pep517
  • python38Packages.pexpect
  • python38Packages.pip
  • python38Packages.pipBuildHook
  • python38Packages.ply
  • python38Packages.pretend
  • python38Packages.prometheus_client
  • python38Packages.py-cpuinfo
  • python38Packages.pyasn1-modules
  • python38Packages.pycairo
  • python38Packages.pydocstyle
  • python38Packages.pyext
  • python38Packages.pygobject3
  • python38Packages.pyhamcrest
  • python38Packages.pylama
  • python38Packages.pyperf
  • python38Packages.pytest (python38Packages.pytest_6)
  • python38Packages.pytest-asyncio
  • python38Packages.pytest-benchmark
  • python38Packages.pytestcov (python38Packages.pytest-cov)
  • python38Packages.pytest-expect
  • python38Packages.pytest-flake8
  • python38Packages.pytest-forked
  • python38Packages.pytest-freezegun
  • python38Packages.pytest-mock
  • python38Packages.pytestrunner (python38Packages.pytest-runner)
  • python38Packages.pytest-subtests
  • python38Packages.pytest-xdist (python38Packages.pytest_xdist)
  • python38Packages.pytestCheckHook
  • python38Packages.pythonImportsCheckHook
  • python38Packages.python_mimeparse
  • python38Packages.pytoml
  • python38Packages.recursivePthLoader
  • python38Packages.send2trash
  • python38Packages.simplejson
  • python38Packages.smartypants
  • python38Packages.sortedcontainers
  • python38Packages.soupsieve
  • python38Packages.sqlparse
  • python38Packages.tappy
  • python38Packages.terminado
  • python38Packages.text-unidecode
  • python38Packages.tornado
  • python38Packages.typogrify
  • python38Packages.u-msgpack-python
  • python38Packages.ukpostcodeparser
  • python38Packages.urllib3
  • python38Packages.vcversioner
  • python38Packages.webencodings
  • python38Packages.wrapPython
  • python38Packages.yarl
  • python38Packages.zope_event
  • python38Packages.zope_interface
  • python39 (python39Packages.python)
  • python39Packages.alabaster
  • python39Packages.appdirs
  • python39Packages.async-timeout
  • python39Packages.atomicwrites
  • python39Packages.attrs
  • python39Packages.automat
  • python39Packages.bootstrapped-pip
  • python39Packages.certifi
  • python39Packages.colorama
  • python39Packages.constantly
  • python39Packages.coverage
  • python39Packages.dateutil (python39Packages.python-dateutil)
  • python39Packages.distlib
  • python39Packages.docutils
  • python39Packages.eradicate
  • python39Packages.extras
  • python39Packages.filelock
  • python39Packages.imagesize
  • python39Packages.importlib-metadata
  • python39Packages.incremental
  • python39Packages.iniconfig
  • python39Packages.itsdangerous
  • python39Packages.linecache2
  • python39Packages.m2r
  • python39Packages.markupsafe
  • python39Packages.mistune (python39Packages.mistune_0_8)
  • python39Packages.mock
  • python39Packages.more-itertools
  • python39Packages.nose
  • python39Packages.olefile
  • python39Packages.outcome
  • python39Packages.packaging
  • python39Packages.pbr
  • python39Packages.pexpect
  • python39Packages.pip
  • python39Packages.pipBuildHook
  • python39Packages.pipInstallHook
  • python39Packages.pkgconfig
  • python39Packages.pluggy
  • python39Packages.pretend
  • python39Packages.psutil
  • python39Packages.ptyprocess
  • python39Packages.purl
  • python39Packages.py
  • python39Packages.pyasn1
  • python39Packages.pycodestyle
  • python39Packages.pycparser
  • python39Packages.pyflakes
  • python39Packages.pygments
  • python39Packages.pyparsing
  • python39Packages.pysocks
  • python39Packages.pythonCatchConflictsHook
  • python39Packages.pythonImportsCheckHook
  • python39Packages.pythonNamespacesHook
  • python39Packages.pythonRemoveTestsDirHook
  • python39Packages.python_mimeparse
  • python39Packages.pytz
  • python39Packages.ruamel-base (python39Packages.ruamel_base)
  • python39Packages.semantic-version
  • python39Packages.setuptools
  • python39Packages.setuptools-rust
  • python39Packages.setuptools-scm (python39Packages.setuptools_scm)
  • python39Packages.setuptoolsBuildHook
  • python39Packages.setuptoolsCheckHook
  • python39Packages.six
  • python39Packages.sortedcontainers
  • python39Packages.sphinxcontrib-devhelp
  • python39Packages.sphinxcontrib-htmlhelp
  • python39Packages.sphinxcontrib-jsmath
  • python39Packages.sphinxcontrib-qthelp
  • python39Packages.sphinxcontrib-serializinghtml
  • python39Packages.sphinxcontrib-websupport
  • python39Packages.testpath
  • python39Packages.toml
  • python39Packages.tornado
  • python39Packages.traceback2
  • python39Packages.typing-extensions
  • python39Packages.u-msgpack-python
  • python39Packages.ukpostcodeparser
  • python39Packages.unittest2
  • python39Packages.wcwidth
  • python39Packages.wheel
  • python39Packages.wheelUnpackHook
  • python39Packages.wrapPython
  • python39Packages.zipp
  • python39Packages.zope_event
  • rtl8723bs-firmware
  • rubyPackages.red-colors (rubyPackages_2_7.red-colors)
  • rubyPackages.rouge (rubyPackages_2_7.rouge)
  • rubyPackages.rubyzip (rubyPackages_2_7.rubyzip)
  • sbclBootstrap
  • shen-sources
  • shhopt
  • source-han-mono
  • speedread
  • subdl
  • tmuxPlugins.fpp
  • tmuxPlugins.nord
  • tmuxPlugins.online-status
  • unicode-character-database
  • unicode-emoji
  • unixtools.arp
  • unixtools.ifconfig
  • unixtools.netstat
  • unixtools.nettools
  • unixtools.route
  • validatePkgConfig
  • volk
  • vscode-extensions.edonet.vscode-command-runner
  • vscode-extensions.gruntfuggly.todo-tree
  • vscode-extensions.justusadam.language-haskell
  • vscode-extensions.timonwong.shellcheck
  • vscode-extensions.zhuangtongfa.material-theme
  • vulkan-headers
  • woof
  • xfce.automakeAddFlags
  • xhtml1
  • xlibsWrapper (xorg.xlibsWrapper)
  • xorg.libX11
  • xorg.libXft
  • xorg.libXinerama
  • xorg.libXt
  • xorg.xcbutilimage
  • xorg.xcbutilkeysyms
  • xorg.xcbutilrenderutil
  • xorg.xcbutilwm
  • xorg.xtrans
  • xxHash
  • yarn
  • zfsnap
  • zsh-z

@Ericson2314 Ericson2314 merged commit 62f3f89 into NixOS:staging-21.05 May 23, 2021
@sternenseemann sternenseemann deleted the pkg-config-mangling-21.05 branch May 23, 2021 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants