Update prepare_rpath_wrappers to enable wrapper shipping with a module (WIP) #4003
Conversation
| if cmdDir is not None and os.path.exists(cmdDir): | ||
| orig_cmd = os.path.join(cmdDir,cmd) | ||
|
|
||
| # TODO: dirty hack to let module-shipped rpath wrappers point to EESSI software layer's ld | ||
| if cmdDir is not None and "ld" in cmd: | ||
| orig_cmd = which(cmd, retain_all=True) | ||
| orig_cmd = [a for a in orig_cmd if "/tmp" not in a][0] | ||
|
|
||
| self.log.debug("orig_cmd: %s", orig_cmd) |
There was a problem hiding this comment.
This section is a bit critical.
The original prepare_rpath_wrappers is using which to lookup ld and compiler commands to wrap with the newly created rpath wrapper.
But we want to wrap the compiler command of the e.g. GCCcore module that currently is build! So we cannot look it up in PATH and instead have to pass a new cmdDir to the function and decide to set it manually.
This might be a first good reason to split this function into two.
Second, in EESSI we ship ld in the software layer on CVMFS. We don't want to pick up any ld commands that exist in the current /tmp locations. They are still around during module building, so we have to put a dirty hack here.
I don't really like putting an EESSI special case into EasyBuild code.
Do we have a general approach to this kind of problem?
Maybe this calls for another manual path, from where to pick up ld?
There was a problem hiding this comment.
I'm confused which usage of prepare_rpath_wrappers you're trying to target here with the exception for ld: the usage by EasyBuild when installing with --rpath? Or is it to prevent the ld wrappers from being installed for EESSI? In the latter case, I'd say we should indeed make the ld wrapper installation part of binutils (as was suggested in easybuilders/easybuild-easyblocks#2638). That means --filter-deps binutils is enough to (also) make sure the RPATH wrappers aren't installed (and I guess we filter binutils already in EESSI? Can't remember exactly...).
If it's more about EasyBuild when installing with --rpath (i.e. you're afraid that this will pick up the ld wrappers from the tempdir that EasyBuild creates), I'd have to think how to elegantly solve this. I agree with the statement that we don't want special case things for EESSI in the EasyBuild source code, it should be something generic.
I'm generally a bit confused what we do currently in EESSI: do we set RPATH with our own ld wrapper? Or do we rely on EasyBuild to set RPATH? I thought it was the latter, but then you'd still want it to generate wrappers when installing with --rpath, right?
|
@stderr-enst I'd like to take over what you've done here for a specific use case, I would like to create an updated |
This WIP PR is part of a suggested feature to ship rpath wrappers e.g. with GCCcore.
More information in related issues and PRs:
This PR contains a first working version, with a couple of dirty hacks, so it is definitely not ready to be merged.
prepare_rpath_wrappersis currently preparing the wrappers in temporary directories. Wrappers are then used in the module's build step so far!Updating
prepare_rpath_wrappersto also install wrappers in the module install is maybe stretching the purpose of a single function.Would you suggest that I refactor the code a bit to produce 2 smaller functions? One prepares wrappers for build, the other does installation. There is probably a bit of shared code though.
I'll also point out a specific section of the code in the comments below.
Tagging @boegel