-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
avoid collisions when downloading source files #455
Comments
I also came across the following one today: http://www.g95.org/downloads.shtml#V0.93 could we perhaps raise a flag in the easyconfigs file, to prepend the filename with version or something like that? alternatively, could we do this automatically whenever there is a collision? |
I think doing this automagically should be possible... We might need an easyconfig parameter though, to specify that the renaming should be done. |
atm, are the downloaded sources flagged read-only? it will at the very least prevent from accidentally overwriting the source (although it should detect that it is there and not redownload, but it will prevent admins from manually overwriting ;). |
I propose to allow for a dictionary in the 'sources' field. sources = [('abinit-%s_x86_64_linux_gnu4.5.bz2' % version, 'tar xfj %s')]
source_urls = ['http://ftp.abinit.org/'] We could keep supporting this, bud also add support for something like we have in the 'extension packages' sources = [{'name': 'abinit-%s_x86_64_linux_gnu4.5.bz2' % version, 'unpack_cmd': 'tar xfj %s', 'rename_to': 'abinit-%s.tar.gz' % version, 'url': 'http://ftp.abinit.org/' )] Note that actually renaming it would also remove the need for the unpack_cmd, but I left it here for illustrative purposes. Switching to a dict would give us much more flexibility in the future, example for adding 'git_id', 'branch', etc... |
On Thu, Feb 14, 2013 at 11:42 AM, Jens Timmerman
I propose a variation: because 'name' is a compulsory item,
ie. the dictionary contains only any extra optional specializations. what do you think? |
makes sense, but we could always add name-version.tar.gz as a default, and then get rid of the compulsory item altogether ;-) |
maybe change rename_to to rename but i think the 'unpack' part should be forbidden. i would also propose that all value strings in easyconfig are templated against {'version':version,
'versionlower':version.lower(),
'name':name,
'namelower':name.lower(),
} and probably a few more (maybe do a short survey on existing configs to see other frequent ones and soome constants like github, googlecode,sourceforge for the homepage/source_url) (the example should then become sources = [('%(namelower)s-%(version)s_x86_64_linux_gnu4.5.bz2', {'rename': '%(namelower)s-%(version)s.tar.bz2' )] and similar for things like url |
@stdweird: The Or are you saying that in that case we should use the rename feature, and then let |
i know it has a vlaid use case now, but if we have the rename possibility, we should rename to proper extenesion or, even better, use some file-like tools to determine what is in the source, irrespective of the the extension. if we rename, the lookup to the local cache should be with the proper renamed name (or both should be tried) |
Point taken, if we have But, we'll have to deprecate is (unless it's not a part of v1.1 yet, I'd have to check, but I think it is), not just remove it... |
Just ran into this issue today with Picard. Older versions had a version number in the downloaded file, but it's distributed as of version 2.5.0 without version number :(. Combined with a lack of a checksum for the sources (checksums are optional :o) this resulted in an easyconfig for version 2.9.x that happily found a previously downloaded version 2.7.x and installed that as 2.9.x: oops. Robust handling of unversioned sources would be welcome and related I wouldn't mind if checksum become mandatory for easyconfigs ;)... |
@pneerincx Making checksums mandatory is something we're considering, but it'll be quite an effort. Also, even though it would have helped to avoid this problem occurring silently, it's still a good idea to rename sources that are not versioned. This is something we should also support in EasyBuild, maybe with a syntax like: sources = ['picard.zip:picard-%(version)s.zip'] EasyBuild would then first go looking for Of course, this should be accompanied with a checksum to avoid downloading and renaming the wrong zip file... |
follow up in #2223 which adds support for renaming sources on download |
We need a proper way to handle software packages that feature non-versioned source files (e.g.
mytool.tar.gz
).The text was updated successfully, but these errors were encountered: