Skip to content

GSOC Ideas

Jump to bottom Edit New page
fgeorgatos edited this page Mar 26, 2013 · 2 revisions

Preamble: EasyBuild is an open source tool, which is very handy for building Scientific Computing software; as such, it is now becoming popular among HPC sites: http://hpcugent.github.com/easybuild/ HPCBIOS is a sibling attempt to define HPC User environments with largely aligning goals: http://hpcbios.readthedocs.org/en/latest/HPC_Baseline_Configuration.html

Here is a list of ideas that are desired to appear in next versions of EasyBuild, or worthy experiments: (their declared importance varies according to who you ask ;-)

Software Mixer - Low-hanging fruit

  • Software Mixer; ie. provide pathways for software among different repositories/projects:
  • .eb easyconfigs for EasyBuild (at least, use this one as target)
  • pkgsrc Makefiles
  • ports Makefiles
  • macports
  • portage
  • (...you name it...)
  • rpms/srpms/specfiles
  • deb

CUDF - Fun/Peripheral

  • Use a "formal" language to describe package info & dependencies, eg. CUDF syntax;
  • http://blog.mancoosi.org
  • http://www.mancoosi.org/papers/cbse11.pdf "MPM: a modular package manager"
  • http://arxiv.org/abs/1109.0456 "Aligning component upgrades"
  • ANSWER: may consider; TBD/postponed.
  • Provide some smart way to handle osdependencies like "tcsh" which may have no meaning on eg. MacOSX # a particularly important case are the boost-devel vs boost-dev differences (rpm vs deb conventions)
  • and for now allow escaping with --ignore-osdependencies ; See #102; FG update: this now is handled with --strict=ignore

Build easyconfigs for missing scientific software - Core development

CDASH - Infrastructure/Automation

  • This is the application of ctest/dash while building software; sounds trivial, but there are >500 modules that now get built automatically, so there is some work here.
  • dependencies make it interesting.
  • Next step would be to combine it with some benchmarking graphs

EasyDoc - Infrastructure/Automation

  • Can we automate the generation of documentation for Scientific Software?
  • Example: http://eniac.cyi.ac.cy/display/UserDoc/GROMACS
  • The idea is about integrating EasyBuild with 3rd party wikis/docs/sites, preferably using markup-intelligent tools like pandoc.
  • Caveat: we need to dynamically import information from the running system(s)!

env-modules cmd extension - Fun/Peripheral:

  • Extend one of the "environment-modules" implementations (tcl/c/lmod)
  • supply features: hierarchies, categories, keywords, whatis, module -d
  • extreme thought: yes, this is an extreme thought: consider module search/install extensions!

Extend auto-generated module namespace - Core development

  • Give freedom to define the namespace format (eg. original Package names OR the lower case version)
  • allow to export namespace in both UpperCase and LowerCase conventions # a post-install hook? TBD
  • this is tracked here: https://github.com/hpcugent/easybuild-framework/issues/173
  • Support more configurable formats of version strings (check below, how other HPC sites do it)
  • nested levels (ie. split version string in sub sections)
  • provide hooks for gnu/gcc/intel/pgi etc strings
  • IDEA: For now, supply a "custom_namespace_function" variable, to define eg. modulename.lower();

Integrate hashdist/EasyBuild/HPCBIOS

  • all of the above attack the reproducibility aspects from different, orthogonal angles
  • nobody has tried yet to synthesize all idea into one unit... TBD!
Add a custom footer
Add a custom sidebar
Clone this wiki locally