Archive easyblocks along with easyconfig

[ocaisa]

Fixes #2633

[boegel]

@ocaisa A bit of logging wouldn't hurt here imho...

[ocaisa]

Logging was being shut down too early, meaning all logging below this was lost

[ocaisa]

@boegel Cleaned up and good to go now

[ocaisa]

@boegel I need some help with how to unit test this, not sure what's required to get a proper app instance

[ocaisa]

...or perhaps what is already tested is good enough?

[boegel]

@ocaisa Once you have a parsed easyconfig (for example for toy-0.0.eb in the tests, you can just create an easyblock instance like MyEasyBlock(parsed_easyconfig) (e.g. Toy(toy_ec)).

But I guess there's no strong need to test this function in isolation, since it's always being called. Just make sure whatever it produces it being checked, which you're already doing I think.

[boegel]

@ocaisa I'll check why @boegelbot isn't picking up on this, but there's a failing test:

ERROR: Test toy build produces expected reproducability files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/453993240/lib/python2.6/site-packages/easybuild_framework-3.7.2.dev0-py2.6.egg/test/framework/toy_build.py", line 1557, in test_reproducability
    for framework_easyblock in FRAMEWORK_EASYBLOCK_MODULES:
NameError: global name 'FRAMEWORK_EASYBLOCK_MODULES' is not defined

[ocaisa]

To address easybuilders/easybuild-easyblocks#1574 , should I be doing the reproducability dump before app.run_all_steps()? That should save me from changes made to the instance by the easyblocks while preserving everything else. I'll have to do it in the temporary space and then move it if we succeed with the build.

[ocaisa]

@boegel After a quick test, the approach in the last comment looks like it would solve easybuilders/easybuild-easyblocks#1574 but I would like your opinion before I implement.

[ocaisa]

Ok, I've implemented the move. This covers the case where modifications come from the command line (or configuration) and dependency resolution but doesn't cover what hooks might do. I think I'd also like to archive those but I don't know how to access them.

[ocaisa]

Urgh, I already see a problem with the hook example https://easybuild.readthedocs.io/en/latest/Hooks.html?highlight=hooks#example-hook-to-inject-a-custom-patch-file
I guess patch files should also be copied over to the reprod folder?

[ocaisa]

I've reverted this back to simply also dumping the used easyblocks. Fixing the deeper problems of this approach (as in easybuilders/easybuild-easyblocks#1574) is for another PR.

[boegel]

lgtm, thanks @ocaisa!