Feature/covalent

[satyaog]

milabench cloud --setup

It creates a system config file and takes a target cloud platform with --run-on.

This starts a local covalent server which is used to manage python code that will be executed on the remote. For now this is only somewhat useful since milabench is mostly using ssh commands anyway and it would take a bit of time to refactor the pipeline I think to instead use the covalent interface to run code. I think this could be an interesting approach but it's a nice to have for now.

So milabench cloud --setup setup the remote and install basic stuff on it like the correct python version (necessary to ensure good serialization/deserialization of python objects between the local and remote machine), pip and venv. venv is used to separate the covalent env and milabench env which have incompatible package requirements versions (sqlalchemy caused problems). On this is done , the covalent server becomes useless

Then system config file should be used in the install, prepare and run commands. In those commands it creates a new standalone config for the tests that will be executed and copies it to the remote before the rest of the pipeline is executed.

At the end of the run command the results are copied to the local machine to allow the generation of a report

At the very end, milabench cloud --teardown should be used to release the cloud resources. The --all argument will release all resources of a target cloud platform specified with --run-on.

milabench report --push

Push the results to a reports branch which as well stores the status svg and summary

Example of reports : #210

[satyaog]

I failed to make this work to be able to push to the reports branch

Suggested change

	with:
	token: ${{ secrets.REPORTS_PAT }}

[satyaog]

There's probably a better way to do this with actions/checkout@v3

[satyaog]

This should probably be simplified, I don't think we'll use the complex features on configs and args paths

[satyaog]

This might not be needed if we set the link with markdown instead of the svg itself to redirect to the summary page

[satyaog]

I kind of feel that using the tag as is would be better even if there's a chance I think it will duplicate the versions? Let me know

Suggested change

	tag = ([
	t.name
	for t in _repo.tags
	if meta[0]["milabench"]["tag"].startswith(t.name)
	] or [meta[0]["milabench"]["tag"]])[0]
	tag = meta[0]["milabench"]["tag"]

[Delaunay]

There is this

I think it would be clearer if we use "branch-commit" because the tag means actually nothing in most cases.
For example, in master the latest tag is still v0.0.6 but stable latest tag is v0.0.10.

[satyaog]

The workflow here cannot access repo's secrest even if I'm an admin. Seams related to this explanation https://stackoverflow.com/a/58740879. Closing in favour of #217 which is a local branch