Add tests for some of the external tools..

[matentzn]

As per slack between @gouttegd and @Clare72, its possible that as the underlying architecture of the ODK are changing (system libraries etc), the behaviours of the tools change. We should consider building a manual test suite using make that invoke such tools (like oort, Konclude, scala scripts etc.).

[gouttegd]

It would be nice if we could also make it so that the test suite does not depend on fetching ontologies on remote servers… Over the weekend I have not been able to successfully run the entire test suite even once because there's always a connection time out when fetching something from purl.obolibrary.org…

Not sure if it is even feasible, though, but if not then we should at least allow the test suite to gracefully skip the current test and go on with the next one instead of aborting completely.

[Clare72]

Over the weekend I have not been able to successfully run the entire test suite even once because there's always a connection time out when fetching something from purl.obolibrary.org…

Is it chebi? I added retries/timeouts for this in dpo because I got fed up of it taking > 30 min going through the release just to fail when it had to download chebi:

mirror/chebi.owl: mirror/chebi.trigger
	curl -L http://purl.obolibrary.org/obo/chebi.owl.gz --create-dirs -o mirror/chebi.owl.gz --retry 4 --max-time 120 &&\
	$(ROBOT) convert -i mirror/chebi.owl.gz -o [email protected] &&\
	mv [email protected] $@

[matentzn]

@Clare72 this is a great idea! We should implement that by default in ODK! Would you be able to make a PR reflecting that change?

[gouttegd]

When running the tests for the ODK, from what I have seen it can be any ontology, really. Most often it seems to be purl.obolibrary.org/obo/ro/annotations.owl, but sometimes it's http://purl.obolibrary.org/obo/ro/bfo-axioms.owl, sometimes yet another one…

[matentzn]

I think if we get that retry logic in there, the failures will go down to a negligible amount..

[Clare72]

@matentzn I don't have write access!

This seems fairly straightforward for the last 4/5 variants of the mirror/{{ ont.id }}.owl goals, but for the {%- if ont.mirror_from %} we don't know whether it will be .owl or .owl.gz and robot needs to be given the correct extension. I am guessing that we would need to get the filename from the ONT-odk.yaml file and give this to curl for output and robot for input, but I am not sure how to do this.

[matentzn]

gave you write access :)

[gouttegd]

As part of #475, we now have a test_odkfull_programs target that checks whether the main programs can be invoked:

% make test_odkfull_programs
Checking for ROBOT... OK
Checking for DOSDP-TOOLS... OK
Checking for OWLTOOLS... OK
Checking for AMMONITE... OK
Checking for KONCLUDE... OK
Checking for SOUFFLE... OK
Checking for JENA... OK
Checking for SPARQL... OK

This should catch the most obvious problems, such as an executable being absent, an executable of the wrong architecture (typically a x86 binary on the arm64 image), and missing shared libraries.

This will not catch more insidious problems such as missing libraries that are dlopened instead of linked or more generally missing any file that is only needed after the program has been started. We'll need functional tests specific to each program to catch such problems. I'm leaving this issue open, since having such functional tests would be nice.

[matentzn]

@gouttegd Was there a reason you didnt add test: test_odkdev_programs to the makefile? I am wondering wether the external tools should be tested in general..

[gouttegd]

No particular reason, except maybe as a reminder that those tests are not perfect (as explained in my previous comment) and that just because they pass, it doesn't mean everything is 100% guaranteed to be OK.

[matentzn]

Thank you :)