The judge needs a better sandboxing

[mquinson]

Hello,

until recently, the judge was running only one exercise before bailing out. Along with the fact that it was running in a docker configured for paranoia, this was probably the most efficient sandboxing thing that we could ever dream of.

But unfortunately, the judge needs almost 10 sec to startup, mostly spent in loading the in-memory compilers and interpreters. We could divide it by 2 or 3 by having specialized judges, loading only what it takes for one language. But still, since running a classical exercise takes only 100ms, and since the loading time takes not only IO but also 300% CPU, there is no way we can manage to have enough judges for a heavy load (such as a run of "sbt test"). Not even for a mild load (such as a few dozen students using the interface at the same time).

So recently, I changed the judge so that it runs 100 requests before leaving. This is really efficient to solve the startup time issue, but unfortunately, we don't have no sandboxing anymore (or almost). We should go for more java-specific sandboxing solutions, as explained in the following links:

• https://tersesystems.com/blog/2015/12/29/sandbox-experiment/
• https://tersesystems.com/blog/2015/12/22/an-easy-way-to-secure-java-applications/
• https://github.com/wsargent/sandboxexperiment
• https://www.securecoding.cert.org/confluence/display/java/SEC54-J.+Create+a+secure+sandbox+using+a+security+manager
• https://blog.jayway.com/2014/06/13/sandboxing-plugins-in-java/

If we ever go for non-JVM languages [again], then this sandboxing will not be enough. There is a plenty of solutions laying around in github, just search for "student code sandboxing" for example. Here is one of them that seem usable in our case: https://github.com/testmycode/tmc-sandbox