end-to-end tests using spark-submit

[erikerlandson]

@ifilonenko pointed out some confusion that was partly involving the differences between directly invoking Client.run() and running through spark-submit.

End to end tests with spark-submit could help capture some of this

[mccheah]

The problem with testing via spark-submit is that spark-submit sets global system properties on the JVM. We could mitigate this by forking the spark-submit process.

[ifilonenko]

I have a working e2e test forking a spark-submit for Kerberos in #391 which we can mimic for future spark-submit e2e tests

[erikerlandson]

would it work better to mimic the java command that spark-submit eventually creates?