Runtime: Allow (enforce?) jobs to be lazily compiled to support Runs

[josephjclark]

Right now, when the runtime executes a workflow, it expects all the expressions (jobs) to be compiled.

The CLI will pre-compile all expressions and log about it before the runtime is invoked.

But Lightning has this notion of a Run, which is the execution of a single job within a workflow.

To properly interface with Lightning, we need compilation to run just before the expression is executed so that it can be associated with a run id. So the job starts, gets compiled, executes, and decides what to do next, all as part of a Run.

At the moment it's kind of impossible to reflect this organisation in the runtime because compilation happens before the job id is created.

This probably isn't necessary for release, but I do think it's really important to make compilation visible.

The solution I think is for the runtime to accept a compile hook, which calls out to whatever runtime manager is calling it. It probably makes sense for the CLI to use lazy compilation as well.