Separate out "job" concept from "workflow" concept

[junjun-zhang]

The current workflow refers to both workflow and job (an instance of a workflow run), it's somewhat confusing. This pull request introduces the job concept, workflow remains and only refers to workflow definition.

The changes were pretty much string replacements, ie, workflow to job (in some cases workflow job) when appropriate.

[junjun-zhang]

The Job concept has actually been used.

[junjun-zhang]

It could simply be just /jobs instead of /workflow-jobs. I personally don't see any problem either way.

[junjun-zhang]

As there is likely a queued period before the job gets run, so submit seems better reflect the situation.

[psafont]

I don't know if job is the best choice, it's usually used as a synonym for task, which might confuse some people who are also using TES.

Unfortunately I haven't found many alternatives that aren't already used in the field (such as stream) and without sounding weird: a (workflow) run, a (workflow) activity, a (workflow) flux.

In any case job is better than using workflow for 2 things at the same time.

[pditommaso]

IMO this is even more confusing. As already said, job is commonly used as a synonym for a workflow task.

Better alternatives could be:

• Run
• Execution
• Instance
• Workflow-run
• Workflow-exec

Another option could be to keep workflows for the actual running instances and introduce workflow-def for the definition.

[geoffjentry]

A couple of years ago the then Containers & Workflows group hashed out how we wanted to refer to these things and the results are here.

The claim was that APIs from this working group would use those terms, so now we don't need to argue about the terminology here :)

[denis-yuen]

@geoffjentry Re-reading that link, it says that an instance of a Task is a TaskInstance but it seems to punt on what an instance of a workflow is.

Out of the proposals so far, I vote for @pditommaso 's "workflow-run" as an instance of a "workflow" mostly for old times sake. "Job" is ok (especially now that Google JES seems to have been retired), but it isn't obvious to me from the naming that a "job" is an instance of a "workflow".

[geoffjentry]

@denis-yuen I noticed that too, i'm 99% sure the decision was to just continue with the instance terminology. I personally never liked the instance terminology but that was what was decided. I suspect I just never got around to updating the doc at the time.

If we're going to deviate from this, and since you brought up Google (it's influenced by them), the Biosphere job manager is referring to a job generically as anything which a user might launch. So from our parlance either a workflow or a task. The physical incarnation becomes a job. As that's likely to be a key way which users are interacting with these APIs it is worth keeping that in mind.

@denis-yuen - Google JES wasn't retired, it was just long ago renamed to the Pipelines API

[denis-yuen]

@geoffjentry I don't remember it (either way), but it sounds fine. Thanks, I meant that as short hand for "the name was retired, freeing the term job for us to use"

In other words ...
For TES, an instance of a Task, TaskInstance
For WES, an instance of a Workflow, WorkflowInstance
For presentations(?), an instance of a tool (a workflow or a task), the proposal is for that to be a "job"

[junjun-zhang]

Thanks for sharing your thoughts, @psafont, @pditommaso, @geoffjentry and @denis-yuen.

My understanding is that standardizing terminology is an integral part of the process defining any standards. As @geoffjentry pointed out this had happened for this group: here

The current situation is that we identified a potential issue in the current WES specification, ie, workflow is used for both workflow and an execution of a workflow (or a workflow run). I hope we are on the same page that we need to do something about it.

It seems the main sticking point is that: what is the best term to be used for an execution of a workflow?

Given that in the field of scientific workflow, the usage of typical terms, such as, workflow, batch, pipeline, job, task, step, work, worker, run, executor, execution, scheduler, queue, orchestrator, launcher etc in different systems has already been diversified. It is a problem on its own (or maybe it's not a problem at all). Either way, it shouldn't be something this group (GA4GH cloud work stream) needs to tackle. Of course, it is relevant to us, we should try to follow most common usage for a given term when possible.

No matter what's the final choice, I hope it's commonly agreeable that the terms must be well defined and unambiguous within the context of GA4GH workflow domain, although the usage may differ from other systems.

Briefly, here is how I see it with relevant terms put together: A workflow is a static definition of a computational process which consists of step(s) with each based on a tool. A tool is an atomic unit of computation usually as a piece of executable code (maybe binary). A workflow run (or execution) is a job and a tool run (or execution) is a task. Defining a workflow involves defining (or specifying) one or more tools with desired execution order among tools and flow of output from one step to the next as input. Workflow and tool are static. Executing (or running) a job involves executing its tasks with predefined (in workflow definition) interdependencies (ie, order of executions) among tasks. Job and task are instances of workflow and tool respectively, they represent work (yet another term, although I don't think we need to formalize it) to be done.

[geoffjentry]

Hi @junjun-zhang - as mentioned above I'm pretty sure that the Instance nomenclature was intended to cover workflows as well.

If people want to break from that pattern, I don't think that this PR is the place to have that convo. Instead it should be taken to the workstream calls

[junjun-zhang]

@geoffjentry, you mentioned never liked the instance terminology, neither do I.

But if we go with Instance, does it mean we will change workflow to workflow-instance when appropriate (I guess using the word instance along sounds weird)? For examples, GetWorkflowLog becomes GetWorkflowInstanceLog; /workflows becomes /workflow-instances; WorkflowRunId becomes WorkflowInstanceId etc.

Don't know how others think about this. It's probably good to discuss it on the workstream calls.

[geoffjentry]

@junjun-zhang My main point is that I don't think this PR is the right venue for the conversation at large.

Ultimately there's never going to be a good answer for this. Nearly every group I interact with uses different (and often contradictory) terminology and many of them feel strongly that their terminology is the correct one.

The one thing I did like about the Instance terminology was that it forced everyone to break out of their comfort zone a bit.

And ultimately I think that unwiedly endpoint/object names are fine as presumably the very extreme majority of interaction with these APIs will be programatic.

[junjun-zhang]

@geoffjentry your point is well taken. I think we still need to address the issue: workflow in the current WES specification refers to two different things. Let's discuss on the workstream call, hopefully a decision may be reached by the group.

[jaeddy]

One other perspective from this presentation about CWL, provenance, and Research Objects

They use "instance" to refer to a parameterized workflow in contrast to the workflow "definition" as a generic recipe. Workflow "run" is then the execution of a workflow instance.

As the Definitions.md file linked a few times here already mentions, "tool" can potentially refer to a task or a workflow, depending on the perspective. Perhaps then, it's best to focus on workflows and tasks, ensure that our nomenclature around those entities are consistent (e.g., some combination of definition, instance, run, job, etc.), and reserve "tool" as a higher level of abstraction for "useful unit of work."

Definitely agree this is worth discussing with the Cloud Workstream.

[geoffjentry]

re "tool", yeah - that was one where we decided the world was already in such a state of disarray that frankly there was no helping it, so punted by saying "we're not going to use it officially for the GA4GH APIs, but it can mean what one needs it to mean in the wild" sort of affair.

@briandoconnor - Do you think it's worth revisiting the nomenclature debate w/ the larger group?

[junjun-zhang]

I am not sure we could get away from the need to well define the term tool. There is a GA4GH standard for Tool Registry Service (https://github.com/ga4gh/tool-registry-service-schemas). Without defining it, how would people know what to register to a TRS and what to expect from a TRS?

In fact, there is already a definition for tool here: https://github.com/ga4gh/tool-registry-service-schemas/blob/aed90afc258136715afff378b73ce5d2c8699e7e/src/main/resources/swagger/ga4gh-tool-discovery.yaml#L5-L16.

As expressed earlier, there is no need for us to harmonize all of the common terms in the broader scientific workflow area, it's already quite messy. However, IMO, all key terms used in the GA4GH Workflow Standards need to be well defined and unambiguous within GA4GH standards. It's totally OK to have different opinions, but within a standard, key terms need to be crystal clear.

[geoffjentry]

@junjun-zhang re TRS note that Dockstore handles both workflows and individual tools (which IMO are just degenerate 1 step workflows, but YMMV)

[junjun-zhang]

Yes, @geoffjentry I noticed that in TRS tool includes both tool and workflow, and had discussion with @denis-yuen about it a while ago. Same as you, I thought a tool could be regarded as a 1 step workflow. In a sense, workflow could be used to represent both tool and workflow (although I am not suggesting that).

The way I find useful to think about it is that both workflow and tool are like functions in a programming language. A workflow is more like a function calls one or more other functions, a tool is a function that does not call other functions. After all, they are functions that take input and produce output. Of course, the difference between workflow and tool is something their authors (and execution engines) need to worry about. However, from a user's perspective, workflow and tool are the same, she just invokes it with needed input and excepts result from the execution.

Just an FYI, I had a feature request in Dockstore about unifying tool and workflow: dockstore/dockstore#1103

[delagoya]

With respect, I think that the original point of this issue was slightly off-the-mark.

The WES, as it stands today, is not intended to store definitions of workflows, only to accept and track run-time requests.

• /workflows - list the submitted workflows that are being tracked
• /workflows/{id} - full information on a running workflow
• /workflows/{id}/status - quick information on a running workflow

Am I missing something?

[jaeddy]

Thanks, @delagoya. Looking at @junjun-zhang's commit I think the PR could be better titled "replace workflow concept with job concept." Indeed, WES would not be storing workflow definitions. While keeping the schema as-is would be fine for using WES in isolation, I think the issue arises when trying to working with multiple GA4GH APIs. For example, if I'm writing an orchestrator that needs to talk to both TRS and WES, I don't to deal with distinguishing between what each considers to be a "workflow."

Others can correct me, if that wasn't the intention.

[geoffjentry]

@jaeddy This was my larger point. We'd already hashed out most of this terminology 2 years ago, at least in a "these are the words we use in the GA4GH APIs". My concern is that the terminology battle is always a painful one and rekindling it on the same words is just going to be a giant PITA

[junjun-zhang]

@jaeddy I do feel terminology in all four GA4GH Workflow API standards should be harmonized. workflow in WES should have exactly same meaning as in TRS. In TRS, as used here, workflow refers to workflow definition.

Within WES, workflow mostly refers to a workflow run (as @delagoya pointed out). However, there are a few places, it refers to workflow definition such as here, here, here and here. Same term is used for different concepts within WES (that is confusing to me).

[junjun-zhang]

@jaeddy it is indeed possible to replace workflow with job except for the places where workflow is meant for workflow definition (there are a few places, see links above).

It's even possible to change Workflow Execution Service (WES) to Job Execution Service (JES) (which I am afraid to suggest given the current state).

[briandoconnor]

@junjun-zhang will this shift to "WorkflowRun"? I like the idea of not including another term (Job for Workflow that's running).

[jaeddy]

Riffing on some thoughts while listening to panel discussions in Toronto...

I feel like the ToolClasses description in the TRS spec is useful for thinking about this discussion:

Describes a class (type) of tool allowing us to categorize workflows, tasks, and maybe even other entities (such as services) separately

Following that idea, we have:

tools
├── tasks: CommandLineTool [CWL], Workflow step [CWL], task [WDL]
├── workflows: Workflow [CWL], workflow [WDL]
├── services (?) (not sure what this would look like, but could be nice)
└── dbs (?)  (same as services — mostly hypothetical)

Basically, a task (for the sake of TRS) is a unit of work that has a command (or baseCommand in CWL). Emphasizing task in TRS also plays nicely with TES (I think...). While standalone tasks are probably more rare for WDL, they do exist. Even if a WDL task isn't executable independent of a workflow, having said task registered via TRS could eventually be useful for workflow composition.

Thus, TRS is still a tool registry service, but we can be more explicit about tool types. This is analogous to how bio.tools classifies entries — although they have some classes that are obviously incompatible with description in a standardized format (CWL/WDL/NFL).

In terms of the executed state of a workflow or task, I'm happy with run for both cases — e.g..:

• CWL CommandLineTool -> WES -> TaskRun
• CWL Workflow, WDL Workflow, Nextflow et al. -> WES -> WorkflowRun

The TaskRun concept would be more satisfying if the WES engine utilized TES to execute individual tasks, steps, OR single-step CWL tools (i.e., CommandLineTool), but that's a bridge we can probably cross later.

[junjun-zhang]

Survey result:

So equal split between:

• WorkflowDefinition + WorkflowRun
• Workflow + WorkflowRun
• Workflow + Job

There were no objection for choosing workflow + WorkflowRun at the Cloud Workstream meeting on Jun 25, 2018.

So closing this PR, move forward with #32