Question: Drake executing stages on remote nodes

[kavink]

I am trying to understand what is best way to use drake when the requirement is being able to run drake on different remote nodes. let me explain with an usecase.

Say we have one pipeline, with multiple drake stages, only functionality for retrigger we use is forcing builds by using tags +^%stagewhichfailed . This works well when its just one pipeline.

but when we need to combine multiple pipelines together, I cannot figure out how to execute them on different nodes. i.e. When pipeline A finishes , Start pipeline B, C, D , Then when pipeline BCD finish run pipeline Z.

One ugly way to do this have jenkins job for each pipeline, Then have a drake pipeline which triggers Jenkins job. But this method makes it difficult to restart the pipeline , Say 2 stages in B and D failed, How do i restart them , etc.

But i am trying to find a much more elegant solution using drake. one way i can think of if i can tell drake to execute a stage or drake file on remote node, then maybe it will help.

Also any ideas how to do complex meta drake workflows joining different workflows, but keeping them separate instead of making one large drake, as each drake workflow can run independently too,

And what i want to avoid is having 2 copies of drake workflows one containing all pipelines and other being each separate workflow.

Appreciate any help i can get on this.

[dirtyvagabond]

Hi @kavink , apologies for the delayed response. it's true that Drake doesn't provide out-of-the-box solutions for the problems you describe. i guess your described problems can be generalized as "trying to manage multiple distributed processes"? Have you made any progress since filing this ticket?

[kavink]

@dirtyvagabond Haven't made a lot of progress, maybe its because of my understanding of drake. But i ended up coming up with a hacky system around it . So now the only real feature i am using in drake forcing triggering by tag which then triggers different jenkins jobs, But something outside drake decides if the stage is run or skipped, Would ideally prefer to leverage as much of drake as possible. But have unblocked myself for now.