"Downstream" build triggering

[tjtelan]

This pattern is particularly prevalent:

Build+Deploy Library repos -> Build+Deploy Service
Where the Service pulls in newer libraries that are in development.

It would be convenient if either/or a Library could trigger a downstream build or a Service could watch upstream build.

---

As a secondary goal: We should consider what would need to be done to display the build-related relationships/ordering of a full microservice architecture.

[tjtelan]

We still need this. We should redesign the config file representation and all of the workflow re: master branch. I still think there is some loop detection that will trip us up. Going to hide the past thinking.

[tjtelan]

I started doing some of the design work for this.

A question I want to be able to answer is: What repos in the org are (explicitly) dependent on Repo X?

I also want the trigger relationship to be defined in the orb.yml. Because of the question above, I need to keep some state in the DB so I can start builds an "upstream" repo is finishing.

There needs to be some attention paid to preventing circular dependencies (at least a single level deep) so we're not perpetually building.

---

Using Orbital as an example, its config might look loosely like this:

  stages:
  [...]
  trigger_on:
    repos:
      - name: git-url-parse 
        watch-branch: main (default)
        action:
          - notify
          - build orbital>main ENV=somethung
          - build orbital>dev
      - name: git-event
        watch-branch: all
        action: build orbital>all
      - name: git-meta

Orbital would build upon the successful builds of these 3 other repos. I imagine that this would neatly fit into the build state machine in the finishing state. We would check in the DB for any triggers defined on this repo and start a build with the JobTrigger type: SUBSCRIBE_TRIGGER (current name)

Additionally, we can specify branches we'd want to trigger on, and also what kind of actions to take. Including sending notifications prior to starting builds, starting multiple builds for different branches, and starting builds with environment variables.

There are a few default behaviors that I also want to include, that will need documenting. Like the git-meta case above, where I am not very specific about watch-branch or action.

One of the bigger details I'll expand now being how changes to this trigger_on block changes behavior in Orbital.

I propose that when we add in repos, that by default, Orbital sets a canonical_branch (using the Git repo's default, unless specified or changed). When the trigger_on block changes in the canonical branch, Orbital will modify the triggers table, but otherwise no changes.

Same behavior if we change the canonical branch through Orbital.

While I've been thinking about it longer, I should point out in the documentation that this is DB-only state with a default value. (I can't think of a story where it adds clarity more than confusion if canonical_branch were in the orb.yml. How would you change the value?)

---

The DB might look like this:

The triggers table is new. I would have a row for each repo+branch combination for which there is a trigger. The repo_id_list column would have the name of a map of other repo+branch combinations to start builds.

Lastly, last_triggered_by would include the build_target id that "pulled the trigger". I am hoping that I can use this info specifically to catch some naive loops but I need to work out more of the details.

[tjtelan]

I will need this functionality in order to monitor changes to the orb config in canonical_branch.

tjtelan/git-meta-rs#4