Verify dependencies between plugins in CI

[dblock]

Is your feature request related to a problem? Please describe.

Coming from #414 (comment), verify dependencies (versions of) between plugins. For example several plugins depend on job-scheduler.

[peternied]

[Triage] @dblock This looks resolved, please reopen if this isn't the case.

[dblock]

We check that plugins depend on the correct version of OpenSearch in https://github.com/opensearch-project/opensearch-build/blob/main/bundle-workflow/src/ci_workflow/ci_check_gradle_dependencies_opensearch.py. This issue is about similarly checking that the version of other dependencies is correct.

[zelinh]

[Triage] @dblock Are you still looking into this issue or do you have a plan to resolve this?

[dblock]

I am not looking into it, but I believe it's still a nice to have feature.

[prudhvigodithi]

We can have the build process to look for dependency logic (depends_on), something like as follows

Manifest: opensearch-2.4.0.yml

components:
  - name: OpenSearch
    repository: https://github.com/opensearch-project/OpenSearch.git
    ref: 2.x
    checks:
      - gradle:publish
      - gradle:properties:version
  - name: common-utils
    repository: https://github.com/opensearch-project/common-utils.git
    ref: 2.x
    platforms:
      - linux
      - windows
    checks:
      - gradle:publish
      - gradle:properties:version
    depends_on:
      - OpenSearch
  - name: alerting
    repository: https://github.com/opensearch-project/alerting.git
    ref: '2.x'
    platforms:
      - linux
      - windows
    checks:
      - gradle:properties:version
      - gradle:dependencies:opensearch.version: alerting

With depends_on logic, the build process should 1st build the depends_on component, example from above case it should start with OpenSearch and proceed to common-utils, if in any case if OpenSearch fails it shouldn't even build common-utils and directly go to alerting, since alerting does not have any depends_on condition it should continue to build.
This way the dependency is handled and other non dependent components does not fail because of some other component dependency failed, having this logic we can even on board all the plugins in one go to the manifest (build yaml) and need not raise any separate PR's.
@bbarani @dblock WDYT?

[dblock]

You could also assume that everything depends on whatever is above to start, without introducing depends_on. So incrementing from 1.3.5 to 1.3.6 you would iterate through that list and make draft PRs and turn them into non-draft PRs 1-by-1 in order when the previous one goes green.

I think maybe the simplest solution would be to keep running the workflow and force pushing daily, until the PR is merged.