publish-snapshot script is publishing the first jar it finds next to a pom.

[mch2]

We have Snapshot publications that are including extra jars. These jars are incorrectly being picked up by the publish-snapshot.sh script and published with mvn deploy-file. As a result it is pushed to Sonatype as the latest publication, but is not the correct jar.

For example
org/opensearch/gradle/build-tools/1.1.0-SNAPSHOT/ publishes two jars:
build-tools-1.1.0-SNAPSHOT-test-fixtures.jar and build-tools-1.1.0-SNAPSHOT.jar

The script is picking up build-tools-1.1.0-SNAPSHOT-test-fixtures.jar and pushing under sonatype as:
/org/opensearch/gradle/build-tools/1.1.0-SNAPSHOT/build-tools-1.1.0-20210910.002130-165.jar

When the dependency is used, the incorrect jar is fetched. Specifically for plugins & build-tools, this results in the following error.

* Where:
Build file '/Users/handalm/workspace/job-scheduler/build.gradle' line: 48

* What went wrong:
A problem occurred evaluating root project 'opensearch-job-scheduler'.
> Plugin with id 'opensearch.opensearchplugin' not found.

Note - this is NOT an issue for release builds that are pushed with the stage-maven-release.sh script. That script using a separate nexus staging plugin that pushes all artifacts that are in the folder. Unfortunately we cannot use that to push snapshots.

[dblock]

Should we be rewriting this as a publish step in build-workflow?

[mch2]

I have mixed feelings on that.

I'd rather that instead of us pushing pre-built snapshot artifacts with this script that we update the build-workflow to conditionally push in component build scripts. Ex. Pass a publish flag to build.sh where artifacts may conditionally invoke a gradle script to push to the snapshot repo & the necessary credentials are provided by CI. For example the core repo has a ./gradlew publishNebulaPublicationsToSnapshotsRepo where this is possible. We would want this toggle-able so we don't break local builds.

I think we should keep the staging functionality with the external script `stage-maven-release.sh` that stages pre-built artifacts. This is because we need to promote them at a much later stage and not at build time, and there is benefit in releasing .
Edit - On second thought, we could totally put the staging script into python as a separate executable vs the shell script, but this script is a lot less brittle because it publishes the entire locally staged folder containing maven deps.

[dblock]

Generally python > bash because we can write tests.

Care to write up a separate issue about what you think we should do longest term?

[peterzhuamazon]

Generally python > bash because we can write tests.

Care to write up a separate issue about what you think we should do longest term?

Hi @mch2 have you looked at this comment from @dblock ?
Thanks.

[zelinh]

@mch2 Can you add acceptance criteria so that we could move on.

[mch2]

This issue has been resolved the best it can with #1188. I will update #1182 with an idea to remove this script entirely.