Publish plugin zips to maven repo

[prudhvigodithi]

Description

With existing setup the plugin jars are only published to maven repo, but not the generated plugin zips.

Related META issue

opensearch-project/opensearch-build#1916

Current Behavior

Existing all maven publications target the jar files to publish to maven, but not the generated distribution zips.

Expected Behavior

Along with plugin jars, plugin zips should also be published to maven repo with right maven coordinates, so user can download these plugins using dependancy model.

Steps To Reproduce

1. Run the gradle assemble task that outputs jar and zips.
2. Using build setup the jars are published to local staging maven repo.
3. Zip's are just added to distribution's folder during build runtime.
4. Finally once build is done, just the jar files are added to local maven repo and updates to the manifest file, but not the zips.

Proposed solution

A custom gradle plugin solution that can create the publication task for the zips to get published to local staging maven repo with right maven coordinates.
Add/Configure the custom gradle plugin openseearch.pluginzip to the gradle project and test the generated plugin zip's to upload to local maven repo.
More details on how to use the Plugin:
https://github.com/opensearch-project/opensearch-plugins/blob/main/BUILDING.md#opensearchpluginzip

[bbarani]

@manishdev-amzn This change is targeted for 2.0 release and it should be a quick change. Can you please make the necessary changes as listed on the document to support publishing your zips to Maven?

[ankitkala]

ack

[ankitkala]

@bbarani Looking at the main issue, this is still not done for most plugins. Since we're almost at the the code freeze date, I doubt whether we're planning to roll this out in this release itself. Let me know if you think otherwise.

[prudhvigodithi]

@bbarani Looking at the main issue, this is still not done for most plugins. Since we're almost at the the code freeze date, I doubt whether we're planning to roll this out in this release itself. Let me know if you think otherwise.

Hey @ankitkala, the mechanism is same for all plugins, just to add the pluginzip plugin to build.gradle file, few plugins does not still use bundlePlugin task (The main task that outputs the zip file), but I would suggest to go ahead with cross-cluster-replication, as the process of adding pluginzip is quite easy.

[ankitkala]

Thanks, I was able to test it out and raise the PR. Since validatePluginZipPom gradle task is failing on 2.0, we'll merge it with 2.1.

I have the PR out but we'll need to wait till we bump up the version to 2.1: https://github.com/opensearch-project/cross-cluster-replication/pull/398/files

[prudhvigodithi]

Hey @ankitkala thanks, it should also work for 2.0.0-SNAPSHOT, can you try ?

[ankitkala]

i had tried with 2.0.0-SNAPSHOT. same issue.

[prudhvigodithi]

can you check this 2.0.0-SNAPSHOT, 2.0.0-SNAPSHOT should have the plugin opensearch.pluginzip
@ankitkala

[ankitkala]

The PR linked above is on 2.0 and failing CI: https://github.com/opensearch-project/cross-cluster-replication/pull/399/files

[prudhvigodithi]

Hey @ankitkala can you please add more details on how its failing?, can you double check the build-tools version that depends on OpenSearch version?
example: https://github.com/opensearch-project/ml-commons/blob/2.0/plugin/build.gradle#L57-L78

[ankitkala]

You can see the failures and the changes done in the PR itself: https://github.com/opensearch-project/cross-cluster-replication/pull/399/files
Here are the failures for your reference: https://github.com/opensearch-project/cross-cluster-replication/runs/6932347344?check_suite_focus=true

[prudhvigodithi]

This could be the issue https://github.com/opensearch-project/cross-cluster-replication/blob/main/build.gradle#L39 can you try with 2.0.1-SNAPSHOT ?
@ankitkala

[ankitkala]

Pushed the changes for 2.1+