Convert freestyle gradle check to Jenkinsfile and migrate to public jenkins

[peterzhuamazon]

Is your feature request related to a problem? Please describe

Convert freestyle gradle check to Jenkinsfile.

Describe the solution you'd like

#74 (comment)

Describe alternatives you've considered

No response

Additional context

No response

[peterzhuamazon]

Seems like builder does not support pipeline syntax as of yet.
jenkinsci/ghprb-plugin#719

[peterzhuamazon]

This probably help:
https://stackoverflow.com/questions/36850485/how-to-trigger-a-jenkins-2-0-pipeline-job-from-a-github-pull-request/45121585#45121585

[dblock]

I labeled this 1.3.0 because OpenSearch gradle check needs to use the same JDK as defined in the .properties file in the branch.

[reta]

@peterzhuamazon I think it is possible to use Jenkinsfile with Github Pull Request builder [1]. At least, when pipeline is created, there is an option to configure Github Pull Request builder and Jenkins file from SCM (I believe it could be set to the base branch like 1.x / main or use the pull request one, have to double check that). Have you tried to configure pipeline with Github Pull Request builder already? If not, I could give you a hand here.

https://medium.com/openwhisk/how-to-make-jenkins-pipeline-jobs-triggered-by-pull-requests-for-apache-projects-2a526f0eb366

[peterzhuamazon]

@peterzhuamazon I think it is possible to use Jenkinsfile with Github Pull Request builder [1]. At least, when pipeline is created, there is an option to configure Github Pull Request builder and Jenkins file from SCM (I believe it could be set to the base branch like 1.x / main or use the pull request one, have to double check that). Have you tried to configure pipeline with Github Pull Request builder already? If not, I could give you a hand here.

https://medium.com/openwhisk/how-to-make-jenkins-pipeline-jobs-triggered-by-pull-requests-for-apache-projects-2a526f0eb366

Thanks @reta I will take a look at this.
Currently a bit busy with new releases and RPM creation.

Thanks.

[peterzhuamazon]

Although ghprb plugin is usable in pipeline workflow, it cannot be saved in the Jenkinsfile itself for backup. However, since it is very easy to setup, this can be a way for later.

[reta]

That is correct, thank you for update.

[bbarani]

@peterzhuamazon @gaiksaya Is this issue still valid?

[peterzhuamazon]

We can wait until the public jenkins is live as ghprb settings would be different to use push instead of pull mode.

[peterzhuamazon]

Issues List:

• [BUG] 'Increment Version' workflow doesn't add new 1.x BWC Version for 2.0 and 2.x branches OpenSearch#3218 (comment)
  • Related: Allow removing the "To honour the JVM settings for this build a new JVM will be forked" message gradle/gradle#11517
• (temp unblock by running on non-container) [BUG] testCreateEmptyDirNoPermissions when running gradle check in docker container OpenSearch#3674
  • Related: [BUG] org.opensearch.monitor.os.OsProbeTests > testLogWarnCpuMessageOnlyOnes FAILED OpenSearch#2592
• [BUG] Gradle Check failed on RPM signature check OpenSearch#3689

[peterzhuamazon]

Pre-requisite PR:

• Add markup formatter as safe/html disable syntax highlighting to allow html opensearch-ci#138

20220623:

• Build Repo: Add gradle check jenkins workflow on public jenkins #2249
• CI Repo: Add Ubuntu Single Host for Gradle Check opensearch-ci#149

20220627:

• Make sure gradle check is using c518xlarge runner #2270
• Raise the gradle check runner to have c518xlarge as in Fork Jenkins opensearch-ci#150
• Add gradle check test for github workflows (Testing again) OpenSearch#3717

20220628:

• Update github action gradle-check to use pull_request_target for accessing token OpenSearch#3728
• Set concurrent build limit for gradle check to 50 #2274

20220706:

• Switch runner of gradle check to c524xlarge for more stable runs #2298
• New cmds reduce AL2 failure and increase gradle check to c524xlarge runner opensearch-ci#151
• Add jdk14 for gradle check 1.x version #2300
• Add jdk14 as it is required by OS 1.x gradle check opensearch-ci#152

20220707:

• Ignore backport / autocut / dependabot branches for gradle checks on push OpenSearch#3816

[peterzhuamazon]

Some explanations to the questions I get:

1. Why do we set 130min for Github Actions timeout?
   Because gradle check jenkinsfile have a 120min timeout, once jenkins abort the run we need extra time for github action to retrieve the information, output the status, cleanup runners, etc. Therefore, adding 10min extra to the github actions timeout to be 130min to be safe.

2. Why do you set 7200seconds timeout within the github actions shell call?
   The jenkins workflow can have zombie process or stuck process during the run, therefore, we need a mechanism to actively curl Jenkins for run information as well as timeout in case Jenkins failed to do so. This 7200seconds value is matching jenkins 120min timeout value.

3. Why are we use agent node directly without using any containers?
   Jenkins gradle check requires a lot of resources from the runner, having a container would definitely impact the performance of the runner. Moreover, gradle check would spawn containers with docker-compose during the run, and running container in container is messy to manage and make it run correctly. Therefore, we directly run the gradle check workflows in agent node.

[peterzhuamazon]

@reta Please be aware Gradle Check now does not need start gradle check or rebasing to restart. You can just re-run it as any other github actions.

Thanks.

[peterzhuamazon]

New Bug per @mch2 as we still need jdk14 for gradle check.

* What went wrong:
Execution failed for task ':distribution:bwc:bugfix:buildBwcDarwinTar'.
> JAVA14_HOME required

[peterzhuamazon]

Andrew Improvement:

• Improve output on gradle check timeout #2913

[peterzhuamazon]

Reta add JDK19:

• Support JDK-19 builds on Jenkins #2808