Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build manifest to include JOB_NAME #1686

Closed
kavilla opened this issue Mar 2, 2022 · 5 comments
Closed

Build manifest to include JOB_NAME #1686

kavilla opened this issue Mar 2, 2022 · 5 comments
Labels
enhancement New Enhancement

Comments

@kavilla
Copy link
Member

kavilla commented Mar 2, 2022

Is your feature request related to a problem? Please describe

Origin: #1653 (comment)

The name of the job that triggers integ-test pipeline along with the build ID is used to figure out which distribution to download and run tests against. This requires hardcoding that value.

Describe the solution you'd like

I propose we include the job name with the build manifest. For example:

---
schema-version: '1.2'
build:
  name: OpenSearch Dashboards
  version: 2.0.0
  platform: linux
  architecture: x64
  distribution: tar
  id: '1417'
  job_name: 'distribution-build-opensearch-dashboards'
components:
  - name: OpenSearch-Dashboards
    repository: https://github.com/opensearch-project/OpenSearch-Dashboards.git
    ref: main
    commit_id: 7cb8297622788b7c074cea1f92e4c2ca9f8f5f5d
    artifacts:
      dist:
        - dist/opensearch-dashboards-min-2.0.0-linux-x64.tar.gz
    version: 2.0.0.0
  - name: functionalTestDashboards
    repository: https://github.com/opensearch-project/opensearch-dashboards-functional-test.git
    ref: main
    commit_id: 24fba9c0cf413db6c16ae4117d0502aae283a019
    version: 2.0.0.0

That way when passing the URL of the build manifest to the test job and downloading the manifest. I can get the value of build.job_name and pass that to run runIntegTestScript.

Describe alternatives you've considered

An assumption could be that the URL passed is the same format https://ci.opensearch.org/ci/dbc/**JOB_NAME**/1.3.0/latest/linux/arm64/builds/opensearch/manifest.yml, and we can do the logic but we have access to the job name while building the build manifest and I think there are no guarantees that the format of this URL will stay the same.

Can also just pass another input in the jenkins params that takes the build job name.

Additional context

I believe adding the job name to the build manifest could provide insight in the future if we have ever change the URLs or if a job was triggered and end up dumping the artifacts to another bucket that does not equal the expect name. Not sure if possible in the current system but if someone hard codes in the Playground it could happen.

@kavilla kavilla added enhancement New Enhancement untriaged Issues that have not yet been triaged labels Mar 2, 2022
@abhinavGupta16 abhinavGupta16 removed the untriaged Issues that have not yet been triaged label Mar 8, 2022
@abhinavGupta16
Copy link
Contributor

[triage] We don't want to add the job_name in the build manifest, since the manifest should only be responsible for building the artifact. We can add job-name as a part of Jenkins pipeline/pipeline.

@kavilla

@bbarani
Copy link
Member

bbarani commented Mar 14, 2022

@kavilla @tianleh Can you please review the triage comments and update this issue accordingly?

kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 14, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 14, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 15, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
@kavilla kavilla linked a pull request Mar 15, 2022 that will close this issue
1 task
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 15, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 15, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 16, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 16, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 17, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 17, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 17, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 17, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit to kavilla/opensearch-build that referenced this issue Mar 19, 2022
While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch.

Issues resolved:
opensearch-project#1688
opensearch-project#1687
opensearch-project#1686

Signed-off-by: Kawika Avilla <[email protected]>
kavilla added a commit that referenced this issue Mar 21, 2022
)

* [Build][Test] OS and OSD improvements plus support for Playground

While using a child folder 'Playground' in Jenkins I had to do a number of
changes to support a successful test run. I also included the ability to
skip publishing the notification not to spam channels from builds from
for example `Playground`. Also ability to skip building docker because
from what I could tell it was publishing docker images from `Playground`
pipelines as well.

If the user attempts to pass an empty string it will skip the integ tests.

Finally, some cleanup to integ tests to use the input manifest to container image.
Also, make integ tests run in docker container for OpenSearch. Use libtester for 
detectTestDockerAgent
 
Issues Resolved
* #1688
* #1687
* #1686
* #1758

Signed-off-by: Kawika Avilla <[email protected]>
@bbarani
Copy link
Member

bbarani commented May 10, 2022

@kavilla Can we close the issue if the PR was already merged and tested?

@bbarani
Copy link
Member

bbarani commented Jul 26, 2022

@kavilla @seanneumann Any updates on this issue?

@gaiksaya
Copy link
Member

Closing this issue. Please reopen if issue still persists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New Enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants