Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLID-262:WIP: Create a separate interface for rebuild catalogs #947

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sherine-k
Copy link
Contributor

Description

This should be merged after #945

PS: independently from the PR content, the test showed that the collector sends 2 catalogs for each catalog in the imagesetconfig. hence, it looks like the catalog is being rebuilt twice, and copied 4 times? This is something to get sorted after the merge with #945 (see test results below)

Fixes CLID-262

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

$ ./bin/oc-mirror --v2 --alpha-ctlg-filter -c config_logs/clid-230.yaml docker://sherinefedora:5000/clid262 --workspace file:///home/skhoury/demo

2024/11/05 16:50:28 [WARN] : ⚠️ --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/11/05 16:50:28 [INFO] : 👋 Hello, welcome to oc-mirror
2024/11/05 16:50:28 [INFO] : ⚙️ setting up the environment for you...
2024/11/05 16:50:28 [INFO] : 🔀 workflow mode: mirrorToMirror
2024/11/05 16:50:28 [INFO] : 🕵️ going to discover the necessary images...
2024/11/05 16:50:28 [INFO] : 🔍 collecting release images...
2024/11/05 16:50:28 [INFO] : 🔍 collecting operator images...
2024/11/05 16:50:30 [INFO] : 🔂 rebuilding catalog (pulling catalog image) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:50:54 [INFO] : ✅ successfully created catalog
2024/11/05 16:51:05 [INFO] : ✅ successfully pushed catalog manifest list
2024/11/05 16:51:05 [INFO] : ✅ completed rebuild catalog docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:05 [INFO] : 🔂 rebuilding catalog (pulling catalog image) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:18 [INFO] : ✅ successfully created catalog
2024/11/05 16:51:18 [INFO] : ✅ successfully pushed catalog manifest list
2024/11/05 16:51:18 [INFO] : ✅ completed rebuild catalog docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:18 [INFO] : 🔍 collecting additional images...
2024/11/05 16:51:18 [INFO] : 🔍 collecting helm images...
2024/11/05 16:51:18 [INFO] : 🚀 Start copying the images...
2024/11/05 16:51:18 [INFO] : images to copy 8
✓ 1/8 : (2s) docker://registry.redhat.io/albo/aws-load-balancer-controller-rhel8@sha256:2e0b9332a44d8d9c23e19c7accab0813a651…
✓ 2/8 : (2s) docker://registry.redhat.io/albo/aws-load-balancer-rhel8-operator@sha256:16e9ffed36107527a37713ac5bd34a7bc20f04…
✓ 3/8 : (10s) docker://registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:f11f71448986aa17abec9caadb568a6cc34ef1a7898e…
✓ 4/8 : (1s) docker://registry.redhat.io/albo/aws-load-balancer-operator-bundle@sha256:01f2ca529d2486f113bcefc9fedce6a6fd07b…
✓ 5/8 : (11s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 6/8 : (9s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 7/8 : (2s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 8/8 : (0s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:30 [INFO] : === Results ===
2024/11/05 16:51:30 [INFO] : ✅ 8 / 8 operator images mirrored successfully
2024/11/05 16:51:30 [INFO] : 📄 Generating IDMS file...
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2024/11/05 16:51:30 [INFO] : 📄 No images by tag were mirrored. Skipping ITMS generation.
2024/11/05 16:51:30 [INFO] : 📄 Generating CatalogSource file...
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2024/11/05 16:51:30 [INFO] : mirror time : 1m1.680298594s
2024/11/05 16:51:30 [INFO] : 👋 Goodbye, thank you for using oc-mirror

Expected Outcome

podman run of the generated catalog doesn' t fail
podman inspect of the generated catalog shows that cmd is serve /configs --cache-dir=/tmp/cache

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Nov 6, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Nov 6, 2024

@sherine-k: This pull request references CLID-262 which is a valid jira issue.

In response to this:

Description

This should be merged after #945

PS: independently from the PR content, the test showed that the collector sends 2 catalogs for each catalog in the imagesetconfig. hence, it looks like the catalog is being rebuilt twice, and copied 4 times? This is something to get sorted after the merge with #945 (see test results below)

Fixes CLID-262

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

$ ./bin/oc-mirror --v2 --alpha-ctlg-filter -c config_logs/clid-230.yaml docker://sherinefedora:5000/clid262 --workspace file:///home/skhoury/demo

2024/11/05 16:50:28 [WARN] : ⚠️ --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/11/05 16:50:28 [INFO] : 👋 Hello, welcome to oc-mirror
2024/11/05 16:50:28 [INFO] : ⚙️ setting up the environment for you...
2024/11/05 16:50:28 [INFO] : 🔀 workflow mode: mirrorToMirror
2024/11/05 16:50:28 [INFO] : 🕵️ going to discover the necessary images...
2024/11/05 16:50:28 [INFO] : 🔍 collecting release images...
2024/11/05 16:50:28 [INFO] : 🔍 collecting operator images...
2024/11/05 16:50:30 [INFO] : 🔂 rebuilding catalog (pulling catalog image) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:50:54 [INFO] : ✅ successfully created catalog
2024/11/05 16:51:05 [INFO] : ✅ successfully pushed catalog manifest list
2024/11/05 16:51:05 [INFO] : ✅ completed rebuild catalog docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:05 [INFO] : 🔂 rebuilding catalog (pulling catalog image) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:18 [INFO] : ✅ successfully created catalog
2024/11/05 16:51:18 [INFO] : ✅ successfully pushed catalog manifest list
2024/11/05 16:51:18 [INFO] : ✅ completed rebuild catalog docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:18 [INFO] : 🔍 collecting additional images...
2024/11/05 16:51:18 [INFO] : 🔍 collecting helm images...
2024/11/05 16:51:18 [INFO] : 🚀 Start copying the images...
2024/11/05 16:51:18 [INFO] : images to copy 8
✓ 1/8 : (2s) docker://registry.redhat.io/albo/aws-load-balancer-controller-rhel8@sha256:2e0b9332a44d8d9c23e19c7accab0813a651…
✓ 2/8 : (2s) docker://registry.redhat.io/albo/aws-load-balancer-rhel8-operator@sha256:16e9ffed36107527a37713ac5bd34a7bc20f04…
✓ 3/8 : (10s) docker://registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:f11f71448986aa17abec9caadb568a6cc34ef1a7898e…
✓ 4/8 : (1s) docker://registry.redhat.io/albo/aws-load-balancer-operator-bundle@sha256:01f2ca529d2486f113bcefc9fedce6a6fd07b…
✓ 5/8 : (11s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 6/8 : (9s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 7/8 : (2s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
✓ 8/8 : (0s) docker://registry.redhat.io/redhat/redhat-operator-index:v4.17
2024/11/05 16:51:30 [INFO] : === Results ===
2024/11/05 16:51:30 [INFO] : ✅ 8 / 8 operator images mirrored successfully
2024/11/05 16:51:30 [INFO] : 📄 Generating IDMS file...
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2024/11/05 16:51:30 [INFO] : 📄 No images by tag were mirrored. Skipping ITMS generation.
2024/11/05 16:51:30 [INFO] : 📄 Generating CatalogSource file...
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2024/11/05 16:51:30 [INFO] : /home/skhoury/demo/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2024/11/05 16:51:30 [INFO] : mirror time : 1m1.680298594s
2024/11/05 16:51:30 [INFO] : 👋 Goodbye, thank you for using oc-mirror

Expected Outcome

podman run of the generated catalog doesn' t fail
podman inspect of the generated catalog shows that cmd is serve /configs --cache-dir=/tmp/cache

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

openshift-ci bot commented Nov 6, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 6, 2024
@sherine-k
Copy link
Contributor Author

/test images

Copy link

openshift-ci bot commented Nov 6, 2024

@sherine-k: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants