Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RHOAIENG-12621] Update the updater github action to the latest branch #716

Merged
merged 4 commits into from
Oct 4, 2024
Merged

[RHOAIENG-12621] Update the updater github action to the latest branch #716

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5ec17c1
Update of workbench images and runtime images digest updater GHAs
jstourac Sep 26, 2024
6cb9472
Small improvements for the runtime digest updater GHA.
jstourac Sep 26, 2024
c34be4d
Update GHA for digest updaters to be closer the content in downstream
jstourac Oct 4, 2024
e81b49f
DONT COMMIT this one
jstourac Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 119 additions & 118 deletions .github/workflows/notebooks-digest-updater-upstream.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ on: # yamllint disable-line rule:truthy
env:
DIGEST_UPDATER_BRANCH: digest-updater-${{ github.run_id }}
BRANCH_NAME: ${{ github.event.inputs.branch || 'main' }}
RELEASE_VERSION_N: 2024a
RELEASE_VERSION_N_1: 2023b
RELEASE_VERSION_N: 2024b
RELEASE_VERSION_N_1: 2024a
jobs:
initialize:
runs-on: ubuntu-latest
Expand All @@ -36,9 +36,9 @@ jobs:
# Create a new branch
- name: Create a new branch
run: |
echo ${{ env.DIGEST_UPDATER_BRANCH }}
git checkout -b ${{ env.DIGEST_UPDATER_BRANCH }}
git push --set-upstream origin ${{ env.DIGEST_UPDATER_BRANCH }}
echo ${{ env.DIGEST_UPDATER_BRANCH }}
git checkout -b ${{ env.DIGEST_UPDATER_BRANCH }}
git push --set-upstream origin ${{ env.DIGEST_UPDATER_BRANCH }}

update-n-version:
needs: [initialize]
Expand All @@ -48,8 +48,8 @@ jobs:
steps:
- name: Configure Git
run: |
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "GitHub Actions"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "GitHub Actions"

# Get latest build commit from the https://github.com/opendatahub-io/notebooks/${release_branch} using this as identifier for the latest tag name
- name: Retrive latest commit hash from the release branch
Expand All @@ -65,64 +65,65 @@ jobs:
with:
ref: ${{ env.DIGEST_UPDATER_BRANCH }}

- name: Update the param.env file
- name: Update the params.env file
run: |
PARAMS_ENV_PATH="manifests/base/params.env"

echo Latest commit is: ${{ steps.hash-n.outputs.HASH_N }} on ${{ env.RELEASE_VERSION_N }}

# Get the complete list of images N-version to update
IMAGES=$(cat "${PARAMS_ENV_PATH}" | grep "\-n=" | cut -d "=" -f 1)

for image in ${IMAGES}; do
echo "CHECKING: '${image}'"
img=$(grep -E "${image}=" "${PARAMS_ENV_PATH}" | cut -d '=' -f2)
registry=$(echo "${img}" | cut -d '@' -f1)

skopeo_metadata=$(skopeo inspect --retry-times 3 "docker://${img}")

src_tag=$(echo "${skopeo_metadata}" | jq '.Env[] | select(startswith("OPENSHIFT_BUILD_NAME=")) | split("=")[1]' | tr -d '"' | sed 's/-amd64$//')
regex="^$src_tag-${{ env.RELEASE_VERSION_N}}-\d+-${{ steps.hash-n.outputs.HASH_N }}\$"
latest_tag=$(echo "${skopeo_metadata}" | jq -r --arg regex "$regex" '.RepoTags | map(select(. | test($regex))) | .[0]')
# use `--no-tags` for skopeo once available in newer version
digest=$(skopeo inspect --retry-times 3 "docker://${registry}:${latest_tag}" | jq .Digest | tr -d '"')
output="${registry}@${digest}"
echo "NEW: ${output}"
sed -i "s|${image}=.*|${image}=${output}|" "${PARAMS_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${PARAMS_ENV_PATH}" && \
git commit -m "Update images for release N via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N }}"
fi
PARAMS_ENV_PATH="manifests/base/params.env"

echo Latest commit is: ${{ steps.hash-n.outputs.HASH_N }} on ${{ env.RELEASE_VERSION_N }}

# Get the complete list of images N-version to update
IMAGES=$(grep "\-n=" "${PARAMS_ENV_PATH}" | cut -d "=" -f 1)

for image in ${IMAGES}; do
echo "CHECKING: '${image}'"
img=$(grep -E "${image}=" "${PARAMS_ENV_PATH}" | cut -d '=' -f2)
registry=$(echo "${img}" | cut -d '@' -f1)

skopeo_metadata=$(skopeo inspect --retry-times 3 "docker://${img}")

src_tag=$(echo "${skopeo_metadata}" | jq '.Env[] | select(startswith("OPENSHIFT_BUILD_NAME=")) | split("=")[1]' | tr -d '"' | sed 's/-amd64$//')
src_tag2=$(echo $src_tag | sed 's/python-3.9/python-3.11/')
regex="^$src_tag2-${{ env.RELEASE_VERSION_N}}-\d+-${{ steps.hash-n.outputs.HASH_N }}\$"
latest_tag=$(echo "${skopeo_metadata}" | jq -r --arg regex "$regex" '.RepoTags | map(select(. | test($regex))) | .[0]')
# use `--no-tags` for skopeo once available in newer version
digest=$(skopeo inspect --retry-times 3 "docker://${registry}:${latest_tag}" | jq .Digest | tr -d '"')
output="${registry}@${digest}"
echo "NEW: ${output}"
sed -i "s|${image}=.*|${image}=${output}|" "${PARAMS_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${PARAMS_ENV_PATH}" && \
git commit -m "Update images for release N via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N }}"
fi

- name: Update the commit.env file
run: |
COMMIT_ENV_PATH="manifests/base/commit.env"

echo Latest commit is: ${{ steps.hash-n.outputs.HASH_N }} on ${{ env.RELEASE_VERSION_N }}
# Get the complete list of images N-1-version to update
COMMIT=$(grep "\-n=" "${COMMIT_ENV_PATH}" | cut -d "=" -f 1)

for val in ${COMMIT}; do
echo "${val}"
sed -i "s|${val}=.*|${val}=${{ steps.hash-n.outputs.HASH_N }}|" "${COMMIT_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${COMMIT_ENV_PATH}" && \
git commit -m "Update image commits for release N via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N }}"
fi
COMMIT_ENV_PATH="manifests/base/commit.env"

echo Latest commit is: ${{ steps.hash-n.outputs.HASH_N }} on ${{ env.RELEASE_VERSION_N }}
# Get the complete list of commits N-version to update
COMMIT=$(grep "\-n=" "${COMMIT_ENV_PATH}" | cut -d "=" -f 1)

for val in ${COMMIT}; do
echo "${val}"
sed -i "s|${val}=.*|${val}=${{ steps.hash-n.outputs.HASH_N }}|" "${COMMIT_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${COMMIT_ENV_PATH}" && \
git commit -m "Update image commits for release N via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N }}"
fi

update-n-1-version:
needs: [initialize, update-n-version]
Expand All @@ -132,8 +133,8 @@ jobs:
steps:
- name: Configure Git
run: |
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "GitHub Actions"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "GitHub Actions"

# Get latest build commit from the https://github.com/opendatahub-io/notebooks/${release_branch} using this as identifier for the latest tag name
- name: Retrive latest commit hash from the release branch
Expand All @@ -151,62 +152,62 @@ jobs:

- name: Update the param.env file
run: |
PARAMS_ENV_PATH="manifests/base/params.env"

echo Latest commit is: ${{ steps.hash-n-1.outputs.HASH_N_1 }} on ${{ env.RELEASE_VERSION_N_1 }}

# Get the complete list of images N-1-version to update
IMAGES=$(cat "${PARAMS_ENV_PATH}" | grep "\-n-1=" | cut -d "=" -f 1)

for image in ${IMAGES}; do
echo "CHECKING: '${image}'"
img=$(grep -E "${image}=" "${PARAMS_ENV_PATH}" | cut -d '=' -f2)
registry=$(echo "${img}" | cut -d '@' -f1)

skopeo_metadata=$(skopeo inspect --retry-times 3 "docker://${img}")

src_tag=$(echo "${skopeo_metadata}" | jq '.Env[] | select(startswith("OPENSHIFT_BUILD_NAME=")) | split("=")[1]' | tr -d '"' | sed 's/-amd64$//')
regex="^$src_tag-${{ env.RELEASE_VERSION_N_1}}-\d+-${{ steps.hash-n-1.outputs.HASH_N_1 }}\$"
latest_tag=$(echo "${skopeo_metadata}" | jq -r --arg regex "$regex" '.RepoTags | map(select(. | test($regex))) | .[0]')
# use `--no-tags` for skopeo once available in newer version
digest=$(skopeo inspect --retry-times 3 "docker://${registry}:${latest_tag}" | jq .Digest | tr -d '"')
output="${registry}@${digest}"
echo "NEW: ${output}"
sed -i "s|${image}=.*|${image}=${output}|" "${PARAMS_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${PARAMS_ENV_PATH}" && \
git commit -m "Update images for release N-1 via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N_1 }}"
fi
PARAMS_ENV_PATH="manifests/base/params.env"

echo Latest commit is: ${{ steps.hash-n-1.outputs.HASH_N_1 }} on ${{ env.RELEASE_VERSION_N_1 }}

# Get the complete list of images N-1-version to update
IMAGES=$(grep "\-n-1=" "${PARAMS_ENV_PATH}" | cut -d "=" -f 1)

for image in ${IMAGES}; do
echo "CHECKING: '${image}'"
img=$(grep -E "${image}=" "${PARAMS_ENV_PATH}" | cut -d '=' -f2)
registry=$(echo "${img}" | cut -d '@' -f1)

skopeo_metadata=$(skopeo inspect --retry-times 3 "docker://${img}")

src_tag=$(echo "${skopeo_metadata}" | jq '.Env[] | select(startswith("OPENSHIFT_BUILD_NAME=")) | split("=")[1]' | tr -d '"' | sed 's/-amd64$//')
regex="^$src_tag-${{ env.RELEASE_VERSION_N_1}}-\d+-${{ steps.hash-n-1.outputs.HASH_N_1 }}\$"
latest_tag=$(echo "${skopeo_metadata}" | jq -r --arg regex "$regex" '.RepoTags | map(select(. | test($regex))) | .[0]')
# use `--no-tags` for skopeo once available in newer version
digest=$(skopeo inspect --retry-times 3 "docker://${registry}:${latest_tag}" | jq .Digest | tr -d '"')
output="${registry}@${digest}"
echo "NEW: ${output}"
sed -i "s|${image}=.*|${image}=${output}|" "${PARAMS_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${PARAMS_ENV_PATH}" && \
git commit -m "Update images for release N-1 via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N_1 }}"
fi

- name: Update the commit.env file
run: |
COMMIT_ENV_PATH="manifests/base/commit.env"

echo Latest commit is: ${{ steps.hash-n-1.outputs.HASH_N_1 }} on ${{ env.RELEASE_VERSION_N_1 }}
# Get the complete list of images N-1-version to update
COMMIT=$(grep "\-n-1=" "${COMMIT_ENV_PATH}" | cut -d "=" -f 1)

for val in ${COMMIT}; do
echo "${val}"
sed -i "s|${val}=.*|${val}=${{ steps.hash-n-1.outputs.HASH_N_1 }}|" "${COMMIT_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${COMMIT_ENV_PATH}" && \
git commit -m "Update image commits for release N via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N }}"
fi
COMMIT_ENV_PATH="manifests/base/commit.env"

echo Latest commit is: ${{ steps.hash-n-1.outputs.HASH_N_1 }} on ${{ env.RELEASE_VERSION_N_1 }}
# Get the complete list of images N-1-version to update
COMMIT=$(grep "\-n-1=" "${COMMIT_ENV_PATH}" | cut -d "=" -f 1)

for val in ${COMMIT}; do
echo "${val}"
sed -i "s|${val}=.*|${val}=${{ steps.hash-n-1.outputs.HASH_N_1 }}|" "${COMMIT_ENV_PATH}"
done

if [[ $(git status --porcelain | wc -l) -gt 0 ]]; then
git fetch origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git pull origin ${{ env.DIGEST_UPDATER_BRANCH }} && \
git add "${COMMIT_ENV_PATH}" && \
git commit -m "Update image commits for release N-1 via ${{ env.DIGEST_UPDATER_BRANCH }} GitHub action" && \
git push origin ${{ env.DIGEST_UPDATER_BRANCH }}
else
echo "There were no changes detected in the images for the ${{ env.RELEASE_VERSION_N_1 }}"
fi

open-pull-request:
needs: [update-n-version, update-n-1-version]
Expand All @@ -221,7 +222,7 @@ jobs:
uses: repo-sync/pull-request@v2
with:
source_branch: ${{ env.DIGEST_UPDATER_BRANCH }}
destination_branch: ${{ env.BRANCH_NAME}}
destination_branch: ${{ env.BRANCH_NAME }}
github_token: ${{ secrets.GITHUB_TOKEN }}
pr_label: "automated pr"
pr_title: "[Digest Updater Action] Update Notebook Images"
Expand All @@ -233,4 +234,4 @@ jobs:
- `manifests/base/params.env` file with the latest updated SHA digests of the notebooks (N & N-1).
- `manifests/base/commit.env` file with the latest commit (N & N-1).

:exclamation: **IMPORTANT NOTE**: Remember to delete the ` ${{ env.DIGEST_UPDATER_BRANCH }}` branch after merging the changes
:exclamation: **IMPORTANT NOTE**: Remember to delete the `${{ env.DIGEST_UPDATER_BRANCH }}` branch after merging the changes
Loading