Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow refactor #588

Merged
merged 60 commits into from
Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
d5ca50b
Unify R and Python versions
ianpittwood Jul 20, 2023
d6ad8b9
Add `get-base-tags` and `get-base-args` just targets
ianpittwood Jul 20, 2023
7cea7fc
Change build-preview-webhook.yaml default to "ubuntu2204"
ianpittwood Jul 20, 2023
7efbfcd
Begin prototyping generic build/test/scan/push workflow
ianpittwood Jul 20, 2023
e229c8d
Fix just unbound variable
ianpittwood Jul 20, 2023
2056c66
Switch to composite action
ianpittwood Jul 20, 2023
140a38c
Specify shell to bash
ianpittwood Jul 20, 2023
42272e8
Use CSV list for tags
ianpittwood Jul 21, 2023
7ebf3ed
Use printf to get line sep build args
ianpittwood Jul 21, 2023
e21d99e
Debug echo build args in action
ianpittwood Jul 21, 2023
346801f
Feed build args as JSON to action
ianpittwood Jul 21, 2023
438f8f6
Add build-args input option
ianpittwood Jul 21, 2023
6688b60
Print and pass build args as block
ianpittwood Jul 21, 2023
d58cdc3
Quote block string
ianpittwood Jul 21, 2023
2095bb4
Debug build-args pass and fix outputs var
ianpittwood Jul 21, 2023
f332d8f
Try multiline string using delimiters
ianpittwood Jul 21, 2023
1b11cb3
Fix build args list for tests and fix first tag cut
ianpittwood Jul 21, 2023
6791357
Quote wrap build args input
ianpittwood Jul 21, 2023
0cff240
Echo variables for debugging
ianpittwood Jul 21, 2023
856a16f
Fix tagging OS variable
ianpittwood Jul 21, 2023
591e305
Pass OS to test command
ianpittwood Jul 21, 2023
3c05c99
Use .env file for tests
ianpittwood Jul 21, 2023
e991dc9
Add `update-quarto-versions` target
ianpittwood Jul 21, 2023
7464d0a
Restrict centos7 to older versions of R and Python
ianpittwood Jul 21, 2023
8c949bc
Trim driver suffix for centos pro builds
ianpittwood Jul 21, 2023
989ad27
Add product builds to pipeline
ianpittwood Jul 21, 2023
d80579a
Add build-prerelease.yaml replacing build-preview.yaml
ianpittwood Jul 21, 2023
b2c4316
Check that $BRANCH is not empty
ianpittwood Jul 21, 2023
b10fa8c
Update context path for preview builds
ianpittwood Jul 24, 2023
8ec90a0
Push base images on dev
ianpittwood Jul 24, 2023
685de43
Fix RSW download url for prerelease
ianpittwood Jul 24, 2023
1896be1
Temporarily push dev base builds for release
ianpittwood Jul 24, 2023
a3e4914
Add PYTHON_VERSION_JUPYTER to ci args
ianpittwood Jul 24, 2023
a09fb09
Unify `PYTHON_VERSION_JUPYTER` naming convention
ianpittwood Jul 24, 2023
e5fde0e
Write Python versions to rstudio-connect.gcfg on build
ianpittwood Jul 24, 2023
e572ed8
Default push-image to false
ianpittwood Jul 24, 2023
617628c
Remove push on base
ianpittwood Jul 24, 2023
71c8234
Debug image pushes
ianpittwood Jul 24, 2023
19e3df9
Remove any reliance on `if` evaluations
ianpittwood Jul 24, 2023
d12c3dd
Remove debugging statements
ianpittwood Jul 24, 2023
25461c5
Add manual build workflow
ianpittwood Jul 25, 2023
88b1c25
Use build-test-scan-push action for content builds
ianpittwood Jul 25, 2023
bb402a3
Trim newlines in image tags
ianpittwood Jul 25, 2023
b49a85e
Revert sweeping release image upgrades
ianpittwood Jul 25, 2023
f8e4982
Write R version to `rstudio-pm.gcfg` on build
ianpittwood Jul 25, 2023
58e6e39
Fix blindspots in Justfile update-versions
ianpittwood Jul 25, 2023
4e6e65b
Update NEWS.md docs
ianpittwood Jul 25, 2023
555f5f5
Always order tags from most specific to most generic
ianpittwood Jul 25, 2023
aa61b55
Fix permissions on rstudio-pm.gcfg
ianpittwood Jul 25, 2023
0753ccd
Revert changes for dynamic rstudio-pm.gcfg
ianpittwood Jul 25, 2023
b9ba1d5
Update Workbench for Azure ML Python versions
ianpittwood Jul 25, 2023
4dace23
Use secrets directly from action.yaml
ianpittwood Jul 25, 2023
459ef9e
Revert "Use secrets directly from action.yaml"
ianpittwood Jul 25, 2023
093e784
Free up additional space in action.yaml
ianpittwood Jul 26, 2023
9f2281b
Remove debugging step
ianpittwood Jul 26, 2023
e12963f
Retry builds once on failure
ianpittwood Jul 27, 2023
179140c
Update OS in just target doc
ianpittwood Jul 27, 2023
accd848
Update ci.Justfile target docs
ianpittwood Jul 27, 2023
58bcbc4
Extract content tags and args creation to ci.Justfile
ianpittwood Jul 27, 2023
5768cef
Note that we can't use "if:" in our action
ianpittwood Jul 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions .github/actions/build-test-scan-push/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
name: 'Build/Test/Scan/Push Image'
inputs:
context:
description: Path to the directory of the Dockerfile
required: true
type: string
os:
description: Target OS to build, the same as the extension of the Dockerfile
default: ubuntu2204
type: string
product:
description: Product being built
type: string
build-args:
description: JSON list of build args for the built image
required: true
type: string
image-tags:
description: List of tags for the built image
required: true
type: string
test-image:
description: Flag to test image once built
default: true
type: boolean
snyk-token:
description: Username for authentication with Snyk for scanning images
type: string
snyk-org-id:
description: Snyk Organization ID to publish scans to
type: string
push-image:
description: Flag to push image once built
default: false
type: boolean
ghcr-token:
description: Username for authentication with GHCR.io
required: true
type: string
dockerhub-username:
description: Username for authentication with DockerHub
required: true
type: string
dockerhub-token:
description: Username for authentication with DockerHub
required: true
type: string

runs:
using: "composite"
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Free Disk space
shell: bash
run: |
sudo rm -rf /usr/local/lib/android # will release about 10 GB
sudo rm -rf /usr/share/dotnet # will release about 20GB

- name: Login to ghcr.io
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ inputs.ghcr-token }}

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ inputs.dockerhub-username }}
password: ${{ inputs.dockerhub-token }}

- name: Build
id: image-build
uses: docker/build-push-action@v4
with:
load: true
context: ${{ inputs.context }}
file: ${{ inputs.context }}/Dockerfile.${{ inputs.os }}
cache-from: type=gha
cache-to: type=gha
build-args: |
${{ inputs.build-args }}
tags: ${{ inputs.image-tags }}

- name: Get first tag
shell: bash
id: first-tag
run: |
IMG_TAGS="${{ inputs.image-tags }}"
FIRST_TAG=$(cut -d "," -f 1 <<< "${IMG_TAGS//$'\n'/}")
echo "$FIRST_TAG"
echo "FIRST_TAG=$FIRST_TAG" >> $GITHUB_OUTPUT

# We have to use bash logic because step "if"s don't work in composite actions
- name: Test - ${{ inputs.test-image }}
shell: bash
run: |
if [[ "${{ inputs.test-image }}" == "true" ]]; then
echo "${{ inputs.build-args }}" > ${{ inputs.context }}/.env
echo "OS=${{ inputs.os }}" >> ${{ inputs.context }}/.env
cat ${{ inputs.context }}/.env
IMAGE_NAME=${{ steps.first-tag.outputs.FIRST_TAG }} docker-compose -f ${{ inputs.context }}/docker-compose.test.yml run sut
fi
ianpittwood marked this conversation as resolved.
Show resolved Hide resolved

- name: Evaluate Snyk command
id: eval-snyk-command
shell: bash
run: |
if [[ "${{ github.ref }}" == "refs/heads/main" ]]; then
SNYK_COMMAND="monitor"
else
SNYK_COMMAND="test"
fi
echo "SNYK_COMMAND=$SNYK_COMMAND" >> $GITHUB_OUTPUT

- name: Run Snyk ${{ steps.eval-snyk-command.SNYK_COMMAND }}
continue-on-error: true
uses: snyk/actions/docker@master
env:
SNYK_TOKEN: ${{ inputs.snyk-token }}
with:
image: ${{ steps.first-tag.FIRST_TAG }}
args: |
--file=${{ inputs.context }}/Dockerfile.${{ inputs.os }} \
--org=${{ inputs.snyk-org-id }} \
--project-name=${{ steps.first-tag.FIRST_TAG }} \
--tags=product=${{ inputs.product }},os=${{ inputs.os }} \
--exclude-base-image-vulns \
--app-vulns
command: ${{ steps.eval-snyk-command.SNYK_COMMAND }}

- name: Push - ${{ inputs.push-image }}
uses: docker/build-push-action@v4
with:
push: ${{ inputs.push-image }}
context: ${{ inputs.context }}
file: ${{ inputs.context }}/Dockerfile.${{ inputs.os }}
cache-from: type=gha
cache-to: type=gha
build-args: ${{ inputs.build-args }}
tags: ${{ inputs.image-tags }}
Loading
Loading