Skip to content

Commit

Permalink
Ensure Github workflows use up-to-date base Docker images (#1951)
Browse files Browse the repository at this point in the history
Relying on a CRON job to update the Antrea base images has proven
sub-optimal: we sometimes push base images manually which do not match
the checked-in Dockerfiles and CI tests run for PRs which update the
Dockerfiles ignore these updates (making the tests worthless).

We now ensure that Github workflows always build the base images before
building the Antrea image, thanks to a new helper script. By relying on
Docker caching (using the Dockerhub registry as the cache), we ensure
that build times are not increased: in the absence of any change, we
only add a handful of seconds to the build time.

For now, we only update CI jobs run as Github workflow. Once this is
merged, we should consider doing the same for Jenkins scripts. We could
add support for DOCKER_REGISTRY to the new helper script
(hack/build-antrea-ubuntu-all.sh).

One question that we could ask now is whether these base images are even
necessary: if caching works well, using one large Dockerfile should be
just as fast, while simplifying build architecture. This is something we
may want to revisit in the future. Maybe using base images only makes
sense if we are going to share them across multiple images.

See #1540
  • Loading branch information
antoninbas authored Mar 22, 2021
1 parent 2c1666d commit cdbd4fc
Show file tree
Hide file tree
Showing 9 changed files with 149 additions and 49 deletions.
9 changes: 6 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,18 @@ jobs:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- name: Build Antrea Docker image
run: make
- name: Push Antrea amd64 Docker image to registry
- name: Build Antrea amd64 Docker image without pushing to registry
if: ${{ github.repository != 'vmware-tanzu/antrea' || github.event_name != 'push' || github.ref != 'refs/heads/main' }}
run: |
./hack/build-antrea-ubuntu-all.sh --pull
- name: Build and push Antrea amd64 Docker image to registry
if: ${{ github.repository == 'vmware-tanzu/antrea' && github.event_name == 'push' && github.ref == 'refs/heads/main' }}
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: |
echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
./hack/build-antrea-ubuntu-all.sh --pull --push-base-images
docker tag antrea/antrea-ubuntu:latest antrea/antrea-ubuntu-amd64:latest
docker push antrea/antrea-ubuntu-amd64:latest
- name: Trigger Antrea arm builds and multi-arch manifest update
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build_tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,13 @@ jobs:
needs: get-version
steps:
- uses: actions/checkout@v2
- name: Build Antrea amd64 Docker image and push to registry
- name: Build and push Antrea amd64 Docker image to registry
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
VERSION: ${{ needs.get-version.outputs.version }}
run: |
make
./hack/build-antrea-ubuntu-all.sh --pull
echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
docker tag antrea/antrea-ubuntu:"${VERSION}" antrea/antrea-ubuntu-amd64:"${VERSION}"
docker push antrea/antrea-ubuntu-amd64:"${VERSION}"
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/kind.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ jobs:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- run: make build-ubuntu-coverage
- name: Build Antrea Docker image with code coverage support
run: |
./hack/build-antrea-ubuntu-all.sh --pull --coverage
- name: Save Antrea image to tarball
run: docker save -o antrea-ubuntu.tar antrea/antrea-ubuntu-coverage:latest
- name: Upload Antrea image for subsequent jobs
Expand Down Expand Up @@ -385,7 +387,9 @@ jobs:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- run: make
- name: Build Antrea Docker image
run: |
./hack/build-antrea-ubuntu-all.sh --pull
- name: Save Antrea image to tarball
run: docker save -o antrea-ubuntu.tar projects.registry.vmware.com/antrea/antrea-ubuntu:latest
- name: Upload Antrea image for subsequent jobs
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/kind_upgrade.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ jobs:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- run: make
- name: Build Antrea Docker image
run: |
./hack/build-antrea-ubuntu-all.sh --pull
- name: Save Antrea image to tarball
run: docker save -o antrea-ubuntu.tar projects.registry.vmware.com/antrea/antrea-ubuntu:latest
- name: Upload Antrea image for subsequent jobs
Expand Down
33 changes: 0 additions & 33 deletions .github/workflows/update_ovs_image.yml

This file was deleted.

20 changes: 13 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ UNAME_S := $(shell uname -s)
USERID := $(shell id -u)
GRPID := $(shell id -g)

# If NO_PULL is set, base Docker images will not be pulled.
# If DOCKER_REGISTRY is set, we always set NO_PULL.
ifneq ($(DOCKER_REGISTRY),)
NO_PULL := 1
endif

.PHONY: bin
bin:
@mkdir -p $(BINDIR)
Expand Down Expand Up @@ -142,7 +148,7 @@ docker-test-unit: $(DOCKER_CACHE)
.PHONY: docker-test-integration
docker-test-integration: .coverage
@echo "===> Building Antrea Integration Test Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/test -f build/images/test/Dockerfile .
else
docker build --pull -t antrea/test -f build/images/test/Dockerfile .
Expand Down Expand Up @@ -266,7 +272,7 @@ codegen:
.PHONY: ubuntu
ubuntu:
@echo "===> Building antrea/antrea-ubuntu Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/antrea-ubuntu:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.ubuntu .
else
docker build --pull -t antrea/antrea-ubuntu:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.ubuntu .
Expand All @@ -279,7 +285,7 @@ endif
.PHONY: build-ubuntu
build-ubuntu:
@echo "===> Building Antrea bins and antrea/antrea-ubuntu Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/antrea-ubuntu:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.ubuntu .
else
docker build --pull -t antrea/antrea-ubuntu:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.ubuntu .
Expand All @@ -291,7 +297,7 @@ endif
.PHONY: build-windows
build-windows:
@echo "===> Building Antrea bins and antrea/antrea-windows Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/antrea-windows:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.windows .
else
docker build --pull -t antrea/antrea-windows:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.windows .
Expand All @@ -303,7 +309,7 @@ endif
.PHONY: build-ubuntu-coverage
build-ubuntu-coverage:
@echo "===> Building Antrea bins and antrea/antrea-ubuntu-coverage Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/antrea-ubuntu-coverage:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.coverage .
else
docker build --pull -t antrea/antrea-ubuntu-coverage:$(DOCKER_IMG_VERSION) -f build/images/Dockerfile.build.coverage .
Expand Down Expand Up @@ -350,7 +356,7 @@ octant-antrea-ubuntu:
.PHONY: flow-aggregator-ubuntu
flow-aggregator-ubuntu:
@echo "===> Building antrea/flow-aggregator Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/flow-aggregator:$(DOCKER_IMG_VERSION) -f build/images/flow-aggregator/Dockerfile .
else
docker build --pull -t antrea/flow-aggregator:$(DOCKER_IMG_VERSION) -f build/images/flow-aggregator/Dockerfile .
Expand All @@ -362,7 +368,7 @@ endif
.PHONY: flow-aggregator-ubuntu-coverage
flow-aggregator-ubuntu-coverage:
@echo "===> Building antrea/flow-aggregator-coverage Docker image <==="
ifneq ($(DOCKER_REGISTRY),"")
ifneq ($(NO_PULL),)
docker build -t antrea/flow-aggregator-coverage:$(DOCKER_IMG_VERSION) -f build/images/flow-aggregator/Dockerfile.coverage .
else
docker build --pull -t antrea/flow-aggregator-coverage:$(DOCKER_IMG_VERSION) -f build/images/flow-aggregator/Dockerfile.coverage .
Expand Down
11 changes: 10 additions & 1 deletion build/images/base/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,23 @@ pushd $THIS_DIR > /dev/null
if $PULL; then
docker pull $PLATFORM_ARG ubuntu:20.04
docker pull $PLATFORM_ARG antrea/openvswitch:$OVS_VERSION
docker pull $PLATFORM_ARG antrea/cni-binaries || true
docker pull $PLATFORM_ARG antrea/base-ubuntu:$OVS_VERSION || true
fi

docker build $PLATFORM_ARG --target cni-binaries \
--cache-from antrea/cni-binaries \
-t antrea/cni-binaries \
--build-arg OVS_VERSION=$OVS_VERSION .

docker build $PLATFORM_ARG \
--cache-from antrea/cni-binaries \
--cache-from antrea/base-ubuntu:$OVS_VERSION \
-t antrea/base-ubuntu:$OVS_VERSION \
-f Dockerfile \
--build-arg OVS_VERSION=$OVS_VERSION .

if $PUSH; then
docker push antrea/cni-binaries:$OVS_VERSION
docker push antrea/base-ubuntu:$OVS_VERSION
fi

Expand Down
2 changes: 2 additions & 0 deletions build/images/ovs/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ pushd $THIS_DIR > /dev/null

if $PULL; then
docker pull $PLATFORM_ARG ubuntu:20.04
docker pull antrea/openvswitch-debs:$OVS_VERSION || true
docker pull antrea/openvswitch:$OVS_VERSION || true
fi

docker build $PLATFORM_ARG --target ovs-debs \
Expand Down
107 changes: 107 additions & 0 deletions hack/build-antrea-ubuntu-all.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env bash

# Copyright 2021 Antrea Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -eo pipefail

# Change this when updating the OVS version!
: "${OVS_VERSION:=2.14.0}"
export OVS_VERSION

function echoerr {
>&2 echo "$@"
}

_usage="Usage: $0 [--pull] [--push-base-images] [--coverage] [--platform <PLATFORM>]
Build the antrea/antrea-ubuntu image, as well as all the base images in the build chain. This is
typically used in CI to build the image with the latest version of all dependencies, taking into
account changes to all Dockerfiles.
--pull Always attempt to pull a newer version of the base images.
--push-base-images Push built images to the registry. Only base images will be pushed.
--coverage Build the image with support for code coverage.
--platform <PLATFORM> Target platform for the images if server is multi-platform capable."

function print_usage {
echoerr "$_usage"
}

PULL=false
PUSH=false
COVERAGE=false
PLATFORM=""

while [[ $# -gt 0 ]]
do
key="$1"

case $key in
--pull)
PULL=true
shift
;;
--push-base-images)
PUSH=true
shift
;;
--coverage)
COVERAGE=true
shift
;;
--platform)
PLATFORM="$2"
shift 2
;;
-h|--help)
print_usage
exit 0
;;
*) # unknown option
echoerr "Unknown option $1"
exit 1
;;
esac
done

THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

pushd "$THIS_DIR/.." > /dev/null

ARGS=""
if $PUSH; then
ARGS="$ARGS --push"
fi
if $PULL; then
ARGS="$ARGS --pull"
fi
if [ "$PLATFORM" != "" ]; then
ARGS="$ARGS --platform $PLATFORM"
fi

cd build/images/ovs
./build.sh $ARGS
cd -

cd build/images/base
./build.sh $ARGS
cd -

export NO_PULL=1
if $COVERAGE; then
make build-ubuntu-coverage
else
make
fi

popd > /dev/null

0 comments on commit cdbd4fc

Please sign in to comment.