Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use buildx docker-container driver for publishing normalization containers #13693

Merged
merged 5 commits into from
Jun 10, 2022
Merged

Use buildx docker-container driver for publishing normalization containers #13693

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
368543c
explicitly use buildx for connector publish
evantahler Jun 10, 2022
67321a2
... and use it
evantahler Jun 10, 2022
54b5e99
use docker-container driver
evantahler Jun 10, 2022
a27157d
use `driver docker-container` only for normalization
evantahler Jun 10, 2022
41f145e
Update tools/integrations/manage.sh
evantahler Jun 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions tools/integrations/manage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,10 @@ cmd_publish() {
echo "Publishing normalization images (version: $versioned_image)"
GIT_REVISION=$(git rev-parse HEAD)

# We use a buildx docker container when building multi-stage builds from one docker compose file
# This works because all the images depend only on already public images
docker buildx create --name connector-buildx --driver docker-container --use

# Note: "buildx bake" needs to be run within the directory
local original_pwd=$PWD
cd airbyte-integrations/bases/base-normalization
Expand All @@ -259,10 +263,13 @@ cmd_publish() {
-f docker-compose.build.yaml \
--push

docker buildx rm connector-buildx

cd $original_pwd
else
# We have to go arch-by-arch locally (see https://github.com/docker/buildx/issues/59 for more info) due to our base images (e.g. airbyte-integrations/bases/base-java)
# Alternative local approach @ https://github.com/docker/buildx/issues/301#issuecomment-755164475
# We need to use the regular docker buildx driver (not docker container) because we need this intermediate contaiers to be available for later build steps

for arch in $(echo $build_arch | sed "s/,/ /g")
do
Expand Down