Support automated ARM builds

[ahoarau]

Currently it is not possible to launch an automated arm build on the Hub because the hosts does not seem to have qemu-user-static.
Is there an alternate solution to this ?

[pchico83]

You could install the package is the post_checkout hook:
https://docs.docker.com/docker-cloud/builds/advanced/
It is also supported in Docker Hub.

[ahoarau]

Cool. So I shall create a docker-compose.test.yml file in my github repo, and docker hub will automatically read it ?

[pchico83]

Yes, that is possible. But you can also defined your own hooks.
For example, you could create a post_checkout file in a hooks folder at the level of your Dockerfile, and that script will be executed after checking out your github repo. You could use it to install apt-get packages (our build machines are Ubuntu Trusty).

[jcsrb]

@ahoarau did you create such a workflow, you link to it?

[ahoarau]

@jcsrb nope did not managed to make it work on the hub. Running locally instead.

[davidecavestro]

@pchico83 it works like a charm!
@ahoarau it turns out to be just a matter of registering qemu-arm-static before launching the build

See mariadb-docker-armhf (WIP):

.
├── Dockerfile
└── hooks
     ├── post_checkout
     └── pre_build

hooks/post_checkout

#!/bin/bash
# downloads a local copy of qemu on docker-hub build machines
curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .

hooks/pre_build

#!/bin/bash
# Register qemu-*-static for all supported processors except the 
# current one, but also remove all registered binfmt_misc before
docker run --rm --privileged multiarch/qemu-user-static:register --reset

Dockerfile

FROM arm32v7/debian:stretch-slim 
COPY qemu-arm-static /usr/bin
...

Probably it can be further reduced.
Now the job fails just for a matter of timeout (huge build) but the arm stuff simply works :-)

[ahoarau]

@davidecavestro excellent ! I guess the issue is closed now :)

[se1exin]

FYI this did not work for me - Docker Hub build cannot find the qemu-arm-static file:

...
Step 6/10 : COPY qemu-arm-static /usr/bin
COPY failed: stat /var/lib/docker/tmp/docker-builder403015795/qemu-arm-static: no such file or directory

I worked around it by removing the post_checkout file and using a builder/mutlistage Dockerfile to download the qemu-arm-static binary inside the container instead.

Example Dockerfile:

FROM golang:1.12.0 AS builder
# ... my go build steps (removed from this example)
WORKDIR /builder/working/directory
RUN curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .

FROM arm32v7/alpine:latest
# Copy across the qemu binary that was downloaded in the previous build step
COPY --from=builder /builder/working/directory/qemu-arm-static /usr/bin
# Now you can tun ARM docker steps.. yay!

Note I'm using golang as my builder image, but I'm sure any amd64 image with curl and tar would work fine. My full Dockerfile is here if it might help anyone: https://github.com/se1exin/Hue-Im-Home/blob/a39ab7c78db2564f767527519d18d8c1faf964cd/docker/Dockerfile.arm32v7

[marklagendijk]

This is related #1874

[Diluka]

didn't work for node

Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.114.3' to the list of known hosts.
Switched to a new branch 'next'
Executing post_checkout hook...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 635 100 635 0 0 3163 0 --:--:-- --:--:-- --:--:-- 3175
qemu-3.0.0+resin-arm/
qemu-3.0.0+resin-arm/qemu-arm-static
100 1678k 100 1678k 0 0 3765k 0 --:--:-- --:--:-- --:--:-- 3765k
Executing pre_build hook...
Unable to find image 'multiarch/qemu-user-static:register' locally
register: Pulling from multiarch/qemu-user-static
d60bca25ef07: Pulling fs layer
7045fbe28d35: Pulling fs layer
ed8a5179ae11: Pulling fs layer
1ec39da9c97d: Pulling fs layer
1ec39da9c97d: Waiting
7045fbe28d35: Download complete
ed8a5179ae11: Verifying Checksum
ed8a5179ae11: Download complete
d60bca25ef07: Verifying Checksum
d60bca25ef07: Download complete
1ec39da9c97d: Verifying Checksum
1ec39da9c97d: Download complete
d60bca25ef07: Pull complete
7045fbe28d35: Pull complete
ed8a5179ae11: Pull complete
1ec39da9c97d: Pull complete
Digest: sha256:0434e870ebbe9d28d254c1e65b46969bef458490be5df04d3266ef87380518e2
Status: Downloaded newer image for multiarch/qemu-user-static:register
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'19.03.8', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2020-03-11T01:24:30.000000000+00:00', u'ApiVersion': u'1.40', u'MinAPIVersion': u'1.12', u'GitCommit': u'afacb8b7f0', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.12.17'}}, {u'Version': u'1.2.13', u'Name': u'containerd', u'Details': {u'GitCommit': u'7ad184331fa3e55e52b890ea95e65ba581ae3429'}}, {u'Version': u'1.0.0-rc10', u'Name': u'runc', u'Details': {u'GitCommit': u'dc9208a3303feef5b3839f4323d9beb36df0a9dd'}}, {u'Version': u'0.18.0', u'Name': u'docker-init', u'Details': {u'GitCommit': u'fec3683'}}]
Arch: amd64
BuildTime: 2020-03-11T01:24:30.000000000+00:00
ApiVersion: 1.40
Platform: {u'Name': u'Docker Engine - Community'}
Version: 19.03.8
MinAPIVersion: 1.12
GitCommit: afacb8b7f0
Os: linux
GoVersion: go1.12.17
Buildkit: Starting build for index.docker.io/diluka/bull-board:next-arm64v8...
#2 [internal] load build definition from Dockerfile.arm64v8
#2 transferring dockerfile:
#2 transferring dockerfile: 474B done
#2 DONE 0.3s
#1 [internal] load .dockerignore
#1 transferring context: 101B done
#1 DONE 0.3s
#3 [internal] load metadata for docker.io/arm64v8/node:14.17-alpine
#3 DONE 0.4s
#5 [internal] load build context
#5 transferring context: 5.77MB 0.5s done
#5 DONE 0.7s
#4 [1/7] FROM docker.io/arm64v8/node:14.17-alpine@sha256:54ce125ebbced038b4...
#4 resolve docker.io/arm64v8/node:14.17-alpine@sha256:54ce125ebbced038b4b54b4c25ab1cb5c79fba1d0e153b23d35c0fc7f0a69101 done
#4 sha256:54ce125ebbced038b4b54b4c25ab1cb5c79fba1d0e153b23d35c0fc7f0a69101 1.16kB / 1.16kB done
#4 sha256:645fc0c6c96611d16cbb3252bf5edccea9124221e16f1407cf483f944e2716ed 6.73kB / 6.73kB done
#4 sha256:535256e01bcb9590f56b9b2a053648a1d98fbe4dc0fd34c1fc3f32afec8c6e7b 2.73MB / 2.73MB 0.5s done
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 14.28MB / 36.26MB 0.7s
#4 sha256:23b960c2560ae1e047a378e94eb41e521c52c1c22860d196f5aaef1a297d74dd 2.30MB / 2.30MB 0.4s done
#4 sha256:2f24eee88333ef6e2bd82a975c7b6dbc8460012bbfb021bc8ddf70d02cb9e88b 281B / 281B 0.6s done
#4 extracting sha256:535256e01bcb9590f56b9b2a053648a1d98fbe4dc0fd34c1fc3f32afec8c6e7b
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 19.78MB / 36.26MB 0.9s
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 23.89MB / 36.26MB 1.0s
#4 extracting sha256:535256e01bcb9590f56b9b2a053648a1d98fbe4dc0fd34c1fc3f32afec8c6e7b 0.3s done
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 30.98MB / 36.26MB 1.1s
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 36.26MB / 36.26MB 1.2s
#4 sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 36.26MB / 36.26MB 1.3s done
#4 extracting sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f
#4 extracting sha256:69982ba3cc3a84e32a718955abee7b677760fecbbf6c71d883a8392ab552164f 3.8s done
#4 extracting sha256:23b960c2560ae1e047a378e94eb41e521c52c1c22860d196f5aaef1a297d74dd
#4 extracting sha256:23b960c2560ae1e047a378e94eb41e521c52c1c22860d196f5aaef1a297d74dd 0.3s done
#4 extracting sha256:2f24eee88333ef6e2bd82a975c7b6dbc8460012bbfb021bc8ddf70d02cb9e88b
#4 extracting sha256:2f24eee88333ef6e2bd82a975c7b6dbc8460012bbfb021bc8ddf70d02cb9e88b done
#4 DONE 6.7s
#6 [2/7] ADD qemu-*-static /usr/bin
#6 DONE 0.1s
#7 [3/7] WORKDIR /usr/app
#7 DONE 0.0s
#8 [4/7] ADD ./package.json .
#8 DONE 0.1s
#9 [5/7] ADD ./yarn.lock .
#9 DONE 0.1s
#10 [6/7] RUN yarn install
#10 0.795 standard_init_linux.go:211: exec user process caused "no such file or directory"
#10 ERROR: executor failed running [/bin/sh -c yarn install]: runc did not terminate sucessfully
------
> [6/7] RUN yarn install:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c yarn install]: runc did not terminate sucessfully
Build failed using Buildkit