Build with custom plugins fails to use new plugins list as a build argument

[kidhasmoxy]

The instructions for building a custom image with a user specified plugin list no longer work with the recent updates to the builder.

I've also tried using the instructions in the readme for a one time build separate from my compose file, and those did not change the plugins installed. These instructions use the new plugin format that was called out in my original issue #76

This section in a compose file used to work perfectly for building a new image with my desired plugins:

---
version: '2'
services:
  caddy:
    build:
      context: https://github.com/abiosoft/caddy-docker.git
      args:
        - plugins=authz,cache,cgi,cors,expires,filter,forwardproxy,git,grpc,ipfilter,jwt,locale,login,minify,nobots,proxyprotocol,ratelimit,realip,upload,tls.dns.googlecloud

Now the image builds, but the verification step lists only standard plugins, and doesn't include any of the extras requested.

Step 11/18 : RUN /usr/bin/caddy -plugins
---> Running in dfb8e94a94ce
Server types:
http

Caddyfile loaders:
short
flag
default

Other plugins:
http.basicauth
http.bind
http.browse
http.errors
http.expvar
http.ext
http.fastcgi
http.gzip
http.header
http.index
http.internal
http.limits
http.log
http.markdown
http.mime
http.pprof
http.proxy
http.push
http.redir
http.request_id
http.rewrite
http.root
http.status
http.templates
http.timeouts
http.websocket
on
shutdown
startup
tls
tls.storage.file

[abiosoft]

Firstly, change tls.dns.googlecloud to googlecloud.

Then can you narrow down the list of plugins to see which ones failed to build.

[abiosoft]

I've narrowed it down.

proxyprotocol and locale are not getting picked up due to a bug in caddyplug, the tool being used to detect packages for plugins. This is within my control and I can work on a fix.

grpc is failing to build with the following error, this is outside my control.

../../pieterlouw/caddy-grpc/server.go:45:54: cannot use director (type func("context".Context, string) (*grpc.ClientConn, error)) as type proxy.StreamDirector in argument to proxy.TransparentHandler

For now, omit forwardproxy, locale and grpc and you should have a successful build.

The other option is https://caddyserver.com/download which only builds with stable version of the plugins. There is no way outside of Caddy dashboard to track stable plugin versions, so all builds are done off master.

[kidhasmoxy]

Thanks for looking into this. The weird thing is that I get the same results when I skip compose and just use your example snippet:

docker build --build-arg \
    plugins=filemanager,git,linode \
    github.com/abiosoft/caddy-docker.git

Here's the full output of that command in case it helps:

docker build --no-cache --build-arg plugins=filemanager,git,linode github.com/abiosoft/caddy-docker.git
Sending build context to Docker daemon 151.6kB
Step 1/18 : FROM abiosoft/caddy:builder as builder
---> 59bb48414d63
Step 2/18 : ARG version="0.10.10"
---> Running in 449a0c4df6aa
---> 1e9f86a4d9c2
Removing intermediate container 449a0c4df6aa
Step 3/18 : ARG plugins="git"
---> Running in 333569b89b05
---> 86020b1eed3c
Removing intermediate container 333569b89b05
Step 4/18 : RUN VERSION=${version} PLUGINS=${plugins} /bin/sh /usr/bin/builder.sh
---> Running in 6c04332492f8
Cloning into '/go/src/github.com/mholt/caddy'...
Note: checking out 'c4dfbb9956095c92d0586a52723748c070c7b459'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

Switched to a new branch 'v0.10.10'
github.com/abiosoft/caddyplug (download)
github.com/abiosoft/errs (download)
github.com/fatih/color (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/abiosoft/errs
github.com/abiosoft/caddyplug/internal
github.com/fatih/color/vendor/github.com/mattn/go-colorable
github.com/fatih/color/vendor/github.com/mattn/go-isatty
golang.org/x/sys/unix
github.com/fatih/color
github.com/abiosoft/caddyplug/caddyplug
Cloning into '/go/src/github.com/caddyserver/builds'...
---> 9d6bddff717d
Removing intermediate container 6c04332492f8
Step 5/18 : FROM alpine:3.6
---> 053cde6e8953
Step 6/18 : LABEL maintainer "Abiola Ibrahim [email protected]"
---> Running in d135a501875d
---> 0ff02514ad93
Removing intermediate container d135a501875d
Step 7/18 : LABEL caddy_version "0.10.10"
---> Running in 80730c66082d
---> 7b4e70e3be70
Removing intermediate container 80730c66082d
Step 8/18 : RUN apk add --no-cache openssh-client git
---> Running in 2eca557eb6e4
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20161130-r2)
(2/8) Installing libssh2 (1.8.0-r1)
(3/8) Installing libcurl (7.56.1-r0)
(4/8) Installing expat (2.2.0-r1)
(5/8) Installing pcre (8.41-r0)
(6/8) Installing git (2.13.5-r0)
(7/8) Installing openssh-keygen (7.5_p1-r1)
(8/8) Installing openssh-client (7.5_p1-r1)
Executing busybox-1.26.2-r7.trigger
Executing ca-certificates-20161130-r2.trigger
OK: 28 MiB in 19 packages
---> 4019cd99f632
Removing intermediate container 2eca557eb6e4
Step 9/18 : COPY --from=builder /install/caddy /usr/bin/caddy
---> 38ffeec912e4
Removing intermediate container 9df91fe32892
Step 10/18 : RUN /usr/bin/caddy -version
---> Running in b38aa3b0bc4e
Caddy 0.10.10 (unofficial)
---> e89bb474eca3
Removing intermediate container b38aa3b0bc4e
Step 11/18 : RUN /usr/bin/caddy -plugins
---> Running in 70b0e413ec68
Server types:
http

Caddyfile loaders:
short
flag
default

Other plugins:
http.basicauth
http.bind
http.browse
http.errors
http.expvar
http.ext
http.fastcgi
http.gzip
http.header
http.index
http.internal
http.limits
http.log
http.markdown
http.mime
http.pprof
http.proxy
http.push
http.redir
http.request_id
http.rewrite
http.root
http.status
http.templates
http.timeouts
http.websocket
on
shutdown
startup
tls
tls.storage.file

---> a87c995c4b26
Removing intermediate container 70b0e413ec68
Step 12/18 : EXPOSE 80 443 2015
---> Running in 84a1f70d19c3
---> a9ae4ac8f010
Removing intermediate container 84a1f70d19c3
Step 13/18 : VOLUME /root/.caddy /srv
---> Running in ae3d1d1938f5
---> 8c009a551742
Removing intermediate container ae3d1d1938f5
Step 14/18 : WORKDIR /srv
---> e6d7bf76cbe0
Removing intermediate container b66689e8bf2e
Step 15/18 : COPY Caddyfile /etc/Caddyfile
---> e7429437c47a
Removing intermediate container f9d043db37af
Step 16/18 : COPY index.html /srv/index.html
---> d8cf55374c33
Removing intermediate container 1ce61c471982
Step 17/18 : ENTRYPOINT /usr/bin/caddy
---> Running in 4ceab8963e06
---> 7ae1a5ae4913
Removing intermediate container 4ceab8963e06
Step 18/18 : CMD --conf /etc/Caddyfile --log stdout
---> Running in c85b4ac1ce1a
---> 58fa9418ea3d
Removing intermediate container c85b4ac1ce1a
Successfully built 58fa9418ea3d

[abiosoft]

Something is probably wrong with your environment. Can you tear down compose and try again or use another environment and --no-cache flag ?

I have no issues building with custom plugins.

[kidhasmoxy]

This seems to be an issue with the way that the build process interacts with the environment on my Synology NAS. I was able to build on a different machine and then use that image as the source on my NAS without issue.

Thanks for your help.

[shaxbee]

Sorry for reanimating old issue - fix for grpc was relatively simple: pieterlouw/caddy-grpc#2
Is there any way to use vendored plugin when building caddy-docker?

Edit:
Simply cloning fork into upstream path in the Dockerfile works, eg:

git clone https://github.com/shaxbee/caddy-grpc.git /go/src/github.com/pieterlouw/caddy-grpc