Add Dockerfile and containerized build #2687

dperny · 2018-07-05T18:36:08Z

- What I did
Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to make intelligent use of Docker.

- How I did it
Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow.

By default, nothing will run inside a container. Set the environment variable SWARMKIT_USE_CONTAINER to use dockerized making.

- How to test it
Set SWARMKIT_USE_CONTAINER and verify that your favorite make targets all work

dperny · 2018-07-05T18:41:58Z

Makefile

+include direct.mk
+else
+include containerized.mk
+endif


for the record, i cannot believe that conditionally including files in make works.

codecov · 2018-07-05T18:48:38Z

Codecov Report

❗ No coverage uploaded for pull request base (master@39d5dc4). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master   #2687   +/-   ##
========================================
  Coverage          ?   61.8%           
========================================
  Files             ?     134           
  Lines             ?   21760           
  Branches          ?       0           
========================================
  Hits              ?   13449           
  Misses            ?    6861           
  Partials          ?    1450

vdemeester

In other project (like docker/app) we put everything containerized in docker.Makefile and try to map the targets from the main one, that way we have make bin/docker-app doing native compile and make -f docker.Makefile bin/docker-app doing the same, in a container

dperny · 2018-07-05T18:57:46Z

That's a good point... I do, however, like the pattern of being able to set an environment variable in my config and just let it happen automagically.

selansen · 2018-07-05T20:06:50Z

This is good idea. One suggestion from me.
As I was trying work on swarmkit first time, I was running into unit tests/integration test failures with latest code on my MAC(without any change). In libnetwork we have options in makefile to run unit tests inside the container with our libnetwork image. this will give one solid environment whoever wants to run swarmkit tests regardless of the platform they run. Pls ignore if you feel this is not necessary.

selansen · 2018-07-05T20:10:39Z

Dockerfile

+  go get -u github.com/stevvooe/protobuild
+
+WORKDIR /go/src/github.com/docker/swarmkit/
+COPY . .


if you are using local machine you dont need to copy entire source code. you can mount the folders when you run container. this will be useful only when we use circleCI2.0 as they dont support mount option now.

right, of course, but i think it's better to copy the whole deal so it's a proper swarmkit image, instead of just a fancy go image.

dperny · 2018-07-05T21:02:36Z

@selansen i'm not sure what you mean... this change does allow you to run unit tests inside of the container. if you set the environment variable, it should mount your local source code directory into the container and run make there.

selansen · 2018-07-05T22:41:34Z

I mis understood. It covers all :)

olljanat · 2018-07-09T18:23:58Z

Tested by running:

export SWARMKIT_USE_CONTAINER=1
git clone https://github.com/dperny/swarmkit-1 -b add-dockerfiles
cd swarmkit-1
docker build . -t docker/swarmkit
make generate

and ended up to error:

running target generate inside a container
docker run -it -v /tmp/swarmkit-1/:/go/src/github.com/docker/swarmkit docker/swarmkit make generate
🐳 bin/protoc-gen-gogoswarm
🐳 protos
protoc -I.:/go/src/github.com/docker/swarmkit/vendor/github.com/gogo/protobuf:/go/src/github.com/docker/swarmkit/vendor:/go/src:/usr/local/include --include_imports --descriptor_set_out=/tmp/descriptors.pb-252013462 --gogoswarm_out=plugins=grpc+deepcopy+storeobject+raftproxy+authenticatedwrapper,import_path=github.com/docker/swarmkit/api,Mgogoproto/gogo.proto=github.com/gogo/protobuf/gogoproto,Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/protoc-gen-gogo/descriptor,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/field_mask.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:/go/src /go/src/github.com/docker/swarmkit/api/ca.proto /go/src/github.com/docker/swarmkit/api/control.proto /go/src/github.com/docker/swarmkit/api/dispatcher.proto /go/src/github.com/docker/swarmkit/api/health.proto /go/src/github.com/docker/swarmkit/api/logbroker.proto /go/src/github.com/docker/swarmkit/api/objects.proto /go/src/github.com/docker/swarmkit/api/raft.proto /go/src/github.com/docker/swarmkit/api/resource.proto /go/src/github.com/docker/swarmkit/api/snapshot.proto /go/src/github.com/docker/swarmkit/api/specs.proto /go/src/github.com/docker/swarmkit/api/types.proto /go/src/github.com/docker/swarmkit/api/watch.proto
sh: 1: protoc: not found
direct.mk:32: recipe for target 'protos' failed
make: *** [protos] Error 127
containerized.mk:10: recipe for target 'generate' failed
make: *** [generate] Error 2

You need include downloading https://github.com/google/protobuf/releases/download/v3.5.0/protoc-3.5.0-linux-x86_64.zip to Dockerfile

dperny · 2018-07-09T18:50:44Z

Killer, thanks for spotting that!

Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow. By default, nothing will run inside a container. Set the environment variable SWARMKIT_USE_CONTAINER to use dockerized making. Signed-off-by: Drew Erny <[email protected]>

dperny · 2018-07-09T19:25:52Z

Added install for protoc version 3.5.0. make generate now works in a container.

selansen · 2018-07-23T15:38:30Z

CI failed below here. I don't think its related to this PR. This PR will be handy to swarmkit new comers like me. Can you guys merge this one ?

time="2018-07-09T19:33:30Z" level=error msg="update failed" error="task tnizi8guwohj72x6hlbxtudha was already shut down when reached by updater" task.id=xnbyrojnmqu79o67qjumb339z
--- FAIL: TestUpdaterRollback (3.86s)
--- FAIL: TestUpdaterRollback/pause/monitor_set/spec_version_set (1.04s)
Error Trace: testutils.go:26
update_test.go:217
update_test.go:20
Error: no task creation

Credit goes to dperny (moby#2687) **- What I did** Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to make intelligent use of Docker. Also made small clean up changes to the Makefile. **- How I did it** Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow. By default, nothing will run inside a container. Set the environment variable `DOCKER_SWARMKIT_USE_CONTAINER` to use dockerized making. Also leverages docker-sync for synchronizing code to the container if the `DOCKER_SWARMKIT_USE_DOCKER_SYNC` env variable is set; comes in handy on Macs, for example. **- How to test it** Set `DOCKER_SWARMKIT_USE_CONTAINER` and verify that your favorite make targets all work!

Credit goes to dperny (moby#2687) **- What I did** Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to make intelligent use of Docker. Also made small clean up changes to the Makefile. **- How I did it** Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow. By default, nothing will run inside a container. Set the environment variable `DOCKER_SWARMKIT_USE_CONTAINER` to use dockerized making. Also leverages docker-sync for synchronizing code to the container if the `DOCKER_SWARMKIT_USE_DOCKER_SYNC` env variable is set; comes in handy on Macs, for example. **- How to test it** Set `DOCKER_SWARMKIT_USE_CONTAINER` and verify that your favorite make targets all work! Signed-off-by: Jean Rouge <[email protected]>

wk8 · 2018-09-10T21:23:21Z

Updated and completed at #2745

Credit goes to dperny (moby#2687) **- What I did** Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to make intelligent use of Docker. Also made small clean up changes to the Makefile. **- How I did it** Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow. By default, nothing will run inside a container. Set the environment variable `DOCKER_SWARMKIT_USE_CONTAINER` to use dockerized making. Also leverages docker-sync for synchronizing code to the container if the `DOCKER_SWARMKIT_USE_DOCKER_SYNC` env variable is set; comes in handy on Macs, for example. **- How to test it** Set `DOCKER_SWARMKIT_USE_CONTAINER` and verify that your favorite make targets all work! Signed-off-by: Jean Rouge <[email protected]>

wk8 · 2018-09-24T23:17:42Z

Should be closed, done in #2745

Credit goes to dperny (moby#2687) **- What I did** Adds a Dockerfile for the swarmkit project, to easily get off the ground. Modifies the Makefile to make intelligent use of Docker. Also made small clean up changes to the Makefile. **- How I did it** Modifies the Makefile to have two paths: containerized.mk, which builds the docker image and forwards any make targets to a container, and direct.mk, which encompasses the old Makefile's workflow. By default, nothing will run inside a container. Set the environment variable `DOCKER_SWARMKIT_USE_CONTAINER` to use dockerized making. Also leverages docker-sync for synchronizing code to the container if the `DOCKER_SWARMKIT_USE_DOCKER_SYNC` env variable is set; comes in handy on Macs, for example. **- How to test it** Set `DOCKER_SWARMKIT_USE_CONTAINER` and verify that your favorite make targets all work! Signed-off-by: Jean Rouge <[email protected]> (cherry picked from commit f8c048c) Signed-off-by: Sebastiaan van Stijn <[email protected]>

dperny commented Jul 5, 2018

View reviewed changes

vdemeester reviewed Jul 5, 2018

View reviewed changes

selansen reviewed Jul 5, 2018

View reviewed changes

dperny mentioned this pull request Jul 9, 2018

Support for Windows named pipes mounts #2691

Merged

dperny force-pushed the add-dockerfiles branch from 35dc876 to fdb2709 Compare July 9, 2018 19:25

wk8 mentioned this pull request Sep 10, 2018

Adding a Dockerfile and making it easy to use it for dev #2745

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Dockerfile and containerized build #2687

Add Dockerfile and containerized build #2687

dperny commented Jul 5, 2018

dperny Jul 5, 2018

codecov bot commented Jul 5, 2018 •

edited

Loading

vdemeester left a comment

dperny commented Jul 5, 2018

selansen commented Jul 5, 2018

selansen Jul 5, 2018

dperny Jul 5, 2018

dperny commented Jul 5, 2018

selansen commented Jul 5, 2018

olljanat commented Jul 9, 2018

dperny commented Jul 9, 2018

dperny commented Jul 9, 2018

selansen commented Jul 23, 2018

wk8 commented Sep 10, 2018

wk8 commented Sep 24, 2018

Add Dockerfile and containerized build #2687

Are you sure you want to change the base?

Add Dockerfile and containerized build #2687

Conversation

dperny commented Jul 5, 2018

dperny Jul 5, 2018

Choose a reason for hiding this comment

codecov bot commented Jul 5, 2018 • edited Loading

Codecov Report

vdemeester left a comment

Choose a reason for hiding this comment

dperny commented Jul 5, 2018

selansen commented Jul 5, 2018

selansen Jul 5, 2018

Choose a reason for hiding this comment

dperny Jul 5, 2018

Choose a reason for hiding this comment

dperny commented Jul 5, 2018

selansen commented Jul 5, 2018

olljanat commented Jul 9, 2018

dperny commented Jul 9, 2018

dperny commented Jul 9, 2018

selansen commented Jul 23, 2018

wk8 commented Sep 10, 2018

wk8 commented Sep 24, 2018

codecov bot commented Jul 5, 2018 •

edited

Loading