Merge pull request #647 from yeya24/feature/add-go-mod

chore: replace govendor with go mod and update docs

.circleci/config.yml

@@ -14,12 +14,12 @@ jobs:
       - run:
           name: use markdownlint v0.5.0 to lint markdown file (https://github.com/markdownlint/markdownlint)
           command: |
-            find  ./ -name  "*.md" | grep -v vendor | grep -v commandline |  grep -v .github |  grep -v swagger |  grep -v api |  xargs mdl -r ~MD002,~MD007,~MD010,~MD013,~MD024,~MD026,~MD029,~MD033,~MD034,~MD036,~MD046
+            find  ./ -name  "*.md" | grep -v commandline |  grep -v .github |  grep -v swagger |  grep -v api |  xargs mdl -r ~MD002,~MD007,~MD010,~MD013,~MD024,~MD026,~MD029,~MD033,~MD034,~MD036,~MD046
       - run:
           name: use markdown-link-check(https://github.com/tcort/markdown-link-check) to check links in markdown files
           command: |
             set +e
-            for name in $(find . -name \*.md | grep -v vendor | grep -v CHANGELOG); do
+            for name in $(find . -name \*.md | grep -v CHANGELOG); do
               if [ -f $name ]; then
                 markdown-link-check -q $name;
                 if [ $? -ne 0 ]; then
@@ -31,11 +31,15 @@ jobs:
       - run:
           name: use opensource tool client9/misspell to correct commonly misspelled English words
           command: |
-            find  ./* -name  "*"  | grep -v vendor | xargs misspell -error
+            find  ./* -name  "*" | xargs misspell -error
       - run:
           name: use ShellCheck (https://github.com/koalaman/shellcheck) to check the validateness of shell scripts in pouch repo
           command: |
-            find ./ -name "*.sh" | grep -v vendor | xargs shellcheck
+            find ./ -name "*.sh" | xargs shellcheck
+      - run:
+          name: validate go mod files
+          command: |
+            make check-go-mod
 
   gocode-check-via-gometalinter-swagger:
     docker:
@@ -50,10 +54,12 @@ jobs:
       - run:
           name: use gometalinter to check gocode of this project.
           command: |
+            make go-mod-vendor
             gometalinter --disable-all --cyclo-over=20 --skip vendor -E gofmt -E goimports -E golint -E misspell -E vet -E goconst -E gocyclo ./...
       - run:
           name: detect deadcode without test folder
           command: |
+            make go-mod-vendor
             gometalinter --disable-all --skip vendor --skip test -E deadcode ./...
 
   unit-test-golang:
@@ -65,12 +71,7 @@ jobs:
       - run:
           name: build client
           command: |
-            cd cmd
-            cd dfget && go build
-            cd ..
-            cd dfdaemon && go build
-            cd ..
-            cd supernode && go build
+            make build
       - run:
           name: unit test
           command: make unit-test
@@ -93,7 +94,7 @@ jobs:
             echo "building..."
             make build > /dev/null
             echo "start integration test..."
-            go test ./test
+            make integration-test
 
 workflows:
   version: 2

CONTRIBUTING.md

@@ -62,7 +62,7 @@ Since you are ready to improve Dragonfly with a PR, we suggest you could take a
 * [Commit Rules](#commit-rules)
 * [PR Description](#pr-description)
 * [Developing Environment](#developing-environment)
-* [Golang Package Vendoring Mechanism](#golang-package-vendoring-mechanism)
+* [Golang Dependency Management](#golang-dependency-management)
 
 ### Workspace Preparation
 
@@ -155,43 +155,32 @@ Here are some dependents with specific version:
 
 When you develop the Dragonfly project at the local environment, you should use subcommands of Makefile to help yourself to check and build the latest version of Dragonfly. For the convenience of developers， we use the docker to build Dragonfly. It can reduce problems of the developing environment.
 
-### Golang Package Vendoring Mechanism
+### Golang Dependency Management
 
-Dragonfly uses tool [govendor](https://github.com/kardianos/govendor) to vendor packages in its own repository. When hacking Dragonfly's code, any developer can use the common way to install `govendor`, and here the community suggest to use govendor v1.0.8 to collaborate:
+The Dragonfly project uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies on external packages. This requires a working Go environment with version 1.12 or greater installed.
 
-```bash
-go get -u github.com/kardianos/govendor
-```
-
-Here are three cases to manage vendored packages with `govendor`:
-
-* [Vendor External Packages](#vendor-external-packages)
-* [Remove Vendored Packages](#remove-vendored-packages)
-* [Update Vendored Packages](#update-vendored-packages)
-
-#### Vendor External Packages
-
-In Dragonfly, when vendoring a package, we take the version of it very seriously. Therefore, almostly every time we use `govendor`, we explicitly pick the specified commit ID or tag.
+To add or update a new dependency, use the `go get` command:
 
-As a result, we seldom use the command `govendor add github.com/a/b`. Instead, we always make use of `govendor add github.com/a/b@8ac97434144a8b58bd39576fc65be42d90257b04` or `govendor add github.com/a/[email protected]`.
-
-When using `govendor`, there are still some tiny tips for us:
-
-* If you pick commit id `8ac97434144a8b58bd39576fc65be42d90257b04` of package `govendor add github.com/a/b`, then for this package in your `GOPATH`, you have to checkout to this specified commit ID before using `govendor`.
-* When there are go files and directories which are all you need to vendor under the root path of package `govendor add github.com/a/b`, you have to use flag `^` to include them all using command `govendor add github.com/a/b/^@8ac97434144a8b58bd39576fc65be42d90257b04`.
-* Assuming that package `github.com/a/b` also has a directory `vendor`, so-called nesting vendor, in most cases we remain these packages.
+```bash
+# Pick the latest tagged release.
+go get example.com/some/module/pkg
 
-#### Remove Vendored Packages
+# Pick a specific version.
+go get example.com/some/module/[email protected]
+```
 
-Removal of vendored package will take no effort. Just execute the following the command:
+Tidy up the `go.mod` and `go.sum` files:
 
 ```bash
-govendor remove github.com/a/b
+make go-mod-tidy
 ```
 
-#### Update Vendored Packages
+You have to commit the changes to `go.mod`, `go.sum` before submitting the pull request.
 
-Update of vendored packages will take a little bit more effort than removal. Although you could execute `govendor update github.com/a/b@a4bbce9fcae005b22ae5443f6af064d80a6f5a55` in which `a4bbce9fcae005b22ae5443f6af064d80a6f5a55` is the new commit ID, you should still keep it in mind that for this package in your `GOPATH`, you have to checkout to this specified commit ID before using `govendor update`.
+```bash
+git add go.mod go.sum
+git commit
+```
 
 ## Engage to help anything
 

Makefile

@@ -19,6 +19,12 @@ USE_DOCKER?=0 # Default: build components in the local environment.
 # Assign the Dragonfly version to DF_VERSION as the image tag.
 DF_VERSION?=latest # Default: use latest as the image tag which built by docker.
 
+# Default: in order to use go mod we have to export GO111MODULE=on.
+export GO111MODULE=on
+
+# Default: use GOPROXY to speed up downloading go mod dependency.
+export GOPROXY=https://goproxy.io
+
 clean:
 	@echo "Begin to clean redundant files."
 	@rm -rf ./bin
@@ -94,13 +100,33 @@ unit-test: build-dirs
 	./hack/unit-test.sh
 .PHONY: unit-test
 
+integration-test:
+	@go test ./test
+.PHONY: integration-test
+
 check:
 	@echo "Begin to check code formats."
 	./hack/check.sh
 	@echo "Begin to check dockerd whether is startup"
 	./hack/check-docker.sh
 .PHONY: check
 
+go-mod-tidy:
+	@echo "Begin to tidy up go.mod and go.sum"
+	@go mod tidy
+.PHONY: go-mod-tidy
+
+# we need this because currently gometalinter can't work in go mod environment.
+go-mod-vendor:
+	@echo "Begin to vendor go mod dependency"
+	@go mod vendor
+.PHONY: go-mod-vendor
+
+check-go-mod: go-mod-tidy
+	@echo "Begin to check for unused/missing packages in go.mod"
+	@git diff --exit-code -- go.sum go.mod
+.PHONY: check-go-mod
+
 docs:
 	@echo "Begin to generate docs of API/CLI"
 	./hack/generate-docs.sh

go.mod

@@ -0,0 +1,44 @@
+module github.com/dragonflyoss/Dragonfly
+
+go 1.12
+
+require (
+	github.com/PuerkitoBio/purell v0.0.0-20170829232023-f619812e3caf // indirect
+	github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect
+	github.com/asaskevich/govalidator v0.0.0-20170903095215-73945b6115bf // indirect
+	github.com/cpuguy83/go-md2man v1.0.7 // indirect
+	github.com/go-check/check v0.0.0-20161208181325-20d25e280405
+	github.com/go-openapi/analysis v0.0.0-20170813233457-8ed83f2ea9f0 // indirect
+	github.com/go-openapi/errors v0.0.0-20170426151106-03cfca65330d
+	github.com/go-openapi/jsonpointer v0.0.0-20170102174223-779f45308c19 // indirect
+	github.com/go-openapi/jsonreference v0.0.0-20161105162150-36d33bfe519e // indirect
+	github.com/go-openapi/loads v0.0.0-20170520182102-a80dea3052f0 // indirect
+	github.com/go-openapi/runtime v0.0.0-20170901133030-bf2ff8f71507 // indirect
+	github.com/go-openapi/spec v0.0.0-20170811033243-3faa0055dbbf // indirect
+	github.com/go-openapi/strfmt v0.0.0-20171222154016-4dd3d302e100
+	github.com/go-openapi/swag v0.0.0-20170606142751-f3f9494671f9
+	github.com/go-openapi/validate v0.0.0-20170705144413-8a82927c942c
+	github.com/golang/mock v1.3.1
+	github.com/gorilla/context v0.0.0-20181012153548-51ce91d2eadd // indirect
+	github.com/gorilla/mux v1.5.0
+	github.com/inconshreveable/mousetrap v1.0.0 // indirect
+	github.com/magiconair/properties v1.8.1 // indirect
+	github.com/mailru/easyjson v0.0.0-20170902151237-2a92e673c9a6 // indirect
+	github.com/mitchellh/mapstructure v1.1.2
+	github.com/pborman/uuid v0.0.0-20180122190007-c65b2f87fee3
+	github.com/pkg/errors v0.8.0
+	github.com/prashantv/gostub v1.0.0
+	github.com/russross/blackfriday v0.0.0-20171011182219-6d1ef893fcb0 // indirect
+	github.com/sirupsen/logrus v1.2.0
+	github.com/spf13/afero v1.2.2
+	github.com/spf13/cobra v0.0.0-20181021141114-fe5e611709b0
+	github.com/spf13/jwalterweatherman v1.1.0 // indirect
+	github.com/spf13/viper v1.4.0
+	github.com/stretchr/testify v1.2.2
+	github.com/valyala/fasthttp v1.3.0
+	github.com/willf/bitset v0.0.0-20190228212526-18bd95f470f9
+	gopkg.in/gcfg.v1 v1.2.3
+	gopkg.in/mgo.v2 v2.0.0-20160818020120-3f83fa500528 // indirect
+	gopkg.in/warnings.v0 v0.1.2
+	gopkg.in/yaml.v2 v2.2.2
+)