Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: Use more accurate test coverage. #134

Merged
merged 8 commits into from
Jan 27, 2022
Merged

ci: Use more accurate test coverage. #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
31cd1ea
Use more accurate test coverage.
shahzadlone Jan 21, 2022
44aa665
Fix makefile to build dependencies properly.
shahzadlone Jan 21, 2022
14163fe
fix circleci build dependencies according to new makefile changes.
shahzadlone Jan 21, 2022
e2d6191
bump version of go to 1.17.5
shahzadlone Jan 21, 2022
f278865
See if coverage is changed by not ignoring test files.
shahzadlone Jan 21, 2022
8c7badd
Add the `-race` flag to tests.
shahzadlone Jan 27, 2022
1c68602
Merge branch 'develop' into lone/ci/more-accurate-test-coverage
shahzadlone Jan 27, 2022
f75285d
Fix running tests in the CI.
shahzadlone Jan 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
docker:
- image: circleci/golang:1.15.8
- image: circleci/golang:1.17.5
# Add steps to the job
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
steps:
Expand All @@ -20,7 +20,7 @@ jobs:
- go-mod-v4-{{ checksum "go.sum" }}
- run:
name: Install Dependencies
command: make deps:circle-ci
command: make deps
- save_cache:
key: go-mod-v4-{{ checksum "go.sum" }}
paths:
Expand All @@ -32,7 +32,7 @@ jobs:
DEFRA_BADGER_FILE=true
DEFRA_MAP=true
mkdir -p /tmp/test-reports
gotestsum --junitfile /tmp/test-reports/unit-tests.xml
gotestsum --junitfile /tmp/test-reports/unit-tests.xml -- ./... -race
- store_test_results:
path: /tmp/test-reports

Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/code-test-coverage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,18 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: '1.17'
go-version: '1.17.5'

- name: Generate coverage report
run: go test ./... -race -coverprofile=coverage.out -covermode=atomic
- name: Generate full test coverage report using go-acc
run: make test:coverage-full

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: true
files: ./coverage.out
flags: unittests
files: ./coverage-full.txt
flags: defra-tests
name: codecov-umbrella
verbose: true
# path_to_write_report: ./coverage/codecov_report.txt
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ cli/defradb/defradb
cli/defradb/defradb.exe
build/defradb*
cover.out
coverage-full.txt
coverage-quick.txt
31 changes: 21 additions & 10 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,17 @@ multi-build:
start: build
./build/defradb start

.PHONY: deps\:circle-ci
deps\:circle-ci:
go mod download
.PHONY: deps\:golangci-lint
deps\:golangci-lint:
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ${GOPATH}/bin v1.43.0

.PHONY: deps\:go-acc
deps\:go-acc:
go install github.com/ory/go-acc@latest

.PHONY: deps
deps: deps\:circle-ci
deps: deps\:golangci-lint deps\:go-acc
go mod download

.PHONY: clean
clean:
Expand All @@ -39,16 +43,23 @@ tidy:

.PHONY: test
test:
go test ./...
go test ./... -race

.PHONY: test\:bench
test\:bench:
go test -bench
go test ./... -race -bench=.

# This also takes integration tests into account.
.PHONY: test\:coverage-full
test\:coverage-full: deps\:go-acc
go-acc ./... --output=coverage-full.txt --covermode=atomic
go tool cover -func coverage-full.txt | grep total | awk '{print $$3}'

.PHONY: test\:coverage
test\:coverage:
go test ./... -coverprofile cover.out
go tool cover -func cover.out | grep total | awk '{print $$3}'
# This only covers how much of the package is tested by itself (unit test).
.PHONY: test\:coverage-quick
test\:coverage-quick:
go test ./... -race -coverprofile=coverage-quick.txt -covermode=atomic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you guys make a descision regarding the -race flag on the coverage-quick make command?

Copy link
Member Author

@shahzadlone shahzadlone Jan 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this coverage-quick rule isn't ran on any CI steps, just there for local coverage report generation of only unit tests (so no integration / cross package coverage).

We did decide to go with -race flag for the benefits. I checked locally the running time difference is like just a few seconds.

I can include the -race flag in the PR and then merge.

go tool cover -func coverage-quick.txt | grep total | awk '{print $$3}'

.PHONY: validate\:codecov
validate\:codecov:
Expand Down
4 changes: 2 additions & 2 deletions codecov.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ ignore:
- ".github"
- "build"
- "**/*.graphql"
- "**/*_test.go"
- "**/*.md"
- "**/*.txt"
- "db/tests"
- "licenses"
- ".gitignore"
- ".golangci.sourceinc.yaml"
- "Makefile"
# - "**/*_test.go"
# - "db/tests"