Skip to content
This repository has been archived by the owner on Jun 20, 2024. It is now read-only.

TestNameservers is flaky #1158

Closed
tomwilkie opened this issue Jul 15, 2015 · 6 comments
Closed

TestNameservers is flaky #1158

tomwilkie opened this issue Jul 15, 2015 · 6 comments
Assignees
@tomwilkie
Labels
bug
Milestone
1.1.0

Comments

@tomwilkie
Copy link
Contributor

Very occasionally (1 in 15 runs so far)

--- FAIL: TestNameservers-4 (2.77s)
        Error Trace:    nameserver_test.go:95
            nameserver_test.go:149
            nameserver_test.go:191
    Error:      Not equal: "51.51.253.7" (expected)
                    != "" (actual)

FAIL
coverage: 60.2% of statements
@rade rade added bug and removed chore labels Jul 15, 2015
@rade rade added this to the current milestone Jul 15, 2015
@paulbellamy
Copy link
Contributor

This test has 5^1000 codepaths, and no output as to which one it picked.

@paulbellamy paulbellamy mentioned this issue Jul 17, 2015
Closed
@tomwilkie
Copy link
Contributor Author

I can't get this to flake:

vagrant@ubuntu-14:~/src/github.com/weaveworks/weave/nameserver$ while true; do go test -cpu 4 -run TestNameserver; done
PASS
ok      github.com/weaveworks/weave/nameserver  5.699s
PASS
ok      github.com/weaveworks/weave/nameserver  5.307s
PASS
ok      github.com/weaveworks/weave/nameserver  4.947s
PASS
ok      github.com/weaveworks/weave/nameserver  4.859s
PASS
ok      github.com/weaveworks/weave/nameserver  4.897s
PASS
ok      github.com/weaveworks/weave/nameserver  6.186s
PASS
ok      github.com/weaveworks/weave/nameserver  6.016s
PASS
ok      github.com/weaveworks/weave/nameserver  5.198s
PASS
ok      github.com/weaveworks/weave/nameserver  5.485s
PASS
ok      github.com/weaveworks/weave/nameserver  6.347s
PASS
ok      github.com/weaveworks/weave/nameserver  5.380s
PASS
ok      github.com/weaveworks/weave/nameserver  5.129s
PASS
ok      github.com/weaveworks/weave/nameserver  5.835s
PASS
ok      github.com/weaveworks/weave/nameserver  5.351s
PASS
ok      github.com/weaveworks/weave/nameserver  5.790s
PASS
ok      github.com/weaveworks/weave/nameserver  5.917s
PASS
ok      github.com/weaveworks/weave/nameserver  5.395s
PASS
ok      github.com/weaveworks/weave/nameserver  6.054s
PASS
ok      github.com/weaveworks/weave/nameserver  5.713s
PASS
ok      github.com/weaveworks/weave/nameserver  5.684s
PASS
ok      github.com/weaveworks/weave/nameserver  5.204s
PASS
ok      github.com/weaveworks/weave/nameserver  5.489s
PASS
ok      github.com/weaveworks/weave/nameserver  5.947s
PASS
ok      github.com/weaveworks/weave/nameserver  5.844s
PASS
ok      github.com/weaveworks/weave/nameserver  5.923s
PASS
ok      github.com/weaveworks/weave/nameserver  5.444s
PASS
ok      github.com/weaveworks/weave/nameserver  5.213s
PASS
ok      github.com/weaveworks/weave/nameserver  5.489s
PASS
ok      github.com/weaveworks/weave/nameserver  5.677s
PASS
ok      github.com/weaveworks/weave/nameserver  6.083s
PASS
ok      github.com/weaveworks/weave/nameserver  5.896s
PASS
ok      github.com/weaveworks/weave/nameserver  5.455s
PASS
ok      github.com/weaveworks/weave/nameserver  5.910s
PASS
ok      github.com/weaveworks/weave/nameserver  5.828s
PASS
ok      github.com/weaveworks/weave/nameserver  5.283s
PASS
ok      github.com/weaveworks/weave/nameserver  6.045s
PASS
ok      github.com/weaveworks/weave/nameserver  6.276s
PASS
ok      github.com/weaveworks/weave/nameserver  6.032s
PASS
ok      github.com/weaveworks/weave/nameserver  6.292s
PASS
ok      github.com/weaveworks/weave/nameserver  5.161s
PASS
ok      github.com/weaveworks/weave/nameserver  5.889s
PASS
ok      github.com/weaveworks/weave/nameserver  5.892s
PASS
ok      github.com/weaveworks/weave/nameserver  7.363s
PASS
ok      github.com/weaveworks/weave/nameserver  5.592s
PASS
ok      github.com/weaveworks/weave/nameserver  5.585s
PASS
ok      github.com/weaveworks/weave/nameserver  5.394s
PASS
ok      github.com/weaveworks/weave/nameserver  5.924s
PASS
ok      github.com/weaveworks/weave/nameserver  6.011s
PASS
ok      github.com/weaveworks/weave/nameserver  5.759s
PASS
ok      github.com/weaveworks/weave/nameserver  7.032s
PASS
ok      github.com/weaveworks/weave/nameserver  7.733s
PASS
ok      github.com/weaveworks/weave/nameserver  8.025s
PASS
ok      github.com/weaveworks/weave/nameserver  5.542s
PASS
ok      github.com/weaveworks/weave/nameserver  5.727s
PASS
ok      github.com/weaveworks/weave/nameserver  5.821s
PASS
ok      github.com/weaveworks/weave/nameserver  7.953s
PASS
ok      github.com/weaveworks/weave/nameserver  9.224s
PASS
ok      github.com/weaveworks/weave/nameserver  8.081s
PASS
ok      github.com/weaveworks/weave/nameserver  7.244s
PASS
ok      github.com/weaveworks/weave/nameserver  6.637s
PASS
ok      github.com/weaveworks/weave/nameserver  5.931s
PASS
ok      github.com/weaveworks/weave/nameserver  7.469s
PASS
ok      github.com/weaveworks/weave/nameserver  7.108s
PASS
ok      github.com/weaveworks/weave/nameserver  7.519s
PASS
ok      github.com/weaveworks/weave/nameserver  5.797s
PASS
ok      github.com/weaveworks/weave/nameserver  7.135s
PASS
ok      github.com/weaveworks/weave/nameserver  8.691s
PASS
ok      github.com/weaveworks/weave/nameserver  -20.906s
PASS
ok      github.com/weaveworks/weave/nameserver  7.213s
PASS
ok      github.com/weaveworks/weave/nameserver  7.254s
PASS
ok      github.com/weaveworks/weave/nameserver  6.347s

@tomwilkie
Copy link
Contributor Author

Reproduction:

https://circleci.com/gh/weaveworks/weave/1989

make: Entering directory `/home/go/src/github.com/weaveworks/weave'
go get ./testing/cover
go build -o testing/cover/cover ./testing/cover
./nameserver
--- FAIL: TestNameservers-4 (7.10s)
    Error Trace:    nameserver_test.go:96
            nameserver_test.go:122
            nameserver_test.go:186
    Error:      Not equal: "43.85.152.248 68.135.81.28" (expected)
                    != "68.135.81.28" (actual)


FAIL
coverage: 60.4% of statements
FAIL    github.com/weaveworks/weave/nameserver  10.016s

@tomwilkie tomwilkie self-assigned this Jul 20, 2015
@tomwilkie tomwilkie mentioned this issue Jul 20, 2015
Merged
@tomwilkie
Copy link
Contributor Author

On my VM with 4 cores I get one failure in about 1 hour:

^Cvagrant@twilkie-test-1:~/src/github.com/weaveworks/weave/nameserver$ while true; do ../test/units.sh -slow; done
ok      github.com/weaveworks/weave/nameserver  53.243s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.687s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.608s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  33.929s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.315s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.691s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  54.456s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.590s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.775s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.322s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.003s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  54.311s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  41.966s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.998s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  51.021s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.524s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.954s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  53.669s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.693s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.573s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  34.318s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  44.519s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  63.501s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.248s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.382s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  54.161s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  54.548s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  52.929s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  53.143s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.980s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  44.573s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  47.935s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  51.575s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  44.216s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  44.320s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.113s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.640s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.743s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  52.525s coverage: 61.4% of statements
--- FAIL: TestNameservers-4 (15.44s)
        Error Trace:    nameserver_test.go:96
            nameserver_test.go:150
            nameserver_test.go:192
    Error:      Not equal: "103.44.176.76 111.223.1.2" (expected)
                    != "111.223.1.2" (actual)

FAIL
coverage: 61.4% of statements
FAIL    github.com/weaveworks/weave/nameserver  20.080s
ok      github.com/weaveworks/weave/nameserver  62.970s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  34.366s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.650s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  41.682s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  43.978s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  51.332s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.106s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.837s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.675s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.002s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.466s coverage: 60.9% of statements
ok      github.com/weaveworks/weave/nameserver  44.117s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  42.523s coverage: 61.4% of statements
ok      github.com/weaveworks/weave/nameserver  53.718s coverage: 61.4% of statements

@bboreham
Copy link
Contributor

Pattern always seems to be it is expecting 1 more address than it gets.

@tomwilkie
Copy link
Contributor Author

Yes I suspect its because a dropped message. I'm going to make it fail more, then try and fix that.

@tomwilkie tomwilkie mentioned this issue Jul 20, 2015
Merged
@rade rade modified the milestones: current, 1.1.0 Jul 21, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@tomwilkie tomwilkie

Labels
bug
Projects
None yet
Milestone
1.1.0
Development

No branches or pull requests
4 participants
@rade @paulbellamy @tomwilkie @bboreham