replace homegrown testing library with something off-the-shelf

[rade]

Our testing util library is growing. It's still quite small, but at some point we should probably bite the bullet and pull in something like gocheck or testify.

[paulbellamy]

Advantage of testify is that it is a drop-in replacement for our custom testing package.

Have asked @peterbourgon if he has any opinion...

[paulbellamy]

Per f2f discussion with @rade, need to see if gocheck provides better failure stacktraces. testify just gives line-number of assertion, which makes refactoring assertions harder.

[peterbourgon]

I fully endorse the Go philosophy of testing as summarized in the FAQ, quote,

[T]esting frameworks tend to develop into mini-languages of their own, with conditionals and controls and printing mechanisms, but Go already has all those capabilities; why recreate them? We'd rather write tests in Go; it's one fewer language to learn and the approach keeps the tests straightforward and easy to understand.

I suppose testify is preferable to gocheck, and either is preferable to goconvey. But (brutal honesty ahead) I don't see much value in any of these functions and would prefer to just inline them in the relevant tests, eliminating the package altogether.

[rade]

prefer to just inline them

Thereby turning every assertion in a three-line if?!?

[peterbourgon]

Thereby turning every assertion in a three-line if?!?

Sure. For example. Neither newlines nor vertical screen space are limited resources :)

[rade]

That is gross, but I guess we'll just have to agree to disagree.

[paulbellamy]

After trying them: unlike testify, gocheck does give stacktraces of failures

[paulbellamy]

But, does come with a lot of other baggage. :(

[peterbourgon]

Curious: you can generate a stacktrace on demand with debug.PrintStack; what does integration with a testing package get you?

[paulbellamy]

It tells us which test failed, in the case where the assertion is not at the top level. e.g.

func TestSomething(t *testing.T) {
  helper(t)
}

func TestSomethingElse(t *testing.T) {
  helper(t)
}

func helper(t *testing.T) {
  t.Fatal("failure!")
}

Edit: and also the line-number within the test which failed.

[paulbellamy]

So, the options so far seem to be:

1. Use gocheck, and deal with it's baggage.
2. Just do all checks inline, with no helper functions
3. Fork testify/assert, and add stacktraces to it

Of those, I prefer 3. By inlining everything, I worry we'll end up re-inventing stuff all over the place, and end up back where we started.

[peterbourgon]

@paulbellamy what about wrapping Fatals with something that invokes stack.Caller?

[paulbellamy]

@peterbourgon Do you mean like this? Or something else in mind? https://github.com/weaveworks/weave/blob/master/testing/util.go#L133

[peterbourgon]

@paulbellamy Ah, I see. Yes. I'd rename that to e.g. FatalWithStack but that's what I was driving at. Single-purpose wrappers like that are, I think, the best available pattern.

[paulbellamy]

@peterbourgon Right. Given that, I'd also be ok with inlining checks, and using the "XxxWithStack" variants in any helpers.