v6 is significantly slower than v5

[Samze]

We just upgraded to counterfeiter v6 and noticed that our fake generation is taking much longer than before.

Here are the time differences between v5 and v6.0.2 to generate 32 fakes:

v5

→ time go generate ./...
Writing `FakeBindingCreateUsecase` to `commandsfakes/fake_binding_create_usecase.go`... Done
Writing `FakeBindingListUsecase` to `commandsfakes/fake_binding_list_usecase.go`... Done
...
...

real	0m2.547s
user	0m2.404s
sys	0m0.888s

v6.0.5

→ time go generate ./...
Writing `FakeBindingCreateUsecase` to `commandsfakes/fake_binding_create_usecase.go`... Done
Writing `FakeBindingListUsecase` to `commandsfakes/fake_binding_list_usecase.go`... Done
...
...

real	0m49.541s
user	1m25.351s
sys	0m21.339s

Approximately 30x slower.

In case it is relevant, this is the repo we are using where the fakes are declared and this is an example of how counterfeiter is being run:

//go:generate counterfeiter . InstanceCreateUsecase

type InstanceCreateUsecase interface {
	Create(name, planName, serviceName, brokerName string) error
}

Thanks
Sam & @jamesjoshuahill

[joefitzgerald]

Thanks @Samze and @jamesjoshuahill, yes this is a known issue. In order to support go modules, counterfeiter (v6) switched to use go/packages. go/packages depends on go list to load a go package. go list invocations are not cached at this time, and are somewhat expensive.

Over time, go list will get more efficient, but this will only partially help. Your issues are compounded by the number of invocations of counterfeiter in your package. https://github.com/cloudfoundry/cli has it worse than you.

Our only other recourse is to implement caching inside counterfeiter. This requires:

• Make invocations 2...n invocations of //go:generate counterfeiter (or //go:generate go run github.com/maxbrunsfeld/counterfeiter/v6, or //go:generate gobin -m -r github.com/maxbrunsfeld/counterfeiter/v6) in a package into no-ops
• Make the first invocation of counterfeiter in a package into something that:
  • Attempts to identify all //go:generate invocations of counterfeiter in a given package (along with their flags) to build up a work list
  • Performs work for each item in the list, caching loaded packages where possible

This is likely to speed things up considerably but is quite a bit of work. I tend to think this may be worth the effort, but I have not had time to prioritize it (I got halfway through an implementation about 5 months ago, but abandoned it).

[williammartin]

Thanks for looking into this @joefitzgerald, unfortunately it's so frustratingly slow I find myself going back to v5.

[joefitzgerald]

The good news is we got a huge speedup from #123, released in v6.1.0.

I am not sure where else to look for speed now. I've profiled counterfeiter and time is almost exclusively spent in go/packages.Load and functions invoked therein.

I'm open to ideas, but perhaps we should open new issues with those ideas?

[joefitzgerald]

This is now fixed in v6.2.0 of counterfeiter, which uses caching and more granular options for package loading.