add experimental support for go.tools

[myitcv]

golang/go#25922 gives guidance on the best practice for adding tool dependencies to a module. In short, the tool dependencies get added to the main module via "fake" imports of the main package that is the tool in a build-ignore file, say tools.go.

There are a number of benefits and issues with this approach; see golang/go#25922 for further discussion on that.

As a trial, let's make gobin understand a go.tools file in the same directory as the main module and take tool versions from that. The format would be very simple:

// go.tools
github.com/rogpeppe/[email protected]
myitcv.io/cmd/[email protected]
golang.org/x/tools/cmd/goimports

golang.org/x/tools/cmd/goimports is special be because the version of golang.org/x/tools/cmd/goimports we want does not resolve to a module. This same restriction applies for any tools that resolve to modules that are incomplete, i.e. go.{mod,sum} are not the result of go mod tidy. The lack of version indicates that for the tool in question we drop down to the main module's go.{mod,sum} to resolve dependencies, i.e. the user also needs to follow the approach outlined in golang/go#25922.

Open questions:

• For a tool that is not part of a module, or where that module is incomplete (go.{mod,sum} not the result of go mod tidy), use of go.tools is a strictly worse experience than the approach in cmd/go: clarify best practice for tool dependencies golang/go#25922. Should we be doing anything more here?
  • The "no version" drop down to the main module seems to work
• Using go.tools, we lose the ability to use our go.sum as a means of verifying any dependencies, tools or otherwise. Instead we end up relying on the go.sum (if there is one) that is distributed with the tool. Perhaps we should capture these go.sum's somewhere too?
  • Maintain a go.tools.sum file that is the concatenation of the go.sum's of the unique set of $module@$version from the set of tools
  • If the go tools starts to understand go.tools, then this would be unnecessary
  • We will need gobin -tidy etc.

cc @rogpeppe @mvdan

[myitcv]

Just noting the detail here, but another potential line of attack on capturing tool dependencies separately from the main module is to use an internal submodule.

For example, consider the module myitcv.io. If we had development-time-only tool dependencies, then we could capture those in the module myitcv.io/internal/tools.

This would likely, however, require a new flag for gobin -m (this only makes sense for gobin -m). Continuing the example from before:

$ go list -m
myitcv.io
$ go list
myitcv.io/gogenerate
$ gobin -m -C myitcv.io/internal/tools -run golang.org/x/tools/cmd/goimports ./...

We are running commands from within the gogenerate subdirectory of the myitcv.io module. Hence gobin would need to be told to resolve its -m dependencies via a specific module, else the main module (myitcv.io) would be assumed, before then executing the command (goimports in this case) in the current directory (i.e. the gogenerate subdirectory).