cmd/go: maintain a GOPATH-wide go.sum

[FiloSottile]

vgo keeps a global cache. In the same way it should keep a global go.modverify file, reminiscent of ssh's known_hosts.

This reduces the TOFU window of attack to just the first time a developer machine fetches a repository, which combined with the repository's go.modverify provides a strong trust waterfall.

When developers maintain multiple projects it will also make the risk of detection propagate to multiple communities, making targeted attacks even more different.

Finally, it could eventually be compared against and merged with any new go.modverify encountered, making verification automatic and making trust also flow from users back to the developer.

Aside from security, it will also provide more pressure against changing tag values, because it will be harder to remove a pin.

While the cache might need size management and/or expiration features, the global go.modverify should not reach meaningful size issues, preventing eviction attack and providing permanent pinning. (And maybe the base for future gossiping features.)

[sdboyer]

Strong agree - the class of information being kept in go.modverify makes more sense in some kind of central location, rather than being split up across projects.

[FiloSottile]

To clarify, I think it's important for it to be both in the repository (#24116) and in a global location, possibly with cross-pollination.

The go.modverify in the repository is critical to ensure users have only a single well-understood point of security failure: fetching the source.

[airylinus]

I agree with that.
For instance, package golang.org/x/net actually repo locates is github.com/golang/net. This is pretty creepy when you clone a new repo and stuck at a go build error.

[rsc]

Not yet. We're fixing one thing at a time. The first thing to fix is management of versions at all. The second thing is verification. There's no need to do both at once. We've gotten by this long with "go get" with no modverify. Let's get versions into go first, and then turn our attention to verifying.

[rsc]

Brad asked me to kill uses of NeedsFixButNotYet, so I'm going to call this NeedsDecision.

[bcmills]

I have a (non-global) counterproposal in #28802. I'll follow up with @FiloSottile to try to better understand how the differences play out under various possible threat models.

[rsc]

Re @bcmills's comment, #28802 would probably help but is not a complete solution. For example, it does not address go get -u. This issue is about a complete solution.

[rsc]

@FiloSottile explained to me off-issue that "global" here really meant "GOPATH-wide".
(Note the mention of "global cache".)

[rsc]

For what it's worth, I think a GOPATH-wide go.sum is a mistake. It introduces a new not-automatically-managed piece of state into what is supposed to be a stateless cache. We've worked hard to eliminate state and state manipulations (like needing to run "go install" to populate the stateful cache that is GOPATH/pkg). The notary should take care of the underlying need here.

The only override for a problem here would be to edit the GOPATH-wide go.sum, which is what I mean by having a not-automatically-managed piece of state. Now the cache is not a cache.

For a similar reason I think we should probably not do #28802: recovering from a problem is too hard and depends on manipulating state outside the current module.

[rsc]

The sumdb support caches results from the checksum database but also reauthenticates them before use. That has the effect of maintaining a GOPATH-wide go.sum but avoids the state problems I mentioned in the last comment. I'm going to close this as addressed by use of the checksum database with this caching (and otherwise not something we want to do).