cmd/go: reject vN.*.*+incompatible as a version if vN/go.mod exists

[bcmills]

(From a golang-nuts thread.)

What version of Go are you using (go version)?

~$ go version
go version devel +7d98da8d31 Tue Jan 7 19:53:19 2020 +0000 linux/amd64

What did you do?

Run go get in module mode, requesting an incompatible major version of a module that has a go.mod file corresponding to that major version using the “major subdirectory” layout, but lacks a go.mod file altogether at the repo root.

~$ go clean -modcache
~$ GOPROXY=direct GO111MODULE=on go get -d github.com/nicksnyder/[email protected]
go: downloading github.com/nicksnyder/go-i18n v2.0.3+incompatible
go: github.com/nicksnyder/go-i18n v2.0.3 => v2.0.3+incompatible

What did you expect to see?

An error message indicating that v2.0.3+incompatible is not valid for github.com/nicksnyder/go-i18n because the repository contains a v2/go.mod.

(See #32695 and #34165.)

What did you see instead?

~$ GOPROXY=direct GO111MODULE=on go get -d github.com/nicksnyder/[email protected]
go: downloading github.com/nicksnyder/go-i18n v2.0.3+incompatible
go: github.com/nicksnyder/go-i18n v2.0.3 => v2.0.3+incompatible

CC @matloob @jayconrod @thepudds @heschik @hyangah @katiehockman

[bcmills]

I've marked this issue as NeedsDecision rather than NeedsFix because although fixing this issue would be more correct, it would potentially invalidate some (small but unknown) fraction of existing dependencies, and (because of the fix for #34165) it is relatively easy for the module owner to mitigate by adding a go.mod file to the latest v0 or v1 release.

[jayconrod]

So with one of these versions, users would not be able to import anything in github.com/nicksnyder/go-i18n/v2, right, since it's a nested module? This means folks are requiring the v0 / v1 module at an incompatible version? It looks like go get -d github.com/nicksnyder/go-i18n@latest will downgrade to a v0 / v1 version at tip, but not in 1.13.5 (I guess that's #34165), which is good.

I'd lean toward fixing this, but we should understand how many module versions this will break. Hopefully a very small number.

[bcmills]

Correct, a user could not import packages from /v2 by depending on the unversioned +incompatible root. The v2/go.mod file gets in the way, but if there are packages in other subdirectories, they could still be imported.

The @latest mitigation helps a bit, but we can still end up with bogus pseudo-versions if the user requests a specific commit hash, which seems like it could lead to confusion.

example.com$ gotip mod init example.com
go: creating new go.mod: module example.com

example.com$ gotip get -d github.com/nicksnyder/[email protected]
go: downloading github.com/nicksnyder/go-i18n v2.0.3+incompatible
go: finding github.com v2.0.3
go: finding github.com/nicksnyder v2.0.3
go: github.com/nicksnyder/go-i18n v2.0.3 => v2.0.3+incompatible

example.com$ gotip list github.com/nicksnyder/go-i18n/...
go: warning: "github.com/nicksnyder/go-i18n/..." matched no packages

example.com$ gotip get -d github.com/nicksnyder/go-i18n@48ecfc2ea7b054b6a1717f2cc40c63d4097d9576
go: finding github.com/nicksnyder/go-i18n 48ecfc2ea7b054b6a1717f2cc40c63d4097d9576
go: github.com/nicksnyder/go-i18n 48ecfc2ea7b054b6a1717f2cc40c63d4097d9576 => v2.0.3-0.20190613172435-48ecfc2ea7b0+incompatible
go: downloading github.com/nicksnyder/go-i18n v2.0.3-0.20190613172435-48ecfc2ea7b0+incompatible

[ianlancetaylor]

This issue is currently in the 1.15 milestone. Should we roll it forward?

[bcmills]

Done.

[ianlancetaylor]

Is there anything to do here for 1.17? Thanks.

[bcmills]

I still think we should add a diagnostic for this, but to my knowledge nobody is planning to do that for a specific release — so moving to Backlog.

[bcmills]

A partial fix for this falls out naturally from #51324: if we don't implicitly add +incompatible to pseudo-versions, then we won't implicitly add +incompatible to release versions either, and we'll pick compatible (instead of +incompatible) pseudo-versions.

However, at this point I do plan to retain compatibility for the +incompatible versions generated by previous versions of the go command. (I'll talk with @rsc and @matloob out-of-band to decide whether we want to take the further step of rejecting those.)

[gopherbot]

Change https://go.dev/cl/387675 mentions this issue: cmd/go: avoid +incompatible major versions if a go.mod file exists in a subdirectory for that version

[gopherbot]

Change https://go.dev/cl/387921 mentions this issue: [release-branch.go1.18] cmd/go: avoid +incompatible major versions if a go.mod file exists in a subdirectory for that version

[gopherbot]

Change https://go.dev/cl/387922 mentions this issue: [release-branch.go1.17] cmd/go: avoid +incompatible major versions if a go.mod file exists in a subdirectory for that version

[gopherbot]

Change https://go.dev/cl/387923 mentions this issue: [release-branch.go1.16] cmd/go: avoid +incompatible major versions if a go.mod file exists in a subdirectory for that version