installation problems with "v2"

[gen2thomas]

latest recognized version seems to be 1.16.0:

go install gobot.io/x/gobot@latest
go: downloading gobot.io/x/gobot v1.16.0

try to install version 2.0.0 will result in an error:

go install gobot.io/x/[email protected]
go: gobot.io/x/[email protected]: gobot.io/x/[email protected]: invalid version: module contains a go.mod file, so module path must match major version ("gobot.io/x/gobot/v2"

For root cause, see e.g.: https://research.swtch.com/vgo-import

the solution: https://github.com/golang/go/wiki/Modules#semantic-import-versioning

example: https://github.com/cpuguy83/go-md2man/blob/master/go.mod

[gen2thomas]

Version v2.0.1 is released to fix this problem, but it don't. The new version can not be found with an external example project.

go list -u -m all
go: unrecognized import path "gobot.io/x/gobot/v2": reading https://gobot.io/x/gobot/v2?go-get=1: 404 Not Found
go: gobot.io/x/gobot/[email protected]: missing go.sum entry; to add it:
	go mod download gobot.io/x/gobot/v2

I have found different resources for v2+ modules:

• https://go.dev/blog/v2-go-modules --> suggests to create a folder "v2" with a copy of code, due to best backward compatibility
• https://donatstudios.com/Go-v2-Modules --> outlines the history of v2+
• https://therootcompany.com/blog/bump-go-package-to-v2/ --> the result can be a caching problem
• working example - there are 52 versions now: https://github.com/google/go-github

[gen2thomas]

v1:

go list -m -json gobot.io/x/gobot@latest
{
	"Path": "gobot.io/x/gobot",
	"Version": "v1.16.0",
	"Query": "latest",
	"Time": "2022-05-02T19:56:19Z",
	"Dir": "/home/user/go/pkg/mod/gobot.io/x/[email protected]",
	"GoMod": "/home/user/go/pkg/mod/cache/download/gobot.io/x/gobot/@v/v1.16.0.mod",
	"GoVersion": "1.13"
}

v2+:

go list -m -json gobot.io/x/gobot/v2@latest
go: gobot.io/x/gobot/v2@latest: unrecognized import path "gobot.io/x/gobot/v2": reading https://gobot.io/x/gobot/v2?go-get=1: 404 Not Found

[gen2thomas]

It seems the simple version increasing will not work in our case, see https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher.

citation: "(Note that this advice regarding incrementing the major version when first adopting modules does not apply to pre-existing repos or packages whose latest versions are v0.x.x or v1.x.x)"

Hi @deadprogram , how we should proceed with this issue? IMO we have some possibilities:

• create a folder v2
• create a branch v2 (see also https://research.swtch.com/vgo-module)
• move v1 code away to another branch/folder (not sure if this will work)
• use a "v1.x.y+incompatible" release instead of v2+

Any suggestions/ideas?

Thanks, Thomas

[deadprogram]

Yes, only if we had started with our various code in a gobot folder inside of the main package would the module naming used by https://github.com/google/go-github work.

Considering our options here... 🤔

[deadprogram]

"create a folder v2" is the most common pattern and would be least surprising to developers.

[gen2thomas]

@deadprogram , thanks for your immediate answer. So I will remove the current releases/tags first, than create the folder and will release the "v2.0.0" again, right?

[deadprogram]

Not sure about the tags, maybe better to just add new one 2.0.3

[gen2thomas]

@deadprogram do you really mean 2.0.3? Normally the next version increment would be 2.0.2.

I would move the code to the new "v2" folder according to this example https://github.com/googleapis/gax-go. What do you think?

[deadprogram]

do you really mean 2.0.3? Normally the next version increment would be 2.0.2.

Correct, I mistyped the number.

[gen2thomas]

Thanks, and I was wrong with my given example. There was done a copy to v2 folder, so the development can also continue on version 0.x/1.x, IMO there is no other reason for this. In our case, I think, nobody wants to maintain the old version v1.x. So I move the code to the v2 folder.

[gen2thomas]

After releasing v2.0.2 it is not working completely. This means go list -m -json gobot.io/x/gobot/v2@latest is still not working, but the following call works (with 2.0.1, 2.0.2 and latest):

go list -m -json  github.com/hybridgroup/gobot/[email protected]
{
	"Path": "github.com/hybridgroup/gobot/v2",
	"Version": "v2.0.1",
	"Time": "2023-05-21T14:05:39Z",
	"GoMod": "/home/user/go/pkg/mod/cache/download/github.com/hybridgroup/gobot/v2/@v/v2.0.1.mod",
	"GoVersion": "1.17",
	"Origin": {
		"VCS": "git",
		"URL": "https://github.com/hybridgroup/gobot",
		"Ref": "refs/tags/v2.0.1",
		"Hash": "20a92a378584154b2cba2f3344e92e30e31c53ac"
	}
}

It seems there is a missing forward from the website or something else. If we can get it to work with v2.0.1 this would be nice, because moving the code to the v2 subfolder in v2.0.2 has some drawbacks.

@deadprogram do you have any idea to fix this?

[gen2thomas]

Now I have tested v2+ with a new branch "v2":

• created from release and released as v2.0.3
• branch content reverted to the tag "v2.0.1" (the one without v2 subfolder) and released as v2.0.4-pre1

both with the same result:

• gobot.io/x/gobot/v2 is not working
• github.com/hybridgroup/gobot/v2 is working

I'm running out of ideas.

[gen2thomas]

It seems "periph.io" runs in the same problem 2017 and solved this with a new start in 2020. See also this version history https://pkg.go.dev/periph.io/x/periph?tab=versions - it begins with a "v2.0.0+incompatible" tag.

Possibly this is the only solution at the moment?

[gen2thomas]

Hi @deadprogram , here for uuid we can find a good description of the problem with a nice solution (read also the later comments to get an idea of the steps to be taken)

I will try to sum up it later for gobot as a base for discussion.

[deadprogram]

Thanks @gen2thomas I will take a look.

[gen2thomas]

@deadprogram here you can find an excellent wrap up of all possible solutions with its pros and cons. This was done by investigation of multiple projects: KateGo520/Hero#1.

(IMO that document deals more or less only with compatibility for GOPATH users)

[gen2thomas]

After some more investigation I'm more wondering than before and still have no idea what went wrong. Here you can find my document: WORKFLOW_V2+.md

Conclusion:

• the transition is almost always done like we have done (change go.mod and adjust internal paths by adding "/v2", or sometimes "/v3" or "/v4")
• I have not found any project, which uses a folder "v2", some uses a branch "v2", "v3", "v4 for smooth transition of GOPATH users (as far I can tell, this do not affect module mode in any form)
• most projects run from the beginning in GOPATH mode, and just introduce v2.0.0 in GOPATH mode, which automatically produce "v2.0.0+incompatible" releases and afterwards, switch with a newer version to module mode (e.g. "/v3"), without any problem
• some projects start with go.mod from the beginning and the transition is the same without any problem
• 1 project was investigated with the same precondition like gobot, but the same transition went well without any problem

[gen2thomas]

Because there is already an "v2.0.0+incompatible" version (GOPATH, but defective), and all v2+ versions (model mode) are not available for download, I will moving forward in this direction:

• revert the move of code to the "v2" sub directory, because this is more common
  * [ ] remove the go.mod file and revert "/v2" extensions in paths
  * [ ] release this state as "2.0.5+incompatible", this should fix the usage of v2 in GOPATH mode
  * [ ] re-add go.mod
• release this as "v2.1.0", with the hope that will then work for v2+ in module mode (last try for v2.x, unless there are more ideas)
• upgrade to go 1.18 and module dependencies
• remove CLI
• prepare code for v3 (go.mod and switch all paths of local files)
• release this state as v3.0.0, hopefully this will resolve our problems for this issue

note: drop the idea to support for GOPATH in v2+, because this will force rollback of all dependencies to GOPATH-compatible versions

[gen2thomas]

Unfortunately, the new v2+ package (v2.1.0) package is still not available. But if you click on Open Source Insights at this page, you can see the package.

On the dependencies there is a problem with tinygo.org/x/bluetooth.

Maybe this prevents publishing the new package, because this is now important in module mode, but was not in GOPATH mode.

So a new idea would be to deactivate/remove the ble adaptor and remove the problematic dependency from gobot for a test with a new version 2.2.0.

@deadprogram what do you think?

[deadprogram]

could not find module for package import: device/arm
could not find module for package import: device/nrf
could not find module for package import: machine
could not find module for package import: runtime/interrupt
could not find module for package import: runtime/volatile

Those are all only included in tinygo.org/x/bluetooth when built to run bare metal on Nordic Semi microcontrollers by TinyGo, and not for Linux/macOS/Windows. The files with those references are protected by build tags like this:

https://github.com/tinygo-org/bluetooth/blob/release/adapter_nrf51.go#L1

I will try to look into what might be the problem with the Gobot build.

[gen2thomas]

problem with the Gobot build.

There is no build problem, but there is a small chance, that the publishing of the "module mode releases" (like 2.1.0) are blocked due to this missing modules. This is only a "static" dependency problem.

[deadprogram]

I think the core problem here is we were missing the needed custom domain redirect to handle the v2 suffix.

$ go get gobot.io/x/gobot/[email protected]+incompatible                                                                                                                             
go: gobot.io/x/gobot/[email protected]+incompatible: unrecognized import path "gobot.io/x/gobot/v2": reading https://gobot.io/x/gobot/v2?go-get=1: 404 Not Found                     

I added this PR to the gobot-site repo:
hybridgroup/gobot-site#81

Now the v2 Gobot package appears to be working as I expect from one of the repos that I have which uses it:
hybridgroup/gophercon-2022@e4e3f1a

This was what I first ran to update the go.mod file in that repo:

$ go get gobot.io/x/gobot/[email protected]+incompatible                                                                                                                             
go: gobot.io/x/gobot/[email protected]+incompatible: invalid version: +incompatible suffix not allowed: major version v2 is compatible                                               

Hmm now it is compatible? Sweet!

$ go get gobot.io/x/gobot/[email protected]                                                                                                                                          
go: downloading gobot.io/x/gobot/v2 v2.1.0                                                                                                                                   
go: upgraded github.com/gofrs/uuid v4.3.0+incompatible => v4.4.0+incompatible                                                                                                
go: added gobot.io/x/gobot/v2 v2.1.0                                                                                                                                         
go: upgraded golang.org/x/sys v0.1.0 => v0.8.0                                                                                                                               

After that I was able to build my code using v2 and Bluetooth:

$ go build -o keyboard-control ./keyboard/

$ ls -l keyboard-control 
-rwxrwxr-x 1 ron ron 6460290 jun  6 12:53 keyboard-control

[gen2thomas]

Hi @deadprogram , Thanks!

It sounds like a hack, but it works, also for pkg.go.dev. Lets see what happen with v3 - depending on gobot-site for all upcoming versions feels wrong to me.

[deadprogram]

It is not a hack, it is how "vanity domains" work.

See https://pkg.go.dev/cmd/go#hdr-Remote_import_paths

Also see https://twitter.com/rakyll/status/892805962867683328

[gen2thomas]

Hi @deadprogram , thanks for the enlightenment, although I have not understand the content of the provided links completely. I thought the redirect is done "automatically" in the background but I see now that there was the same for the v0/v1 present at gobot-site. So my earlier guess, was not so bad 😄:

It seems there is a missing forward from the website or something else

Thanks again, so we are prepared for v3 and I can close this issue immediately.

[deadprogram]

I wish I had noticed your earlier guess, you had the right idea about what was going wrong!