cmd/go: support vendoring in workspace mode #60056
Comments
cherrymui
added
GoCommand
|
The lack of vendor support makes go workspaces very difficult for Kubernetes to adopt. I don't care if it is exactly identical to classic vendor support, but we need something...
|
|
Change https://go.dev/cl/495801 mentions this issue: |
matloob
changed the title
matloob
removed
the
NeedsInvestigation
|
Change https://go.dev/cl/499977 mentions this issue: |
We might be introducing vendoring for workspace mode. Set GOWORK=off when running go mod vendor in a single module context to make sure it has the right behavior and doesn't return an error. For golang/go#60056 Change-Id: I703d74d579aec6e4dad86ca092e3651e0b2e4eb0 Reviewed-on: https://go-review.googlesource.com/c/tools/+/499977 TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Michael Matloob <[email protected]> gopls-CI: kokoro <[email protected]> Reviewed-by: Robert Findley <[email protected]>
|
This proposal has been added to the active column of the proposals project |
|
Based on the discussion above, this proposal seems like a likely accept. |
|
No change in consensus, so accepted. 🎉 |
rsc
changed the title
In most cases this change removes assumptions that there is a single main module in vendor mode and iterates over the workspace modules when doing checks. The go mod vendor command will now, if in workspace mode, create a vendor directory in the same directory as the go.work file, containing the packages (and modules in modules.txt) loaded from the workspace. When reassembling the module graph from the vendor directory, an edges are added from each of the main modules to their requirements, plus additionally to a fake 'vendor/modules.txt' module with edges to all the modules listed in vendor/modules.txt. For #60056 Change-Id: I4a485bb39836e7ab35cdc7726229191c6599903e Reviewed-on: https://go-review.googlesource.com/c/go/+/495801 Reviewed-by: Bryan Mills <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Michael Matloob <[email protected]>
|
Change https://go.dev/cl/513756 mentions this issue: |
modules.txt gets a new ## workspace line at the start of the file if it's generated in workspace mode. Then, when deciding whether the go command runs in mod=vendor, we only do so if we're in the same mode (workspace or not) as the modules.txt specifies. For #60056 Change-Id: If478a9891a7135614326fcb80c4c33a431e4e531 Reviewed-on: https://go-review.googlesource.com/c/go/+/513756 TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Bryan Mills <[email protected]> Run-TryBot: Bryan Mills <[email protected]>
|
@bcmills what is remaining to do here? |
|
Is your concern about "depending on the environment" regarding switching between workspace mode on vs. off? I believe all Go invocations in k8s will be done in workspace mode. @thockin knows best, but I've been following along and was trying to help at one point by implementing a hacky solution for a pseudo-vendor dir in workspace mode. That hack thankfully became obsolete with this feature here! |
|
Sigh, I really need to finish this work. Kubernetes has a top-level module and workspace. |
|
Per discussion with @rsc, we think this is done. |
|
Fine to close now. @thockin if you do run into any problems, please let us know. |
Yes, I was worried this change could interfere with some existing workspace mode users' current workflow who want to keep vendor in non-workspace mode while keeping their go.work files in the repo root. @rsc informed me that the latest change cl 513756 addressed it. The I tested with my existing workflow, and it seems ok so far. :-) |
|
Change https://go.dev/cl/548815 mentions this issue: |
For #61422. Updates #60056. Change-Id: Ie7c7e12acc173f1ff1644555016e51a52509bd6d Reviewed-on: https://go-review.googlesource.com/c/go/+/548815 Auto-Submit: Bryan Mills <[email protected]> Reviewed-by: Michael Matloob <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
For golang#61422. Updates golang#60056. Change-Id: Ie7c7e12acc173f1ff1644555016e51a52509bd6d Reviewed-on: https://go-review.googlesource.com/c/go/+/548815 Auto-Submit: Bryan Mills <[email protected]> Reviewed-by: Michael Matloob <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
If go.work is present and a vendor/ is present, then the module is using go workspaces with vendoring, recently introduced with Go 1.22 (see golang/go#60056 for details). When using Go workspaces, you must use `go work vendor` instead of `go mod vendor`.
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
This is a proposal to support vendoring in workspace mode.
We originally didn't support vendoring of a workspace to simplify the implementation of workspaces, but there's recently been demand by users of vendoring who want to workspaces for that support, so now is a good time to consider adding that support in.
go work vendorsubcommandUnder this proposal, a new subcommand,
go work vendor, will be added to thegocommand.go work vendorwill check to make sure that workspace mode is active when it starts, (and return an error if it isn't) and if it is, it will write a vendor directory for the workspace. The vendor directory will be created in the same directory as thego.workfile. The vendor directory will contain packages taken from the modules in the build list of the workspace. vendor/modules.txt will contain the set of packages from each of the modules in the build list as usual, will list modules with# explicitannotations for each module required by one of the workspace modules, and will annotate replacements for each of the effective replacements in the workspace (That is, if thego.workfile overrides replacements in workspace only the replacement in thego.workfile will be annotated.)go work vendorotherwise behaves the same as and takes the same flags asgo mod vendor.go mod vendorsubcommandgo mod vendorwill explicitly fail if ago.workfile is found andGOWORKis not set tooff. See discussion below.-mod=vendorin workspace mode.Under this proposal, when workspace mode is active, if
-mod=vendoris set explicitly, or it is set implicitly through the existence of a vendor directory in the same directory as thego.workThe go command will enter workspace mode and interpret the vendor directory as reflecting the workspace's modules. The go command will use the vendor directory in the same directory asgo.work.vendor/modules.txtis interpreted as reflecting the modules in the workspace. That interpretation is done through the vendoring consistency check, and the structure of the internal module graph built.The vendoring consistency check would check
vendor/modules.txtagainst the workspace modules. It will check that all explicitly required modules appear invendor/modules.txtwith an# explicitannotation and all modules invendor/modules.txtwith# explicitare required by a workspace module. Similarly, all effective replacements in the workspace are expected to appear in vendor/modules.txt and all replacements in vendor/modules.txt are expected to appear in a workspace modulego.modor thego.workfile.The way the module graph is built is similar to to that of an unpruned module. The module graph contains the main modules as its roots, and each of the main modules would require their module requirements in addition to a fake
vendor/modules.txtmodule that in turn requires the set of modules in thevendorList(the set of modules in vendor/modules.txt` that provide packages.Otherwise, vendor mode with a workspace behaves the same as vendor mode with a module. The packages in the vendor directory are used to satisfy imports in the main modules when loading packages and doing builds.
Other considerations.
go work vendorandgo mod vendor.Under the proposal as it is,
go mod vendorworks to vendor the module it's in, but fails if there's ago.workfile present andGOWORKis not set to off. If that wasn't the case, there could be confusion if ago.modandgo.workfiles exist in the same directory. Thengo mod vendorandgo work vendorwould create vendor directories in the same place, but with different contents. Since in workspace mode, the vendor directory is interpreted as a reflecting the workspace, ifgo mod vendoris run before another command in workspace mode, the vendor consistency checks will likely fail. When they fail the error message will give the user the appropriate command to rungo mod vendororgo work vendordepending on whethermodules.txtwas checked against a single module or a the workspace. Even so, the user might still get confused about why runninggo mod vendorand immediately running another command might result in an error. To reduce the confusion, we require a user to not have ago.workfile present or haveGOWORKset tooffwhengo mod vendoris run, so that when a following command is run under the same conditions, it doesn't interpret the vendor directory as reflecting the workspace.The behavior might break already existing workflows where
go mod vendoris run when ago.workis present and module vendoring is being used, but it seems better to do so than to introduce the potential confusion of the user creating a vendor directory that fails consistency checks.The text was updated successfully, but these errors were encountered: