Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

idea: break away from GOPRIVATE? #276

Closed
mvdan opened this issue Mar 15, 2021 · 2 comments
Closed

idea: break away from GOPRIVATE? #276

mvdan opened this issue Mar 15, 2021 · 2 comments
Labels
enhancement New feature or request

Comments

@mvdan
Copy link
Member

mvdan commented Mar 15, 2021

Piggybacking off of GOPRIVATE is great for a number of reasons:

  • People tend to obfuscate private code, whose package paths will generally be in GOPRIVATE already
  • Its meaning and syntax are well understood
  • It allows all the flexibility we need without adding our own env var or config option

However, it has one drawback. It's fairly common to also want to obfuscate public dependencies, even including the standard library, to make the code in private packages even harder to follow. However, using something like GOPRIVATE=* will result in two main downsides:

  • GONOPROXY defaults to GOPRIVATE, so the proxy would be entirely disabled. This means that downloading modules, such as when adding or updating dependencies, or when the local cache is cold, can be less reliable.
  • GONOSUMDB defaults to GOPRIVATE, so the sumdb server would be entirely disabled. This means that adding entries to go.sum, such as when adding or updating dependencies, can be less secure.

In other words, if someone manages to hijack a github repository for a public dependency of yours, or the original upstream simply re-tags a previous version with bad code, neither proxy.golang.org nor sum.golang.org would save you.

I will admit that the second isn't really a problem in practice, because starting in Go 1.16, go build won't modify go.mod or go.sum by default. We don't have garble mod or garble get, so the only way to get a bad build would be garble build -mod=mod. We could potentially fix this by rejecting the use of -mod=mod and only accepting the default of -mod=readonly, but this still leaves the issue with the proxy.

Not using the proxy by itself isn't terrible, as one can't end up with bad/malicious builds, but it can still mean slower downloads (git clones download more data) and potential failures (e.g. if a VCS host is down).

I think we should add our own env var, with the same syntax as GOPRIVATE, but which only affects the decisions on what packages to obfuscate. We could call it GOGARBLE, but I'm open to ideas. Our logic would be as follows, in order:

  1. If GOGARBLE is set and non-empty, use it as a package prefix list
  2. Otherwise, if GOPRIVATE is set, use it as if it were GOGARBLE
  3. Otherwise, use the main module path as GOGARBLE

Right now, we just do steps 2 and 3. Adding step 1 would allow using GOGARBLE=* without having to give up on the proxy and sumdb entirely. Users should still set GOPRIVATE though, to ensure that they don't try to fetch private packages via the public servers.

Because step 2 and 3 are still there, this change should be fully backwards compatible. We could consider removing step 2 in the future, though I think it works OK as a first default.

@mvdan mvdan added the enhancement New feature or request label Mar 15, 2021
@mvdan mvdan added this to the v0.3.0 milestone Apr 8, 2021
@mvdan
Copy link
Member Author

mvdan commented Apr 17, 2021

I'm now thinking that, instead, we want to just obfuscate all packages all the time. Why wouldn't we? It's overall stronger obfuscation, fewer knobs for the user make the tool easier to use, and the tool itself is easier to maintain and test.

That will require us to be able to obfuscate all packages though, and we currently have some bugs preventing that. See #193 for example.

For now, I'm putting this back in the backlog. Hopefully we can remove this option entirely, without needing to introduce GOGARBLE.

@mvdan
Copy link
Member Author

mvdan commented Jun 20, 2021

I'm now thinking that, instead, we want to just obfuscate all packages all the time. Why wouldn't we? It's overall stronger obfuscation, fewer knobs for the user make the tool easier to use, and the tool itself is easier to maintain and test.

Here's one reason not to: wanting to publish code for an obfuscated library, which has dependencies on open source libraries such as the standard library. You could potentially also publish obfuscated versions of those, but that could get hairy - you could end up with duplicate std packages, for example. And I don't think it's possible to "publish" the runtime package that way.

One such example is https://github.com/unidoc/unipdf, which seems to use an obfuscator of some sort: https://www.unidoc.io/uniguard

mvdan added a commit to mvdan/garble-fork that referenced this issue Dec 3, 2021
Piggybacking off of GOPRIVATE is great for a number of reasons:

* People tend to obfuscate private code, whose package paths will
  generally be in GOPRIVATE already

* Its meaning and syntax are well understood

* It allows all the flexibility we need without adding our own env var
  or config option

However, using GOPRIVATE directly has one main drawback.
It's fairly common to also want to obfuscate public dependencies,
to make the code in private packages even harder to follow.
However, using "GOPRIVATE=*" will result in two main downsides:

* GONOPROXY defaults to GOPRIVATE, so the proxy would be entirely disabled.
  Downloading modules, such as when adding or updating dependencies,
  or when the local cache is cold, can be less reliable.

* GONOSUMDB defaults to GOPRIVATE, so the sumdb would be entirely disabled.
  Adding entries to go.sum, such as when adding or updating dependencies,
  can be less secure.

We will continue to consume GOPRIVATE as a fallback,
but we now expect users to set GOGARBLE instead.
The new logic is documented in the README.

Fixes burrowers#276.
mvdan added a commit to mvdan/garble-fork that referenced this issue Dec 3, 2021
Piggybacking off of GOPRIVATE is great for a number of reasons:

* People tend to obfuscate private code, whose package paths will
  generally be in GOPRIVATE already

* Its meaning and syntax are well understood

* It allows all the flexibility we need without adding our own env var
  or config option

However, using GOPRIVATE directly has one main drawback.
It's fairly common to also want to obfuscate public dependencies,
to make the code in private packages even harder to follow.
However, using "GOPRIVATE=*" will result in two main downsides:

* GONOPROXY defaults to GOPRIVATE, so the proxy would be entirely disabled.
  Downloading modules, such as when adding or updating dependencies,
  or when the local cache is cold, can be less reliable.

* GONOSUMDB defaults to GOPRIVATE, so the sumdb would be entirely disabled.
  Adding entries to go.sum, such as when adding or updating dependencies,
  can be less secure.

We will continue to consume GOPRIVATE as a fallback,
but we now expect users to set GOGARBLE instead.
The new logic is documented in the README.

While here, rewrite some uses of "private" with "to obfuscate",
to make the code easier to follow and harder to misunderstand.

Fixes burrowers#276.
@lu4p lu4p closed this as completed in fceb19f Dec 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

1 participant