cmd/go: "go build" fails if go.sum is not writable

[atombender]

What version of Go are you using (go version)?

$ go version
1.11.5

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/alex/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/alex/.go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/alex/Trash/gotest/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qs/wpmg19r12_9_nz7pvvs2_82r0000gn/T/go-build642111807=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a Dockerfile that runs go mod download followed by go build. Both the download and the build are run as an unprivileged user against a read-only codebase whose files are owned by root and not writable by anyone else.

A rudimentary dockerfile:

FROM golang:1.11-alpine AS build
RUN apk add --no-cache curl git ca-certificates \
  && addgroup -S build && adduser -S -G build build
WORKDIR /srv
COPY . ./
USER build
RUN go mod download
RUN go mod verify
RUN CGO_ENABLED=0 go build -mod=readonly -o ./server ./cmd/server

Possibly related to #29452?

What did you expect to see?

I expect go build to not attempt to write go.sum, especially if -mod=readonly is provided. I can obviously chmod the files to be writable, but this still shouldn't fail.

What did you see instead?

go: writing go.sum: open /srv/go.sum: permission denied

[agnivade]

/cc @bcmills

[dsymonds]

This was biting me just now. I believe the source of the difference is that my local dev machine has a newer version of Go than the continuous build I'm using, so when it runs go get -d ./... it gets a slightly different set of dependencies, then fails because it can't write go.sum.

I don't think failing to write go.sum should cause the go tool to exit unsuccessfully.

[dsymonds]

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

[bcmills]

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

That part is necessary in general:

• To ensure that we don't lose existing information if the go command is killed partway through a write, we write to a temporary file and then rename it over the existing file.
• That rename operation fails if the temporary file is not on the same filesystem as the destination.
• The most portable way to ensure that both files are on the same filesystem is to construct them in the same directory.

[dsymonds]

I'm not sure I see why the go tool is so different to the multitude of tools that have existed for decades that quite happily use $TMPDIR for the same effect.

[griesemer]

Related to #34860?

[bcmills]

Somewhat? They're mostly orthogonal, I think, but this issue can make the failure mode for #34860 worse (or less explicable).

[dowgird]

This makes guru fail on (some) projects fetched from Github. Reproduction & error message pasted below to make discovery easier.

go get github.com/GoogleCloudPlatform/osconfig
guru "-scope" "github.com/GoogleCloudPlatform/osconfig" "-tags=" "callstack" "$GOPATH/src/github.com/GoogleCloudPlatform/osconfig/inventory/inventory.go:#1629"

Result (snippet):

 [redacted]/go/pkg/mod/github.com/golang/[email protected]/ptypes/duration/duration.pb.go:8:8: could not import github.com/golang/protobuf/proto (go/build: importGo github.com/golang/protobuf/proto:      exit status 1
    error writing go.mod: open [redacted]/go/pkg/mod/github.com/golang/[email protected]/go.mod298498081.tmp: permission denied

[gopherbot]

Change https://golang.org/cl/204521 mentions this issue: cmd/go: default to mod=readonly when the go.mod file is read-only

[bcmills]

The use of $TMPDIR will be addressed incidentally by the fix for #34634.

The remainder is working as designed per #30667.

[bcmills]

Duplicate of #34054