Release Pipeline failing

[dibyom]

Expected Behavior

Release pipeline works

Actual Behavior

Pipeline fails at the run-ko step:

2022/07/19 18:39:13 Unexpected error running "go build": exit status 2
# github.com/tektoncd/pipeline/cmd/entrypoint
cmd/entrypoint/main.go:142:4: unknown field 'stdoutPath' in struct literal of type realRunner
cmd/entrypoint/main.go:143:4: unknown field 'stderrPath' in struct literal of type realRunner

Steps to Reproduce the Problem

Additional Info

Full logs: https://gist.github.com/dibyom/81be5cb43e342cbcb77b9bea0926e0a3

Run: https://dashboard.dogfooding.tekton.dev/#/namespaces/default/pipelineruns/pipeline-release-run-z8vht?pipelineTask=publish-images

• Tekton Pipeline version: v0.37.1

Go version: 1.17.8

[dibyom]

I tried the failing command ko resolve --platform=linux/amd64,linux/arm,linux/arm64,linux/s390x,linux/ppc64le,windows/amd64 --preserve-import-paths -t v0.38.0 -R -f config/ locally on go version go1.18.3 darwin/amd64 and it worked

[dibyom]

cc @jerop

[jerop]

another release pipeline failure reported in #5175

[abayer]

Ok, via docker run -it gcr.io/tekton-releases/dogfooding/ko@sha256:aafcfa5c22281f9e882664c3e5f234a2069a0050674c0c8d9908ccae6f0e8457 and manually replicating the commands leading up to the ko resolve and then running ko resolve ... with --push=false, I've reproduced the error, and narrowed it down to having windows/amd64 in the platforms. And more specifically, it's only when .ko.yaml is updated to use gcr.io/tekton-releases/github.com/tektoncd/pipeline/combined-base-image@sha256:cabda55b027488777f06576dcb615779640ea9e768784b2720215b91e14cc9e0 as the base image for cmd/entrypoint, which is done by

pipeline/tekton/publish.yaml

Lines 97 to 113 in 94055d9

	COMBINED_BASE_IMAGE=$(go run ./vendor/github.com/tektoncd/plumbing/cmd/combine/main.go \
	ghcr.io/distroless/static \
	mcr.microsoft.com/windows/nanoserver:ltsc2022 \
	${CONTAINER_REGISTRY}/$(params.package)/combined-base-image:latest)
	cat <<EOF > ${PROJECT_ROOT}/.ko.yaml
	# This matches the value configured in .ko.yaml
	defaultBaseImage: ghcr.io/distroless/static
	baseImageOverrides:
	# Use the combined base image for images that should include Windows support.
	$(params.package)/cmd/entrypoint: ${COMBINED_BASE_IMAGE}
	$(params.package)/cmd/nop: ${COMBINED_BASE_IMAGE}
	$(params.package)/cmd/workingdirinit: ${COMBINED_BASE_IMAGE}
	# This matches values configured in .ko.yaml
	$(params.package)/cmd/git-init: ghcr.io/distroless/git
	EOF

That said, it doesn't even try to build Windows images without that base image override.

I'm trying rebuilding the ko image with Go 1.18.3 (it's based on Go 1.18.1 currently) and will see if I get the same result.

[abayer]

Dang, still failed with 1.18.3.

[abayer]

Ah-ha! cmd/entrypoint/runner.go, where realRunner and its fields are defined, has //go:build !windows - so a0c7d31#diff-d9a50b24352d0a286b3e339692447eb55a219654e06e0fd71ad373de71b8c265R141-R144 broke compilation on Windows, period. I wonder if cmd/entrypoint/main.go ever actually worked on Windows, but changing from

Runner:              &realRunner{},

to

Runner: &realRunner{
	stdoutPath: *stdoutPath,
	stderrPath: *stderrPath,
},

definitely breaks compilation.

I think we may need to revert #4882, if there isn't a quick fix.

[abayer]

cc @tektoncd/core-maintainers to see what people's thoughts are as to how we should move forward.

[afrittoli]

The windows entrypoint struct has not been changed, so I think an option could be to make the code able to cope with the windows version too, and declare in the release notes that the feature is not available on Windows for now.

Alternatively we could revert the change.
Testing this on Windows would take longer as we don't have windows CI infrastructure.

[imjasonh]

To get this to build, add stdoutPath and stderrPath to realRunner defined in https://github.com/tektoncd/pipeline/blob/main/cmd/entrypoint/runner_windows.go#L34

Then to @afrittoli's point we probably need to add code specific to Windows to fail if these fields are set with a helpful error message indicating they're not supported on Windows.

[dibyom]

Hmmm....I'm surprised we did not catch this in our nightly builds

[abayer]

@dibyom We don’t build Windows images anywhere other than releases, I think. Well, until #5178 at least.

[dibyom]

I thought the nightly build pipeline and our release pipeline were the exact same..but looks like that's not true
https://dashboard.dogfooding.tekton.dev/#/namespaces/default/pipelines/pipeline-release-nightly?view=yaml
https://dashboard.dogfooding.tekton.dev/#/namespaces/default/pipelines/pipeline-release?view=yaml

Opened #5191