Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a http.response.Body leak on a permission error #2363

Merged
merged 2 commits into from
Apr 8, 2024

Conversation

mtrmac
Copy link
Collaborator

@mtrmac mtrmac commented Apr 2, 2024

On the needsRetryWithUpdatedScope code path, we didn’t close the response body.

See individual commit messages for details.

@mtrmac mtrmac added the kind/bug A defect in an existing functionality (or a PR fixing it) label Apr 5, 2024
The idea of a StatusTooManyRequests retry loop,
or the needsRetryWithUpdatedScope logic,
only makes sense if we do get a response; on other errors,
we can exit immediately. So do that, and simplify the
code.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
Not doing so can keep the HTTP client code waiting forever,
keeping some goroutines running, and the socket open.

Signed-off-by: Miloslav Trmač <[email protected]>
@mtrmac
Copy link
Collaborator Author

mtrmac commented Apr 5, 2024

Testing this with

package main

import (
	"context"
	"fmt"
	"io/fs"
	"os"
	"runtime/debug"

	"github.com/containers/image/v5/docker"
	"github.com/containers/image/v5/types"
	"github.com/containers/image/v5/version"
	"github.com/sirupsen/logrus"
)

func inspectImage(ref types.ImageReference) error {
	ctx := context.Background()
	src, err := ref.NewImageSource(ctx, nil)
	if err != nil {
		return err
	}
	defer src.Close()
	_, _, err = src.GetManifest(ctx, nil)
	return err
}

func listOpenFiles() {
	fd, err := os.Open("/dev/fd")
	if err != nil {
		panic(err)
	}
	defer fd.Close()
	entries, err := fd.ReadDir(-1)
	fmt.Printf("%d open files:\n", len(entries))
	for _, e := range entries {
		fmt.Printf("\t%s\n", fs.FormatDirEntry(e))
	}
}

func main() {
	debug.SetTraceback("all")
	c := make(chan struct{})
	go func() { <-c }() // A sanity check to show the traceback does list all goroutines
	fmt.Printf("c/image version: %s\n", version.Version)
	logrus.SetLevel(logrus.DebugLevel)
	ref, err := docker.ParseReference("//docker.io/library/none:latest")
	if err != nil {
		panic(err)
	}
	for i := 0; i < 10; i++ {
		_ = inspectImage(ref)
	}
	listOpenFiles()
	panic("listing goroutines:")
}

before this PR, I see

goroutine 1 [running]:
main.main()
	/Users/mitr/Go/src/github.com/containers/image/main.go:54 +0x14c

goroutine 35 [chan receive]:
main.main.func1()
	/Users/mitr/Go/src/github.com/containers/image/main.go:43 +0x24
created by main.main in goroutine 1
	/Users/mitr/Go/src/github.com/containers/image/main.go:43 +0x7c
…
goroutine 75 [select]:
net/http.(*persistConn).readLoop(0x1400013fe60)
	/usr/local/go/src/net/http/transport.go:2261 +0xb74
created by net/http.(*Transport).dialConn in goroutine 56
	/usr/local/go/src/net/http/transport.go:1799 +0x1018
…
goroutine 165 [select]:
net/http.(*persistConn).writeLoop(0x140000006c0)
	/usr/local/go/src/net/http/transport.go:2444 +0xa0
created by net/http.(*Transport).dialConn in goroutine 147
	/usr/local/go/src/net/http/transport.go:1800 +0x1060

with 10 readLoop and 10 writeLoop goroutines.

After this PR, it’s just the first two goroutines, as expected.

@rhatdan
Copy link
Member

rhatdan commented Apr 6, 2024

LGTM
@giuseppe PTAL

@kwilczynski
Copy link
Member

/approve
/lgtm

Copy link
Member

@giuseppe giuseppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants