Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

podman logs -f hangs with no error when container is completed #11461

Closed
ctron opened this issue Sep 6, 2021 · 9 comments
Closed

podman logs -f hangs with no error when container is completed #11461

ctron opened this issue Sep 6, 2021 · 9 comments
Assignees
@vrothberg
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@ctron
Copy link

ctron commented Sep 6, 2021

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running podman logs -f <id> for a container which is terminated blocks forever.

Steps to reproduce the issue:

  1. podman run docker.io/library/hello-world

  2. Wait for container to finish (withing milliseconds)

  3. podman logs -f <container id> 💥

Describe the results you received:

The logs -f commands hangs forever. Not returning any logs.

Describe the results you expected:

Same behavior as docker logs -f, at least show the logs.

Additional information you deem important (e.g. issue happens only occasionally):

This currently blocks podman support in testcontainers-rs: testcontainers/testcontainers-rs#299

Output of podman version:

Version:      3.3.1
API Version:  3.3.1
Go Version:   go1.16.6
Built:        Mon Aug 30 22:46:36 2021
OS/Arch:      linux/amd64

Same for 3.2.x!

Output of podman info --debug:

panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
panic(0x55eecd403360, 0xc000a3b368)
        /usr/lib/golang/src/runtime/panic.go:1065 +0x565 fp=0xc00097b2f0 sp=0xc00097b228 pc=0x55eecbadd9e5
runtime.goPanicIndex(0x1, 0x1)
        /usr/lib/golang/src/runtime/panic.go:88 +0xa5 fp=0xc00097b338 sp=0xc00097b2f0 pc=0x55eecbadada5
github.com/containers/podman/pkg/cgroups.getCgroupPathForCurrentProcess(0x0, 0x0, 0x0, 0x0)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/pkg/cgroups/cgroups.go:234 +0x365 fp=0xc00097b448 sp=0xc00097b338 pc=0x55eecc29e2a5
github.com/containers/podman/pkg/cgroups.getAvailableControllers(0x0, 0x1, 0x0, 0xc00097b5a0, 0x55eeccb6a785, 0x55eeccecd20e, 0x22)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/pkg/cgroups/cgroups.go:135 +0x30f fp=0xc00097b530 sp=0xc00097b448 pc=0x55eecc29d56f
github.com/containers/podman/pkg/cgroups.GetAvailableControllers(0x0, 0x1, 0x0, 0x0, 0x2, 0x55eecbaf29b7, 0x8)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/pkg/cgroups/cgroups.go:184 +0x45 fp=0xc00097b5b0 sp=0xc00097b530 pc=0x55eecc29d9a5
github.com/containers/podman/libpod.(*Runtime).hostInfo(0xc0002b4a80, 0x5, 0xc000139e95, 0x5)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/info.go:105 +0x174 fp=0xc00097b9f8 sp=0xc00097b5b0 pc=0x55eeccb02114
github.com/containers/podman/libpod.(*Runtime).info(0xc0002b4a80, 0x0, 0x0, 0x2)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/info.go:39 +0x16d fp=0xc00097bc28 sp=0xc00097b9f8 pc=0x55eeccb0196d
github.com/containers/podman/libpod.(*Runtime).Info(...)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/runtime.go:875
github.com/containers/podman/pkg/domain/infra/abi.(*ContainerEngine).Info(0xc0006107a8, 0x55eecd561528, 0xc000162ff0, 0x55eecc1316aa, 0xc00060dc00, 0xc00097bd08)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/pkg/domain/infra/abi/system.go:31 +0x34 fp=0xc00097bc98 sp=0xc00097bc28 pc=0x55eecccdc894
github.com/containers/podman/cmd/podman/system.info(0x55eece091be0, 0xc00039e3e0, 0x0, 0x1, 0x0, 0x0)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/cmd/podman/system/info.go:72 +0x78 fp=0xc00097bd40 sp=0xc00097bc98 pc=0x55eecce64858
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).execute(0x55eece091be0, 0xc00003c1a0, 0x1, 0x1, 0x55eece091be0, 0xc00003c1a0)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:856 +0x472 fp=0xc00097be00 sp=0xc00097bd40 pc=0x55eecc130432
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x55eece094b60, 0xc000046150, 0x55eecd2ce400, 0x55eece156338)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:974 +0x375 fp=0xc00097bee0 sp=0xc00097be00 pc=0x55eecc131055
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:902
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:895
main.Execute()
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/cmd/podman/root.go:90 +0xe9 fp=0xc00097bf48 sp=0xc00097bee0 pc=0x55eecce816e9
main.main()
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/cmd/podman/main.go:39 +0x94 fp=0xc00097bf88 sp=0xc00097bf48 pc=0x55eecce80ff4
runtime.main()
        /usr/lib/golang/src/runtime/proc.go:225 +0x263 fp=0xc00097bfe0 sp=0xc00097bf88 pc=0x55eecbae0663
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00097bfe8 sp=0xc00097bfe0 pc=0x55eecbb16721

goroutine 2 [force gc (idle)]:
runtime.gopark(0x55eecd502778, 0x55eece11f130, 0x1411, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000094fb0 sp=0xc000094f90 pc=0x55eecbae0aa6
runtime.goparkunlock(...)
        /usr/lib/golang/src/runtime/proc.go:342
runtime.forcegchelper()
        /usr/lib/golang/src/runtime/proc.go:276 +0xc5 fp=0xc000094fe0 sp=0xc000094fb0 pc=0x55eecbae0905
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000094fe8 sp=0xc000094fe0 pc=0x55eecbb16721
created by runtime.init.7
        /usr/lib/golang/src/runtime/proc.go:264 +0x37

goroutine 3 [GC sweep wait]:
runtime.gopark(0x55eecd502778, 0x55eece121980, 0x140c, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc0000957a8 sp=0xc000095788 pc=0x55eecbae0aa6
runtime.goparkunlock(...)
        /usr/lib/golang/src/runtime/proc.go:342
runtime.bgsweep(0xc0000500e0)
        /usr/lib/golang/src/runtime/mgcsweep.go:182 +0x145 fp=0xc0000957d8 sp=0xc0000957a8 pc=0x55eecbacb845
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000957e0 sp=0xc0000957d8 pc=0x55eecbb16721
created by runtime.gcenable
        /usr/lib/golang/src/runtime/mgc.go:217 +0x5e

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x55eecd502778, 0x55eece122460, 0x140d, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000095f78 sp=0xc000095f58 pc=0x55eecbae0aa6
runtime.goparkunlock(...)
        /usr/lib/golang/src/runtime/proc.go:342
runtime.bgscavenge(0xc0000500e0)
        /usr/lib/golang/src/runtime/mgcscavenge.go:314 +0x2a5 fp=0xc000095fd8 sp=0xc000095f78 pc=0x55eecbac99c5
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000095fe0 sp=0xc000095fd8 pc=0x55eecbb16721
created by runtime.gcenable
        /usr/lib/golang/src/runtime/mgc.go:218 +0x85

goroutine 5 [finalizer wait]:
runtime.gopark(0x55eecd502778, 0x55eece156248, 0xc0001e1410, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000094758 sp=0xc000094738 pc=0x55eecbae0aa6
runtime.goparkunlock(...)
        /usr/lib/golang/src/runtime/proc.go:342
runtime.runfinq()
        /usr/lib/golang/src/runtime/mfinal.go:175 +0xab fp=0xc0000947e0 sp=0xc000094758 pc=0x55eecbac08ab
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000947e8 sp=0xc0000947e0 pc=0x55eecbb16721
created by runtime.createfing
        /usr/lib/golang/src/runtime/mfinal.go:156 +0x66

goroutine 6 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0004169a0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000096760 sp=0xc000096740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0000967e0 sp=0xc000096760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000967e8 sp=0xc0000967e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 7 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000116000, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000096f60 sp=0xc000096f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000096fe0 sp=0xc000096f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000096fe8 sp=0xc000096fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 18 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580000, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000090760 sp=0xc000090740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0000907e0 sp=0xc000090760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000907e8 sp=0xc0000907e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 34 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580020, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000588760 sp=0xc000588740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005887e0 sp=0xc000588760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005887e8 sp=0xc0005887e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 8 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0004169c0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000097760 sp=0xc000097740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0000977e0 sp=0xc000097760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000977e8 sp=0xc0000977e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 35 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580040, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000588f60 sp=0xc000588f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000588fe0 sp=0xc000588f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000588fe8 sp=0xc000588fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 36 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0004169e0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000589760 sp=0xc000589740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005897e0 sp=0xc000589760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005897e8 sp=0xc0005897e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 37 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000116020, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000589f60 sp=0xc000589f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000589fe0 sp=0xc000589f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000589fe8 sp=0xc000589fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 19 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416a00, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000090f60 sp=0xc000090f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000090fe0 sp=0xc000090f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000090fe8 sp=0xc000090fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 9 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000600000, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000097f60 sp=0xc000097f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000097fe0 sp=0xc000097f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 50 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000600020, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000584760 sp=0xc000584740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005847e0 sp=0xc000584760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005847e8 sp=0xc0005847e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 51 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580060, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000584f60 sp=0xc000584f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000584fe0 sp=0xc000584f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000584fe8 sp=0xc000584fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 10 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580080, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000518760 sp=0xc000518740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005187e0 sp=0xc000518760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005187e8 sp=0xc0005187e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 52 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0005800a0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000585760 sp=0xc000585740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005857e0 sp=0xc000585760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005857e8 sp=0xc0005857e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 53 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416a20, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000585f60 sp=0xc000585f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000585fe0 sp=0xc000585f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000585fe8 sp=0xc000585fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 38 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416a40, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc00058a760 sp=0xc00058a740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc00058a7e0 sp=0xc00058a760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00058a7e8 sp=0xc00058a7e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 54 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416a60, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000586760 sp=0xc000586740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005867e0 sp=0xc000586760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005867e8 sp=0xc0005867e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 55 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0005800c0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000586f60 sp=0xc000586f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000586fe0 sp=0xc000586f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000586fe8 sp=0xc000586fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 11 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc0005800e0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000518f60 sp=0xc000518f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000518fe0 sp=0xc000518f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000518fe8 sp=0xc000518fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 56 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580100, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000587760 sp=0xc000587740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005877e0 sp=0xc000587760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005877e8 sp=0xc0005877e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 39 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416a80, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc00058af60 sp=0xc00058af40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc00058afe0 sp=0xc00058af60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00058afe8 sp=0xc00058afe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 57 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000416aa0, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000587f60 sp=0xc000587f40 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc000587fe0 sp=0xc000587f60 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000587fe8 sp=0xc000587fe0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 58 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580120, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000514760 sp=0xc000514740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc0005147e0 sp=0xc000514760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005147e8 sp=0xc0005147e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 40 [GC worker (idle)]:
runtime.gopark(0x55eecd502610, 0xc000580140, 0x1418, 0x0)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc00058b760 sp=0xc00058b740 pc=0x55eecbae0aa6
runtime.gcBgMarkWorker()
        /usr/lib/golang/src/runtime/mgc.go:1911 +0x105 fp=0xc00058b7e0 sp=0xc00058b760 pc=0x55eecbac4625
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00058b7e8 sp=0xc00058b7e0 pc=0x55eecbb16721
created by runtime.gcBgMarkStartWorkers
        /usr/lib/golang/src/runtime/mgc.go:1835 +0x3a

goroutine 13 [chan receive]:
runtime.gopark(0x55eecd502570, 0xc0000c0ad8, 0xc00050170e, 0x2)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000517ed0 sp=0xc000517eb0 pc=0x55eecbae0aa6
runtime.chanrecv(0xc0000c0a80, 0xc000517fb0, 0xc0004e2801, 0xc0000c0a80)
        /usr/lib/golang/src/runtime/chan.go:576 +0x37a fp=0xc000517f60 sp=0xc000517ed0 pc=0x55eecbaad07a
runtime.chanrecv2(0xc0000c0a80, 0xc000517fb0, 0x55eecbfa74eb)
        /usr/lib/golang/src/runtime/chan.go:444 +0x2b fp=0xc000517f90 sp=0xc000517f60 pc=0x55eecbaacceb
github.com/containers/podman/vendor/k8s.io/klog/v2.(*loggingT).flushDaemon(0x55eece1247e0)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/k8s.io/klog/v2/klog.go:1164 +0x8d fp=0xc000517fd8 sp=0xc000517f90 pc=0x55eecc77416d
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000517fe0 sp=0xc000517fd8 pc=0x55eecbb16721
created by github.com/containers/podman/vendor/k8s.io/klog/v2.init.0
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/vendor/k8s.io/klog/v2/klog.go:418 +0xdf

goroutine 21 [select, locked to thread]:
runtime.gopark(0x55eecd5027c8, 0x0, 0x1809, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc000517610 sp=0xc0005175f0 pc=0x55eecbae0aa6
runtime.selectgo(0xc0005177a8, 0xc0005177a0, 0x0, 0x0, 0x2, 0x1, 0x1, 0x1)
        /usr/lib/golang/src/runtime/select.go:327 +0xef7 fp=0xc000517748 sp=0xc000517610 pc=0x55eecbaf1e77
runtime.ensureSigM.func1()
        /usr/lib/golang/src/runtime/signal_unix.go:889 +0x1cd fp=0xc0005177e0 sp=0xc000517748 pc=0x55eecbb0e86d
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005177e8 sp=0xc0005177e0 pc=0x55eecbb16721
created by runtime.ensureSigM
        /usr/lib/golang/src/runtime/signal_unix.go:872 +0xd7

goroutine 14 [syscall]:
runtime.notetsleepg(0x55eece156fc0, 0xffffffffffffffff, 0x0)
        /usr/lib/golang/src/runtime/lock_futex.go:235 +0x38 fp=0xc00051af98 sp=0xc00051af68 pc=0x55eecbab2f98
os/signal.signal_recv(0x0)
        /usr/lib/golang/src/runtime/sigqueue.go:168 +0xa5 fp=0xc00051afc0 sp=0xc00051af98 pc=0x55eecbb12e25
os/signal.loop()
        /usr/lib/golang/src/os/signal/signal_unix.go:23 +0x25 fp=0xc00051afe0 sp=0xc00051afc0 pc=0x55eecc243345
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00051afe8 sp=0xc00051afe0 pc=0x55eecbb16721
created by os/signal.Notify.func1.1
        /usr/lib/golang/src/os/signal/signal.go:151 +0x46

goroutine 15 [select]:
runtime.gopark(0x55eecd5027c8, 0x0, 0x1809, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc0000a5d80 sp=0xc0000a5d60 pc=0x55eecbae0aa6
runtime.selectgo(0xc0000a5fb0, 0xc00051b708, 0x0, 0x0, 0x2, 0x1, 0x0, 0x0)
        /usr/lib/golang/src/runtime/select.go:327 +0xef7 fp=0xc0000a5eb8 sp=0xc0000a5d80 pc=0x55eecbaf1e77
github.com/containers/podman/libpod/shutdown.Start.func1()
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/shutdown/handler.go:45 +0xab fp=0xc0000a5fe0 sp=0xc0000a5eb8 pc=0x55eecca7688b
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000a5fe8 sp=0xc0000a5fe0 pc=0x55eecbb16721
created by github.com/containers/podman/libpod/shutdown.Start
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/shutdown/handler.go:44 +0x116

goroutine 66 [sleep]:
runtime.gopark(0x55eecd5027b0, 0xc0005de000, 0xc000281313, 0x1)
        /usr/lib/golang/src/runtime/proc.go:336 +0xe6 fp=0xc00051b610 sp=0xc00051b5f0 pc=0x55eecbae0aa6
time.Sleep(0x5f5e100)
        /usr/lib/golang/src/runtime/time.go:193 +0xd8 fp=0xc00051b650 sp=0xc00051b610 pc=0x55eecbb137d8
github.com/containers/podman/libpod.(*Runtime).libimageEvents.func2(0xc0002bc360, 0x55eecd4fdc98, 0xc0002b4a80)
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/runtime.go:744 +0x25f fp=0xc00051b7c8 sp=0xc00051b650 pc=0x55eeccb9a2df
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00051b7d0 sp=0xc00051b7c8 pc=0x55eecbb16721
created by github.com/containers/podman/libpod.(*Runtime).libimageEvents
        /builddir/build/BUILD/podman-3.3.1/_build/src/github.com/containers/podman/libpod/runtime.go:721 +0x99
fish: Job 1, 'podman info --debug' terminated by signal SIGABRT (Abort)

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.3.1-1.fc34.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

No

Additional environment details (AWS, VirtualBox, physical, etc.):

Issue on Fedora 34 and Ubuntu GitHub actions runners. Worked before (unknown which version).
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 6, 2021
@mheon
Copy link
Member

mheon commented Sep 6, 2021

@vrothberg Mind looking at this one?

Otherwise - I'm a lot more concerned about that podman info panic - does that consistently reproduce?

@flouthoc
Copy link
Collaborator

flouthoc commented Sep 6, 2021

Something seems really off here why is podman process without a cgroup entry and this should error instead of panic afaik I made PR for Cgroup check in info. Could you share env details and content of /proc/self/cgroup.

@flouthoc
Copy link
Collaborator

flouthoc commented Sep 6, 2021

@mheon I think this should fix panic #11463 @ctron could you please share value of /proc/self/cgroup unrelated to your issue but i want to check what values are there and why did we hit panic case.

@vrothberg vrothberg self-assigned this Sep 7, 2021
@ctron
Copy link
Author

ctron commented Sep 7, 2021

Otherwise - I'm a lot more concerned about that podman info panic - does that consistently reproduce?

Yes

@ctron
Copy link
Author

ctron commented Sep 7, 2021

@mheon I think this should fix panic #11463 @ctron could you please share value of /proc/self/cgroup unrelated to your issue but i want to check what values are there and why did we hit panic case.

➜  ~ cat /proc/self/cgroup 
1:net_cls:/
0::/user.slice/user-110147.slice/[email protected]/app.slice/app-org.kde.konsole-ff464820bf9b43a88d1019d22cdd7209.scope

@flouthoc
Copy link
Collaborator

flouthoc commented Sep 7, 2021

@ctron thanks this should fix #11463 panic and info for you since first line is not the correct line it should process the next line.

@giuseppe giuseppe mentioned this issue Sep 7, 2021
Merged
@Luap99
Copy link
Member

Luap99 commented Sep 7, 2021

The PR was not linked correctly, the fix is merged into main.

@Luap99 Luap99 closed this as completed Sep 7, 2021
@mheon
Copy link
Member

mheon commented Sep 7, 2021

We fixed the panic - did we fix logs -f? I didn't see a PR for that merge?

@vrothberg
Copy link
Member

#11464 fixes it. I missed adding the hash for cross referencing and auto closing.

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@vrothberg vrothberg

Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ctron @mheon @flouthoc @vrothberg @Luap99