podman play kube --replace : timeout

[edsantiago]

Only one logged flake, but I've seen this one multiple times in the last two days on other PRs:

         Running: podman-remote [options] pod ps --format '{{.ID}}'
         @ - - [17/Oct/2021:21:33:52 -0500] "GET /v4.0.0/libpod/_ping HTTP/1.1" 200 2 "" "Go-http-client/1.1"
         @ - - [17/Oct/2021:21:33:52 -0500] "GET /v4.0.0/libpod/pods/json HTTP/1.1" 200 581 "" "Go-http-client/1.1"
         'f2d0ede34702'
         output: 'f2d0ede34702'
         Running: podman-remote [options] pod inspect testPod --format '{{len .Containers}}'
         Running: podman-remote [options] play kube --replace /tmp/podman_test035144678/kube.yaml
         @ - - [17/Oct/2021:21:33:52 -0500] "GET /v4.0.0/libpod/_ping HTTP/1.1" 200 2 "" "Go-http-client/1.1"
         @ - - [17/Oct/2021:21:33:52 -0500] "GET /v4.0.0/libpod/_ping HTTP/1.1" 200 2 "" "Go-http-client/1.1"
         [AfterEach] Podman play kube
           /var/tmp/go/src/github.com/containers/podman/test/e2e/play_kube_test.go:1089
         Running: podman-remote [options] stop -a --time 0
         @ - - [17/Oct/2021:21:35:22 -0500] "GET /v4.0.0/libpod/_ping HTTP/1.1" 200 2 "" "Go-http-client/1.1"
         
         

         ? Failure [183.193 seconds]
         Podman play kube
         /var/tmp/go/src/github.com/containers/podman/test/e2e/play_kube_test.go:1069
           podman play kube replace [It]
           /var/tmp/go/src/github.com/containers/podman/test/e2e/play_kube_test.go:2803
         
           Timed out after 90.000s.
           Expected process to exit.  It did not.
         
           /var/tmp/go/src/github.com/containers/podman/test/utils/utils.go:327

Podman play kube [It] podman play kube replace

• fedora-34 : int remote fedora-34 root host [remote]
  • PR System tests: new checkpoint test #11957
    • 10-17 22:43
    • 10-17 22:43
    • 10-17 22:43

[vrothberg]

Is that remote only?

[edsantiago]

It seems to be. Here are the logged failures since my initial report:

Podman play kube [It] podman play kube replace

• fedora-33 : int remote fedora-33 root host [remote]
  • PR remove need to download pause image #11956
    • 10-26 08:43
    • 10-26 08:43
    • 10-26 08:43
• fedora-34 : int remote fedora-34 root host [remote]
  • PR Generate Kube should not print default structs #12021
    • 10-19 09:22
  • PR fix remote checkpoint/restore #12281
    • 11-16 09:43

From that list I removed this one because it looks like a timeout fetching from k8s.io. That suggested that this bug might just be that timeout, misreported in the remote client, but the failure yesterday (in a PR that, presumably, includes #12280) implies otherwise.

[vrothberg]

Did it happen again?

[edsantiago]

Only two instances since my last post:

Podman play kube [It] podman play kube replace

• fedora-34 : int remote fedora-34 root host [remote]
  • PR generate systemd: add --start-timeout flag #12380
    • 11-23 04:26
  • PR Do not store the exit command in container config #12354
    • 11-18 15:20

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@edsantiago Still an issue?

[edsantiago]

Twice in the last month:

Podman play kube [It] podman play kube replace

• fedora-35 : int remote fedora-35 root host [remote]
  • PR Add --context-dir option to podman play kube #12913
    • 01-18 18:43
    • 01-18 18:43
    • 01-18 18:43
  • PR Remove two GetImages functions from API #12572
    • 01-14 08:24
    • 01-14 08:24
    • 01-14 08:24

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[umohnani8]

@edsantiago have you seen this happen again recently?

[edsantiago]

Last one in my logs is 05-23.

[vrothberg]

That is good enough for me to close it. We fixed some deadlocks in the recent past, so it may be related.

Let's reopen if it pops up again.