Add count-based checkpoint retention policies #5

Parthiba-Hazra · 2024-06-23T17:19:38Z

Introduced global, container, pod, and namespace-level policies for checkpoint retention, all based on count limits. Introduced a flag applyPoliciesImmediately to enable immediate application of policies

The resource specific policies like container, pod and namespace policy will override the global policy.

rst0git · 2024-07-05T13:42:16Z

@Parthiba-Hazra To summarize the comments from the meeting earlier today:

It would be good to add a test for CI to validate the functionality in this pull request.
The individual container count limit takes precedence over Pod, namespace, and global policy.
It might be useful to add a markdown file (e.g., in a docs/ folder) that describes how the checkpoint retention policy works and how it can be configured.

adrianreber · 2024-07-09T14:59:25Z

Nice to see CI complaining about the shell script formatting. So CI works 👍 Good

.github/workflows/tests.yaml

config/manager/kustomization.yaml

.github/workflows/tests.yaml

Makefile

Parthiba-Hazra · 2024-07-24T18:50:17Z

@rst0git I think now it's working properly :)

but in local test this was failing -

Show log

``` ➜ checkpoint-restore-operator git:(ci-test) ✗ act -W .github/workflows/tests.yaml push INFO[0000] Using docker host 'unix:///var/run/docker.sock', and daemon socket 'unix:///var/run/docker.sock' [tests.yaml/build-and-deploy] 🚀 Start image=catthehacker/ubuntu:act-latest [tests.yaml/build-and-deploy] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true [tests.yaml/build-and-deploy] 🐳 docker create image=catthehacker/ubuntu:act-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host" [tests.yaml/build-and-deploy] 🐳 docker run image=catthehacker/ubuntu:act-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host" [tests.yaml/build-and-deploy] ☁ git clone 'https://github.com/actions/setup-go' # ref=v5 [tests.yaml/build-and-deploy] ⭐ Run Main Checkout code [tests.yaml/build-and-deploy] 🐳 docker cp src=/home/ubuntu/os/checkpoint-restore-operator/. dst=/home/ubuntu/os/checkpoint-restore-operator [tests.yaml/build-and-deploy] ✅ Success - Main Checkout code [tests.yaml/build-and-deploy] ⭐ Run Main Set up Go [tests.yaml/build-and-deploy] 🐳 docker cp src=/home/ubuntu/.cache/act/actions-setup-go@v5/ dst=/var/run/act/actions/actions-setup-go@v5/ [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[node /var/run/act/actions/actions-setup-go@v5/dist/setup/index.js] user= workdir= | Setup go version spec 1.21 [tests.yaml/build-and-deploy] 💬 ::debug::isExplicit: [tests.yaml/build-and-deploy] 💬 ::debug::explicit? false [tests.yaml/build-and-deploy] 💬 ::debug::isExplicit: 1.21.12 [tests.yaml/build-and-deploy] 💬 ::debug::explicit? true [tests.yaml/build-and-deploy] 💬 ::debug::evaluating 1 versions [tests.yaml/build-and-deploy] 💬 ::debug::matched: 1.21.12 [tests.yaml/build-and-deploy] 💬 ::debug::checking cache: /opt/hostedtoolcache/go/1.21.12/arm64 [tests.yaml/build-and-deploy] 💬 ::debug::Found tool in cache go 1.21.12 arm64 | Found in cache @ /opt/hostedtoolcache/go/1.21.12/arm64 | Added go to the path | Successfully set up Go version 1.21 [tests.yaml/build-and-deploy] 💬 ::debug::which go :/opt/hostedtoolcache/go/1.21.12/arm64/bin/go: [tests.yaml/build-and-deploy] 💬 ::debug::go env GOPATH :/root/go: [tests.yaml/build-and-deploy] 💬 ::debug::creating /root/go [tests.yaml/build-and-deploy] 💬 ::debug::creating /root/go/bin [tests.yaml/build-and-deploy] 💬 ::debug::add bin true | [command]/opt/hostedtoolcache/go/1.21.12/arm64/bin/go env GOMODCACHE | [command]/opt/hostedtoolcache/go/1.21.12/arm64/bin/go env GOCACHE | /root/go/pkg/mod | /root/.cache/go-build [tests.yaml/build-and-deploy] 💬 ::debug::followSymbolicLinks 'true' [tests.yaml/build-and-deploy] 💬 ::debug::followSymbolicLinks 'true' [tests.yaml/build-and-deploy] 💬 ::debug::implicitDescendants 'true' [tests.yaml/build-and-deploy] 💬 ::debug::matchDirectories 'true' [tests.yaml/build-and-deploy] 💬 ::debug::omitBrokenSymbolicLinks 'true' [tests.yaml/build-and-deploy] 💬 ::debug::Search path '/home/ubuntu/os/checkpoint-restore-operator/go.sum' [tests.yaml/build-and-deploy] 💬 ::debug::/home/ubuntu/os/checkpoint-restore-operator/go.sum [tests.yaml/build-and-deploy] 💬 ::debug::Found 1 files to hash. [tests.yaml/build-and-deploy] 💬 ::debug::primary key is setup-go-Linux-ubuntu20-go-1.21.12-84b075fa9a22072cff932a3c7017f7f85dba0189d1ed11f8bf84aeec1d606a97 [tests.yaml/build-and-deploy] 💬 ::debug::Resolved Keys: [tests.yaml/build-and-deploy] 💬 ::debug::["setup-go-Linux-ubuntu20-go-1.21.12-84b075fa9a22072cff932a3c7017f7f85dba0189d1ed11f8bf84aeec1d606a97"] [tests.yaml/build-and-deploy] 💬 ::debug::Checking zstd --quiet --version [tests.yaml/build-and-deploy] 💬 ::debug::1.4.8 [tests.yaml/build-and-deploy] 💬 ::debug::zstd version: 1.4.8 [tests.yaml/build-and-deploy] 💬 ::debug::Resource Url: http://192.168.64.5:41577/_apis/artifactcache/cache?keys=setup-go-Linux-ubuntu20-go-1.21.12-84b075fa9a22072cff932a3c7017f7f85dba0189d1ed11f8bf84aeec1d606a97&version=80c03ad9bc0f7552c128ae36020fe5294ef09c552e168fbe45ea237491c60cae [tests.yaml/build-and-deploy] ⚙ *** [tests.yaml/build-and-deploy] 💬 ::debug::Cache Result: [tests.yaml/build-and-deploy] 💬 ::debug::{"archiveLocation":"***","cacheKey":"setup-go-linux-ubuntu20-go-1.21.12-84b075fa9a22072cff932a3c7017f7f85dba0189d1ed11f8bf84aeec1d606a97","result":"hit"} [tests.yaml/build-and-deploy] 💬 ::debug::Archive Path: /tmp/ceb5ea98-917d-41b8-8a72-44ffe796d9f3/cache.tzst [tests.yaml/build-and-deploy] 💬 ::debug::Use Azure SDK: false [tests.yaml/build-and-deploy] 💬 ::debug::Download concurrency: 8 [tests.yaml/build-and-deploy] 💬 ::debug::Request timeout (ms): 30000 [tests.yaml/build-and-deploy] 💬 ::debug::Cache segment download timeout mins env var: undefined [tests.yaml/build-and-deploy] 💬 ::debug::Segment download timeout (ms): 600000 [tests.yaml/build-and-deploy] 💬 ::debug::Lookup only: false | Cache Size: ~142 MB (148825117 B) | [command]/usr/bin/tar -xf /tmp/ceb5ea98-917d-41b8-8a72-44ffe796d9f3/cache.tzst -P -C /home/ubuntu/os/checkpoint-restore-operator --use-compress-program unzstd | Cache restored successfully | Cache restored from key: setup-go-linux-ubuntu20-go-1.21.12-84b075fa9a22072cff932a3c7017f7f85dba0189d1ed11f8bf84aeec1d606a97 [tests.yaml/build-and-deploy] ❓ add-matcher /run/act/actions/actions-setup-go@v5/matchers.json | go version go1.21.12 linux/arm64 | [tests.yaml/build-and-deploy] ❓ ::group::go env | GO111MODULE='' | GOARCH='arm64' | GOBIN='' | GOCACHE='/root/.cache/go-build' | GOENV='/root/.config/go/env' | GOEXE='' | GOEXPERIMENT='' | GOFLAGS='' | GOHOSTARCH='arm64' | GOHOSTOS='linux' | GOINSECURE='' | GOMODCACHE='/root/go/pkg/mod' | GONOPROXY='' | GONOSUMDB='' | GOOS='linux' | GOPATH='/root/go' | GOPRIVATE='' | GOPROXY='https://proxy.golang.org,direct' | GOROOT='/opt/hostedtoolcache/go/1.21.12/arm64' | GOSUMDB='sum.golang.org' | GOTMPDIR='' | GOTOOLCHAIN='auto' | GOTOOLDIR='/opt/hostedtoolcache/go/1.21.12/arm64/pkg/tool/linux_arm64' | GOVCS='' | GOVERSION='go1.21.12' | GCCGO='gccgo' | AR='ar' | CC='gcc' | CXX='g++' | CGO_ENABLED='1' | GOMOD='/home/ubuntu/os/checkpoint-restore-operator/go.mod' | GOWORK='' | CGO_CFLAGS='-O2 -g' | CGO_CPPFLAGS='' | CGO_CXXFLAGS='-O2 -g' | CGO_FFLAGS='-O2 -g' | CGO_LDFLAGS='-O2 -g' | PKG_CONFIG='pkg-config' | GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2246315361=/tmp/go-build -gno-record-gcc-switches' | [tests.yaml/build-and-deploy] ❓ ::endgroup:: [tests.yaml/build-and-deploy] ✅ Success - Main Set up Go [tests.yaml/build-and-deploy] ⚙ ::set-output:: cache-hit=true [tests.yaml/build-and-deploy] ⚙ ::set-output:: go-version=1.21.12 [tests.yaml/build-and-deploy] ⚙ ::add-path:: /opt/hostedtoolcache/go/1.21.12/arm64/bin [tests.yaml/build-and-deploy] ⚙ ::add-path:: /root/go/bin [tests.yaml/build-and-deploy] ⭐ Run Main Install dependencies [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/2] user= workdir= | % Total % Received % Xferd Average Speed Time Time Time Current | Dload Upload Total Spent Left Speed 100 47.6M 100 47.6M 0 0 5364k 0 0:00:09 0:00:09 --:--:-- 6887k [tests.yaml/build-and-deploy] ✅ Success - Main Install dependencies [tests.yaml/build-and-deploy] ⭐ Run Main Prepare Checkpoints Directory in Host [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/3] user= workdir= | /var/lib/kubelet/checkpoints [tests.yaml/build-and-deploy] ✅ Success - Main Prepare Checkpoints Directory in Host [tests.yaml/build-and-deploy] ⭐ Run Main Generate Checkpoint Tar Files [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/4] user= workdir= | Checkpoint tar file created at /var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-2024-07-24T18:09:34.tar | Checkpoint tar file created at /var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-2024-07-24T18:09:35.tar | Checkpoint tar file created at /var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-2024-07-24T18:09:36.tar | Checkpoint tar file created at /var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-2024-07-24T18:09:37.tar | Checkpoint tar file created at /var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-2024-07-24T18:09:38.tar [tests.yaml/build-and-deploy] ✅ Success - Main Generate Checkpoint Tar Files [tests.yaml/build-and-deploy] ⭐ Run Main Generate Kind Config [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/5] user= workdir= [tests.yaml/build-and-deploy] ✅ Success - Main Generate Kind Config [tests.yaml/build-and-deploy] ⭐ Run Main Delete Existing Kind Cluster [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/6] user= workdir= | Deleting cluster "kind" ... | Deleted nodes: ["kind-control-plane"] [tests.yaml/build-and-deploy] ✅ Success - Main Delete Existing Kind Cluster [tests.yaml/build-and-deploy] ⭐ Run Main Create Kubernetes cluster [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/7] user= workdir= | Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.30.0) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 | Set kubectl context to "kind-kind" | You can now use your cluster with: | | kubectl cluster-info --context kind-kind | | Thanks for using kind! 😊 [tests.yaml/build-and-deploy] ✅ Success - Main Create Kubernetes cluster [tests.yaml/build-and-deploy] ⭐ Run Main Load Docker image into kind cluster [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/8] user= workdir= | Untagged: checkpoint-restore-operator:ci | Deleted: sha256:01baf479c6c940deca0fd057b8669e80d0376d5af8b53eac2cf3988a4961a48e | mkdir -p /home/ubuntu/os/checkpoint-restore-operator/bin | test -s /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen && /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen --version | grep -q v0.15.0 || \ | GOBIN=/home/ubuntu/os/checkpoint-restore-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] | go: downloading sigs.k8s.io/controller-tools v0.15.0 | go: sigs.k8s.io/[email protected] requires go >= 1.22.0; switching to go1.22.5 | go: downloading go1.22.5 (linux/arm64) | go: downloading github.com/spf13/cobra v1.8.0 | go: downloading github.com/gobuffalo/flect v1.0.2 | go: downloading k8s.io/apiextensions-apiserver v0.30.0 | go: downloading k8s.io/apimachinery v0.30.0 | go: downloading golang.org/x/tools v0.20.0 | go: downloading github.com/fatih/color v1.16.0 | go: downloading k8s.io/api v0.30.0 | go: downloading sigs.k8s.io/yaml v1.4.0 | go: downloading k8s.io/klog/v2 v2.120.1 | go: downloading github.com/mattn/go-colorable v0.1.13 | go: downloading github.com/mattn/go-isatty v0.0.20 | go: downloading golang.org/x/sync v0.7.0 | go: downloading github.com/go-logr/logr v1.4.1 | go: downloading golang.org/x/sys v0.19.0 | go: downloading golang.org/x/mod v0.17.0 | go: downloading golang.org/x/net v0.24.0 | /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases | /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." | go fmt ./... | go vet ./... | test -s /home/ubuntu/os/checkpoint-restore-operator/bin/setup-envtest || GOBIN=/home/ubuntu/os/checkpoint-restore-operator/bin go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest | go: downloading sigs.k8s.io/controller-runtime v0.18.4 | go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20240717190548-1ed345090869 | go: sigs.k8s.io/controller-runtime/tools/[email protected] requires go >= 1.22.0; switching to go1.22.5 | go: downloading github.com/go-logr/logr v1.4.2 | go: downloading github.com/go-logr/zapr v1.3.0 | go: downloading github.com/spf13/afero v1.6.0 | go: downloading go.uber.org/zap v1.26.0 | go: downloading go.uber.org/multierr v1.10.0 | go: downloading golang.org/x/text v0.16.0 | KUBEBUILDER_ASSETS="/home/ubuntu/os/checkpoint-restore-operator/bin/k8s/1.27.1-linux-arm64" go test ./... -coverprofile cover.out | ? github.com/checkpoint-restore/checkpoint-restore-operator/api/v1 [no test files] | ? github.com/checkpoint-restore/checkpoint-restore-operator/cmd [no test files] | ok github.com/checkpoint-restore/checkpoint-restore-operator/internal/controller 0.016s coverage: 0.0% of statements | docker build -t checkpoint-restore-operator:ci . [+] Building 0.2s (17/17) FINISHED docker:default | => [internal] load .dockerignore 0.1s | => => transferring context: 160B 0.0s | => [internal] load build definition from Dockerfile 0.1s | => => transferring dockerfile: 1.32kB 0.0s | => [internal] load metadata for gcr.io/distroless/static:nonroot 0.0s | => [internal] load metadata for docker.io/library/golang:1.21 0.0s | => [builder 1/9] FROM docker.io/library/golang:1.21 0.0s | => [stage-1 1/3] FROM gcr.io/distroless/static:nonroot 0.0s | => [internal] load build context 0.0s | => => transferring context: 63.13kB 0.0s | => CACHED [builder 2/9] WORKDIR /workspace 0.0s | => CACHED [builder 3/9] COPY go.mod go.mod 0.0s | => CACHED [builder 4/9] COPY go.sum go.sum 0.0s | => CACHED [builder 5/9] RUN go mod download 0.0s | => CACHED [builder 6/9] COPY cmd/main.go cmd/main.go 0.0s | => CACHED [builder 7/9] COPY api/ api/ 0.0s | => CACHED [builder 8/9] COPY internal/controller/ internal/controller/ 0.0s | => CACHED [builder 9/9] RUN CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go bui 0.0s | => CACHED [stage-1 2/3] COPY --from=builder /workspace/manager . 0.0s | => exporting to image 0.0s | => => exporting layers 0.0s | => => writing image sha256:9543ec8a74c4e3fe38c64738eab5e633141e0921e1082 0.0s | => => naming to docker.io/library/checkpoint-restore-operator:ci 0.0s | checkpoint-restore-operator ci 9543ec8a74c4 2 days ago 53.4MB | remontada007/checkpoint-restore-operator t10 981b592e461f 2 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t9 da82b2344091 2 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t8 fb6dcef76f76 2 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t7 1f44068f3f6a 2 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t6 5d8f7941dfbb 4 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t5 56844c050f15 4 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t4 16dd8fbdbd7c 4 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t3 927736bc2bc4 4 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t2 51e5ee868846 4 weeks ago 53.4MB | remontada007/checkpoint-restore-operator t1 badb1044695a 4 weeks ago 53.3MB | Image: "checkpoint-restore-operator:ci" with ID "sha256:9543ec8a74c4e3fe38c64738eab5e633141e0921e1082539504d96d73e5d791d" not yet present on node "kind-control-plane", loading... [tests.yaml/build-and-deploy] ✅ Success - Main Load Docker image into kind cluster [tests.yaml/build-and-deploy] ⭐ Run Main Replace Placeholder in Manager [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/9] user= workdir= [tests.yaml/build-and-deploy] ✅ Success - Main Replace Placeholder in Manager [tests.yaml/build-and-deploy] ⭐ Run Main Deploy to Kubernetes [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/10] user= workdir= | test -s /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen && /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen --version | grep -q v0.15.0 || \ | GOBIN=/home/ubuntu/os/checkpoint-restore-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] | /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases | test -s /home/ubuntu/os/checkpoint-restore-operator/bin/kustomize || GOBIN=/home/ubuntu/os/checkpoint-restore-operator/bin GO111MODULE=on go install sigs.k8s.io/kustomize/kustomize/[email protected] | go: downloading sigs.k8s.io/kustomize/kustomize/v5 v5.4.2 | go: downloading sigs.k8s.io/kustomize/api v0.17.2 | go: downloading sigs.k8s.io/kustomize/cmd/config v0.14.1 | go: downloading sigs.k8s.io/kustomize/kyaml v0.17.1 | go: downloading github.com/go-errors/errors v1.4.2 | go: downloading github.com/blang/semver/v4 v4.0.0 | go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc | go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 | go: downloading github.com/xlab/treeprint v1.2.0 | go: downloading gopkg.in/evanphx/json-patch.v4 v4.12.0 | go: downloading google.golang.org/protobuf v1.33.0 | go: downloading github.com/go-openapi/swag v0.22.4 | go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 | go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 | /home/ubuntu/os/checkpoint-restore-operator/bin/kustomize build config/crd | kubectl apply -f - | customresourcedefinition.apiextensions.k8s.io/checkpointrestoreoperators.criu.org created | test -s /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen && /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen --version | grep -q v0.15.0 || \ | GOBIN=/home/ubuntu/os/checkpoint-restore-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] | /home/ubuntu/os/checkpoint-restore-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases | cd config/manager && /home/ubuntu/os/checkpoint-restore-operator/bin/kustomize edit set image controller=checkpoint-restore-operator:ci | /home/ubuntu/os/checkpoint-restore-operator/bin/kustomize build config/default | kubectl apply -f - | # Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically. | namespace/checkpoint-restore-operator-system created | customresourcedefinition.apiextensions.k8s.io/checkpointrestoreoperators.criu.org unchanged | serviceaccount/checkpoint-restore-operator-controller-manager created | role.rbac.authorization.k8s.io/checkpoint-restore-operator-leader-election-role created | clusterrole.rbac.authorization.k8s.io/checkpoint-restore-operator-manager-role created | clusterrole.rbac.authorization.k8s.io/checkpoint-restore-operator-metrics-reader created | clusterrole.rbac.authorization.k8s.io/checkpoint-restore-operator-proxy-role created | rolebinding.rbac.authorization.k8s.io/checkpoint-restore-operator-leader-election-rolebinding created | clusterrolebinding.rbac.authorization.k8s.io/checkpoint-restore-operator-manager-rolebinding created | clusterrolebinding.rbac.authorization.k8s.io/checkpoint-restore-operator-proxy-rolebinding created | service/checkpoint-restore-operator-controller-manager-metrics-service created | deployment.apps/checkpoint-restore-operator-controller-manager created | checkpoint-podname_namespace-containername-2024-07-24T18:09:34.tar | checkpoint-podname_namespace-containername-2024-07-24T18:09:35.tar | checkpoint-podname_namespace-containername-2024-07-24T18:09:36.tar | checkpoint-podname_namespace-containername-2024-07-24T18:09:37.tar | checkpoint-podname_namespace-containername-2024-07-24T18:09:38.tar [tests.yaml/build-and-deploy] ✅ Success - Main Deploy to Kubernetes [tests.yaml/build-and-deploy] ⭐ Run Main Wait for deployments to be ready [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/11] user= workdir= | Checking if the checkpoint-restore-operator-controller-manager deployment is ready | Waiting for checkpoint-restore-operator-controller-manager deployment to report a ready status (Attempt 2/20) | Waiting for checkpoint-restore-operator-controller-manager deployment to report a ready status (Attempt 3/20) | Waiting for checkpoint-restore-operator-controller-manager deployment to report a ready status (Attempt 4/20) | Waiting for checkpoint-restore-operator-controller-manager deployment to report a ready status (Attempt 5/20) | Available Progressing | The checkpoint-restore-operator-controller-manager deployment is ready [tests.yaml/build-and-deploy] ✅ Success - Main Wait for deployments to be ready [tests.yaml/build-and-deploy] ⭐ Run Main Check resources [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/12] user= workdir= | NAME READY STATUS RESTARTS AGE | pod/checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j 2/2 Running 0 21s | | NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE | service/checkpoint-restore-operator-controller-manager-metrics-service ClusterIP 10.96.188.90 8443/TCP 21s | | NAME READY UP-TO-DATE AVAILABLE AGE | deployment.apps/checkpoint-restore-operator-controller-manager 1/1 1 1 21s | | NAME DESIRED CURRENT READY AGE | replicaset.apps/checkpoint-restore-operator-controller-manager-5dbc4bb588 1 1 1 21s [tests.yaml/build-and-deploy] ✅ Success - Main Check resources [tests.yaml/build-and-deploy] ⭐ Run Main Wait for Checkpoint Tar Files to Reduce [tests.yaml/build-and-deploy] 🐳 docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/13] user= workdir= | checkpointrestoreoperator.criu.org/checkpointrestoreoperator-sample created | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Checkpoint tar files count is 5 (waiting for 2) | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | Timeout reached: Checkpoint tar files count is still 5 (should be 2) | Fetching checkpoint files for debugging: | total 60 | -rw-r--r-- 1 root root 10240 Jul 24 18:09 checkpoint-podname_namespace-containername-2024-07-24T18:09:34.tar | -rw-r--r-- 1 root root 10240 Jul 24 18:09 checkpoint-podname_namespace-containername-2024-07-24T18:09:35.tar | -rw-r--r-- 1 root root 10240 Jul 24 18:09 checkpoint-podname_namespace-containername-2024-07-24T18:09:36.tar | -rw-r--r-- 1 root root 10240 Jul 24 18:09 checkpoint-podname_namespace-containername-2024-07-24T18:09:37.tar | -rw-r--r-- 1 root root 10240 Jul 24 18:09 checkpoint-podname_namespace-containername-2024-07-24T18:09:38.tar | Fetching logs of operator pods: | Logs for pod checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j: | 2024-07-24T18:13:10Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"} | 2024-07-24T18:13:10Z INFO setup starting manager | 2024-07-24T18:13:10Z INFO Existing file {"name": "ORIGINAL_NAME"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:38+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:39+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-08T20:36:41+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-2024-07-11T21:58:21+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:44+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:45+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:46+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-06-25T00:48:47+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-2024-07-04T20:37:52+05:30.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:06.tar"} | 2024-07-24T18:13:10Z INFO Existing file {"name": "checkpoint-podname_namespace-containername-2024-07-24T23:22:07.tar"} | 2024-07-24T18:13:10Z INFO Watching {"directory": "/var/lib/kubelet/checkpoints"} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerContainer {"maxCheckpointsPerContainer": 10} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerPod {"maxCheckpointsPerPod": 20} | 2024-07-24T18:13:10Z INFO MaxCheckpointsPerNamespace {"maxCheckpointsPerNamespace": 30} | 2024-07-24T18:13:10Z INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} | 2024-07-24T18:13:10Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"} | I0724 18:13:10.248098 1 leaderelection.go:250] attempting to acquire leader lease checkpoint-restore-operator-system/f1499914.criu.org... | 2024-07-24T18:13:10Z DEBUG events checkpoint-restore-operator-controller-manager-5dbc4bb588-hwf5j_5bea18c9-092c-4754-a91e-10f8443cec97 became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"checkpoint-restore-operator-system","name":"f1499914.criu.org","uid":"36dc2c61-ae47-4671-ad93-1e41bfa67bef","apiVersion":"coordination.k8s.io/v1","resourceVersion":"705"}, "reason": "LeaderElection"} | I0724 18:13:10.268995 1 leaderelection.go:260] successfully acquired lease checkpoint-restore-operator-system/f1499914.criu.org | 2024-07-24T18:13:10Z INFO Starting EventSource {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "source": "kind source: *v1.CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting Controller {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator"} | 2024-07-24T18:13:10Z INFO Starting workers {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "worker count": 1} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerContainer {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerContainer": 5} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerPod {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerPod": 10} | 2024-07-24T18:13:20Z INFO Changed MaxCheckpointsPerNamespace {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "maxCheckpointsPerNamespace": 10} | 2024-07-24T18:13:20Z INFO Applying policies immediately {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-q2sv5"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-q2sv5_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "containername"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-podname_namespace-containername-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global container-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "container": "nginx"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-nginx-[0-9][0-9][0-9][0-9]-[0-2][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-6][0-9]*.tar"} | 2024-07-24T18:13:20Z INFO Applying global pod-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pod": "nginx-deployment-85bfd45d84-hq752"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-nginx-deployment-85bfd45d84-hq752_default-*-*.tar"} | 2024-07-24T18:13:20Z INFO Applying global namespace-level policy {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "namespace": "default"} | 2024-07-24T18:13:20Z INFO Looking for checkpoint archives {"controller": "checkpointrestoreoperator", "controllerGroup": "criu.org", "controllerKind": "CheckpointRestoreOperator", "CheckpointRestoreOperator": {"name":"checkpointrestoreoperator-sample","namespace":"default"}, "namespace": "default", "name": "checkpointrestoreoperator-sample", "reconcileID": "a66b28e3-3a92-44cf-bff3-2b7be29faed8", "pattern": "/var/lib/kubelet/checkpoints/checkpoint-*_default-*-*.tar"} [tests.yaml/build-and-deploy] ❌ Failure - Main Wait for Checkpoint Tar Files to Reduce [tests.yaml/build-and-deploy] exitcode '1': failure [tests.yaml/build-and-deploy] 🏁 Job failed Error: Job 'build-and-deploy' failed ```

adrianreber · 2024-07-28T09:38:43Z

I think it would be helpful if the script test/wait_for_checkpoint_reduction.sh would be a bit more verbose. Maybe how many checkpoint archives it found initially and what it is waiting for. Currently the output is very terse.

adrianreber · 2024-07-28T09:40:59Z

I am also a fan of shellcheck and shfmt but we can add that in a later step.

test/generate_checkpoint_tar.sh

test/test_checkpointrestoreoperator.yaml

.github/workflows/tests.yaml

config/samples/_v1_checkpointrestoreoperator.yaml

docs/retention_policy.md

internal/controller/checkpointrestoreoperator_controller.go

test/data/config.dump

test/run_tests.sh

docs/retention_policy.md

.github/workflows/tests.yaml

docs/retention_policy.md

config/samples/_v1_checkpointrestoreoperator.yaml

- Introduced global, container, pod, and namespace-level policies for checkpoint retention, all based on count limits. - Updated CRD definitions to store the newly added policies. - Added a flag `applyPoliciesImmediately` to enable immediate application of policies upon their update or creation. - Implemented logic to prioritize container, pod, and namespace-specific policies over global policies. - Updated the sample (`/config/samples/_v1_checkpointrestoreoperator.yaml`) with new policies. Signed-off-by: Parthiba-Hazra <[email protected]>

Implemented a GitHub Actions workflow to test the checkpoint restore operator's ability to correctly delete checkpoints through garbage collection. - Added `wait_for_checkpoint_reduction.sh` script to check the number of checkpoint tar files and ensure it is reduced to target value within a 60-second timeout. - Added `generate_checkpoint_tar.sh` script to correctly create and move checkpoint tar files. - Added `run_tests.bats` to test different garbage collection policies of the operator Signed-off-by: Parthiba-Hazra <[email protected]>

- Configure workflow to use local registry to build and push the image. - Add `configure_local_registry.sh` for connecting the registry with kind cluster so that the image can be pulled from the cluster. Signed-off-by: Parthiba-Hazra <[email protected]>

- Describe how to configure the checkpoint retention policy - Provide sample YAML for applying retention policies - Explain the hierarchy and specificity of policies Signed-off-by: Parthiba-Hazra <[email protected]>

rst0git

LGTM

@Parthiba-Hazra Great work on the implementation of count-based policies!

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 4ee200b to e668f80 Compare June 24, 2024 07:13

Parthiba-Hazra force-pushed the count-based-policies branch 4 times, most recently from abd701f to 6674bbe Compare July 8, 2024 18:56

Parthiba-Hazra force-pushed the count-based-policies branch from 6674bbe to 1f479ed Compare July 9, 2024 16:50

snprajwal reviewed Jul 9, 2024

View reviewed changes

.github/workflows/tests.yaml Outdated Show resolved Hide resolved

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 60e9c4c to bcca817 Compare July 9, 2024 17:21

rst0git reviewed Jul 10, 2024

View reviewed changes

config/manager/kustomization.yaml Outdated Show resolved Hide resolved

rst0git reviewed Jul 17, 2024

View reviewed changes

.github/workflows/tests.yaml Outdated Show resolved Hide resolved

rst0git reviewed Jul 17, 2024

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 3ff3897 to 7fb34d3 Compare July 24, 2024 18:25

Parthiba-Hazra force-pushed the count-based-policies branch 6 times, most recently from e4354cc to ec2b390 Compare July 27, 2024 16:41

Parthiba-Hazra requested review from adrianreber, snprajwal and rst0git July 27, 2024 17:06

rst0git mentioned this pull request Jul 27, 2024

make: update kustomize to v5.4.3 #21

Merged

rst0git reviewed Jul 28, 2024

View reviewed changes

test/generate_checkpoint_tar.sh Outdated Show resolved Hide resolved

rst0git reviewed Jul 28, 2024

View reviewed changes

test/test_checkpointrestoreoperator.yaml Outdated Show resolved Hide resolved

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 0d95d72 to b074f6b Compare July 30, 2024 15:21

Parthiba-Hazra requested review from adrianreber and rst0git July 30, 2024 15:27

Parthiba-Hazra force-pushed the count-based-policies branch from b074f6b to a3f4cd6 Compare July 30, 2024 15:30

snprajwal reviewed Aug 1, 2024

View reviewed changes

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 6a341a0 to 3e2edf0 Compare August 2, 2024 15:49

snprajwal reviewed Aug 2, 2024

View reviewed changes

test/run_tests.sh Outdated Show resolved Hide resolved

Parthiba-Hazra force-pushed the count-based-policies branch 2 times, most recently from 34ea36c to 061494b Compare August 6, 2024 10:22

rst0git reviewed Aug 7, 2024

View reviewed changes

.github/workflows/tests.yaml Show resolved Hide resolved

rst0git reviewed Aug 7, 2024

View reviewed changes

docs/retention_policy.md Outdated Show resolved Hide resolved

docs/retention_policy.md Outdated Show resolved Hide resolved

docs/retention_policy.md Outdated Show resolved Hide resolved

Parthiba-Hazra force-pushed the count-based-policies branch 3 times, most recently from 5aca925 to 79e782c Compare August 7, 2024 19:39

Parthiba-Hazra mentioned this pull request Aug 10, 2024

feat: Introduced storage/size based global, container, pod, and namespace-level policies #6

Open

Parthiba-Hazra force-pushed the count-based-policies branch from 79e782c to e8a925f Compare August 12, 2024 14:14

rst0git reviewed Aug 13, 2024

View reviewed changes

config/samples/_v1_checkpointrestoreoperator.yaml Outdated Show resolved Hide resolved

Parthiba-Hazra added 4 commits August 13, 2024 16:06

docs: Add checkpoint retention policy documentation

d5bc0de

- Describe how to configure the checkpoint retention policy - Provide sample YAML for applying retention policies - Explain the hierarchy and specificity of policies Signed-off-by: Parthiba-Hazra <[email protected]>

Parthiba-Hazra force-pushed the count-based-policies branch from e8a925f to d5bc0de Compare August 13, 2024 10:38

rst0git approved these changes Aug 13, 2024

View reviewed changes

rst0git merged commit 2d9695c into checkpoint-restore:main Aug 13, 2024
6 checks passed

rst0git mentioned this pull request Aug 13, 2024

config/manager: update default image name #29

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add count-based checkpoint retention policies #5

Add count-based checkpoint retention policies #5

Parthiba-Hazra commented Jun 23, 2024

rst0git commented Jul 5, 2024

adrianreber commented Jul 9, 2024

Parthiba-Hazra commented Jul 24, 2024

adrianreber commented Jul 28, 2024

adrianreber commented Jul 28, 2024

rst0git left a comment

Add count-based checkpoint retention policies #5

Add count-based checkpoint retention policies #5

Conversation

Parthiba-Hazra commented Jun 23, 2024

rst0git commented Jul 5, 2024

adrianreber commented Jul 9, 2024

Parthiba-Hazra commented Jul 24, 2024

adrianreber commented Jul 28, 2024

adrianreber commented Jul 28, 2024

rst0git left a comment

Choose a reason for hiding this comment