kubeadm can't pull coredns:v1.8.6 from registry.k8s.io #2761
Labels
kind/support
Categorizes issue or PR as a support question.
Comments
older k8s versions have a default of gcr.l8s.io during upgrade the registry.k8s.io migration will be handled for you. |
|
@neolit123 That would be great, indeed. Except it's the registry migration is not handled at all. Or the recommended way is to ignore the preflight errors? |
|
@neolit123 I guess the real issue is that the new registry |
|
what are the contents of: did you manually edit this cm?
we have upgrade ci that is passing. all coredns images should be under a /coredns path. the key here is that custom registry, different from the one hardcoded in the kubeadm binary would result in a path without /coredns. i mentioned this above. |
|
if you have edited the registry to registry.k8s.io manually the cm please revert it to k8s.gcr.io and try upgrade again. the 1.25 upgrade will do the migration for you. |
I'm not upgrading to The However the image repository seems to be already upgraded, I'm not sure how that happened (I haven't done that manually). $ kubectl -n kube-system get cm kubeadm-config -o yaml | grep imageRepository
imageRepository: registry.k8s.io |
|
|
please use |
|
I've reverted the and now the upgrade seems to be working. |
weizhouapache
added a commit
to weizhouapache/cluster-api-provider-cloudstack
that referenced
this issue
May 8, 2023
with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io This change can be reverted when we upgrade to k8s 1.25+
weizhouapache
added a commit
to weizhouapache/cluster-api-provider-cloudstack
that referenced
this issue
May 10, 2023
The image repository has been changed to registry.k8s.io by commit 8c1e614 However, with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ``` ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 ``` this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io setting the image respository to "" so that capi/kubeadm will determine the default repository by kubernetes version.
hrak
pushed a commit
to hrak/cluster-api-provider-cloudstack
that referenced
this issue
May 15, 2023
The image repository has been changed to registry.k8s.io by commit 8c1e614 However, with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ``` ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 ``` this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io setting the image respository to "" so that capi/kubeadm will determine the default repository by kubernetes version.
hrak
pushed a commit
to hrak/cluster-api-provider-cloudstack
that referenced
this issue
May 15, 2023
The image repository has been changed to registry.k8s.io by commit 8c1e614 However, with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ``` ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 ``` this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io setting the image respository to "" so that capi/kubeadm will determine the default repository by kubernetes version.
vignesh-goutham
pushed a commit
to vignesh-goutham/cluster-api-provider-cloudstack
that referenced
this issue
Jul 12, 2023
The image repository has been changed to registry.k8s.io by commit 8c1e614 However, with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ``` ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 ``` this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io setting the image respository to "" so that capi/kubeadm will determine the default repository by kubernetes version.
vignesh-goutham
pushed a commit
to vignesh-goutham/cluster-api-provider-cloudstack
that referenced
this issue
Jul 12, 2023
The image repository has been changed to registry.k8s.io by commit 8c1e614 However, with registry.k8s.io, the control plane vm cannot be booted to Ready state due to error below ``` ubuntu@disk-offering-gzojvb-control-plane-gwbnt:~$ tail -f /var/log/cloud-init-output.log [2023-05-08 11:09:23] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [2023-05-08 11:09:28] error execution phase preflight: [preflight] Some fatal errors occurred: [2023-05-08 11:09:28] [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns:v1.8.4: output: time="2023-05-08T11:09:28Z" level=fatal msg="pulling image: rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/coredns:v1.8.4\": failed to resolve reference \"registry.k8s.io/coredns:v1.8.4\": registry.k8s.io/coredns:v1.8.4: not found" [2023-05-08 11:09:28] , error: exit status 1 ``` this is same as kubernetes/kubeadm#2761 The new registry should be supported in k8s 1.25+. However, we still use 1.22/1.23/1.24 templates, so we need to use k8s.gcr.io setting the image respository to "" so that capi/kubeadm will determine the default repository by kubernetes version.
ghost
mentioned this issue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
BUG REPORT
kubeadmcan't upgradev1.24.3->v1.24.6Versions
kubeadm version (use
kubeadm version):Environment:
kubectl version):v1.24.3uname -a):5.10.0-17containerdflannelWhat happened?
Failed to pull image "registry.k8s.io/coredns:v1.8.6".
What you expected to happen?
Download the image or use local cache.
How to reproduce it (as minimally and precisely as possible)?
Pulling images works fine
However it's pulling from
k8s.gcr.iowhile the "real" upgrade is usingregistry.k8s.io.When
registry.k8s.iois specified, thecoredns:v1.8.6image is missing.Anything else we need to know?
Seems to be related to #2671.
The text was updated successfully, but these errors were encountered: