Replies: 3 comments 31 replies
-
|
Please attach log bundle |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the long delay, I had an other issue, the host had NIC bond with balance-rr, NICs where connected to two different switchs, which lead to packet loss. I sorted that issue out, but still having the same problem with bootkube.service not starting. I tired to pull the logs with I am able to connect with SSH on to the bootsrap with the privat key, what tells me that I have the right key. Maybe I am being stupid here but I redone everything severel times and still having the same issue. If you want me to post the manually gathered logs, I am currently getting them. |
Beta Was this translation helpful? Give feedback.
-
|
I may have found the issue, it seems that the download of the OpenShift Release Image is not working. If I try to open the URL, I get 403: Unauthorized Tried to login to quay.io seems to work but gives me this: {"error": "CSRF token was invalid or missing."} I have setup a second account on redhat and still the same issue with a different pull secret. Ever seen this before? |
Beta Was this translation helpful? Give feedback.
-
|
That means you're attempting to install OCP. Double check you're using OKD installer. If you're intending to install OCP please contact support instead |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, I swapped the installer as I was troubleshooting last week and forgot to change it back. Got me a bit further but still having issues... |
Beta Was this translation helpful? Give feedback.
-
|
log-bundle-20210805141205.tar.gz I created a new self signed RSA certificate and created new ignitionfiles but getting stuck on master not pulling the image.... This is what the master shows: Service machinge "wait-for bootstrap-complete" I attached you the log bundle, please tell me if I can provide anything else. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Bad node naming could be a cause. You will need to make sure each of the nodes has a DNS PTR record defined. From reviewing the control plane journals, I do think a good next step would be fixing the node names by defining the mentioned PTR records. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks I will try later this day, I've already all 3 PTR records in my DNS conf "cxw-tst58.eng.local.", "cxw-tst58.cxw-tst-okd.eng.local." and "etcd-0.cxw-tst-okd.eng.local." for all the masters. So I guess for the installation only "cxw-tst58.eng.local." is needed? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, otherwise the node name will be somewhat random based on which PTR record is chosen. Also, the node which was |
Beta Was this translation helpful? Give feedback.
-
|
Now I only have one PTR record per node "cxw-tstXX.cxw-tst-okd.eng.local." "nodename.clustername.domain." but still not completing... all masters have the correct hostname now so thats a plus I guess. log-bundle-20210818144848.tar.gz Had to run the command twice until it looked like all masters finished setting up from monitoring CPU load. root@cxw-tst56:/home/cxw_admin# openshift-install --dir=/home/cxw_admin/okd/install_d wait-for bootstrap-complete --log-level=debug |
Beta Was this translation helpful? Give feedback.
-
|
It definitely looks like it is getting much further along. It appears to be failing because the master nodes are not becoming ready. They appear to not be ready due to timeouts while communicating with the API via the load balancer: The master nodes need to be ready for the bootstrapping process to continue. From looking at the log-bundle, it appears that etcd is having extreme latency. Reads are taking over 20 seconds when they should be less than 100ms: This is usually caused by inadequate storage performance or inadequate CPU capacity. Until etcd performance issues are resolved you likely will not be able to complete an installation on this configuration. |
Beta Was this translation helpful? Give feedback.
-
hi, i got stuck on setting up bootstrap, masters and workers. I am not sure if bootkube.service not starting is the exact issue here, but that is the conclusion i got from troubleshooting.
some details about my installation (bootstrap and masters are the same):
Node OS: FCOS 34.20210626.3.1
Node Specs: 16GB RAM, 4 vCPUs, 120GB disk
following my troubleshoot, tell me if you need more info BR.
what i see on my master:
ignition[533]: GET error: Get "https://........:22623/config/master": EOF
ssh in to bootstrap:
Manually gathering logs:
[core@localhost ~]$ journalctl -b -f -u release-image.service -u bootkube.service
-- Journal begins at Wed 2021-07-21 12:40:41 UTC. --
Jul 21 12:55:22 localhost systemd[1]: Starting Download the OpenShift Release Image...
Jul 21 12:55:22 localhost release-image-download.sh[848]: Pulling quay.io/openshift-release-dev/ocp-release@sha256:eafdac268e1f65053de423ba4a028e8de5133ab78e7954d76ed838bcf5f4f666...
Jul 21 12:55:32 localhost podman[850]: 2021-07-21 12:55:32.863448543 +0000 UTC m=+10.077218247 system refresh
Jul 21 12:55:32 localhost podman[850]: 2021-07-21 12:55:32.864988287 +0000 UTC m=+10.078757975 image pull quay.io/openshift-release-dev/ocp-release@sha256:eafdac268e1f65053de423ba4a028e8de5133ab78e7954d76ed838bcf5f4f666
Jul 21 12:55:57 localhost release-image-download.sh[850]: 7df5b82029357ab3601bec214038b683e20603bbbd2e99e447aba5e50e56d607
Jul 21 12:55:57 localhost systemd[1]: Finished Download the OpenShift Release Image.
Jul 21 12:56:00 localhost systemd[1]: Dependency failed for Bootstrap a Kubernetes cluster.
Jul 21 12:56:00 localhost systemd[1]: bootkube.service: Job bootkube.service/start failed with result 'dependency'.
^C
[core@localhost ~]$
Beta Was this translation helpful? Give feedback.
All reactions