What configures the serviceNetwork?

[Nadyita]

I'm trying to setup OKD with 3 control planes and 3 workers on VPS servers of GoDaddy. There is 0 k8s infrastructure or help provided, not private networks between the servers, or even load balancers. It's even challenging to boot CoreOS at all, because you cannot boot from arbitrary iso files, but I managed to get 2 load balancers (haproxy), the bootstrap server and master0 up following this guide. However, I have no idea where the serviceNetwork (172.30.0.0/16 as per default) is configured. I don't see this network being brought up on any of the hosts. I'm doing regular networking for all machines via kernel cmdline rd.neednet=1 ip=<lotsofthings>.

Am I supposed to set this up myself somehow (or have it provided by my hoster, haha!), or should okd create this network automatically? My machine-config-server on master0 currently gives a 500 when querying a master ignition file for master1, and the logs are spamming with network errors.

My install-config.yaml:

apiVersion: v1
baseDomain: nadybot.org
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 3
metadata:
  name: okd
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
platform:
  none: {}
pullSecret: '{"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}}'
sshKey: 'ssh-ed25519 …'

log-bundle-20231226231811.tar.gz

[vrutkovs]

We recommend using Assisted Installer on "new" clouds - this would validate host details before starting installation. See OKD instructions

However, I have no idea where the serviceNetwork (172.30.0.0/16 as per default) is configured

This would be a CNI config setting and won't show up in ip a list and such.

should okd create this network automatically

It would be created automatically

My machine-config-server on master0 currently gives a 500 when querying a master ignition file for master1, and the logs are spamming with network errors

I see master0 fetching Ignition (and provisioning correctly), the other masters should point to the same exact URL

[Nadyita]

I see master0 fetching Ignition (and provisioning correctly), the other masters should point to the same exact URL

master0 gets its configuration from the bootstrap server, but the haproxy-config (from the docs)

listen machine-config-server-22623 
  bind *:22623
  mode tcp
  server bootstrap bootstrap.ocp4.example.com:22623 check inter 1s backup 
  server master0 master0.ocp4.example.com:22623 check inter 1s
  server master1 master1.ocp4.example.com:22623 check inter 1s
  server master2 master2.ocp4.example.com:22623 check inter 1s

configures the bootstrap server to be used only as a backup if all master-servers are down. So master1 would get its config from master0, but master0 gives a 500 when requesting the ignition file. For workers, the ignition file would be created correctly, but we don't need workers yet…

We recommend using Assisted Installer on "new" clouds - this would validate host details before starting installation. See OKD instructions

The problem is that the installer requires a "Machine network", but the IPs of my servers are in completely different networks. Some are 68.0.0.0, some are 72.0.0.0, so what am I supposed to enter here? Should I maybe create a separate VPN network? If so, how am I supposed to do that with the installer? If not: what is this range used for?

[vrutkovs]

Ah, yes, now I see MCS on master throwing errors:

I1226 21:53:51.449689       1 log.go:194] http: TLS handshake error from 208.109.215.121:57422: local error: tls: bad record MAC
I1226 21:53:56.319340       1 api.go:116] Pool master requested by address:"208.109.215.121:49238" User-Agent:"curl/8.2.1" Accept-Header: "*/*"
E1226 21:53:56.319387       1 api.go:135] couldn't get config for req: {master 0xc0005d21c0}, error: could not fetch config , err: machineconfig.machineconfiguration.openshift.io "" not found

which is weird, do you turn on masters one by one? They are expected to start simultaneously so that all of them fetch Ignition from bootstrap.