KGPE-D16 : Do we want a headless system until dom0 kernel boots?

[tlaurion]

https://github.com/osresearch/heads/blob/e62362ddcc6b0ea295b138c91c0b99805714ea13/boards/kgpe-d16/kgpe-d16.config#L27

That line could also add standard terminal by adding console=tty0
@flammit What do you think?

For some reason, reflashing OpenBMC disabled it. Will dig into it, but meanwhile, I needed this.

[flammit]

This kgpe-d16.config is currently configured to be headless, so maybe it's worthwhile making two separate configs kgpe-d16-headless.config w/ network recovery and BMC console for Heads & OS and kgpe-d16-local.config w/o network recovery and the Heads & OS console on tty0. @osresearch any thoughts?

[osresearch]

Is the only difference the kernel command line?

a) it seems that keeping the kernel command line in the bzImage is a bad plan long term
b) having a way to update the command line (that still goes through measurements, etc) is necessary
c) coreboot can deliver the command line; can we have it stored in an updatable part of cbfs?

[flammit]

It's a combination of the coreboot/heads kernel command line as well as the target kernel command line additions to set both consoles to have the correct init stdin/stdout:

https://github.com/osresearch/heads/blob/e62362ddcc6b0ea295b138c91c0b99805714ea13/config/coreboot-kgpe-d16.config#L669

[tlaurion]

@osresearch @flammit : Actually, following "LUKSerror: luks device has no key/passphrase encountered bug from QubesOS text-based installer, it is currently impossible to create a LUKS container on a headless computer from QubesOS installation if not hacking your way and providing a TBD kickstart recipe. Note the the second phase install would also not be covered and would required the user to finish installation through manual qubesctl salt calls. Kickstart OEM installation is desirable, but not a "Just works ™" approach and shouldn't be the default.

I propose to make the default kgpe-d16 board configuration dual output to both serial console and local tty.

[tlaurion]

Having issues having dual console to boot QubesOS graphical installation. The installer complains about hvc1 not being accessible but fires up in FB mode, as opposed to the text only installer.

Unfortunately, I have no time for that and needed my server to reinstall QubesOS right away so I splitted up the configuration in precedent WiP PR and used kgpe-d16_workstation to reinstall QubesOS, and flashed back the kgpe-d16.config resulting rom to manage it remotely through OpenBMC. As a result, now, I cannot run qvm-run -a --pass-io sys-firewall "echo test" which doesn't even fails after a timeout or anything.

Will try to play around Xen console recommendations later on when I'll come visit back my server physically.

EDIT: Actually it was my error. Launching qvm-run --no-gui is required when not logged into X...

Installing QubesOS on headless system is still not possible.

[tlaurion]

Actually it was my error. Launching qvm-run --no-gui is required when not logged into X...

The recovery shell was not working on my kgpe-d16 last time I tried.

EDIT: If someone has an idea to be able to launch that Qubes Graphical installation remotely...
SSH->OpenBMC->serial to host with a working tty? Someone has done that before?

[awokd]

Haven't tried it myself, but would this work for what you are doing?

I let the default (not text) install fail, unable to start Xorg.
I press Alt-Tab until I get the shell window. I find the network
interface name by running networkctl. I run dhclient with the
interface name as the sole argument. I use ifconfig to find the
IP address that DHCP assigned. I remove /var/run/anaconda.pid.
I run

anaconda --vnc

On a remote computer behind the same router I start a vnc client and
connect to the Qubes target IP::5901. I think the client (tightvnc
on Windows 8.1) defaults to port 5900, and the anaconda sets up 5901.

[tlaurion]

@awokd: Awesome. Haven't thought of doing that. Thanks. So headless it will stay!

[tlaurion]

@marmarek: QubesOS installation falling back into text mode only should ask the user if he wants to continue through VNC. Want me to open a ticket for that?

[marmarek]

@tlaurion Qubes installer have disabled network on purpose - to allow safe installation, even if computer is plugged into hostile network, so this definitely should not automatically fallback into VNC install. But indeed, with a confirmation, it would be much easier than with kickstart.
On the other hand, if X server failed to start during installation, most likely it will also fail to start on the installed system, which greatly reduce usefulness of such system (QubesOS being client system).
But it may be worth adding such option anyway.

[tlaurion]

@awokd @marmarek :

Well. Since we connect to the KGPE-D16 through OpenBMC by ssh and from there, access to host terminal by screen /dev/ttyS0 115200

When arriving at the installer, we need to type CTRL-ALT-TAB to switch to shell there, then launch dhclient, ip addr and then anaconda --vnc. Voila Installer works.

[Tonux599]

Assuming most people will be using a secondary GPU if using Qubes on this board. If we include the required drivers for the GPU (i.e. nouveau for Nvidia cards) in the kernel, and set the jumper to disable the onboard GPU on the motherboard, Linux will only see this secondary video card and so will the installer.

I used this method to install Qubes recently on the board with coreboot. Qubes installer initialised the secondary GPU fine and as the primary display. The only thing the installer complained about was that there was no IOMMU (even though enabled in coreboot). Appending 'iommu=force' to Xen's command line rectified this.

[tlaurion]

@Tonux599 we would then talk about a third board config. Headless, aspeed and nouveau.

[Tonux599]

@tlaurion Not necessarily, as has been mentioned before we could have two configs.

1. A headless server config, End user could manage the system remotely via OpenBMC.
2. A workstation config, contains the display drivers for Aspeed and other popular display drivers.

If the jumper is at default, the Aspeed GPU is initialised by Linux as the primary display. If the jumper is set at disable, Linux doesn't even see the Aspeed GPU and as such any secondary GPU is initialised as the primary display. Qubes installer (and booting into Qubes) runs great with the internal GPU disabled.

[tlaurion]

@Tonux599 my point here is that you use nouveau, I use radeon...

This is a server board having ASPEED by default, which can be configured to be used (ugly) but as is, without external additinal hardware.

Kernel modules are not heavy and can be added in a workstation config and should be use case based.

If you have PR, please propose.

Else I will work on this this week cause I want to use this board, but again, use case of mine would be server+radeon for local display + Libremkey_notpm.

I'm more challenging the good way to do this then anything else. In present thinking, I think i'll have to create kgpe_d16-bmc-radeon-libremkey_notpm, which will have console output to both bmc and tty0, linux drivers for radeon/aspeed, while not depending on network-recovery script but launching generic-init and having code to measure rom and validate through libremkey. This use case will permits disk unlocking via console/radeon while validating rom integrity without a TPM.