cannot boot linux_latest_hardened on lvm-on-luks anymore

[cyplo]

Describe the bug
Since switching from 19.09 to 20.03 I needed to switch from pkgs.linuxPackages_latest_hardened to pkgs.linuxPackages_latest, otherwise the system does not boot anymore.

To Reproduce
Steps to reproduce the behavior:

1. Install system on lvm-inside-luks using either 20.03 or unstable channel
2. use pkgs.linuxPackages_latest_hardened
3. sudo nixos-rebuild switch --upgrade
4. reboot
5. enter password - gets accepted correctly - LUKS seems to be working
6. system does not boot - complains about missing root device

Expected behavior
Would boot :)

Screenshots
latest_hardened:

latest:

Additional context
All my config for this machine starts from here - I link this file in place of /etc/nixos/configuration.nix

Notify maintainers
@thoughtpolice maybe ?

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.6.7, NixOS, 20.09pre223023.fce7562cf46 (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`
 - channels(cyryl): `"home-manager, unstable-20.09pre223023.fce7562cf46"`
 - channels(root): `"nixos-20.09pre223023.fce7562cf46"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

^ using unstable channel with latest kernel right now

Happy to help debug this if needed :)

[eadwu]

What filesystem are you using?

If it's a mainlined filesystem, I don't believe hardened should've broken that. But it's probably depending on CONFIG_AIO which is disabled in hardened.

[cyplo]

Thanks so much for the reponse :)

It's ext4 on lvm on luks for /, hopefully still mainlined ? ;)
I can try to compile the kernel with CONFIG_AIO set but otherwise with hardened settings enabled and see if it helps ?

[emilazy]

Sorry, this is my fault!

Yeah, my guess is that we need to re-enable AIO. I did notice some additional harmless-seeming warnings at boot that seemed AIO-related when writing #84522 (which switched to a patchset that disables AIO by default), but I guess it didn't affect any of my systems because they use ZFS.

I was hoping AIO was only required by marginal things (like Oracle...), but LVM is pretty important. Given that nginx also uses AIO per #84522 (comment), it might be worth explicitly turning it on in the default shared Kconfig (as I did for System V IPC in the original PR). I'm not sure if the AIO code is known to be particularly crufty by Linux standards or if it's just a general attack surface reduction thing; a few of the linux-hardened defaults are a bit better-suited to the Android environment than a desktop Linux distro. kconfig-hardened-check credits CONFIG_AIO=n to the original linux-hardened author's Android-based GrapheneOS which makes me suspect this might be the case here.

@cyplo Here's a nixpkgs diff to re-enable AIO:

diff --git a/pkgs/os-specific/linux/kernel/common-config.nix b/pkgs/os-specific/linux/kernel/common-config.nix
index 0a58ed87e6c..06fd21bb87b 100644
--- a/pkgs/os-specific/linux/kernel/common-config.nix
+++ b/pkgs/os-specific/linux/kernel/common-config.nix
@@ -615,6 +615,8 @@ let
 
       SYSVIPC            = yes;  # System-V IPC
 
+      AIO                = yes;  # POSIX asynchronous I/O
+
       UNIX               = yes;  # Unix domain sockets.
 
       MD                 = yes;     # Device mapper (RAID, LVM, etc.)

It'd definitely be good to know if this fixes things for you. nixos-rebuild -I nixpkgs=/path/to/patched/nixpkgs should be enough to test this. If you don't want to use a custom nixpkgs then you can do a fiddly linuxPackagesFor (linux_latest_hardened.override { ... }) thing like the definition of hardenedLinuxPackagesFor in all-packages.nix.

[emilazy]

Also, this would be pretty easy to test for, so if this change fixes it then we should also update the hardened test to use LVM.

[cyplo]

Heya, good news ! One kernel compilation later - I'm back to running hardened thanks to you :)
I tried the sudo nixos-rebuild switch -I nixpkgs=/path route and indeed it worked.

For the test - I think that would be awesome, to catch things like this in the future :)