[FEATURE]: Add /boot/firmware/cmdline.txt to OS Customization

[phreed]

Is your feature request related to a problem? Please describe.

There are settings which should be used the first time the Pi is booted.
The Raspberry Pi uses a configuration file instead of the BIOS you would expect to find on a conventional PC. The system configuration parameters, which would traditionally be edited and stored using a BIOS, are stored instead in an optional text file named config.txt.
Specifically, /boot/firmware/cmdline.txt controls things like the setting of cgroups.

• https://www.raspberrypi.com/documentation/computers/config_txt.html
• https://www.raspberrypi.com/documentation/computers/configuration.html#cmdline-txt
• https://www.raspberrypi.com/documentation/computers/configuration.html#the-kernel-command-line

Describe the solution you would like to see implemented

Add the ability to edit (or supply) an /boot/firmware/cmdline.txt as part of the 'OS Customization'.

Describe alternatives you've considered

Once the sdcard has been written I edit the mounted /boot/firmware/cmdline.txt to add cgroup settings.

Additional context

It would be helpful to have a button on the main screen to open the 'OS Customization'.
I know there is a hot-key 'Ctrl-Shift-X' to do this but a button on the main screen would be helpful.

Version

1.8.5 (Default)

[lurch]

Bear in mind that Raspberry Pi Imager is capable of flashing many different OSes, and not all of them may use a /boot/firmware/cmdline.txt ?

It would be helpful to have a button on the main screen to open the 'OS Customization'.

The OS-customisation step is now part of the image-flashing workflow instead of being part of the image-selection workflow; see the screenshots at https://www.raspberrypi.com/documentation/computers/getting-started.html#raspberry-pi-imager

[tdewey-rpi]

As I have stated elsewhere in this issue tracker - my bias for accepting features and PRs for Raspberry Pi Imager is overwhelmingly in favour of inexperienced users - to the point where I will reject anything that may compromise their experience (even if it would improve my experience).

I am therefore minded to reject this Feature request. I cannot see how exposing this can be done in a manner where it's clear to the new user what it is, what it does, and how to verify it's not going to produce a broken system. I consider this feature in the same bucket as the run arbitrary script as part of OS customisation requests.

That said, I don't have a monopoly on good ideas, and I'm prepared to be swayed if there is evidence that this feature would benefit a first-time Raspberry Pi user. In the spirit of discussion, then, I will withhold judgement until March 12th 2024 - and I am prepared to accept counterpoints until that time.

[phreed]

#828 (comment)
Regarding the placement in the workflow.
To me it seems to be too late makes sense there.
Why is it not at the same level as the three main selections?
If I am making headless devices, things like providing ssh keys seems equally important.

[lurch]

@phreed I'm not sure I understand what you're saying. Could you explain why it's "too late"? If you look at the link I provided, you'll see that it does have an option for providing an SSH key.

[phreed]

#828 (comment)

I see your point.
It is reasonably easy for me to actually make an "arbitrary script" that I can run before ejecting the storage device.
Changing the cmdline.txt could be considered correcting a defect in the image.
That being said,

@phreed I'm not sure I understand what you're saying. Could you explain why it's "too late"? If you look at the link I provided, you'll see that it does have an option for providing an SSH key.

Here is the thought process I went through using the tool:

1. Context, e.g. I want to make this Raspberry Pi a Headless Server (specifically, a micok8s node)

2. I want to pick the Raspberry Pi hardware I am writing, cool, here is a button for that.

3. I want to pick the OS I will be running on it, cool, here is a button for that.

4. I want to pick the Storage media, cool, here is a button for that.

5. I want to pick the Access method, umm, there is no button... I guess I will pick Next and hope for the best.
   This is what I meant by "too late". I am feeling the slightest degree of apprehension at this point.

6. Ah, maybe it is in the "Settings", yup there it is. Oh look it will run keygen for me, that would have been good to know about earlier.

7. Yikes, there is a "No, Clear Settings", I sure do not want to pick that. (Maybe "No, Ignore Settings" would be less hostile).

8. Hmm, the image is slightly "broken" for Kubernetes. Maybe I should write an issue.

All in all not a bad workflow, just two small confusions.
I can do everything I needed and it is certainly smoother than a tool that just burns an image onto media.

[phreed]

#828 (comment)
@tdewey-rpi
Maybe the right place would be on the front end (rather than the backend, as I suggested).
The issue is that the image I pick is, in some sense, broken (the cmdline.txt is missing cgroup_enable=memory which is needed by microk8s).
If I understand right the RaspberryPi imager is pulling a configuration file which describes all the devices and os images.
If that is correct, where is that file retrieved from, i.e. I would like to look at it.
Could some image patching mechanism be provided in the imager configuration file?
Essentially, providing patched versions of images? This would provide a place for run arbitrary script NOT part of OS Customization but part of OS image presentation.
To your point, you want to keep most complexity away from the user and put it in the hands of the image creators.

[tdewey-rpi]

@phreed

A completely fair request. I'd typically expect something like this to be done using pi-gen: https://github.com/RPi-Distro/pi-gen

Assuming you're just using a fairly vanilla Raspberry Pi OS, I'd hope you could use that tool to assemble a "deeper" customisation of the OS (for example, including your microk8s changes), which you could then use locally - and potentially contact me to get added as a Raspberry Pi Imager distro.

Between that, and the rpi-imager --cli mode, which allows scripting and more arbitrary firstrun.sh changes, I think there's a way to satisfy your requirements in a slightly more robust manner. Let me know your thoughts.

[lurch]

If I understand right the RaspberryPi imager is pulling a configuration file which describes all the devices and os images.
If that is correct, where is that file retrieved from, i.e. I would like to look at it.

https://downloads.raspberrypi.com/os_list_imagingutility_v4.json

[phreed]

@tdewey-rpi
Thanks for the pointers. Yes, I think those two paths will do just what I need.

$ rpi-imager --cli
Usage: --cli [--disable-verify] [--sha256 <expected hash> [--cache-file <cache file>]] [--first-run-script <script>] [--debug] [--quiet] <image file to write> <destination drive device>

I found https://forums.raspberrypi.com/viewtopic.php?t=326006
Specifically, I am wondering when the --first-run-script is run.
I see --cli works on Windows as well as Linux.

Feel free to close this issue as complete.
I would like to see a button on the main page for OS Configuration though.

Apparently, there is another (linux only) tool (in addition to pi-gen) which can be used to make custom images.
https://github.com/gitbls/sdm/blob/master/QuickStart.md

[phreed]

If I understand right the RaspberryPi imager is pulling a configuration file which describes all the devices and os images.
If that is correct, where is that file retrieved from, i.e. I would like to look at it.

https://downloads.raspberrypi.com/os_list_imagingutility_v4.json

Just to complete the thought:

Here is a fragment of the current json.

"os_list": [
{
"name": "Raspberry Pi OS (64-bit)",
"description": "A port of Debian Bookworm with the Raspberry Pi Desktop (Recommended)",
"icon": "https://downloads.raspberrypi.com/raspios_armhf/Raspberry_Pi_OS_(32-bit).png",
"url": "https://downloads.raspberrypi.com/raspios_arm64/images/raspios_arm64-2023-12-06/2023-12-05-raspios-bookworm-arm64.img.xz",
"extract_size": 5838471168,
"extract_sha256": "66d2bc469bc2531097039616fcf91169ffd861981e1f881030b178da1ffd64df",
"image_download_size": 1162186752,
"release_date": "2023-12-05",
"init_format": "systemd",
"devices": [
"pi5-64bit",
"pi4-64bit"
]
},

I was thinking something like the following:

{
"name": "Raspberry Pi Container OS (64-bit)",
"description": "Raspberry Pi OS (64-bit) with settings needed for containers",
"base_image": "66d2bc469bc2531097039616fcf91169ffd861981e1f881030b178da1ffd64df",
"files": [{
   "path": "/boot/firmware/cmdline.txt"
   "content": "console=serial0,115200 dwc_otg.lpm_enable=0 cgroup_enable=memory cgroup_enable=cpuset console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash"
   }]
},
...

Where the new OS is based on another other OS.
The provided file and its content are added to the sdcard after everything else is complete.

[lurch]

Specifically, I am wondering when the --first-run-script is run.

https://github.com/search?q=repo%3ARPi-Distro%2Fraspberrypi-sys-mods+firstrun&type=code

Apparently, there is another (linux only) tool (in addition to pi-gen) which can be used to make custom images.
https://github.com/gitbls/sdm/blob/master/QuickStart.md

That's a tool by a 3rd-party user, and is therefore outside of our control.

[phreed]

Specifically, I am wondering when the --first-run-script is run.

https://github.com/search?q=repo%3ARPi-Distro%2Fraspberrypi-sys-mods+firstrun&type=code

#554
Covers the use of first_run.sh.

[tdewey-rpi]

If I understand right the RaspberryPi imager is pulling a configuration file which describes all the devices and os images.
If that is correct, where is that file retrieved from, i.e. I would like to look at it.

https://downloads.raspberrypi.com/os_list_imagingutility_v4.json

Just to complete the thought:

Here is a fragment of the current json.

"os_list": [
{
"name": "Raspberry Pi OS (64-bit)",
"description": "A port of Debian Bookworm with the Raspberry Pi Desktop (Recommended)",
"icon": "https://downloads.raspberrypi.com/raspios_armhf/Raspberry_Pi_OS_(32-bit).png",
"url": "https://downloads.raspberrypi.com/raspios_arm64/images/raspios_arm64-2023-12-06/2023-12-05-raspios-bookworm-arm64.img.xz",
"extract_size": 5838471168,
"extract_sha256": "66d2bc469bc2531097039616fcf91169ffd861981e1f881030b178da1ffd64df",
"image_download_size": 1162186752,
"release_date": "2023-12-05",
"init_format": "systemd",
"devices": [
"pi5-64bit",
"pi4-64bit"
]
},

I was thinking something like the following:

{
"name": "Raspberry Pi Container OS (64-bit)",
"description": "Raspberry Pi OS (64-bit) with settings needed for containers",
"base_image": "66d2bc469bc2531097039616fcf91169ffd861981e1f881030b178da1ffd64df",
"files": [{
   "path": "/boot/firmware/cmdline.txt"
   "content": "console=serial0,115200 dwc_otg.lpm_enable=0 cgroup_enable=memory cgroup_enable=cpuset console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash"
   }]
},
...

Where the new OS is based on another other OS. The provided file and its content are added to the sdcard after everything else is complete.

A fair suggestion - but also one that might suggest there's freedom to modify the ext4 filesystem inside of OS customisation. Unfortunately, that capability doesn't exist today (critically, we'd need to implement an ext4 modification flow for Windows, which we don't have).

With that in mind, I'll add this to the feature considerations for 2.0. In the meantime, I hope pi-gen can get you some satisfaction, and I'll keep an eye on the issue tracker there in case I see problems in this area.

Finally, I think this FR is probably 'Won't address', at least in the near term. I believe there's auxiliary tooling that's better suited to produce OS images that can then be written by Raspberry Pi Imager, and for customisations beyond the simple variety offered in Raspberry Pi Imager, users should consult that tooling until further notice.

[calphool]

As I have stated elsewhere in this issue tracker - my bias for accepting features and PRs for Raspberry Pi Imager is overwhelmingly in favour of inexperienced users - to the point where I will reject anything that may compromise their experience (even if it would improve my experience).

I am therefore minded to reject this Feature request. I cannot see how exposing this can be done in a manner where it's clear to the new user what it is, what it does, and how to verify it's not going to produce a broken system. I consider this feature in the same bucket as the run arbitrary script as part of OS customisation requests.

That said, I don't have a monopoly on good ideas, and I'm prepared to be swayed if there is evidence that this feature would benefit a first-time Raspberry Pi user. In the spirit of discussion, then, I will withhold judgement until March 12th 2024 - and I am prepared to accept counterpoints until that time.

You guys are obviously way closer to this than me, but I thought I'd just comment that the reason I'm here is BECAUSE I'm a relatively inexperienced user and this thread seemed like it applies to my situation.

I've got a Raspberry Pi 5, and of course it defaults to HDMI output only (with an annoying micro-hdmi interface, but that's not your problem). I'm using it to build full sized arcades for people's rec rooms, and I often have to convert HDMI to VGA for the monitors being used (people like the nostalgia of CRTs), while separating the audio stream to send it to an amplifier. When I connect the HDMI to VGA adapter, sometimes something whacko happens with the EDID conversation between the adapter and the RP5, and I end up with either strange video resolution, or no audio. The work around seems to be to build a custom EDID binary set up with the resolution you want and basic audio turned on, and then changing the cmdline.txt so that it loads this EDID binary at boot up. This is a ridiculously complex way to say "I want you to boot up in 1024x768 and please make sure you stream audio over the HDMI interface."

Moving all that stuff so that can be done through something a lot more friendly than trying to build EDID binaries sure seems like it would be a step in the right direction, which seems to align with what the OP was asserting. Though, as I say, you all are far closer to this space than me.

[tdewey-rpi]

As I have stated elsewhere in this issue tracker - my bias for accepting features and PRs for Raspberry Pi Imager is overwhelmingly in favour of inexperienced users - to the point where I will reject anything that may compromise their experience (even if it would improve my experience).
I am therefore minded to reject this Feature request. I cannot see how exposing this can be done in a manner where it's clear to the new user what it is, what it does, and how to verify it's not going to produce a broken system. I consider this feature in the same bucket as the run arbitrary script as part of OS customisation requests.
That said, I don't have a monopoly on good ideas, and I'm prepared to be swayed if there is evidence that this feature would benefit a first-time Raspberry Pi user. In the spirit of discussion, then, I will withhold judgement until March 12th 2024 - and I am prepared to accept counterpoints until that time.

You guys are obviously way closer to this than me, but I thought I'd just comment that the reason I'm here is BECAUSE I'm a relatively inexperienced user and this thread seemed like it applies to my situation.

I've got a Raspberry Pi 5, and of course it defaults to HDMI output only (with an annoying micro-hdmi interface, but that's not your problem). I'm using it to build full sized arcades for people's rec rooms, and I often have to convert HDMI to VGA for the monitors being used (people like the nostalgia of CRTs), while separating the audio stream to send it to an amplifier. When I connect the HDMI to VGA adapter, sometimes something whacko happens with the EDID conversation between the adapter and the RP5, and I end up with either strange video resolution, or no audio. The work around seems to be to build a custom EDID binary set up with the resolution you want and basic audio turned on, and then changing the cmdline.txt so that it loads this EDID binary at boot up. This is a ridiculously complex way to say "I want you to boot up in 1024x768 and please make sure you stream audio over the HDMI interface."

Moving all that stuff so that can be done through something a lot more friendly than trying to build EDID binaries sure seems like it would be a step in the right direction, which seems to align with what the OP was asserting. Though, as I say, you all are far closer to this space than me.

Thanks for the feedback, @calphool.

I completely sympathise with your point regarding the difficulty of this path. I must, however, point out this is not captured inside the typical use case cone - while there are a significant number of people building retrogaming systems and other systems using 'older' display technology (I should point out I do have a fondness for CRT displays, though!), I still cannot see a way to expose such a function that doesn't introduce more confusion for people who are coming to a Raspberry Pi for the first time.

An additional counterargument is in the form of visibility and debug. I routinely see issues raised against this project chastising rpi-imager for failing, or performing, actions that it never has - most likely because the user doesn't understand what they are trying to do, or what the tooling is doing. The function you describe is, I believe, of sufficient gravity that if someone were to misconfigure it, they would have significant problems trying to debug the problem - most notably because they wouldn't have a viable display!

I suspect your better path forward is to investigate tooling to automate the extraction & modification of an EDID.