Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] test: test against reference images lib generated manifest #186

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f6ed83f
Makefile: add "make externals" to have a quick way to build externals
mvo5 Sep 2, 2024
ae5147c
examples: update otk-gen-partition-table example to "work"
mvo5 Sep 2, 2024
39e198d
[RFC] test: test against reference images lib generated manifest
mvo5 Sep 2, 2024
ee5518b
fixup! integrate otk-{gen-partition-table,make-fstab-stage}
mvo5 Sep 2, 2024
c9ebe37
fixup! use otk.external.otk-make-partition-stages to generate partiti…
mvo5 Sep 2, 2024
486e11b
fixup! use otk-make-partition-mounts-devices
mvo5 Sep 2, 2024
8949b09
fixup! use filesystem.const.partition_map.root.uuid
mvo5 Sep 3, 2024
84a5bca
fixup! use filesystem.const.partition_map.boot.uuid
mvo5 Sep 3, 2024
067fe3d
fixup! use ${filesystem.const.filename}
mvo5 Sep 3, 2024
6ad7247
example: import centos-9-x86_64-ami
mvo5 Sep 3, 2024
843af8a
fixup! restore hardcoded part_uuid/partition table UUID
mvo5 Sep 3, 2024
7ac1bdf
example: use otk.external.otk-gen-partition-table in centos-9-x86_64-…
mvo5 Sep 3, 2024
4635037
examples: extract common centos parts
mvo5 Sep 3, 2024
d7834d2
examples/centos: remove everything that is not used currently
mvo5 Sep 3, 2024
771987f
fixup! add another todo
mvo5 Sep 3, 2024
f3dd7c0
fixup! fix incorrectly generated centos-9-x86_64-ami.yaml
mvo5 Sep 3, 2024
494d5f8
centos: extract build pipeline
mvo5 Sep 3, 2024
7e0f61b
fixup! extract kernel-cmdline fragemnt
mvo5 Sep 3, 2024
7ee91a0
centos: extract grub stages into fragments
mvo5 Sep 3, 2024
28fef79
centos: extract image pipeline
mvo5 Sep 3, 2024
239b7fa
examples: merge fedora-minimal
mvo5 Sep 3, 2024
5754525
fixup! extract more from fedora-minimal
mvo5 Sep 3, 2024
2430d24
fixup! add another fixme
mvo5 Sep 3, 2024
ec10b6d
examples: remove everything fedora that is not used
mvo5 Sep 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ git-diff-check:
@git diff --cached --exit-code

## Package building
SRCDIR ?= $(abspath .)
COMMIT = $(shell (cd "$(SRCDIR)" && git rev-parse HEAD))
RPM_SPECFILE=rpmbuild/SPECS/otk-$(COMMIT).spec
RPM_TARBALL=rpmbuild/SOURCES/otk-$(COMMIT).tar.gz
Expand All @@ -55,3 +56,15 @@ rpm: git-diff-check $(RPM_SPECFILE) $(RPM_TARBALL)
rpmbuild -bb \
--define "_topdir $(CURDIR)/rpmbuild" \
$(RPM_SPECFILE)

# XXX: strawman target name
.PHONY: externals
IMAGES_REF ?= github.com/osbuild/images
externals:
mkdir -p "$(SRCDIR)/externals"
for otk_cmd in gen-partition-table \
make-fstab-stage \
make-partition-mounts-devices \
make-partition-stages; do \
GOBIN="$(SRCDIR)/externals" go install "$(IMAGES_REF)"/cmd/otk-$${otk_cmd}@latest ; \
done
168 changes: 135 additions & 33 deletions example/centos/centos-9-x86_64-ami.yaml
Original file line number Diff line number Diff line change
@@ -1,37 +1,139 @@
otk.version: "1"

otk.define.default:
default:
language: "en_US.UTF-8"

otk.define.centos-9-x86_64-ami:
architecture: "x86_64"
version: "9"
kernel_opts_list:
- "rd.blacklist=nouveau"
- "nvme_core.io_timeout=4294967295"
sshd_password_authentication: true
modfication:
language: "en_US.UTF-8"
timezone: "UTC"
enabled_services:
- sshd
- NetworkManager
- nm-cloud-setup.service
- nm-cloud-setup.timer
- cloud-init
- cloud-init-local
- cloud-config
- cloud-final
- reboot.target
- tuned
# todo: should this go into centos-9-common.yaml ? and we customize via normal "otk.defines" ?
otk.define:
# XXX: de-dup
distro: centos
version: 9
kernel_commandline: "console=tty0 console=ttyS0,115200n8 net.ifnames=0 nvme_core.io_timeout=4294967295"
filesystem:
# this will expand to "root: ...\nboot: \n"etc (i.e. a real file-systems) and "kernel_opts_list:
# [list_item1, list_item2, ..]"
# (i.e. define the partition table as a map of root/boot/boot_efi vars)
# *and* it needs to undstand modifications (e.g. adding extra parititions somehow, size contraints)
otk.external.osbuild.create_partition_table_defines:
total_size: 10G
modifications:
filename: "image.raw"
otk.include: "common/gen-partition-table-x86_64.yaml"

otk.include.centos-9-x86_64: "common/centos-9.yaml"
otk.target.osbuild:
pipelines:
- otk.include: "pipeline/build.yaml"
- name: os
build: name:build
stages:
- otk.include: "fragment/kernel-cmdline.yaml"
- type: org.osbuild.rpm
- type: org.osbuild.fix-bls
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extracting: org.osbuild.fix-bls,org.osbuild.keymap, org.osbuild.timezone would work here but because it's in a list it will look quite awkward

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't be that awkward, I believe we already use otk.op.join in some places and I used it 'everywhere' in my branch just so fragments could define one or more stages :)

options:
prefix: ""
- type: org.osbuild.locale
options:
language: en_US.UTF-8
- type: org.osbuild.keymap
options:
keymap: us
x11-keymap:
layouts:
- us
- type: org.osbuild.timezone
options:
zone: UTC
- type: org.osbuild.chrony
options:
servers:
- hostname: 169.254.169.123
minpoll: 4
maxpoll: 4
iburst: true
prefer: true
leapsectz: ""
- type: org.osbuild.sysconfig
options:
kernel:
update_default: true
default_kernel: kernel
network:
networking: true
no_zero_conf: true
network-scripts:
ifcfg:
eth0:
bootproto: dhcp
device: eth0
ipv6init: false
onboot: true
peerdns: true
type: Ethernet
userctl: true
- type: org.osbuild.systemd-logind
options:
filename: 00-getty-fixes.conf
config:
Login:
NAutoVTs: 0
- type: org.osbuild.cloud-init
options:
filename: 00-rhel-default-user.cfg
config:
system_info:
default_user:
name: ec2-user
- type: org.osbuild.modprobe
options:
filename: blacklist-nouveau.conf
commands:
- command: blacklist
modulename: nouveau
- type: org.osbuild.modprobe
options:
filename: blacklist-amdgpu.conf
commands:
- command: blacklist
modulename: amdgpu
- type: org.osbuild.dracut.conf
options:
filename: sgdisk.conf
config:
install_items:
- sgdisk
- type: org.osbuild.dracut.conf
options:
filename: ec2.conf
config:
add_drivers:
- nvme
- xen-blkfront
- type: org.osbuild.systemd.unit
options:
unit: nm-cloud-setup.service
dropin: 10-rh-enable-for-ec2.conf
config:
Service:
Environment:
- key: NM_CLOUD_SETUP_EC2
value: "yes"
- type: org.osbuild.authselect
options:
profile: sssd
- type: org.osbuild.sshd.config
options:
config:
PasswordAuthentication: false
- otk.external.otk-make-fstab-stage:
${filesystem}
- otk.include: "fragment/grub2.yaml"
- type: org.osbuild.systemd
options:
enabled_services:
- sshd
- NetworkManager
- nm-cloud-setup.service
- nm-cloud-setup.timer
- cloud-init
- cloud-init-local
- cloud-config
- cloud-final
- reboot.target
- tuned
default_target: multi-user.target
- type: org.osbuild.selinux
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be split into a fragement as it's use with the same options everywhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely, we'd need separate fragments for the build and root pipelines as the build pipeline also sets install_t on cp, so maybe two fragments? :)

options:
file_contexts: etc/selinux/targeted/contexts/files/file_contexts
- otk.include: "pipeline/image.yaml"
sources:
org.osbuild.curl:
78 changes: 62 additions & 16 deletions example/centos/centos-9-x86_64-qcow2.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,66 @@
otk.version: "1"

otk.define.centos-9-x86_64-qcow2:
architecture: "x86_64"
version: "9"
kernel_opts_list: []
sshd_password_authentication: true
modfication:
language: "C.UTF-8"
timezone: "America/New_York"
enabled_services: []
otk.define:
# XXX: dedup
distro: centos
version: 9
kernel_commandline: "console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0"
filesystem:
# this will expand to "root: ...\nboot: \n"etc (i.e. a real file-systems) and "kernel_opts_list:
# [list_item1, list_item2, ..]"
# (i.e. define the partition table as a map of root/boot/boot_efi vars)
# *and* it needs to undstand modifications (e.g. adding extra parititions somehow, size contraints)
otk.external.osbuild.create_partition_table_defines:
total_size: 10G
modifications:
# empty

Check warning on line 10 in example/centos/centos-9-x86_64-qcow2.yaml

View workflow job for this annotation

GitHub Actions / lint

10:7 [comments-indentation] comment not indented like content
otk.include: "common/gen-partition-table-x86_64.yaml"

otk.include.centos-9-x86_64: "common/centos-9.yaml"
otk.target.osbuild:
pipelines:
- otk.include: "pipeline/build.yaml"
- name: os
build: name:build
stages:
- otk.include: "fragment/kernel-cmdline.yaml"
- type: org.osbuild.rpm
- type: org.osbuild.fix-bls
options:
prefix: ""
- type: org.osbuild.locale
options:
language: C.UTF-8
- type: org.osbuild.timezone
options:
zone: America/New_York
- type: org.osbuild.sysconfig
options:
kernel:
update_default: true
default_kernel: kernel
network:
networking: true
no_zero_conf: true
- otk.external.otk-make-fstab-stage:
${filesystem}
- otk.include: "fragment/grub2.yaml"
- type: org.osbuild.systemd
options:
default_target: multi-user.target
- type: org.osbuild.selinux
options:
file_contexts: etc/selinux/targeted/contexts/files/file_contexts
- otk.include: "pipeline/image.yaml"
- name: qcow2
build: name:build
stages:
- type: org.osbuild.qemu
inputs:
image:
type: org.osbuild.files
origin: org.osbuild.pipeline
references:
name:image:
file: ${filesystem.const.filename}
options:
# XXX: provide a way to derive the name from the filesystem one?
filename: disk.qcow2
format:
type: qcow2
compat: "1.1"
sources:
org.osbuild.curl:
114 changes: 0 additions & 114 deletions example/centos/common/centos-9.yaml

This file was deleted.

Loading
Loading