Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v255 batch #426

Merged
merged 43 commits into from
Jul 24, 2024
Merged

v255 batch #426

merged 43 commits into from
Jul 24, 2024

Conversation

bluca
Copy link
Member

@bluca bluca commented Jul 24, 2024

No description provided.

keszybz and others added 30 commits July 24, 2024 13:51
By itself, this is not useful. I'm making this a separate commit to
make debugging easier. It turns out that meson does static libraries
using references, so the "static library" a tiny stub stub that refers
to the object files on disk and this has negligible cost:
$ ls -lhd build/src/core/libsystemd-core-257.{a,so}
-rw-r--r-- 1 zbyszek zbyszek  36K Jul  3 16:54 build/src/core/libsystemd-core-257.a
-rwxr-xr-x 1 zbyszek zbyszek 6.1M Jul  3 16:54 build/src/core/libsystemd-core-257.so

(cherry picked from commit d0689ee5fbfafa736e6eca89bc80cb2d372f2229)
(cherry picked from commit c3b4032fc3153684c9917f23e08e2928f8972f0d)
The new link-executor-shared option is similar to the existing
link-udev-shared: when set to false, we link to the static versions of our
internal libraries.

The resulting exuctor binary is fairly large, about as large as libsystemd-core
(14 MB without lto, 8 with lto).

This is intended as a workaround for the fuckup with the pinned executor
binary:
when an upgrade is performed, the package manager will install new version of
the libraries and new version of the code, and some time later reexecute the
managers. This creates a window when the pinned executor binary will fail to
execute. There are two factors which make the issue easier to hit:

- when the distribution uses a finely-grained shared-lib-tag. E.g. Fedora
  uses version-release as the tag, which means that the issue occurs on
  every package upgrade. This is the right thing to do, because the
  ABI of our internal libraries is not stable at all, so replacing the
  library from a different version in place creates a window where our
  programs may crash or misbehave.

- when the distribution doesn't immediately reexec all the managers after
  upgrade. In early versions of systemd, we used to hammer the machine during
  upgrade, doing daemon-reexecs repeatedly. This works, but is ugly and
  wasteful. Doing the reexecs while the upgrade is in progres also creates a
  window where a mix of old and new configs or both is loaded. Users are
  particularly annoyed by those reloads if there is some issue in the
  configuration causing us to emit warnings on every reexec. Doing the
  reexecs once after the new configuration and libraries have been put
  in place is nicer.

The pinning of the executor binary breaks upgrades and in particular
it penalizes the distributions which make use of the features which
were previously added to avoid bugs and inefficiency during upgrades.

When the executor is linked statically, there is a smaller chance that it'll
fail to load libraries. The issue can still occur because other libraries, not
our own, are linked dynamically.

(cherry picked from commit d59cae6cebd0fc25a16a020bd28e5303901f1b19)
(cherry picked from commit d28aa922fdee5c5c438ca9b485b92b21180482d3)
(cherry picked from commit bffd3c52ad7113f21523568120d84326151f4600)
(cherry picked from commit ab11d7e177378704859703f4821784462c8966a5)
This is only possible since a recent kernel version, and fails otherwise,
like on CentOS 9

(cherry picked from commit ff8c89aa5a84733dd777f3e9113df33ce6c1ab1e)
(cherry picked from commit a8a7a6716ef2a8e6f10410b9af92fafc3ce204d4)
…able

dhcpd is not available on CentOS Stream 10

See systemd/systemd#33717

(cherry picked from commit 985d5b4bc23f791bdc79d4c2a9a949cc5d3bc27a)
(cherry picked from commit 89904fc10c57b28d064f7abbde97a636000e1323)
$SYSTEMD_REPART_OVERRIDE_FSTYPE is too invasive. Often you want to
override the fstype only for a specific designator, so let's support
that as well.

(cherry picked from commit 90a255779d1f8e6697e08e91918df88bb52274ad)
(cherry picked from commit a63e82ca4afc2e663e81d1a8cffb035b5e5922f9)
Copied directly from a1d6dbb.

(cherry picked from commit a669091a436c4f0c8537c83bba05a1e33d263dad)
(cherry picked from commit 8b3bedd821bad917c3ddf8914b84da54230854bd)
On Fedora, with crypto policy TEST-FEDORA41, sha1 is not allowed:

$ SYSTEMD_LOG_LEVEL=debug build/systemd-measure sign
  --linux=/lib/modules/6.9.7-200.fc40.x86_64/vmlinuz
  --osrel=/tmp/tmp.osrelbl2sr77f
  --cmdline=/tmp/tmp.cmdlineouc7hqtj
  --uname=/tmp/tmp.unamecbjgesty
  --pcrpkey=/tmp/tmpufiadu8l
  --initrd=/boot/3a9d668b4db749398a4a5e78a03bffa5/6.9.7-200.fc40.x86_64/initrd
  --sbat=/tmp/tmp.sbataz9arpy0
  --private-key=/tmp/tmppyf0gx6w
  --public-key=/tmp/tmpufiadu8l
  --bank=sha1
Measuring boot phases: enter-initrd, enter-initrd:leave-initrd, enter-initrd:leave-initrd:sysinit, enter-initrd:leave-initrd:sysinit:ready
Loaded 'libtss2-esys.so.0' via dlopen()
Loaded 'libtss2-rc.so.0' via dlopen()
Loaded 'libtss2-mu.so.0' via dlopen()
PolicyPCR calculated digest: cec1a2ccb188ddd171a2be7bfa6b31cb9148776647354eb1069e0f891ed2dbe7
Failed to initialize signature context: error:03000098:digital envelope routines::invalid digest
Failed to sign PCR policy: Input/output error

(cherry picked from commit 87204601df305c270ffa05430081bc5b76dede04)
(cherry picked from commit 88264411b674f4d0b260c4b9ac7dbb1b96879209)
They might not be readable to the unprivileged user running the tests
and it shouldn't really matter what is used. OTOH, we need a real kernel
because we look at the header.

(cherry picked from commit 987f4bce938e790622a4b4b89d37daa7adfdc141)
(cherry picked from commit bcb13a3fa258cc179cef4db4f77014560ba627e5)
(cherry picked from commit 399646faac8ca91a46287e40280f56055c9e0fe8)
(cherry picked from commit bb71d5dfb45bb96ce4a6e535e3a93c14b7074b62)
(cherry picked from commit 701bd9d08ac1d16f74e2b453ca0826e85b1c8491)
(cherry picked from commit 67e0d093682d763ff1d27027a5040571fc039193)
(cherry picked from commit 09d6038d833468ba7c24c658597387ef699ca4fd)
(cherry picked from commit f414ca0ee3bed9b67f76a67d8eb569fda99f5fde)
(cherry picked from commit dd6b325a05c4caccd1a17dd4147f48a916eee386)
(cherry picked from commit 908edce5b6d4fd5243eab3e7fc8ec57d0e8da130)
Currently, install_info_apply() only updates r if it's 0,
meaning that if one of the earlier install_info_symlink_alias/wants()
calls returns > 0, errors generated by later calls will be discarded.
Fix that.

(cherry picked from commit a159aa07e1548367d2fde80cb0d45b869c591864)
(cherry picked from commit bb83650f96af1a7f803696fab5dd06442c789b1c)
Follow-up for b2751cf

Also make the conditions consistent for install_info_symlink_wants().

Fixes #33411

(cherry picked from commit 4441cf330b3847d6c553fb230e8e4c86aa75ebb9)
(cherry picked from commit a42db16a1cd0e080d972bd778ea44e981ea31dfc)
(cherry picked from commit 9fb5a8ca24e677e10f8c2b8973b5e2a11676bda0)
(cherry picked from commit 7684f528392ddb94455c924ec820be2c8de6989a)
(cherry picked from commit 4442aef08e0fe8ba381b580455f7eb281c5a28a1)
(cherry picked from commit 06c2ee39799064a82d2af1bee7a5c72ebdd66090)
…33632)

* Remove extra period at end of unit description.

Having an extra period at the end of this unit description makes log entries pertaining to it appear weirdly, as it seems the default expectation is that there is not to be a period at the end of a unit description.

e.g.: `systemd[1]: Started Displays emergency message in full screen..`

(cherry picked from commit 496b4fa0e974d7a1b10b8af966da445a28f512c5)
(cherry picked from commit 35e2f6296782a7aadd52baad33c17b350a564a09)
When watching a given pathspec, systemd unconditionally installs
IN_ATTRIB watches to track the link count of the resolved file. This
way, we are notified if the watched path disappears, even if the
resolved file inode is not removed.

Similarly, systemd installs inotify watches on each parent directory, to
be notified when the specified path appears. However, for these watches
IN_ATTRIB is an unnecessary addition to the mask. In inotify, IN_ATTRIB
on a directory is emitted whenever the attributes of any child changes,
which, for many paths, has the potential to cause a high number of
spurious wakeups in systemd. Let's remove IN_ATTRIB from the mask when
installing watches on the parent directories of the specified path.

(cherry picked from commit 8bf8c7d83dcffffa55b5f534fb98db6b01315dc1)
(cherry picked from commit fa2b2da1466ff225363c1a0492b1b43c1d01dd8a)
Symlink created by Alias will use the value as the file name.

(cherry picked from commit 3f0e7fd4fd1d20e3f4be18f485c76d25ce10f41b)
(cherry picked from commit a68188e985d29e46cfa6eb2e17419fad90f0b287)
Otherwise they will pull down the disk too, which we don't want on soft-reboot

(cherry picked from commit bbb0b72849ebbeeb8e252d9aeed94521df4f0ae8)
(cherry picked from commit 1747350ffd05e2588b808d17befbc36072207c3c)
Fixes #32918

(cherry picked from commit f9572d2b89341dfb224aa2c7222a316e59627bc9)
(cherry picked from commit bcda6d46373a7cf071d86e91917bbdd31a5597dd)
Addons are called addons, say so. And some other fixes.

(cherry picked from commit 40d9c16d1ee98c16af5804bab256b62c37feac3b)
(cherry picked from commit 78fcf31f08664371d2df58f9cf43d68a74a347ec)
This is VFAT world after all.

(cherry picked from commit 764faf60400bafb1764b728aafe0dcf4cbf07364)
(cherry picked from commit 18143edf3e582d6b8c2933f5c181c9b29146023a)
… does

(cherry picked from commit 35451a32043504013eed5725c8be46b36ccdf71a)
(cherry picked from commit 3736e21341500d98d878b84a34cc5b9d7cd9125f)
When boot counting is enabled, adding a new loader entry or UKI can conflict
with an existing one that has booted successfully and therefore has its boot
counter removed. systemd-bless-boot will fail to bless the new successful boot,
since a file without a boot counter already exists. Since kernel-install will
clobber existing files without boot counting, we should therefore remove files
without a boot count as well, when we add a file with one.

Fixes: #33504
(cherry picked from commit 99d4575e541fa1fb00dc80f7aad572f3a66db461)
(cherry picked from commit b78618540659a40c4c26aa588b3cd8b9c46116d1)
Same as the other aliases. Allows chaining commands like:

$ systemd-id128 show -P root-$(dpkg-architecture --query DEB_HOST_ARCH)
4f68bce3e8cd4db196e7fbcaf984b709

(cherry picked from commit f0b151ce864371da06a4d4a63a2a8b5282817b7e)
(cherry picked from commit b60d5bc1b774f900dc5c5d45faed17e919bdf0b3)
Prompted by #33737

The intention of b37e818
is to expose sd_id128_get_app_specific() on command line.
But combining that with GPT type list makes little sense.

(cherry picked from commit fa96c55b7b0d19a7f72908ee7d3f8a1ef630be96)
(cherry picked from commit 86ec58a55c62e974a4f0cc345fac3fd84817399f)
This is a common case, and nothing noteworthy at all. For example, if we
establish an enumerator for listing all devices tagged by some tag, then
the per-tag dir is not going to exist if there are currently no devices
tagged that way, but that's a really common case, and doesn't really
deserve any mention, not even at debug level.

(cherry picked from commit a68c97a54527cacaeeac0c117493639fc455ef5e)
(cherry picked from commit 8aa9e60f89f84a90fb364ee66cf62432a6b877ba)
(cherry picked from commit 0d8b38415c4b192c1a4e608da8dc16c7340e31ba)
(cherry picked from commit d870f2335b4e233cf67c9e9011711051889bc100)
lucaswerkmeister and others added 13 commits July 24, 2024 14:16
Make the warning for oneshot services (where RuntimeMaxSec= has no
effect) more actionable by pointing to the directive people can use
instead to effectively limit their runtime.

(cherry picked from commit 8c4aa0f1c6a78b35712fa6a7acf6d755d0c0bd86)
(cherry picked from commit 468b0646342986c6cc9bd797b4ba189dc488ee8d)
(cherry picked from commit af63b4b769bfb86ff7848af980268901e9c0b47d)
(cherry picked from commit e274de4b744b3d189e297b38e558771cd8806dc9)
If building with clang and clang does not support bpf, then enabling
-Dbpf-framework=enabled would silently drop the feature (even printing
bpf-framework: enabled in the meson build recap, and no message anywhere
that'd hint at the failure!)

This is unexpected, so add check to fail hard in this case.

All other code paths (gcc, missing bpftool) properly check for the
option, but it is not as easy for a custom command so check explicitly

(cherry picked from commit 8da20e3fe2a544979922cea457de3031aa74d64c)
(cherry picked from commit d6f8575f1e771ec667ed6821fa89ac679dab119d)
If there is an error with the execv call in fork_agent the
program exits without any meaningful log message. Log the
command and errno so the user gets more information about
the failure.

Fixes: #33418

Signed-off-by: Mauri de Souza Meneguzzo <[email protected]>
(cherry picked from commit a408d4453145621902b9a3ef78a552f83b09bd8d)
(cherry picked from commit 7fcfb73d71ed1d4230f58de1a94790e0c28719ea)
(cherry picked from commit 8b6de9e6381b39f59c936d2b0c6ce47f1b70a19e)
(cherry picked from commit f81659f5f37eec39182e98ae02608c28de0ed292)
…eed_reload

Follow-up for 19a44df

If a drop-in is set from upper level, e.g. global unit_type.d/,
even if a unit is masked, its dropin_paths would still be partially
populated. However, unit_need_daemon_reload() would always
compare u->dropin_paths with empty strv in case of masked units,
resulting in it always returning true. Instead, let's ignore
dropins entirely here.

Fixes #33672

(cherry picked from commit 11b3775f514f521f353741ff6ac4d66cf0e928e8)
(cherry picked from commit 6a3cb4cd11119cf8d3ed29d076c223b0fe491f98)
We'll *always* hit ENEOENT when iterating through SMBIOS type systemd#11
fields, on the last one. it's very confusing to debug log about that,
let's just not do it.

(cherry picked from commit 5202ee42d5da0ae3a6655d2bc959a19d8c347e9d)
(cherry picked from commit 995c702a347d16cfad4605f3982d5278616ea1f8)
It is unnecessary, which will mess the completion.

(cherry picked from commit 733518b41350ce781c7e41a4c866eafb9e549e1f)
(cherry picked from commit fd2a6ea0a8d4b535e4ac3645772b946906e02c7d)
Certainly on systemd 252 at least a configuration of
```
MemorySwapMax=40%
```
is supported but this was missing from the man page.
Only MemoryMax was documented as supporting a %.

(cherry picked from commit 8af38e5b0475f514141d314088dcf9fffd7edc37)
(cherry picked from commit 766af3f782299a7cbfba24a4333444ac008c17d2)
The page was written when systemd-repart was primarily intended to be used on a
running system. But nowadays it's more often used to create images, so extend
that part of the description.

While at it, fix some whitespace issues and trim some overly complicated sentences.

(cherry picked from commit d202ea57549248c4246c8f453a2ff88a4c2a7e1e)
(cherry picked from commit e60d01bdbf0d31d32d4bf1e36d5c40e16c84fafb)
Follow-up for 6906c02

The mentioned commit uses access() to check if varlink socket
already exists in the filesystem, but that isn't sufficient.

> Varlink sockets are not serialized until v252, so upgrading from
> v251 or older means we will not listen anymore on the varlink sockets.
>
> See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1074789
> for more details as this was found when updating from Debian Bullseye to a new version.

After this commit, the set up of varlink_server is effectively
split into two steps. manager_varlink_init_system(), which is
called after deserialization, would no longer skip listening
even if Manager.varlink_server is in place, but actually
check if we're listening on desired sockets.
Then, manager_deserialize() can be switched back to using
manager_setup_varlink_server().

Alternative to #33817

Co-authored-by: Luca Boccassi <[email protected]>
(cherry picked from commit d4e5c66ed469c822ca5346c7a445ec1446b1d17f)
(cherry picked from commit b825a8be0b7b857a715e982cee861e8ae6995ee8)
(cherry picked from commit 2cf425ec573b8f67025c5e74cd267015129e7349)
(cherry picked from commit a78a52465298e8f5a927da9c9fc56c41837018aa)
git restore -s origin/main hwdb.d/ test/hwdb.d

(cherry picked from commit b731debea9221ca43edc49f85be23db2fde79492)
@bluca bluca merged commit 6c260ee into systemd:v255-stable Jul 24, 2024
41 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.