Normal cephadm setups use different labels than osism

[boekhorstb1]

This issue is related to #94.

It seems e.g. cephadm setups use the fsid of the cluster for their systemd units:

ceph-5cce5876-5f97-11ef-aa89-5bc6fcba3be8@alertmanager.vm1.service
[email protected]
[email protected]
ceph-5cce5876-5f97-11ef-aa89-5bc6fcba3be8@mds.mds.vm1.jwyofk.service
[email protected]
[email protected]
ceph-5cce5876-5f97-11ef-aa89-5bc6fcba3be8@node-exporter.vm1.service
[email protected]
[email protected]
ceph-5cce5876-5f97-11ef-aa89-5bc6fcba3be8.target
ceph.target

because of this migration with rookify fails e.g. here:

Command: 'sudo systemctl disable --now ceph-mon.target'

Exit code: 1

[NotTheEvilOne]

Thank you very much for your findings. Do I read your list of systemd units correctly that there's no unit file named [email protected] (or similar)?

See https://docs.ceph.com/en/reef/rados/operations/operating/#starting-all-daemons-by-type

[boekhorstb1]

ok I ran my testsetup again, on this cephadm version:

root@master:~# cephadm version
cephadm version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)

looking for targets like in the link you send me gives this:

vagrant@master:~$ sudo systemctl status ceph\*.target
● ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0.target - Ceph cluster fe0d9064-95e1-11ef-9868-67dc24888cb0
     Loaded: loaded (/etc/systemd/system/ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0.target; enabled; vendor preset: enabled)
     Active: active since Tue 2024-10-29 12:16:33 UTC; 53min ago

Oct 29 12:16:33 master systemd[1]: Reached target Ceph cluster fe0d9064-95e1-11ef-9868-67dc24888cb0.

● ceph.target - All Ceph clusters and services
     Loaded: loaded (/etc/systemd/system/ceph.target; enabled; vendor preset: enabled)
     Active: active since Tue 2024-10-29 12:16:33 UTC; 53min ago

Oct 29 12:16:33 master systemd[1]: Reached target All Ceph clusters and services.

systemd services look like this, the one targets has a "wants" which in turn has all services:

vagrant@master:~$ ls /etc/systemd/system/
ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0.target        final.target.wants              paths.target.wants            snap.lxd.user-daemon.unix.socket
ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0.target.wants  getty.target.wants              rescue.target.wants           snapd.mounts.target.wants
[email protected]      iscsi.service                   sleep.target.wants            sockets.target.wants
ceph.target                                             k3s.service                     snap-core20-2318.mount        [email protected]
ceph.target.wants                                       k3s.service.env                 snap-lxd-29351.mount          sshd.service
cloud-final.service.wants                               mdmonitor.service.wants         snap-snapd-21759.mount        sudo.service
cloud-init.target.wants                                 multi-user.target.wants         snap.lxd.activate.service     sysinit.target.wants
dbus-org.freedesktop.resolve1.service                   multipath-tools.service         snap.lxd.daemon.service       syslog.service
dbus-org.freedesktop.timesync1.service                  network-online.target.wants     snap.lxd.daemon.unix.socket   timers.target.wants
emergency.target.wants                                  open-vm-tools.service.requires  snap.lxd.user-daemon.service  vmtoolsd.service
vagrant@master:~$ ls /etc/systemd/system/ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0.target.wants
ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@alertmanager.master.service    ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@mgr.master.hhdsag.service
ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@ceph-exporter.master.service   [email protected]
[email protected]           ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@node-exporter.master.service
[email protected]         [email protected]
ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@mds.mds.master.folfpu.service  ceph-fe0d9064-95e1-11ef-9868-67dc24888cb0@prometheus.master.service
vagrant@master:~$ 

so a mon service looks like this:

[email protected]

note: the number after ceph-xxx is the fsid of the cluster (ceph -s returns that number)

[boekhorstb1]

I do wonder if that has something todo with my setup though :) checking...
otherwise one maybe try something sudo systemctl disable --now ceph-mon\*.service as a second option? Or is that too brutish. The link you send me notes both options :)

[boekhorstb1]

It could be some ceph realted bug, looks a bit like this: https://bugzilla.redhat.com/show_bug.cgi?id=1324589#c4

[boekhorstb1]

ok @NotTheEvilOne I found out.
For those targets to work, you need to install extra tooling.
For ubuntu I installed ceph-base ceph-mds ceph-osd ceph-volume for the tarets to appear for example.

[boekhorstb1]

So should we adapt the README to inform that such postinstalls for targets to appear are a requirement?

[NotTheEvilOne]

I'll add a configurable solution as well.