Sidecar rewrite

[misohu]

Context

We rewrite all of our charms using the sidecar pattern instead of the old podspec.

What needs to get done

Rewrite the charm using sidecar with base charm pattern.

Definition of Done

Charm is rewritten with sidecar with base charm pattern.
All of the tests are rewritten and passing.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5256.

This message was autogenerated

[DnPlas]

For completing this task we will need to consider the following

Base charm usage

Because we are using the base charm standard to refactor this charm, the proposed components are:

• MlmdPebbleService - for creating and updating the Pebble layer. This component will make sure to run the bin/metadata_store_server command with all required flags.
• LeadershipGateComponent - to make sure the unit is the leader

Since this is a relatively easy charm, nothing else should be required.

Sqlite to mysql migration

The current implementation of this charm uses sqlite and a volume to store all ML metadata. Ideally we will move away from that approach and integrate with the mysql-k8s charm.
For this, we'll need to make sure the mysql-k8s charm adapts to the requirements of the mlmd workload and shares the information it expects (db host, port, user, password and db name). More investigation is required for completing this sub-task.

k8s-service relation interface replacement

Because this relation interface is based on the old SDI implementation, the proposal is to replace it with the newer version that's worked in canonical/mlops-libs#2. This sub-task requires us to potentially change all the charms that relate to mlmd to ensure a correct data sharing.

[DnPlas]

Update on the mysql-k8s integration

We have tried using mysql-k8s to replace sqlite as a database. We have found issues in the past and decided not to go with that approach due to the expectations of both the mlmd workload and the mysql-k8s restrictions. For more details about this, please refer to #64.

Because of the above, we may have to stick with the sqlite solution, which means that we'll need to "manually" configure the volumes of the application containers:

1. Keeping the filesystem storage defined already in metadata.yaml
2. Using pebble to push the metadata_store_server_config_file to /config/config.proto

Please note that this solution completely deviates from the upstream deployment, where they are using mysql as their DB.

[kimwnasptd]

@DnPlas since MLMD Charm is using a PVC for holding the data. Then how will the upgrade look like for this Charm (podspec to sidecar) if we need to remove/re-apply the Charm?

If it's indeed a problem we don't have to tackle it in this issue, but let's have a follow-up then and track it in the 1.9 release

[DnPlas]

@DnPlas since MLMD Charm is using a PVC for holding the data. Then how will the upgrade look like for this Charm (podspec to sidecar) if we need to remove/re-apply the Charm?

If it's indeed a problem we don't have to tackle it in this issue, but let's have a follow-up then and track it in the 1.9 release

@kimwnasptd I'm not entirely sure what you mean by "we need to remove/re-apply the Charm`. Are you saying this because in the past we had to remove some charms because of incompatibilities?

My understanding for the upgrade story is that the storage and its data should not be lost during the process. When we remove a charm that has a storage attached, the default action juju takes is to detach the storage, and not remove it entirely, which means that after re-deploying the application we must be able to attach it to it. I will test this is the case for this effort.