Implement optimization migrations

[sosthene-nitrokey]

I'm not sure this should be merged before the FIDO2 certification is done.

Depends on:

• read_dir_first: stop at the first file that is alphabetically "after" not_before trussed-dev/trussed#151
• ClientFilestore: make the dat folder optional trussed-dev/trussed#150
• deserialize_identifier: add support for all types expected by serde trussed-dev/cbor-smol#8
  Replaced by:
  • PR for upstream unreviewed PRs cbor-smol#2
• Add migration mechanism without relying on the staging backend. admin-app#23
• Reduce space taken by credentials, using a migration step fido-authenticator#55
• Remove the dat folders to save littlefs space trussed-dev/trussed-auth#39
• Add migration for data trussed-se050-backend#16
• Use trussed-auth 0.3.0 trussed-secrets-app#109
• Update trussed-auth to 0.3.0 opcard-rs#201
• Update trussed staging and trussed-auth piv-authenticator#48

[nitrokey-ci]

No significant changes.

Insignifcant changes

metric	value		change
binary-size-nk3am	1194682	✅	-2962 (-0.00%)
binary-size-nk3am-test	2018799	🔴	+17064 (+0.01%)
binary-size-nk3xn	429864	✅	-1500 (-0.00%)
binary-size-nk3xn-test	546620	✅	-884 (-0.00%)
binary-size-nkpk	731876	✅	-5098 (-0.01%)

[sosthene-nitrokey]

I made the migrations optional for for the trussed-auth and trussed-se050-backend and disabled them in this PR. I also removed the FIDO migrations from this PR.

That way we can merge this without merging the migrations until the FIDO certification is done.

[robin-nitrokey]

I will need some more time to properly review and think through the admin-app changes. IMHO we could merge a first version that updates trussed-auth, trussed-se050-backend, trussed and the affected dependencies but keeps the V0 version unconditionally as soon as the use_raw fields are replaced with a filesystem version . Then we could have a second PR that introduces the migrations and updates admin-app. This would make sure that we don’t have too many open PRs, can make a new release soon without pulling in the migrations and have enough time to review the migrations.

[sosthene-nitrokey]

Ok, i'll see if that's possible. Since admin-app is only a dependency of trussed-auth for the tests it might be fine.

[robin-nitrokey]

Of course we can also keep it in one PR if that’s easier. I just wanted to mention that option because I think it should not be a big change and it would reduce the amount of unmerged changes.

[sosthene-nitrokey]

The trussed-auth PR bumps to v0.3.0 so it needs to update all the apps that depend on it anyways. So that would still be 2 huge PRs.

[robin-nitrokey]

I wonder if we could also use the FirmwareVersion from the migrated crates to select the executed migrations. Something along the lines of:

enum FilesystemLayout {
    V0,
    V1,
}

impl FilesystemLayout {
    fn trussed_auth(&self) -> trussed_auth::FilesystemLayout {
        match self {
            Self::V0 => trussed_auth::FilesystemLayout::V0,
            Self::V1 => trussed_auth::FilesystemLayout::V1,
        }
    }
}

// ...
    Migrator {
        migrate: |ifs, _efs| {
            trussed_auth::migrate::migrate_remove_dat(
                ifs,
                &[
                    path!("opcard"),
                    path!("webcrypt"),
                    path!("secrets"),
                    path!("piv"),
                ],
            )
        },
        enable: |old: FilesystemLayout, new: FilesystemLayout| {
            trussed_auth::migrate::enable_migration(old.trussed_auth(), new.trussed_auth())
        },
    },
// ...

This would reduce the number of magic constants in the runner and would keep the details about the migration inside the migrated crates.

[robin-nitrokey]

I think it would be clearer to skip creating the relevant apps if migration failed instead of skipping them in apdu_dispatch and ctaphid_dispatch, similar to what we do with opcard on a config error.

[robin-nitrokey]

Do we really want to migrate on a config error? I would prefer to just trigger a migration error in that case and disable the affected apps.

[robin-nitrokey]

nit: can just be pub instead of pub(crate) since the module is private

[robin-nitrokey]

LGTM.