revisit parsing of configuration

[trinity-1686a]

Description

fix #3472
Change how configuration is parsed so validation and replacement of default values happens during deserialization.

Sadly Serde doesn't provide anything for manipulating multiple fields (or a whole struct) at once, and I couldn't find a crate to do that either, so I wrote a procmacro to generate a mirror of our strucs for parsing, and a TryFrom/Into impl to use that for serializing/deserializing our config structs, which are now entirely validated.

I've done some special handling of utoipa, but I don't really know how to verify

[fulmicoton]

@trinity-1686a The description does not explain what this is improving, whether this is no-op or not etc. Can you explain?

[trinity-1686a]

it simplifies some handling of invalid configuration. Getting a configuration struct means it's already validated, and default values are already put there in case they need to be.

This means serializing (and storing in the metastore) is guaranteed to keep the same value over time, even if the default change from a version to the next.

It also means tests which don't set some values will now fail if the default for this values changes (this is what changes in *.expected.json is about).

From a black box point of view, it works like if deserialization was unchanged (everything that was okay before is okay after), but serialization fills empty fields with their default value

[fulmicoton]

@trinity-1686a That's indeed the behavior that we want for most of the values.

[fulmicoton]

#[serde(flatten)] does what you described. However it does not work in combination with deny_unknown_fields (and probably ruins the error messages, although I am not sure)

[fulmicoton]

LGTM... Thanks for dealing with the tricky ticket.

[trinity-1686a]

#[serde(flatten)] does what you described. However it does not work in combination with deny_unknown_fields

yes, and that's exactly why I decided not to use it. Not rejecting unknown fields in config is imo not an option (it can catch important typos), so #[serde(flatten)] wasn't either. While the way it works is different, the result is overall similar