Serde flatten

[glademiller]

🐛 Bug description

Using the flatten attribute from serde almost works but breaks in the case of non string values in flattened structs. In this case config always parses size as a string. However, if I put the size attribute directly in Config then everything works.

🤔 Expected Behavior

The usize value in the flattened struct should parse

👟 Steps to reproduce

#[derive(Deserialize)]
struct Config {
   #[serde(flatten)]
   pub subconfig: Subconfig
}

#[derive(Deserialize)]
struct Subconfig {
   pub size: usize
}

🌍 Your environment

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.33.0-nightly (a8a2a887d 2018-12-16)

envy version:
latest

[xoac]

I have also noticed this bug with bool variables.

[blechatellier]

@glademiller @xoac running into the same issue, did you find a workaround?

[neysofu]

Also looking for a solution to this.

[softprops]

I'm open to pull requests to add this feature

[neysofu]

I tried giving it a look but couldn't even locate the bug, any tips?

[jaboatman]

I spent some time trying to trace down where the issue is with this. Turns out #[serde(flatten)] deserializes the sub-structure as a map, which means it only works well for self-describing formats (e.g. JSON), since it will always defer to deserialize_any.

I haven't been able to come up with a good workaround yet.

[Pzixel]

Faced this isssue today, went with manual deserializing:

use serde::de;
use std::{fmt, fmt::Display, marker::PhantomData, str::FromStr};

pub fn deserialize_stringified_any<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
    D: de::Deserializer<'de>,
    T: FromStr,
    T::Err: Display,
{
    deserializer.deserialize_any(StringifiedAnyVisitor(PhantomData))
}

pub struct StringifiedAnyVisitor<T>(PhantomData<T>);

impl<'de, T> de::Visitor<'de> for StringifiedAnyVisitor<T>
where
    T: FromStr,
    T::Err: Display,
{
    type Value = T;

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        formatter.write_str("a string containing json data")
    }

    fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
    where
        E: de::Error,
    {
        Self::Value::from_str(v).map_err(E::custom)
    }
}

And then:

#[derive(Deserialize)]
struct Config {
   #[serde(flatten)]
   pub fluentd_config: FluentdConfig 
}

#[derive(Deserialize, Debug)]
pub struct FluentdConfig {
    pub fluentd_host: String,
    #[serde(deserialize_with = "deserialize_stringified_any")]
    pub fluentd_port: u16,
    pub fluentd_environment: String,
    pub fluentd_tag: String,
}

Not ideal but works to me

[nazar-pc]

I worked around by creating non-flattened struct and mapping that to desired after parsing environment variables, seemed easier than writing custom deserializers.

[SeedyROM]

Any new advice besides implementing deserialize_stringified_any and it's visitor? Thanks @Pzixel for the quick fix, did you just know that would work because you know serde well or is that some kind of escape hatch?

[SeedyROM]

I decided to go with not making my root configuration struct deserializable, and then using a prefix for each nested struct inside something like this:

#[derive(Deserialize, Serialize, Debug)]
pub struct DBConfig {
    host: String,
    port: u16,
    user: String,
    pass: String,
}

#[derive(Debug)]
pub struct Env {
    pub db: DBConfig,
}

impl Env {
    pub fn from_env() -> Result<Self, Report> {
        let db = envy::prefixed("PG").from_env::<DBConfig>()?;

        Ok(Self { db })
    }
}

(eg.) I pass in values as standard PostgreSQL envs and get back the correct values!: PGPASS=***

2022-03-19T09:56:16.719152Z  INFO example_envy: env=Env { db: DBConfig { host: "localhost", port: 5432, user: "psql", pass: "psql" } }

I think @nazar-pc might have been hinting at something similar?

[bhoudebert]

Any update on that one, it seems like if the nested type is always expected to be a string,

Tried to put an u64 but got the parsing error actual String not a u64.

[rollo-b2c2]

@Pzixel

Couldn't we just call that on every field? I get it'd be slow, but if your bottleneck is parsing envars you probably have another problem