Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit array serialization in serde #52

Merged
merged 1 commit into from
Jun 17, 2020

Conversation

Schuwi
Copy link
Contributor

@Schuwi Schuwi commented Jun 17, 2020

This PR aims to implement Byte/Int/Long array serialization support for serde.
Right now every Vec<i8> or similar sequence types get serialized to a List of Bytes instead of the ByteArray type. This PR implements a serde serialize_with function to tag fields of collection types to serialize into ByteArray/IntArray/LongArray (called i8_array/i32_array/i64_array respectively).

For example:

#[derive(Serialize)]
struct MyStruct {
    #[serde(serialize_with="hematite_nbt::i8_array")]
    byte_array: Vec<i8>,
}

This will serialize to a compound containing a ByteArray instead of a List of Bytes.

Current limitations

  • As this approach uses a bit of a hack internally because of serde limitations, fields tagged to use the array serializer can only be serialized with hematite-nbt. When serializing with a different serializer this will try to serialize as a serde tuple_struct with the name __hematite_nbt_i8_array__ (or i32/i64 respectively) and the array elements as fields of that tuple.
  • Serde currently doesn't support using serialize_with on inner elements (of Option, Vec, ...) and thus nested arrays are not trivially supported (see Using de/serialize_with inside of an Option, Map, Vec serde-rs/serde#723). A workaround for this as also currently used in hematite_nbt tests is not that clean, but it can be made to work:
    #[derive(Debug, PartialEq, Serialize, Deserialize)]
    struct NestedArrayNbt {
        #[serde(serialize_with = "nested_i32_array")]
        data: Vec<Vec<i32>>,
    }
    
    fn nested_i32_array<S>(outer_arr: &Vec<Vec<i32>>, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        #[derive(Debug, Serialize)]
        struct Wrapper(#[serde(serialize_with = "nbt::i32_array")] Vec<i32>);
    
        // clone should be optimized to a no-op
        serializer.collect_seq(outer_arr.iter().map(|vec| Wrapper(vec.clone())))
    }
    

More details and discussion can also be found in the PR of a previous, different approach on implicitely converting all NBT lists that can be represented as arrays: #51

Resolves #27
Resolves #32

/// to_writer(
/// &mut serialized,
/// &Sheep {
/// byte_data: vec![0x62, 0x69, 0x6E, 0x61, 0x72, 0x79, 0x20, 0x73, 0x68, 0x65, 0x65, 0x70],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a "real" example? If so, it'd be neat to mention where it comes from.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nah, just had a bit of fun with those examples. it's "binary sheep" in ASCII ;)

"__hematite_nbt_i8_array__"
| "__hematite_nbt_i32_array__"
| "__hematite_nbt_i64_array__" => Compound::for_seq(self.outer, len as i32, true),
_ => Err(Error::UnrepresentableType(stringify!(tuple_struct))),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this just be UnrepresentableType("tuple struct")?

Copy link
Contributor Author

@Schuwi Schuwi Jun 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I basically just 1:1 copied what the macro did before, but I can change that if that's more readable
Edit: so stringify! does resolve to either "tuple_struct" or "tuple struct" dunno whether it removes the underscore

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay will do

@atheriel
Copy link
Collaborator

Can you squash this and rebase against master?

@Schuwi
Copy link
Contributor Author

Schuwi commented Jun 17, 2020

Never done either of them. Squashing is putting multiple commits into one and rebase is that it looks like it branches off the current master HEAD?
I'll find out how to do it, but I'll have to do it later this evening.

@Schuwi Schuwi force-pushed the explicit-array-serializing branch from f157196 to f820d92 Compare June 17, 2020 18:10
@Schuwi
Copy link
Contributor Author

Schuwi commented Jun 17, 2020

Alright, it's done

@atheriel atheriel merged commit aac96f9 into PistonDevelopers:master Jun 17, 2020
@atheriel
Copy link
Collaborator

Thanks for all your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants