Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix serializer and deserializer not symmetric #98

Closed
wants to merge 2 commits into from

Conversation

yihuang
Copy link
Contributor

@yihuang yihuang commented Dec 17, 2019

These deserializers intents to parse some empty data as None, because golang version of tendermint would encode these empty values as some default values.
But since the serializers would encode None as null, which makes them not symmetric.
So the solution is make these deserializers also support decoding null as None.

let commit = TmpCommit::deserialize(deserializer)?;
if commit.block_id.is_none() || commit.precommits.is_none() {
Ok(None)
if let Some(commit) = <Option<TmpCommit>>::deserialize(deserializer)? {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks! I want to double check if the tendermint rpc will return an empty commit (likely), or, if this can be treated as invalid input instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An empty commit happens in the last_commit field in the first block.

@ebuchman
Copy link
Member

Thanks for more fixes. Do you think you could add some tests to help illustrate the issue better? Would probably be good to get a bunch of json data for blocks so we can tests all these encoding issues for first block, blocks with/without other hashes. Maybe in tendermint/tests/support/rpc/block.json ?

@yihuang
Copy link
Contributor Author

yihuang commented May 1, 2020

Thanks for more fixes. Do you think you could add some tests to help illustrate the issue better? Would probably be good to get a bunch of json data for blocks so we can tests all these encoding issues for first block, blocks with/without other hashes. Maybe in tendermint/tests/support/rpc/block.json ?

I've added the unit tests which reveal the issue, and rebased the PR against current master.
Currently, I only concern with Block and TrustedState, but I believe there are other several cases of a similar issue.
This issue happens when you want to serialization some data types into storage and load/deserialize them later.
I think we really need something to make our encoding more robust, almost all of these issues are related to the serialization of empty values.

Some optional types encode to null but don't parse null.
- Block.last_commit
- CommitSig.validator_address
bors bot added a commit to crypto-com/thaler that referenced this pull request May 2, 2020
1524: Problem (Fix #1523): sync state deserialization fails when nil vote happens r=tomtau a=yihuang

Solution:
- Make parse_non_empty_id parse null in tendermint-rs

Related upstream PR: informalsystems/tendermint-rs#98

Co-authored-by: yihuang <[email protected]>
liamsi pushed a commit that referenced this pull request May 27, 2020
* Symmetric serialization tests

* CHANGES.md update

* Fix serializer and deserializer not symmetric - reimpl from yihuang PR #98
@greg-szabo
Copy link
Member

This is implemented with #283 . Thanks @yihuang for your patience with this. Closing.

@greg-szabo greg-szabo closed this May 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants