Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataclasses #3969

Merged
merged 71 commits into from
May 17, 2019
Merged

Dataclasses #3969

merged 71 commits into from
May 17, 2019

Conversation

rakanalh
Copy link
Contributor

@rakanalh rakanalh commented Apr 30, 2019

Resolves #3869, #3253

Follow-up PRs should:

  1. Use frozen=True on dataclasses
  2. Use the pyrsistent library for nested lists / dicts
  3. Fix repr for dataclasses

TODO:

  • Fix tests
  • Cleanup utils/serialization
  • Cleanup storage/serialize

Notes to the reviewer(s)

  • Some changes included adding keyword arguments to function calls, because the order of which dataclasses arguments in the constructor is very specific, which can be overcome with passing kwargs.

  • All fields in any dataclass should NOT be field(init=False) as this would prevent them from being serialized and included with JSON. So all fields should be included in the constructor with a default instead.

  • Dataclasses should NOT use "quoted" types. Example:

secrethashes_to_task: Dict[SecretHash, "TransferTask"]

Because marshmallow will fail to look up that string from its registry in case the class hasn't been imported yet. So we should strive for a proper code organization that doesn't cause cyclic imports and therefore, not have to use quoted types.

  • Constructor calls with positional arguments should have the arguments ordered by starting with parent arguments then the current class arguments. Preferrably, always use keyword args

@codecov
Copy link

codecov bot commented May 16, 2019

Codecov Report

Merging #3969 into develop will increase coverage by 0.18%.
The diff coverage is 84.2%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3969      +/-   ##
===========================================
+ Coverage    78.13%   78.32%   +0.18%     
===========================================
  Files          113      117       +4     
  Lines        15625    14434    -1191     
  Branches      2182     2290     +108     
===========================================
- Hits         12209    11305     -904     
+ Misses        2661     2385     -276     
+ Partials       755      744      -11
Impacted Files Coverage Δ
raiden/transfer/channel.py 90.65% <ø> (ø) ⬆️
raiden/utils/cli.py 49.61% <ø> (ø) ⬆️
raiden/raiden_service.py 86.95% <100%> (+0.12%) ⬆️
raiden/api/rest.py 74.82% <100%> (ø) ⬆️
raiden/storage/migrations/v19_to_v20.py 92.04% <100%> (-0.09%) ⬇️
raiden/api/v1/encoding.py 91.28% <100%> (ø) ⬆️
raiden/constants.py 100% <100%> (ø) ⬆️
raiden/transfer/mediated_transfer/mediator.py 90.96% <100%> (ø) ⬆️
raiden/api/python.py 68.23% <100%> (+0.2%) ⬆️
raiden/utils/typing.py 94.39% <100%> (ø) ⬆️
... and 47 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f651457...7b339b6. Read the comment docs.

Copy link
Contributor

@hackaugusto hackaugusto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first round

raiden/messages.py Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
raiden/messages.py Outdated Show resolved Hide resolved
Copy link
Contributor

@hackaugusto hackaugusto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

round2

raiden/api/rest.py Show resolved Hide resolved
@@ -111,6 +120,7 @@ def handle_channel_new(raiden: "RaidenService", event: Event):
)
raiden.handle_and_track_state_change(new_channel)

# pylint: disable=E1101
partner_address = channel_state.partner_state.address
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, in our case this happens when field(repr=False) is used and the field that is another dataclass. Shouldn't we just remove the field(repr=False) and have the lint working?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing field(repr=False) in the case where we only disable repr for these fields is possible. However, there will be many different places where we need to override the field with something else.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but this would fix roughly 32 instances that are just repr=True. Counted with:

ag --python 'field(repr=False)' --nobreak --nofilename | wc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this point out. Most of the disable=E1101 were added because partner_state and our_state are defined as field(repr=False). However, i would argue that setting them to be included in repr would be a bit too much because of the amount of data that will end up being logged.
Do you think we should accept this and remove field assignment to those attributes?

raiden/messages.py Show resolved Hide resolved
raiden/messages.py Show resolved Hide resolved
raiden/messages.py Show resolved Hide resolved
@@ -605,4 +605,5 @@ def run_test_mediated_transfer_with_node_consuming_more_than_allocated_fee(
initiator_task = app0_chain_state.payment_mapping.secrethashes_to_task[secrethash]

msg = "App0 should have never revealed the secret"
assert initiator_task.manager_state.initiator_transfers[secrethash].revealsecret is None
transfer_state = initiator_task.manager_state.initiator_transfers[secrethash].transfer_state
assert transfer_state != "transfer_secret_revealed"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not strictally equivalent to the previous test, although I don't think it is a problem, why not just use the flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We both agreed that instead of using nesting the SendSecretReveal event into this object as revealsecret, it's better to either use a flag, or set the status of the transfer similar to one of your previous PRs. So i opted to go for introducing the state

@@ -44,7 +44,7 @@ def test_v1_event_payment_sent_failed_schema():

expected = {"event": "EventPaymentSentFailed", "log_time": log_time, "reason": "whatever"}

assert all(dumped.data.get(key) == value for key, value in expected.items())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so it does look like marshmallow interface changed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, answered previously.

@@ -356,7 +358,9 @@ def test_deposit_must_wait_for_confirmation():
)
partner_model2 = partner_model1

# pylint: disable=E1101
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not what you want, it will disable that given warning all the way to the end of the block:

https://pylint.readthedocs.io/en/latest/user_guide/message-control.html#block-disables

That is not good. Example:

https://gist.github.com/hackaugusto/2266871f69dac442e0bd210365c38d7a

A2(A1(1)).a.a is a false positive that has to be silenced, B(1).a is a bug that should not. However because of a block annotation like this the warning about the bug is silenced.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, there are other instances of the same pattern that need to be fixed in this PR

raiden/transfer/architecture.py Show resolved Hide resolved
Copy link
Contributor

@hackaugusto hackaugusto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a third round on this PR. I did not find any obvious errors/bugs, and since this is a huge PR that takes a lot of time to rebase, I'm approving it so that we can merge. Improvements can be done on follow ups.

@rakanalh rakanalh marked this pull request as ready for review May 17, 2019 13:08
@rakanalh rakanalh changed the base branch from master to develop May 17, 2019 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add dataclasses (related to "Spring Cleaning")
4 participants