Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: pattern based compression #6371

Closed
wants to merge 1 commit into from
Closed

Conversation

erights
Copy link
Member

@erights erights commented Oct 1, 2022

When the only specimens you store are those that match a pattern, store only the information which is not literally the same as the pattern.

Added to the store module

  • compress(specimen, pattern) - returns bindings array, or undefined to indicate match failure
  • mustCompress(specimen, pattern, label) - indicates failure with diagnostic including label
  • decompress(bindings, pattern) - undoes the others.

Modified SwingSet's virtual mapStores so that if they have a valueShape option, they also use it for compression.

Modified ERTP's paymentLedger to provide the amountShape as the valueShape option for the durable ledger mapStore.

Verified in the debugger that we now only store a singleton array around the value itself, dropping the enclosing amount structure and the brand which is always the same for a given ledger mapStore. IOW, instead of storing the serialization of

{
   brand: moolaBrand,
   value: 2n,
}

for every live payment, with this PR we store the serialization of

[2n]

for every live payment. This is not just a savings because it is smaller. We also avoid needing to adjust the refcount of their common moolaBrand.

Once we've switched to smallcaps, the serialization of [2n] is tiny.

@erights erights requested a review from FUDCo October 1, 2022 06:13
@erights erights self-assigned this Oct 1, 2022
@erights erights force-pushed the markm-pattern-compress branch 4 times, most recently from c7e0dd3 to 4192893 Compare October 8, 2022 02:56
@erights erights changed the title WIP: pattern based compression fix: pattern based compression Oct 8, 2022
@erights erights requested a review from gibson042 October 8, 2022 03:09
@erights erights marked this pull request as ready for review October 8, 2022 03:10
@erights erights requested a review from dtribble October 8, 2022 03:12
@erights
Copy link
Member Author

erights commented Oct 8, 2022

Ready for review!

@erights erights force-pushed the markm-pattern-compress branch 2 times, most recently from 1d142b9 to 2fb1fb1 Compare October 8, 2022 03:28
Copy link
Member Author

@erights erights left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for @FUDCo (Nevermind. Resolved)

packages/SwingSet/src/liveslots/collectionManager.js Outdated Show resolved Hide resolved
@erights erights force-pushed the markm-pattern-compress branch 3 times, most recently from 259ec5e to 44be107 Compare October 9, 2022 04:44
@erights
Copy link
Member Author

erights commented Oct 10, 2022

Closing in favor of #6432

@erights erights closed this Oct 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant