Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine performance impact of enforcing Method Schemas #6098

Closed
Tartuffo opened this issue Aug 31, 2022 · 4 comments
Closed

Determine performance impact of enforcing Method Schemas #6098

Tartuffo opened this issue Aug 31, 2022 · 4 comments
Assignees
Labels
performance Performance related issues security SwingSet package: SwingSet
Milestone

Comments

@Tartuffo
Copy link
Contributor

We need a perf measure comparing calling methods with and without input validation, to demonstrate that using the method schemas will not cause significant slowdown.

Ideally we'd do this w/o relying on end-to-end testing, because there is so much machinery involved it can be difficult to tell what the impact is.

@Tartuffo Tartuffo added performance Performance related issues security Zoe package: Zoe labels Aug 31, 2022
@Tartuffo Tartuffo changed the title Determine performance impact of Method Schemas implementation Determine performance impact of enforcing Method Schemas Aug 31, 2022
@Tartuffo Tartuffo added SwingSet package: SwingSet and removed Zoe package: Zoe labels Aug 31, 2022
@Tartuffo Tartuffo added this to the Mainnet 1 RC0 milestone Aug 31, 2022
@dtribble
Copy link
Member

dtribble commented Sep 6, 2022

Initial plan:

  • build a test to run a set of interface guard checks (iteratively)
  • time it in nodejs
  • for best fidelity with XS, use NODE_OPTIONS=--jitless
  • if optimization seems necessary, analyze with node profiling tools

@erights
Copy link
Member

erights commented Sep 6, 2022

We need a perf measure comparing calling methods with and without input validation, to demonstrate that using the method schemas will not cause significant slowdown.

This is not a sensible comparison. We should compare methods using interface guards for input validation vs methods that do input validation themselves with manually written code. No matter what the perf says, we do not have the option of exposing methods that are not doing input validation somehow.

@warner
Copy link
Member

warner commented Sep 6, 2022

If you can make the code-under-test self-contained enough to survive bundleSource, then we can build a harness to run it under XS (but outside of a vat: no SwingSet). I think that will give us the most meaningful results.

@erights
Copy link
Member

erights commented Sep 6, 2022

Regarding our desire to benchmark manual input validation vs pattern-based input validation, #6114 (comment) is an extreme but realistic example. All the input validation it did was mandatory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance related issues security SwingSet package: SwingSet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants