feat: base measureFunction API #271

mdjastrzebski · 2022-11-25T13:58:18Z

Summary

Resolves #224

async function measureFunction(
  fn: () => void,
  options?: { runs?: number, dropWorst?: number },
): Promise<MeasureResults>

API for measuring execution time of regular JS functions. It works similarly to render measures, but instead of using React.Profiler to measure renders stats of component, it just measures the execution duration of JS function.

It's intended to be used for any JS apps (not only React/RN) or libraries.

Scope:

Test plan

Added unit tests and perf tests using measureFunction

changeset-bot · 2022-11-25T13:58:21Z

⚠️ No Changeset found

Latest commit: 30c0767

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2022-11-25T14:08:20Z

Performance Comparison Report

Current: 1e4e6b5 - 2023-10-11 20:04:23Z
Baseline: main (8c69317) - 2023-10-11 20:03:35Z

Significant Changes To Duration

Name	Type	Duration	Count
Async Component	render	301.8 ms → 310.8 ms (+9.0 ms, +3.0%)	7 → 7

Show details

Name	Type	Duration	Count
Async Component	render	Baseline Mean: 301.8 ms Stdev: 11.0 ms (3.6%) Runs: 317 313 309 307 303 302 299 297 292 279 Current Mean: 310.8 ms Stdev: 15.9 ms (5.1%) Runs: 337 331 325 314 313 303 299 299 296 291	Baseline Mean: 7 Stdev: 0 (0.0%) Runs: 7 7 7 7 7 7 7 7 7 7 Current Mean: 7 Stdev: 0 (0.0%) Runs: 7 7 7 7 7 7 7 7 7 7

Meaningless Changes To Duration

Show entries

Name	Type	Duration	Count
Other Component 20	render	181.6 ms → 182.7 ms (+1.1 ms, +0.6%)	4 → 4
Other Component 10 legacy scenario	render	182.5 ms → 182.9 ms (+0.4 ms, ±0.0%)	4 → 4
Other Component 10	render	178.2 ms → 174.3 ms (-3.9 ms, -2.2%)	4 → 4

Show details

Name	Type	Duration	Count
Other Component 20	render	Baseline Mean: 181.6 ms Stdev: 9.7 ms (5.3%) Runs: 202 200 193 191 188 187 183 182 181 181 181 178 178 176 175 175 173 172 171 164 Current Mean: 182.7 ms Stdev: 8.4 ms (4.6%) Runs: 197 195 193 193 191 186 185 185 184 184 182 182 181 179 177 176 175 170 170 169	Baseline Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Current Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Other Component 10 legacy scenario	render	Baseline Mean: 182.5 ms Stdev: 7.2 ms (4.0%) Runs: 195 190 185 185 184 183 181 177 175 170 Current Mean: 182.9 ms Stdev: 12.2 ms (6.7%) Runs: 203 200 189 184 183 183 179 174 169 165	Baseline Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4 Current Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4
Other Component 10	render	Baseline Mean: 178.2 ms Stdev: 4.7 ms (2.6%) Runs: 185 184 183 179 179 177 176 175 173 171 Current Mean: 174.3 ms Stdev: 8.5 ms (4.9%) Runs: 190 182 181 177 175 172 171 167 166 162	Baseline Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4 Current Mean: 4 Stdev: 0 (0.0%) Runs: 4 4 4 4 4 4 4 4 4 4

Changes To Count

There are no entries

Added Scenarios

Name	Type	Duration	Count
fib 32	function	388.9 ms	1
fib 31	function	244.9 ms	1
fib 30	function	151.0 ms	1

Show details

Name	Type	Duration	Count
fib 32	function	Current Mean: 388.9 ms Stdev: 2.6 ms (0.7%) Runs: 391.8 391.5 390.5 390.4 390.2 388.8 388.5 387.0 386.3 383.4	Current Mean: 1 Stdev: 0 (0.0%) Runs: 1 1 1 1 1 1 1 1 1 1
fib 31	function	Current Mean: 244.9 ms Stdev: 0.6 ms (0.2%) Runs: 245.5 245.5 245.3 245.3 245.1 244.9 244.9 244.5 244.5 243.6	Current Mean: 1 Stdev: 0 (0.0%) Runs: 1 1 1 1 1 1 1 1 1 1
fib 30	function	Current Mean: 151.0 ms Stdev: 1.5 ms (1.0%) Runs: 153.7 152.6 152.5 151.3 151.2 151.1 150.0 149.9 149.2 149.1	Current Mean: 1 Stdev: 0 (0.0%) Runs: 1 1 1 1 1 1 1 1 1 1

Removed Scenarios

There are no entries

Generated by 🚫 dangerJS against 30c0767

Xiltyn

Where are we with this exactly? It's marked as WIP and I was just wandering if that's something you're still working on? @mdjastrzebski

packages/reassure-measure/src/measure-execution.tsx

mdjastrzebski · 2023-02-08T10:01:45Z

@Xiltyn This is an new feature/direction for Reassure, where it allow for testing performance of node apps and/or JS libraries (e.g. open source). This PR is about prototyping the API, and since I am limited on OSS time, it is progressing slowly. However, I think that this is very interesting niche to explore

packages/reassure-measure/src/measure-function.tsx

thymikee

Looks like a good direction. Thinking about whether we should incorporate some warmup code – but does it make sense in our case where we disable JIT and force GC?

mdjastrzebski · 2023-04-27T09:29:22Z

@thymikee what do you mean by warmup code? The worst (first) run of the function will be dropped, similar to how we process render measurements.

thymikee · 2023-04-27T11:14:45Z

@mdjastrzebski e.g. here: https://github.com/krausest/js-framework-benchmark#about-the-benchmarks. AFAIK warmup is helpful when preparing the JS engine to fire up JIT, but maybe there's more to it

packages/reassure-measure/src/measure-function.tsx

packages/reassure-measure/src/types.ts

fix: lint refactor: rename `measureExecution` to `UNSTABLE_measurePerformance` feat: measurePerformanceInternal executs code number of times feat: allow setting custom measure code refactor: re-structure measure functions refactor: cleanup tests refactor: rename measure results type

packages/reassure-measure/src/measure-function.tsx

adhorodyski · 2023-10-10T15:36:54Z

@mdjastrzebski now ready for review, with a fallback value of render as a migration path for the missing 'type' property in any existing tests :)

mdjastrzebski · 2023-10-11T20:13:29Z

Looks good! Let's merge it! 🦙🏄‍♂️

Xiltyn reviewed Feb 7, 2023

View reviewed changes

packages/reassure-measure/src/measure-execution.tsx Outdated Show resolved Hide resolved

packages/reassure-measure/src/measure-execution.tsx Outdated Show resolved Hide resolved

mdjastrzebski force-pushed the feature/measure-execution branch from f47dd59 to 7c9d232 Compare April 25, 2023 20:34

mdjastrzebski changed the title ~~feat: base measureExecution API (WIP)~~ feat: base measureFunction API Apr 25, 2023

mdjastrzebski requested review from Xiltyn and thymikee April 25, 2023 20:39

thymikee reviewed Apr 27, 2023

View reviewed changes

packages/reassure-measure/src/measure-function.tsx Outdated Show resolved Hide resolved

thymikee reviewed Apr 27, 2023

View reviewed changes

mdjastrzebski marked this pull request as ready for review April 27, 2023 15:22

mdjastrzebski force-pushed the feature/measure-execution branch from 14dd779 to e0766cf Compare May 9, 2023 14:44

mdjastrzebski commented Sep 29, 2023

View reviewed changes

packages/reassure-measure/src/measure-function.tsx Outdated Show resolved Hide resolved

mdjastrzebski commented Sep 29, 2023

View reviewed changes

packages/reassure-measure/src/types.ts Outdated Show resolved Hide resolved

adhorodyski force-pushed the feature/measure-execution branch from e0766cf to d0d175f Compare September 29, 2023 11:08

adhorodyski added 4 commits September 29, 2023 13:31

fix: build errors due to invalid imports

d19738b

chore: rename dropWorst to warmupRuns for consistency

9e19197

chore: adjust naming convention to drop render prefix where possible

c696349

refactor: collect current time using performance api

e5975f8

adhorodyski reviewed Sep 29, 2023

View reviewed changes

packages/reassure-measure/src/measure-function.tsx Show resolved Hide resolved

adhorodyski mentioned this pull request Oct 9, 2023

[HOLD for payment 2023-10-24] [Reassure] Migrate some of the GooglePlacesUtilsTest to reassure test framework Expensify/App#28585

Closed

adhorodyski added 6 commits October 10, 2023 09:41

feat: add type property to each measurement

5cad008

feat: render test type within the output

a5b94d5

docs: align methodology categories with the new output

c74fdcd

docs: add measureFunction api, fix formatting

994cb0d

fix: unit tests for compare module

0a60da0

fix: save compare test snapshot with a matching node version

e6335dd

fix: default to render when validating performance entries

f9009b4

mdjastrzebski added 12 commits October 11, 2023 11:56

docs: tweaks

9ed6790

docs: more tweaks

4644476

refactor: code review change

327d0e2

refactor: cleanup

5f4cd7e

refactor: code review changes

9c97a8a

refactor: code review changes

64b89aa

chore: add tests

9365419

docs: tweak

d4bb666

chore: fix build

8e08125

refactor: cleanup

ccca977

docs: tweaks

0febcd6

refactor: tweaks

30c0767

mdjastrzebski merged commit 933a5a2 into main Oct 11, 2023
2 checks passed

mdjastrzebski deleted the feature/measure-execution branch October 11, 2023 20:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: base measureFunction API #271

feat: base measureFunction API #271

mdjastrzebski commented Nov 25, 2022 •

edited by adhorodyski

Loading

changeset-bot bot commented Nov 25, 2022 •

edited

Loading

github-actions bot commented Nov 25, 2022 •

edited

Loading

Xiltyn left a comment

mdjastrzebski commented Feb 8, 2023

thymikee left a comment

mdjastrzebski commented Apr 27, 2023

thymikee commented Apr 27, 2023

adhorodyski commented Oct 10, 2023

mdjastrzebski commented Oct 11, 2023

feat: base measureFunction API #271

feat: base measureFunction API #271

Conversation

mdjastrzebski commented Nov 25, 2022 • edited by adhorodyski Loading

Summary

Test plan

changeset-bot bot commented Nov 25, 2022 • edited Loading

⚠️ No Changeset found

github-actions bot commented Nov 25, 2022 • edited Loading

Performance Comparison Report

Significant Changes To Duration

Meaningless Changes To Duration

Changes To Count

Added Scenarios

Removed Scenarios

Xiltyn left a comment

Choose a reason for hiding this comment

mdjastrzebski commented Feb 8, 2023

thymikee left a comment

Choose a reason for hiding this comment

mdjastrzebski commented Apr 27, 2023

thymikee commented Apr 27, 2023

adhorodyski commented Oct 10, 2023

mdjastrzebski commented Oct 11, 2023

mdjastrzebski commented Nov 25, 2022 •

edited by adhorodyski

Loading

changeset-bot bot commented Nov 25, 2022 •

edited

Loading

github-actions bot commented Nov 25, 2022 •

edited

Loading