Automated perf tests

[ayumi]

#10115

Submitter Checklist:

• Submitted a ticket for my issue if one did not already exist.
• Used Github auto-closing keywords in the commit message.
• Added/updated tests for this change (for new code or code which already has tests).
• Ran git rebase -i to squash commits (if needed).
• Tagged reviewers and labelled the pull request as needed.

Test Plan:

Reviewer Checklist:

Tests

• Adequate test coverage exists to prevent regressions
• Tests should be independent and work correctly when run individually or as a suite ref
• New files have MPL2 license header

[NejcZdovc]

We just remove this dependency in #10514. Do we need it back?

[ayumi]

oops, was a mis-rebase. good catch!

[NejcZdovc]

👍

[luixxiul]

Should this be updated as well?

browser-laptop/package.json

Lines 230 to 232 in 836d391

	"engines": {
	"node": ">= 7.9.0"
	}

[NejcZdovc]

@ayumi are you working on this one right now? 😃 @bsclifton said that you have some problems with travis, so I wanted to check it out and help out

[ayumi]

Currently blocked on fixing --debug on Linux:
brave/muon#292

[luixxiul]

Is it not worth adding a large amount of cache to see how much it affects the performance?

[ayumi]

@luixxiul what do you mean?

this is no longer blocked on linux muon; i'm working around the --debug linux crash by using muon debug builds rather than using the prod builds.

[luixxiul]

I meant loading a cache file to calculate how much it can slow down the browser.

[cezaraugusto]

@ayumi is this ready for review? please put the work-in-progress label otherwise, thanks

[ayumi]

@cezaraugusto yes, this is ready for review

the tests have been reasonably deterministic.

[bsclifton]

@ayumi I'm trying to test this out and can use some help 😄

1. Would you be able to create some docs to accompany this? Maybe a new wiki entry:
   https://github.com/brave/browser-laptop/wiki

2. I checked out this branch on an Ubuntu 14.04 VM, removed node_modules, and re-did the npm install. After that completed, I ran the following:

curl -sL https://raw.githubusercontent.com/travis-ci/artifacts/master/install | bash
CXX=g++-4.8 NODE_ENV=test TEST_DIR=performance ARTIFACTS_REGION=us-east-1 npm run testsuite

When running this, I'm getting a few errors (some tests are timing out). However, it all seems great on the Travis side (which is more important 😄 ):
https://travis-ci.org/brave/browser-laptop/jobs/286826649

Any advice on what to try the setup and test for?

3. You had shown me the Grafana instance (would be good to document some details about that). This is uploading the results of a run to S3. Once merged, this should be collecting all of the perf data about each PR, correct?

[bsclifton]

Comments left 😄 I think most (if not all) of the concerns I had can be solved with documentation. The actual changes look great! It's really exciting seeing the progress you made. Great job! 😄 👍

[bsclifton]

BTW- per discussion on Slack, the --debug option should work on Linux builds for the latest Muon. If you get a chance, please retry that and let us know how it works (if resolved, we can close brave/muon#292). Until we get a chance to test that, we can still keep the separate debug version of Muon 😄

[ayumi]

1. 👍

2. travis artifacts uploader isn't needed outside of travis

3. Yes, after this is merged we will start collecting data on every Travis run for browser-laptop; and it will automatically go to the https://github.com/brave/perfaderp web UI.

4. --debug still doesn't work for me on Linux. browser-laptop master 3ee0c9543f582794117b3ce98832da339285429d

[bsclifton]

@ayumi the last commit has some failing tests. I restarted the travis job- if that passes, this looks great; will merge 😄

[ayumi]

@bsclifton I might need to create a new muon debug build. however, I can update it without code changes (just by uploading the builds to S3)

[bsclifton]

@ayumi OK great, sounds good 😄 Let's merge this for now and you can investigate that 😄

[ayumi]

tests working again with a new muon debug build