Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make desktop perf metric score curves identical everywhere #9436

Closed
RustedBones opened this issue Jul 23, 2019 · 4 comments · Fixed by #9911
Closed

Make desktop perf metric score curves identical everywhere #9436

RustedBones opened this issue Jul 23, 2019 · 4 comments · Fixed by #9911

Comments

@RustedBones
Copy link

When running tests on Lighthouse and PageSpeed Insight, we spotted some inconsistencies in the speed index classification of what is a 'fast', i.e. green/orange/red:

  • In Lighthouse, a speed index of 3.2s is green

lighthouse

  • In Page Speed Insights, a speed index of 1.6s is orange

pagespeed_insights

Both results are observed in Desktop mode.

Since speed index is a Lab Data field, I expected the 2 tools to give the same classification.

@patrickhulce
Copy link
Collaborator

Thanks for filing @RustedBones! The score scale for desktop in Page Speed Insights is adjusted for the faster connection type, but the score scale is not automatically adjusted in other Lighthouse environments.

We would like to fix this discrepancy in other environments but doing so while supporting all the other available flags in an intuitive way is challenging.

It looks like you were running in DevTools though which should be the most straightforward. @connorjclark do you think this is something we could tackle with an upcoming roll? (Using the desktop config when the "Desktop" radio is selected instead of just setting emulatedFormFactor?)

@RustedBones
Copy link
Author

RustedBones commented Jul 23, 2019

If my understanding is right, you are meaning that PageSpeed Insights is using the lighthouse-core/config/lr-desktop-config.js with

    // 75th and 95th percentiles -> median and PODR
    // SELECT QUANTILES(SpeedIndex, 21) FROM [httparchive:summary_pages.2018_12_15_desktop] LIMIT 1000
    {path: 'metrics/speed-index', options: {scorePODR: 1100, scoreMedian: 2300}},

that changes the underlying score from page speed hence its color, but Lighthouse doesn't.
Lighthouse then scores the metrics the same way for desktop and mobile with the default values.

Is this correct ?

@patrickhulce
Copy link
Collaborator

Yes exactly correct @RustedBones !

@paulirish paulirish changed the title Lighthouse and PageSpeed Insight have inconsistent classification Make desktop perf metric score curves identical everywhere Jul 23, 2019
@paulirish
Copy link
Member

Updated title. Our planned action is to use the lr-desktop-config.js score configuration everywhere (dt, cli, etc) when desktop mode is set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants