Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The web vitals reported by k6 browser, seems to be very different than what is reported by google's pagespeed API #1396

Open
RP451 opened this issue Jul 3, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@RP451
Copy link

RP451 commented Jul 3, 2024

Brief summary

I am running the a browser script and the web vital numbers seem to vary significantly from Google’s pagespeed reports. I am not sure at this point if this has something to do with the k6 framework or the implementation in our script.

Example - For the URL - https://freshprints.com), both the LCP is significantly lower than what is reported by pagespeed insights. And on mobile the numbers deviate even more.

xk6-browser version

using the version included in k6/experimental/browser branch

OS

windows 10

Chrome version

using the version included in k6/experimental/browser branch

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Even a basic script like this is throwing wildly different numbers for all core metrics

import { browser } from 'k6/experimental/browser';

export const options = {
scenarios: {
browser: {
executor: 'constant-vus',
exec: 'browserTest',
vus: 5,
duration: '60s',
options: {
browser: {
type: 'chromium',
}
}
}
},
};

export async function browserTest() {
const page = browser.newPage();

try {
await page.goto('https://freshprints.com/home');
page.waitForLoadState(networkidle);
}
finally {
page.close();
}
}

Expected behaviour

The numbers reported by k6 browser should match that of Google's pagespeed report or atleast should be within tolerable values

image

Actual behaviour

All the core metrics vary significantly compared to the numbers reported by pagespeed's API.

image

@RP451 RP451 added the bug Something isn't working label Jul 3, 2024
@RP451
Copy link
Author

RP451 commented Jul 3, 2024

This problem seems to be worse for mobile devices, the difference there is even higher. Not sure if its just because of the throttling on google's report, but adding that to my script doesn't help much. The numbers on mobile from K6's script seem to be almost the same as on desktop

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant