tests(lantern): rebase #10129
tests(lantern): rebase #10129
Conversation
connorjclark
requested review from
brendankenny,
patrickhulce and
paulirish
as code owners
| @@ -29,7 +29,7 @@ rm -rf lantern-data/ | |||
| mkdir -p lantern-data/ && cd lantern-data | |||
| echo $VERSION > version | |||
|
|
|||
| curl -o lantern-traces.tar.gz -L https://storage.googleapis.com/lh-lantern-data/lantern-traces-$VERSION.tar.gz | |||
| curl -o golden-lantern-traces.zip -L https://storage.googleapis.com/lh-lantern-data/golden-lantern-traces-$VERSION.zip | |||
There was a problem hiding this comment.
renamed file b/c we are also keeping the big zip files in this bucket, and I wanted to reuse the naming that the collection scripts use.
|
wowzah am I reading this right that the median error for LCP is 108% (though it seems like the p50 and p90 are flipped and it should be 78%?, still...) |
There was a problem hiding this comment.
would be nice to get a before shot in the PR that has the full dataset for posterity instead of just a couple overlap URLs :)
just curious do these traces also have CLS events for us too? I think the plan is to not have a separate lantern impl for it but it would be nice to have a corpus of those in our golden set already
LGTM 💯
we're living in a new world of chrome changes I guess we have some work ahead of us to get back down! :D
good point. also updated the side-by-side ..
none of these traces have |
yeah its high. "should" be? It's backwards for sure, but I don't think the percentile maths is wrong or anything. |
"should be" meaning if it's just flipped then the p50 error is 78%, not 108%. |
What is looks like today:
This is with the new traces, comparing with the old data. I wouldn't give this too much concern. See the side-by-side instead.
After rebase:
Side by side (before - after):