Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core(manifest-parser): handle blob manifests #9088

Merged
merged 5 commits into from
Jun 1, 2019
Merged

Conversation

patrickhulce
Copy link
Collaborator

Summary
A manifest can be fetched over a blob: URL but all of its URLs will be invalid. Handle the case where we have invalid URLs and add a toplevel manifest warning when it's bloby

Related Issues/PRs
closes #9087

Copy link
Member

@brendankenny brendankenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice fix.

Also shows how the devtools manifest errors piped in by the parser could be a lot more helpful (humorously the application panel warns on the icons and shows them...it seems to be using the document URL as a fallback base URL)

@@ -151,7 +151,7 @@ function parseStartUrl(jsonInput, manifestUrl, documentUrl) {
return {
raw,
value: documentUrl,
warning: 'ERROR: invalid start_url relative to ${manifestUrl}',
warning: `ERROR: invalid start_url relative to ${manifestUrl}`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol, that's certainly one big problem with testing via assert.ok(parsedUrl.warning)...

lighthouse-core/lib/manifest-parser.js Outdated Show resolved Hide resolved
lighthouse-core/lib/manifest-parser.js Outdated Show resolved Hide resolved
manifestUrlWarning = `WARNING: manifest URL not available over a valid network protocol`;
}
} catch (_) {
manifestUrlWarning = `ERROR: invalid manifest URL ${manifestUrl}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the url was completely invalid, it seems like this should be throwing, a la the undefined check at the top?

(in theory it really should never happen since something was fetched in order to get here)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, yeah I agree in principle and it should never happen but seeing as it tanks the entire LH run, that seems like a bad outcome and it was reasonable to warn when the rest of the manifest can still be parsed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, yeah I agree in principle and it should never happen but seeing as it tanks the entire LH run, that seems like a bad outcome and it was reasonable to warn when the rest of the manifest can still be parsed?

arg, very good point. I forgot it's not in a gatherer anymore.

I can't find anywhere that we surface the warning if value is defined, but we can think about that with the other manifest error stuff.


it('should warn on valid but non-HTTPS manifest parser URL', function() {
const parsedManifest = manifestParser('{}', EXAMPLE_MANIFEST_BLOB_URL, EXAMPLE_DOC_URL);
assert.ok(parsedManifest.warning);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably learn our lesson and do some kind of real assertion :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heh, alright alright

const parsedManifest = manifestParser(manifestSrc, EXAMPLE_MANIFEST_BLOB_URL,
EXAMPLE_DOC_URL);
const icons = parsedManifest.value.icons;
assert.equal(icons.value.length, 0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check the warning?

Copy link
Member

@brendankenny brendankenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

thanks for the cleanup

lighthouse-core/lib/manifest-parser.js Outdated Show resolved Hide resolved
lighthouse-core/test/lib/manifest-parser-test.js Outdated Show resolved Hide resolved
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failed to Construct: Invalid URL
3 participants