Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scalars: set data download filenames on backend, not a tag #3782

Closed
wants to merge 8 commits into from

Conversation

nfelt
Copy link
Contributor

@nfelt nfelt commented Jul 2, 2020

Scalar chart data download URLs use custom filenames that include the user's run and tag names, for ease of identification when downloading from multiple charts. This PR moves that custom filename support from the frontend, where it used the download attribute on <a> tags, to the backend, where we use the Content-Disposition header.

The intent is to sidestep issues with trying to sanitize the strings going into the download attribute so that they aren't intercepted by Polymer Resin; the easiest way I determined to do that was to stop binding to the attribute entirely. It's a little less elegant in that if we had multiple download widgets for various charts, now each backend location must set the filename rather than having it centralized in the UI component, but right now it's only used in one place anyway and it doesn't seem too onerous even if we had multiple places.

In addition, I actually still implemented some sanitization of the attribute, to eliminate or replace with _ all characters other than [-_=A-Za-z0-9]. Chrome's auto-downloading behavior already seems to do something fairly similar (using _ to replace /, for instance) but it seemed safest to ensure we're generating innocuous filenames in the first place.

Lastly, the scalar plugin tests for this were a mess, so I unrolled and flattened them a fair bit and added two new tests that explicitly test the format= URL behavior end-to-end.

(The first commit on this PR is split out as #3781 since it includes an unrelated behavior change.)

Copy link
Contributor

@wchargin wchargin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the scalar plugin tests for this were a mess, so I unrolled and
flattened them a fair bit

Thank you!

@@ -184,6 +189,8 @@ def Respond(
headers.append(("X-Content-Type-Options", "nosniff"))
if content_encoding:
headers.append(("Content-Encoding", content_encoding))
if filename:
headers.append(("Content-Disposition", 'filename="%s"' % filename))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want attachment; filename=…?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I specifically left out attachment since with the current behavior it's possible in Chrome to open the .json file in the browser in a new tab if you want (the .csv is always forcibly downloaded; I think this is content-type specific browser behavior). As a user I like it when it's possible to do this rather than requiring that I always download the file, so that's why I did it this way.

Setting the download attribute in the HTML tag but omitting attachment means that the behavior depends on whether you just click the link (download) or click to open in a new tab (browser dependent, but may stay in the browser), so that seemed like a nice option to preserve.

That said I agree it's overall a bit less consistent to reason about, so if you prefer I'm happy to add attachment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s fine with me (I also like that behavior). In that case, let’s fix
the newly added docs to not mention Content-Disposition: attachment?

@@ -184,6 +189,8 @@ def Respond(
headers.append(("X-Content-Type-Options", "nosniff"))
if content_encoding:
headers.append(("Content-Encoding", content_encoding))
if filename:
headers.append(("Content-Disposition", 'filename="%s"' % filename))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use werkzeug.http.quote_header_value(filename) here?
Callers need to sanitize the logical filename, but shouldn’t need to
know about the RFC 2616 quoted-string grammar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooh yes, thanks for pointing that out. Done and test added.

@@ -127,7 +121,7 @@ def index_impl(self, experiment=None):

return result

def scalars_impl(self, tag, run, experiment, output_format):
def scalars_impl(self, tag, run, experiment):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like legit test failure in the custom scalars plugin due to the
removal of this parameter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will fix shortly.

mapping[ord(char)] = "_"
for char in allowed_chars:
mapping[ord(char)] = char
print(mapping[ord(" ")])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, thanks


This discards all characters that are not ASCII letters, digits, or
punctuation, and then converts any remaining characters that are not in
[-_=,A-Za-z0-9] to "_".
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc comment includes comma as allowed character, but code doesn’t?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, thanks. Updated the code to match docs.

"punct___________-_____=______________uation",
)
check("w h\ti\nt\re\x0bs\x0cpace", "whitespace")
check("emo\U0001F95Dji", "emoji")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i’m honored

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sadly github probably doesn't allow sending arbitrary emoji to their comment react API

@nfelt
Copy link
Contributor Author

nfelt commented Jul 10, 2020

Update: I'm withdrawing this PR because after some discussion we decided that this isn't a great approach, since in the long term we want the flexibility to be able to generate the data to download wholly on the frontend (e.g. so that it can more easily reflect client-side filtering done by the user, or see also the discussion in #1845) and because it also didn't fix other cases of this problem - e.g. in the custom scalars download URLs (which we'd also have had to change) or in the scalars dashboard "Download chart as SVG" option which also uses a dynamic filename (the tag name) and is generated client-side, making it ineligible for this approach to sidestepping the problem.

Instead, after some discussion with the Polymer folks, it turns out they were fine changing Resin's policy for the download attribute to skip sanitization since the attribute value is only a hint to the browser anyway (which can more appropriately sanitize or reject the proposed filename in the context of actually writing it to disk, if it so chooses).

Note I already broke the still-useful test cleanup from this PR out into #3793.

@nfelt nfelt closed this Jul 10, 2020
@nfelt nfelt deleted the fix-download-links branch July 10, 2020 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants