HTTP HEAD and GET requests return different headers for media/r0/download endpoints

[julianfoad]

Description

Synapse returns different headers for HTTP HEAD and GET requests, for media/r0/download endpoints.

According to HTTP/1.1 spec https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html section 9.4 HEAD:

"The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request."

Steps to reproduce

• make a plain text file, e.g. "echo hello > hello.txt".
• using Riot-web 1.3.3, in room #test:matrix.org, send that file using the "Upload file" paperclip button.
• copy the URL from the displayed "Download hello.txt" link.
• compare the results of "curl --head " and "curl -i "

$ curl --head https://matrix.org/_matrix/media/r0/download/matrix.org/WozzzqyaQXYsxVYlRePaGtIZ
HTTP/2 500 
date: Tue, 10 Sep 2019 12:17:55 GMT
content-type: text/html
set-cookie: __cfduid=d8bae399428628ffcc58058da4277761f1568117875; expires=Wed, 09-Sep-20 12:17:55 GMT; path=/; domain=.matrix.org; HttpOnly
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 5141576ec83ece17-LHR

versus

$ curl -i https://matrix.org/_matrix/media/r0/download/matrix.org/WozzzqyaQXYsxVYlRePaGtIZ
HTTP/2 200 
date: Tue, 10 Sep 2019 12:18:00 GMT
content-type: text/plain
set-cookie: __cfduid=d2ea5ebdf19d7719715ee5c80be8dcb341568117880; expires=Wed, 09-Sep-20 12:18:00 GMT; path=/; domain=.matrix.org; HttpOnly
access-control-allow-origin: *
access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
content-security-policy: sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; media-src 'self'; object-src 'self';
content-disposition: inline; filename=hello.txt
cache-control: public,max-age=86400,s-maxage=86400
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 5141578e3aadbbca-LHR

hello

Version information

• Homeserver: matrix.org
• Version: Synapse "1.3.1 (b=matrix-org-hotfixes,9240622c1,dirty)"