Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc.rust-lang.org: add caching headers #62

Open
jsha opened this issue Feb 20, 2021 · 4 comments
Open

doc.rust-lang.org: add caching headers #62

jsha opened this issue Feb 20, 2021 · 4 comments

Comments

@jsha
Copy link
Contributor

jsha commented Feb 20, 2021

(moving from rust-lang/rust#82286)

Steps to reproduce:

  1. Load any doc.rust-lang.org page, e.g. https://doc.rust-lang.org/std/str/index.html.
  2. Open Developer Console.
  3. Click the URL bar and hit enter to load the page again.

Expected result:

Most resources are cached and do not hit the network.

Actual result:

Many objects require a network fetch, but get a 304 Not Modified. These should be served with long caching headers.

image

Current headers:

$ curl -I https://doc.rust-lang.org/SourceCodePro-Regular.woff
HTTP/2 200 
content-type: font/woff
content-length: 55472
last-modified: Thu, 11 Feb 2021 14:27:46 GMT
x-amz-version-id: R.xOjlR7eKEsQwzm9EIrHs9eeCIDeXYq
server: AmazonS3
date: Thu, 18 Feb 2021 13:27:31 GMT
etag: "957fa8c030f8116bea59c13df470e4e8"
x-cache: Hit from cloudfront
via: 1.1 4b84530d7a095b58fb7a1d20b7f0cbe0.cloudfront.net (CloudFront)
x-amz-cf-pop: HIO50-C2
x-amz-cf-id: QpMYsW6BVcTPpDmi8Pg9HiMz_1iSOIpdYv60MMzNqVoV707yqsv_Dw==
age: 63023
@Nemo157
Copy link
Member

Nemo157 commented Feb 20, 2021

All the files with the version numbers embedded in them, or files hosted under the versioned doc paths (e.g. https://doc.rust-lang.org/1.50.0/std/) seem like they could be marked as cache-control: immutable (and all the other headers that go with it). That still leaves the font-files on the root/named docs as not really being cacheable since they may change when a release is made; maybe these files should have the fonts version number embedded in their filename?

@jsha
Copy link
Contributor Author

jsha commented Feb 21, 2021

I think it would be reasonable to set the fonts to have a TTL of a few weeks and not worry about versioning them. They're not required to be strongly consistent with the rest of the content. If the fonts do change, and it takes someone a while to get the new version, no big deal.

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Mar 4, 2021
…Gomez

Improve page load performance in rustdoc

Add an explicit height to icons (which already had an explicit width) to allow browsers to lay out the page more accurately before the icons have been loaded. https://web.dev/optimize-cls/.

Add min-width: 115px to the crate search dropdown. When the HTML first loads, this dropdown includes only the text "All crates." Later, JS loads the items underneath it, some of which are wider. That causes the dropdown to get wider, causing a distracting reflow. This sets a min-width based on the size that the dropdown eventually becomes based on the crates on doc.rust-lang.org, reducing page movement during load.

Add font-display: swap. Per https://web.dev/font-display/, this prevents "flash of invisible text" during load by using a system font until the custom font is available. I've noticed this flash of invisible text occasionally when reading Rust docs. Note that users without cached fonts will see text, and then see it reflow. For `docs.rust-lang.org`, [setting caching headers will help a lot](rust-lang/simpleinfra#62).

Generated output at https://jacob.hoffman-andrews.com/rust/flow-improvements/std/string/struct.String.html.
JohnTitor added a commit to JohnTitor/rust that referenced this issue Mar 4, 2021
…Gomez

Improve page load performance in rustdoc

Add an explicit height to icons (which already had an explicit width) to allow browsers to lay out the page more accurately before the icons have been loaded. https://web.dev/optimize-cls/.

Add min-width: 115px to the crate search dropdown. When the HTML first loads, this dropdown includes only the text "All crates." Later, JS loads the items underneath it, some of which are wider. That causes the dropdown to get wider, causing a distracting reflow. This sets a min-width based on the size that the dropdown eventually becomes based on the crates on doc.rust-lang.org, reducing page movement during load.

Add font-display: swap. Per https://web.dev/font-display/, this prevents "flash of invisible text" during load by using a system font until the custom font is available. I've noticed this flash of invisible text occasionally when reading Rust docs. Note that users without cached fonts will see text, and then see it reflow. For `docs.rust-lang.org`, [setting caching headers will help a lot](rust-lang/simpleinfra#62).

Generated output at https://jacob.hoffman-andrews.com/rust/flow-improvements/std/string/struct.String.html.
JohnTitor added a commit to JohnTitor/rust that referenced this issue Mar 4, 2021
…Gomez

Improve page load performance in rustdoc

Add an explicit height to icons (which already had an explicit width) to allow browsers to lay out the page more accurately before the icons have been loaded. https://web.dev/optimize-cls/.

Add min-width: 115px to the crate search dropdown. When the HTML first loads, this dropdown includes only the text "All crates." Later, JS loads the items underneath it, some of which are wider. That causes the dropdown to get wider, causing a distracting reflow. This sets a min-width based on the size that the dropdown eventually becomes based on the crates on doc.rust-lang.org, reducing page movement during load.

Add font-display: swap. Per https://web.dev/font-display/, this prevents "flash of invisible text" during load by using a system font until the custom font is available. I've noticed this flash of invisible text occasionally when reading Rust docs. Note that users without cached fonts will see text, and then see it reflow. For `docs.rust-lang.org`, [setting caching headers will help a lot](rust-lang/simpleinfra#62).

Generated output at https://jacob.hoffman-andrews.com/rust/flow-improvements/std/string/struct.String.html.
@jsha
Copy link
Contributor Author

jsha commented Mar 27, 2021

A friendly bump on this. I think it would make a noticeable improvement to the experience of people reading docs on rust-lang.org, particularly if they have slow connections.

@jsha
Copy link
Contributor Author

jsha commented Nov 30, 2021

Did some more thinking and research in this Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/266220-rustdoc/topic/caching.20for.20doc.2Erust-lang.2Eorg.20docs/near/262957610


For the stdlib docs on doc.rust-lang.org, here's the current situation for caching: everything is in S3, so it gets ETags and Last-Modified automatically. The Cache-Control headers are controlled by what's set when uploading to S3, which is currently nothing. That means it's up to browsers to apply a heuristic about how long to cache.

According to https://paulcalvano.com/2018-03-14-http-heuristic-caching-missing-cache-control-and-expires-headers-explained/, that heuristic is "10% of the time since Last-Modified." So for anything in the current stable docs, released 29 days ago, the heuristic cache time is 2.9 days. After the next stable is released, the heuristic cache time will be ~0 days, and slowly go up.

It'd be nice to do better! In particular, for versioned static assets like CSS, JS, and fonts (especially fonts), we'd like to do as docs.rs does and set a long max-age plus "immutable." For nightly, this doesn't really work, since those static assets get overwritten (with the same version number) every night. However, for the stable docs, this could work.

For HTML in the stable docs: We could set "Expires" to the scheduled release of the next stable version. But that would create a problem when there needs to be a patchlevel release in between. During an uneventful 6-week release cycle with no patchlevel releases, the heuristic cache time would get as high as 4.2 days. So we could safely set max-age=4.2 days and have not much worse than the current situation. Or we could round up a bit in the name of more caching and say 7 days. We could also set stale-while-revalidate to some period on the order of months.

For HTML in the nightly docs: Here we can quite confidently set Expires to the time of the next planned nightly build, since it's unlikely there will be another build in between.


Here's my proposal:

  • For nightly uploads, set the Expires header to the typical deploy time the next night.
  • For beta uploads, 1 week.
  • For stable uploads, set the Expires header to the next scheduled release date (assume the regular 6 week cycle).
  • For uploads to versioned paths (https://doc.rust-lang.org/1.56.0/), set Cache-Control: max-age=1 year, immutable.
  • For fonts, set Cache-Control: max-age=1 year, immutable.

@jsha jsha changed the title doc.rust-lang.org: add caching headers for static content doc.rust-lang.org: add caching headers Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants