Skip to content
This repository has been archived by the owner on Aug 3, 2023. It is now read-only.

[sites] Don't delete unused assets #2096

Merged
merged 3 commits into from
Oct 12, 2021
Merged

Conversation

threepointone
Copy link
Contributor

@threepointone threepointone commented Oct 8, 2021

We have a consistency issue when after uploading a new site, it takes a while for the worker to propagate, so incoming requests will try to fetch older assets, and 404. This isn't an issue just with us, it's a common scenario. Besides, there's no real reason to delete older assets, we don't have limits on how many keys we use (https://developers.cloudflare.com/workers/platform/limits#kv-limits). Maybe someone could write a script that does a cleanup, but there's no reason to delete previous deployments assets immediately. (This matches Pages behaviour too).

This PR removes the functionality where we delete unused assets on fresh uploads.

(This is my first rust PR! Please verify I didn't screw anything up!)

We have a consistency issue when after uploading a new site, it takes a while for the worker to propagate, so incoming requests will try to fetch older assets, and 404. This isn't an issue just with us, it's a common scenario. besides, there's no real reason to delete older assets, we don't have limits on how many keys we use (https://developers.cloudflare.com/workers/platform/limits#kv-limits). Maybe someone could write a script that does a cleanup, but there's no reson to delete previous deployments assets immediately. (This matches Pages behaviour too).

This PR removes the functionality where we delete unused assets on fresh uploads.
@threepointone threepointone requested a review from a team as a code owner October 8, 2021 08:22
@koeninger
Copy link
Contributor

Storage is cheap but we still need some way of deleting unused assets eventually.

If you don't delete them here (or overwrite them with an expiration time), how will you identify them later for deletion?

@threepointone
Copy link
Contributor Author

Honestly I don't think we should delete static assets by default. It's fine. In my experience, it's incredibly rare that folks will keep uploading large fresh assets AND not reference previous older assets. Further, with what we're planning separately in other projects, it becomes even more important to keep previously built assets around. And finally, this is a temporary solution until we migrate sites to Pages. So I think it's ok.

@Electroid Electroid merged commit 3c604a5 into master Oct 12, 2021
@delete-merged-branch delete-merged-branch bot deleted the dont-delete-sites-assets branch October 12, 2021 10:17
@hkochniss
Copy link

hkochniss commented Oct 19, 2021

🎉 thanks for that.. we were getting occational problems due to missing asssets (e.g. dynamic lazy-loaded hash-stamped JS modules not existing anymore on server side), so I think the delete action on publish was naive to begin with.
I was creating my own node upload script and mangled the worker to circumvent wrangler publish completely.. 8 hours gone ;) Still pursuing that route to allow for dnamically switching website versions on same domain, but I have a feeling that is on the roadmap for pages...

threepointone added a commit that referenced this pull request Mar 8, 2022
This reverts the changes made in #2096 that would preserve already uploaded assets with `[site]`, even if they weren't being used. This lads to longer upload times for sites with rapidly changing assets, and a burgeoning kv store. We're reverting this for now, and will add a proper fix later that expires unused assets on upload.
threepointone added a commit that referenced this pull request Mar 8, 2022
This reverts the changes made in #2096 that would preserve already uploaded assets with `[site]`, even if they weren't being used. This lads to longer upload times for sites with rapidly changing assets, and a burgeoning kv store. We're reverting this for now, and will add a proper fix later that expires unused assets on upload.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants