Optimize Workers Sites Asset Manfest

[ashleymichal]

The user report in #896 reveals an issue with using Workers Sites with large-scale static sites; the user had a project with over 60k assets in it. The asset manifest wrangler generates in order to optimize cache invalidation on the edge is included in the overall bundle size for a worker script, and so can blow up the 1MB limitation when applied to such a large scale project. We should look at ways to optimize the asset manifest to increase the capacity for larger projects like this.

[SamuelPauloBarros]

Any development in this subject? I'm having the same problem as reported in #896 we also have around 70k files. I even tried to reduce the number of files to 48k and then to 24k and still run into the same problem, so far the only website I was able to host with wrangler publish was one with 2k files

Is there any work-around at least?

[stasb]

I am having the same issue with a site containing 20,000 pages, of 500mb build folder size.

Reducing the pages to ~15,000 makes it work. Would be great if this can be fixed, as it looks to be a pretty big limitation with worker sites.

[EverlastingBugstopper]

closing this because #1145 should help a lot with this problem and also we have some other more detailed issues such as #1151 #1004 #896

[ashleymichal]

sgtm