Updates are larger than necessary (bad .ZIP implementation)

[ericlaw1979]

Did you search for similar issues before submitting this one?
Of course.

Describe the issue you encountered:
Observe network traffic while downloading Brave update.

Expected behavior:
Small Update package using an efficient format like 7z/lzma

Actual: Zip file (nupkg) which is not optimally compressed; even a good .ZIP compressor can save a few megabytes.

• Platform (Win7, 8, 10? macOS? Linux distro?):
  Win7
• Brave Version:
  Latest
• Steps to reproduce:
  1. Check for update
  2. Watch download size

[neeklamy]

The issue here is upstream with Electron, which uses Squirrel to deliver updates, which itself doesn’t support anything other than the zip format. Squirrel is two different projects, Squirrel.Mac and Squirrel.Windows (and the Windows version uses NuGet) – Linux is currently unsupported, it’s left to each distribution’s software manager.

Brave’s autoUpdates documentation and Electron’s autoUpdater documentation explain more.

There’s an issue requesting delta updates, Squirrel/Squirrel.Mac#5 and another asking for .patch support, Squirrel/Squirrel.Mac#162

But it looks like, until Squirrel and NuGet support anything other than plain zips, or unless the Brave developers were to switch solutions, this won’t be readily fixed. And it looks like the NuGet devs aren’t in any hurry to change that…

[ericlaw1979]

this won’t be readily fixed.

I'm not really sure I understand. At a minimum, they can shave off a portion of the size simply by recompressing the ZIP file.

[neeklamy]

And I’m explaining that the Brave uses a service (Electron) that uses another service (Squirrel) that in turn (on Windows at least) uses yet another service (NuGet) to package up the archives.

If NuGet doesn’t support anything other than vanilla zips, then neither can anything that exclusively uses their services.

See the complaints about NuGet here for example, NuGet/Home#890.

That’s the way it looks from here, anyway.

[ericlaw1979]

A zip file is a zip file. If you compress it well, it's still a zip file.

[neeklamy]

Did you read anything that I linked to?

This, Squirrel/Squirrel.Windows#620 specifically speaks to the same issue regarding the current large size of Electron (the thing that Brave is built on):

I'm wondering if it would be possible as a feature request for this project for it to be able to use a different compression method such as LZMA2 to create much smaller installers?

The reply from a Squirrel developer?

This isn't possible because we have to remain compatible with NuGet. While on full installs I'm getting similar results as you on the Slack Desktop app, on delta updates it seems to have almost no effect.

It's definitely worth thinking about, but killing NuGet compatibility (and by extension, compatibility with the entire NuGet ecosystem of tools) is a real drag. I'll think about it.

[ericlaw1979]

Yes, LZMA2 is a compression method.
No, LZMA2 is not a compression method used for ZIP.
No, you do not need to use LZMA2 to improve compression over what is being used now (as shown in the original screenshot.)

[neeklamy]

And @Ivoz was using LZMA2 as an example,

using say LZMA2 instead of zip/DEFLATE you can get from 84.8MB compressed size to 48.7MB. That's approaching 50% smaller file size!

(Emphasis mine.)

The answer from @paulcbetts from last March?

This isn't possible because we have to remain compatible with NuGet.

You obviously know a lot more about compression tech than I do, but of course I’ve searched up and found out the source of the current Brave/Electron implementation. Maybe you could take up the issue with the NuGet developers?

[anaisbetts]

On the Windows side, we're planning on moving to LZMA2 for at least the initial install. LZMA2 actually doesn't win us much of anything for delta updates (<2%). Trying to do this while maintaining compatibility is pretty tricky too (i.e. you're updating from a version of Squirrel that doesn't understand LZMA2 => a version that does).

Brave should definitely enable delta updates on Windows though, it would save them a lot of bandwidth