PNG file of smaller resolution is bigger than a smaller one? - size optimization mystery.

[AndKe]

I need to transfer fast lots of images over a limited link. Color count is <10
When converting 553x288 svg to .png (compression=6), I had an average filesize of 36kB
then I reduced the resolution of the svg to 472x288 (trimmed off a black area)
now the average file size is 43kB.

Is png more efficient at some specific resolutions/aspect ratios? , when it can be divided by some specific number?

[jcupitt]

Hi, yes, compression is complex and depends on a lot of factors. Do you have some sample images which show this?

[AndKe]

png-mystery.zip
please see attached zip, the results are the opposite of what I tried to achieve.
I hope you can suggest some changes, it seems the resolution is the key to success.

[AndKe]

@jcupitt I used this script to do the testing:
do.zip
I ran it like this:
$ time ./do.sh 6 && du -ach *.png

• then it outputs an average filesize too.

[jcupitt]

Try bitdepth, since you have so few colours:

john@banana ~/pics/553x288 $ vips copy test122.svg x.png[bitdepth=4,filter=8,compression=9]
john@banana ~/pics/553x288 $ ls -l x.png
-rw-r--r-- 1 john john 5996 May  3 23:00 x.png

[AndKe]

@jcupitt You are truly good at this ! :)
this really makes a lot of difference - I am about to do some performnce-testing on Rpi4 too , but it runs vips-8.7.4 - and says: "pngsave: no property named `bitdepth' - so I am trying my best to build a decent version for RPI4.

[AndKe]

@jcupitt I did performance testing on the receiving end of this data (google glass) - and it is a wonderful improvement.
It allowed me to increase the frame rate from 4hz to 10hz at the same CPU load of the "glass browser" - and even reach 20Hz without thermal throttling.
A big "Thank you."

[jcupitt]

Oh, great! You could try turning off compression=9 too, it's a lot of CPU work for a very small gain.

[AndKe]

Yes, all my findings and actual use is compression=6 (which I suspect is the default) - it produces nice results.

Also, I managed to run it fine on the Rpi. (the currently performance-limiting device)
converting svg->png with compression=6 takes on average 210ms
with "bitdepth=4,filter=8,compression=6" it takes 190ms

I do create one svg at a time (based on telemetry input), so it does not help to throw in multiple cores unless vips is multithreaded itself.
If there is something you can think of to speed up the re-compression, please say so - otherwise, my backup plan is to replace the Rpi with something with more CPU power.

[jcupitt]

libpng spends almost all of its time in deflate compression, so swapping libz out for something faster is an easy win. People usually suggest:

https://github.com/ebiggers/libdeflate

[AndKe]

Thank you. I will definitely have a look at it another day soon.
when I compile it, is it a matter of symlinking or directly replacing a library , or is a recompilation of vips required to make use of it?

[AndKe]

@jcupitt I cloned, make, and make install the libdeflate library. (the speed is the same)
do I need to symlink it somewhere, change/uninstall something, and/or make vips again to make use of this library?

[jcupitt]

Sorry, I've never actually done this. You might be able to symlink it to libz, or you might need to rebuild libpng. I'd look up some guides.

[AndKe]

For your information:
ebiggers/libdeflate#129

While it sounded like a superb performance improvement, it does not seem to be very stright-forward, because
the quote libz and libdeflate has different API. Does vips support libdeflate's?

I did a fast grep for libdeflate in vips's source, and it does not seem to have the API for it.(or at least does not mention the word "libdeflate")

[jcupitt]

Oh darn, sorry, I though it was the same API.

You should take a look at libpng -- that's what would benefit from using libdeflate.

[AndKe]

Thanks, let me get this right:
libpng should be the one to use libdeflate, .. and then libvips would "just work" because it uses libpng - which in turn would be using libdeflate?

[jcupitt]

That's it. libvips uses libz a little, but almost all the heavy use is inside the various format libraries.

[kleisauke]

You could try to use zlib-ng. It's API / ABI compatible with zlib (when configured with --zlib-compat or -DZLIB_COMPAT=ON), so it can be used as a drop-in replacement (for e.g. with the LD_PRELOAD environment variable).

fwiw, sharp uses this as the default zlib library in the pre-built libvips binaries since v0.28.0, so it might be worth experimenting with that. According to lovell/sharp#2604 this improves PNG de/compression performance by ~20%.

[AndKe]

@kleisauke Thank you.
my actual usage looks like this(pyvips)

x = pyvips.Image.svgload_buffer(content.encode('utf8'))
        buf = x.write_to_buffer('.png', bitdepth=4,filter=8,compression=6)

...on a Rpi4 (need to build for arm, like I did libvips)

I am not sure whatever your advice is that I use zlib-ng (and then recompile libvips, or somehow make it use zlib-ng)
or
if you are advising me on using sharp - which seems already compiled for arm https://github.com/lovell/sharp/releases/download/v0.28.1/sharp-v0.28.1-napi-v3-linux-arm64.tar.gz ... but only contain one "sharp.node" file. - but does the sharp thing somehow work with my current vips/pyvips ?

[kleisauke]

I'm not sure if the pre-built libvips binaries provided by sharp are compatible with pyvips (since libvips.so.42 was statically-linked into libvips-cpp.so.42). I thought it might be easier to use these instead of compiling them yourself. Anyways, they can be found here: https://github.com/lovell/sharp-libvips/releases/tag/v8.10.6.

I think the easiest way is to "just" compile zlib-ng from source with the flags mentioned above. By using the LD_PRELOAD trick, you could use it immediately without having to recompile every dependency that uses zlib.

[AndKe]

@kleisauke thank you. Could you please reveal the whole "LD_PRELOAD trick" ? I mean: what should this variable be set to?

[kleisauke]

Please see: https://github.com/zlib-ng/zlib-ng#install.

[AndKe]

@kleisauke Thank you, that last one was embarrassing to overlook :) I'll report here any performance improvement/difference.

[AndKe]

Oddly, the zlib-ng is significantly slower:

pi@pfd:/run/user/1000 $ time vips copy out--12.png  x.png

real	0m0.130s
user	0m0.104s
sys	0m0.048s

pi@pfd:/run/user/1000 $ time LD_PRELOAD=/home/pi/zlib-ng/libz.so.1.2.11.zlib-ng vips copy out--12.png  x.png

real	0m0.364s
user	0m0.073s
sys	0m0.306s

At first I thought that it could be somehow just an overhead for the preload, but when testing a whole batch
LD_PRELOAD=/home/pi/zlib-ng/libz.so.1.2.11.zlib-ng ./do.sh 6
vs
./do.sh 6

then the results are clear, - the libz-ng is much slower.

• it was worth a try :)