composite operation create a gray outline with the latest 0.30.2

[rawpixel-vincent]

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

• Running npm install sharp completes without error.
• Running node -e "require('sharp')" completes without error.

Are you using the latest version of sharp?

• I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

  System:
    OS: macOS 12.2.1
    CPU: (8) arm64 Apple M1
    Memory: 131.50 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm
  npmPackages:
    sharp: 0.30.2 => 0.30.2 

What are the steps to reproduce?

take a transparent png,
apply a composite with tiles to the background,
output to jpeg or webp,
a gray outline is visble (obvious on white images like attached)

inputFile

inputTile

example code 1:

let image = sharp(inputFile);
image.composite([{ input: inputTile, tile: true, blend: 'dest-over', gravity: 'northwest' }]);
image.resize({ width: 400, fit: 'outside' });
image.jpeg({ quality: 100 });
OR
image.webp({ quality: 100 });

example code 2:

let image = sharp(inputFile);
image.resize({ width: 400, fit: 'outside' });
image = sharp(await image.png().toBuffer());
image.composite([{ input: inputTile, tile: true, blend: 'dest-over', gravity: 'northwest' }]);
image.jpeg({ quality: 100 });
OR
image.webp({ quality: 100 });

output with 0.30.2

output with 0.30.1

[lovell]

These kind of fringe effects around edges usually relate to premultiplication.

Commit c620025 in v0.30.2 switched from libvips' composite2 to composite as well as passing responsibility for premultiplication to libvips.

Does this only occur for dest-over? Perhaps the problem fixed by libvips/libvips#1301 needs to include that blend also?

[rawpixel-vincent]

using image.composite([{ input: inputTile, tile: true, blend: 'dest-in', gravity: 'northwest' }]);

doesn't seems to produce that gray line, I can test other options if that's helpful

edit: I only use composite with those options, so didn't spot anything else

[rawpixel-vincent]

do you think I should transfer this issue to https://github.com/libvips/libvips and reference the change in libvips/libvips#1301 ?

[lovell]

Thanks for confirming, if you're able to reproduce using vips composite2 in1.png in2.png out.png dest-over at the command line then yes please.

[rawpixel-vincent]

thank you, here is my results using vips-8.12.2-Tue Jan 25 09:34:32 UTC 2022 installed with brew install libvips

output of --vips-config

enable debug: no
enable deprecated library components: yes
enable modules: no
use fftw3 for FFT: yes
accelerate loops with orc: yes
ICC profile support with lcms: yes (lcms2)
zlib: yes
text rendering with pangocairo: yes
font file support with fontconfig: yes
RAD load/save: yes
Analyze7 load/save: yes
PPM load/save: yes
GIF load:  yes
GIF save with cgif: yes
EXIF metadata support with libexif: yes
JPEG load/save with libjpeg: yes (pkg-config)
JXL load/save with libjxl: yes (dynamic module: no)
JPEG2000 load/save with libopenjp2: yes
PNG load with libspng: yes
PNG load/save with libpng: yes (pkg-config libpng >= 1.2.9)
quantisation to 8 bit: yes
TIFF load/save with libtiff: yes (pkg-config libtiff-4)
image pyramid save: yes
HEIC/AVIF load/save with libheif: yes (dynamic module: no)
WebP load/save with libwebp: yes
PDF load with PDFium:  no
PDF load with poppler-glib: yes (dynamic module: no)
SVG load with librsvg-2.0: yes
EXR load with OpenEXR: yes
OpenSlide load: yes (dynamic module: no)
Matlab load with matio: yes
NIfTI load/save with niftiio: no
FITS load/save with cfitsio: yes
Magick package: MagickCore (dynamic module: no)
Magick API version: magick7
load with libMagickCore: yes
save with libMagickCore: yes

I don't reproduce the issue using vips composite2 inputFile.png inputTile.jpg output.png dest-over

inputFile.png

inputTile.png

output.png

[rawpixel-vincent]

when using vips composite2 inputFile.png inputTile.jpg output.png dest-over --premultiplied
the output give me an white outline border

output with --premultiplied

[rawpixel-vincent]

fwiw I got the same results when sharp use the globally available libvip instead of the prebuilt binaries
here is the complete test

const { readFileSync, writeFileSync, unlinkSync } = require('fs');
const path = require('path');
const sharp = require('sharp');
let assert = require('assert');

describe('composite', () => {
  it(`create a tiled png composite output from sharp pipeline`, async () => {
    try {
      unlinkSync(path.resolve(`${__dirname}/output/test2.jpg`));
    } catch (error) {}
    let image = sharp(readFileSync(path.resolve(`${__dirname}/inputFile.png`)));
    image.composite([{ input: readFileSync(path.resolve(`${__dirname}/inputTile.jpg`)), tile: true, blend: 'dest-over', gravity: 'northwest' }]);
    image.resize({ width: 400, fit: 'outside' });
    image.jpeg({ quality: 100 });

    writeFileSync(path.resolve(`${__dirname}/output/test2.jpg`), await image.toBuffer());
  });
});

outputs:

[lovell]

Thanks for the all the extra examples, I can reproduce this locally with sharp, but I've yet to try to reproduce with vips at the command line.

If sharp premultiplies both images before compositing and sets premultiplied to true (as it used to prior to this change) then the problem goes away.

I think we're seeing this due to libvips only premultiplying one of the images when compositing, which is usually the "lower" image, but with the dest-* blends I think it might be the "upper" image that needs this instead/as well.

https://github.com/libvips/libvips/blob/330ebf3cd7114afd6bc634b450bea20d73fd864f/libvips/conversion/composite.cpp#L508-L512

Possibly the calculation for the dest-over resulting colour should somehow involve the resulting alpha?

https://github.com/libvips/libvips/blob/330ebf3cd7114afd6bc634b450bea20d73fd864f/libvips/conversion/composite.cpp#L566

@jcupitt When you get a mo, please can you check the current VIPS_BLEND_MODE_DEST_OVER logic to see if we're missing anything obvious, thank you.

[jcupitt]

Hi @lovell, this is an odd one. I tried a small test program

#!/usr/bin/python3

import sys
import pyvips

image = pyvips.Image.new_from_file(sys.argv[1])
tile = pyvips.Image.new_from_file(sys.argv[2])

background = tile \
    .replicate(1 + image.width / tile.width,
               1 + image.height / tile.height) \
    .crop(0, 0, image.width, image.height)

over = background.composite(image, "over")
dest_over = image.composite(background, "dest-over")

diff = (over - dest_over).abs().max()

print(f"maximum difference = {diff}")

With git master libvips and with 8.12 I see:

$ ./composite-tile.py ~/pics/heart.png ~/pics/tile.jpg
maximum difference = 0.0

So over and dest-over seem to work. I saved the images and checked by hand as well, they look OK.

I think I would patch sharp to save the argument images to composite just before calling it, probably as .v format to make sure the saver doesn't change anything. You can then check the argument images to verify that sharp hasn't done any premultiplication (vipsdisp is handy for this, right-click, turn on the info bar and you can see the exact pixel values in the file). Then run composite at the command-line and check the result.

If the input images are wrong I suppose sharp might have a stray premultiply still. If they are OK and the output is wrong, we have a reproducer for a libvips bug.

[lovell]

Thanks John, I think I've found the problem, there was a extra unpremultiply taking place in sharp after composite that affects some of the dest-* blend modes.

Rather annoyingly the unit tests did pick this up at the time but I incorrectly assumed the "new" behaviour was more accurate. c620025#diff-7a6322e019754e2e5f843a65aa8cfb3635d77a757b88ef4c6a9450a022e5678f

[lovell]

I've reverted the test fixtures to their v0.30.1 state, as well as removing the unpremultiply operation when compositing, in commit 23033e2

This will be in v0.30.3, thanks for reporting.

[lovell]

v0.30.3 now available.

[rawpixel-vincent]

Hi,
I've tried with sharp 0.30.3

> npm view sharp dist-tags.latest
0.30.3

>npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 12.2.1
    CPU: (8) arm64 Apple M1
    Memory: 78.98 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm
  npmPackages:
    sharp: 0.30.3 => 0.30.3 

for the following code, with the same input files

describe('bg mosaic', () => {
  it(`create a tiled png output from sharp pipeline`, async () => {
    try {
      unlinkSync(path.resolve(`${__dirname}/output/test2.jpg`));
    } catch (error) {}
    let image = sharp(readFileSync(path.resolve(`${__dirname}/inputFile.png`)));
    image.composite([{ input: readFileSync(path.resolve(`${__dirname}/inputTile.jpg`)), tile: true, blend: 'dest-over', gravity: 'northwest' }]);
    image.resize({ width: 400, fit: 'outside' });
    image.jpeg({ quality: 100 });

    writeFileSync(path.resolve(`${__dirname}/output/test2.jpg`), await image.toBuffer());
  });
});

I still have the same output

edit: I don't have libvips installed, it's using sharp prebuilt binaries
edit2: I don't seem to be able to re-open the issue, I let it rest for a while and open a new one in case you're not notified, fwiw it's possible I'm doing something stupid... but I've tried multiple time, cleaning everything, deleting node_modules, checking my working path..
edit3: re-attaching the input files I'm testing with in case

[lovell]

v0.30.3 works on my machine:

node -p "require('sharp')('heart.png').composite([{ input: 'checks.jpg', blend: 'dest-over' }]).toFile('out.png')"

[rawpixel-vincent]

yes I messed up my test files... and was testing with a wrong input, it works fine, thank you again