Possible bug in resampler

[DTL2020]

It looks like some error on some edge-conditions in calculations. At least it simply reproductible in SincResize: Given input image data as 'ideal video/image point' for example defined with 2D cross-sections as (16),27,132,235,132,27,(16) samples (not gamma-corrected, linear, 16..235 video levels coded) sequence and 2D full (sinc-interpolated using radius-vector) 5x5 array

16,20,27,20,16 
20,76,132,76,20 
27,132,235,132,27 
20,76,132,76,20 
16,20,27,20,16

We can set its values in 'ms-paint' to get image-formatted file:
1point_27_132.zip

And load into avisynth and SincResize:

ImageReader("1point_27_132.bmp") 
Crop(20,20,-20,-20) 
SincResize(width*8,height*8,taps=20)

With above Crop() parameters output image size is 1032x904 and output shows 4 errors artifacts (marked with circles)

If we lower resizer input image size the bug looks like disappear and output is clear:

ImageReader("1point_27_132.bmp") 
Crop(50,50,-50,-50) 
SincResize(width*8,height*8,taps=20)

To make bug much more visible it can be added
Levels(0, 1, 25, -300, 255)

The distance to buggy output samples from center of filter depends on 'taps' parameter. With taps=10

Version 3.6.1 has it too. Attempt to SetMaxCPU("none") does not helps.

First reported at https://forum.doom9.org/showthread.php?p=1919151#post1919151

[DTL2020]

For further bug isolation and easier step-by step walking through code tried to lower number of samples in resizer's input as much as possible: 1 sample (230 at field of 16) input
1sample_230.zip

retrives kernel itself image output (acting as delta-function input to filter, Size x 8, taps=20).

And ringing at its edges masks the bug.

The minimum good bug visible input is 3 non-zero samples (for 1d horizontal resampler and taps 10..20),(127,230,127):
3sample_127_230_127.zip

It shows how sincs (3 sincs waves from 3 non-zero input samples) sum already good converges/balances to zero close enough to samples position. But something buggy happens at the edges of truncated sincs. May the resampler engine losts some members of kernel at the very edges of processing so the very edges of sum lost ability to converges to zero and stays in small negative values.

Script is

ImageReader("3sample_127_230_127.bmp") 
#ImageReader("1sample_230.bmp") 
Crop(20,20,-20,-21) 
ConvertToYV12()  
SincResize(width*8,height*8,taps=20) 
Crop(700,600,800,800) 
Levels(0, 1, 25, -400, 255)

So comes the easiest (temporal) bugfix or at least workaround - just truncate resampler's sums a bit so we will get good converged central output. Like perform taps=20/support=20 processing and use only 16..18 output max which are not buggy.

[DTL2020]

Some more finding:

At fast image resizers we use very truncated sinc kernels to a few taps (lobes). So when resampler stepping engine make step to next sample there occur real lost of a (part because we upsize and kernel lobes cover a number of output samples i think) of kernel lobe at the very edge of a kernel that was required for good convergence at that output sample. So at each step of resampler core we got ouput edge sample value error. If image-object data is less in size in compare with 'taps' parameter that cause even forming of 'ghost' parasitic image at 'taps' distance from the object (sometime in many directions - V and/or H).

Here is sample of the script:

Loadplugin("ResampleMT.dll")  
function Ast2(clip c, int isize) {  return Subtitle(c, "7",font="Arial",size=isize,x=10,y=20,halo_color=$FF000000, text_color=$00e0e0e0) }  
BlankClip(100,200,180,"RGB24",25,color=$00202020)  
Animate(last, 0,100,"Ast2",  45,  180)  AddBorders(100,100,100,100,color=$00202020)  SinPowResizeMT(last,width/4,height/4,p=3.1)  
SincResize(width*8,height*8,taps=20)  
Levels(25, 1, 55, 0, 255)

And at frame about 40 it produces output with 3 'ghosts' to the right/top/bottom but no ghost to the left:

And the 'ghost' looking almost independent of frequency spectrum of useful data (defined with p-param of SinPowResize()).
The simple but almost non-possible at current computer's performance solution is to make taps > output_image_size/2 (or may be even output_image_size). So we will move our non-converged defective output samples out of valuable image data array. The other solutions is think how to workaround with current possible very few taps about tenths maximum. May be try to weight edge of kernel to zero starting from for example taps/2 with linear or whatever interpolation function. So loosing part of last kernel lobe will give less errors and may they finally drops lower 8bit precision output at least (for HDR work they will be still visible). So with SincResize(taps=20) we will have full sinc kernel from tap=0 to tap=10 and fading sinc (like Lanczos) from tap=11 to tap=20. Or think how to supply resampler with required part of full sinc kernel at edge in other way.

In the above example parasitic 'ghost' non-exist at the left side - and it is close to frame's edge. May be some frame-edge workaround already exist in the resampler code but simply do not work in the middle of frame ? So the bugfix may be simple ?

[DTL2020]

Have finally spend more time with checking this bug in debugger. It looks caused with combined reasons:
1.Too high amplitude of edge lobes of sinc function
2. Too few taps used even for 8bit-coded output.

At the edges if image buffer bug is not exist because current resampling program simply copy 'taps' (or may be 2x taps) number of edge samples of image buffer and supply resampler with static input (the changing is kernel shifted samples) untill it travels on taps (or 2x taps) distance from frame edge. So at that places if we have 'convergible' samples at resampler;s input it always output correct. But then convolution 'reading frame' starts to travel from image buffer edge it, at some step, losts for example 1 of 3 input sample required for exact convergence of 3 sinc input_samples-weighted sum and we got an error in output. If this error > 1/255 for 8-bit encoding we got 'ghost' output error and it follows image object at 'taps' distance. The most of other resize kernels are weighted to almost or full zero at the edge of kernel so this error falls below 1/255 and not visible at output.

So I see 2 ways of workarounding:
The bug in C-based integer short 16 bit resampler looks like disappears at taps parameter about 70 or more. Though the processing speed is slow enough ofcourse.
We can try to weight the edge of Sinc kernel so to get bug disappear with lower number of taps (at least with 8bit output samples) while keeping this filter as much closer to SincResize instead of LanczosResize with similair number of taps as possible. Because LanczosResize is simply Sinc-kernel full-size weighted with ‘Lanczos-weighting’ that is just wide sinc half-lobe too.
I made a tests and found one possible weighting more sharper in compare with ‘full-Lanczos’ — it keeps more Sinc lobes unchanged so gets more sharpness etc. It is simple linear fading starting from 0.5 of kernel width (taps).
So I add one new function SincLin2Resize and it works at 3-sample test and other quick tests without exposing edge-bug with taps as lower as 15. Though for varios cases I increase available ‘taps’ from 20 to 30 if sometime it will be needed. And default taps to 15. Also I think in 2020 yea and avisynth+ we can increase limit of ‘taps’ paramerter from 20 to 70 or more to ‘pure/reference SincResize filter’ so users can test ‘pure/reference SincResize’ with possible workaround using just significant increasing of taps-number though loosing speed processing.
Current sources with x64-build .dll are in attach
ResampleMT-master_SincLin2Resize.zip

Kernel func:

/***********************
*** SincLin2 filter ***
***********************/
SincLin2Filter::SincLin2Filter(int _taps)
{
	taps = (double)clamp(_taps, 1, 30);
}

double SincLin2Filter::sinc(double value)
{
	if (value > 0.000001)
	{
		value *= M_PI;
		return sin(value) / value;
	}
	else
	{
		return 1.0;
	}
}

double SincLin2Filter::f(double value)
{
	value = fabs(value);

	if (value < taps / 2)
		return sinc(value);
	else
		return sinc(value)*((2 - (2 * value / taps)));

}

Header

class SincLin2Filter : public ResamplingFunction
	/**
	* SincLin2 filter, used in SincLin2Resize
	**/
{
public:
	SincLin2Filter(int _taps = 15);
	double f(double x);
	double support() { return taps; };

private:
	double sinc(double value);
	double taps;
};

In addition to 'ghosting' effect outside object's image the old SincResize has also 'grid' effect at high levels inside object:

Loadplugin("ResampleMT.dll")

function Ast2(clip c, int isize)
{
 return Subtitle(c, "7",font="Arial",size=isize,x=10,y=20,halo_color=$FF000000, text_color=$00e0e0e0)
}

BlankClip(100,200,180,"RGB24",25,color=$00202020)

Animate(last, 0,100,"Ast2",  45,  180)

AddBorders(150,140,100,100,color=$00202020)

SinPowResizeMT(last,width/4,height/4,p=3.1)

SincResizeMT(width*8,height*8,taps=20)

Levels(0,0.3,255,0,255)

The new method fixes this too.