WebGL lens distortion reversal applied unequally to x and y, only works on square images

[jywarren]

Problem

The distortion only works properly for square images -- if the height and width are different, it doesn't correctly remove the lens distortion.

We need to account for the dimensions of the image. #2 links to another project which does this properly. Note that in the fragment shader, field of view is present, but never used:

https://github.com/jywarren/fisheyegl/blob/master/shaders/fragment.glfs#L21-L23

	float L = length(vec3(vPosition.xy/scale, F));

	vec2 vMapping = vPosition.xy * F / L;

F should be applied differently vertically and horizontally, or something. (guessing here)

Theory: maybe the image is "clamped" to the edges of a square WebGL buffer before being distorted, so any non-square image is smushed before being distorted. This actually makes a bit more sense to me but I'm not sure it's what's going on.

In any case, until we can apply F differently for x vs. y, we won't be able to correctly undistort images.

Possible solutions

I believe the key is "length" calculated on this line, but am not sure: https://github.com/jywarren/fisheyegl/blob/master/shaders/fragment.glfs#L21

I think it might also work to not cover the entire square WebGL working area, and to crop out "letterboxing" after WebGL's done it's work.

[jywarren]

I've added some more detail, test images, and discussion over at https://publiclab.org/notes/warren/12-30-2016/web-based-fisheye-removal-prototype-for-correcting-lens-distortion

We'd love help from anyone with some WebGL experience!

[jywarren]

I posted a bounty for this at BountySource! https://www.bountysource.com/issues/40827605-webgl-lens-distortion-reversal-applied-unequally-to-x-and-y-only-works-on-square-images

[Azorlogh]

I tried my best to fix it with a modified algorithm for the different x and y corrections.
#5

[jywarren]

YES! It's fixed. This is perfect. Thank you.

[jywarren]

I think you have to mark it complete before I can award the bounty?

[jywarren]

Also if you'd like to link to this solution (#5) from StackOverflow I can mark your answer accepted there too!

https://stackoverflow.com/questions/45110817/webgl-lens-distortion-reversal-applied-unequally-to-x-and-y-only-works-on-squar?noredirect=1#comment77194226_45110817