-
-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
implement p5.Camera.slerp(), 3x3Matrix functions and minor spec-change of orbitControl() #6251
implement p5.Camera.slerp(), 3x3Matrix functions and minor spec-change of orbitControl() #6251
Conversation
The value of mouseWheel is machine-dependent, so I'll just use only the sign to remove that effect. Also, I replaced with 'cam' where it could be replaced with 'cam'. In addition, in the current specification, the projMatrix of the camera is not updated when scaling with ortho(), so after updating it, I rewrote it to update uPMatrix.
Implement slerp(). It interpolates the views of the two cameras. For example, by interpolating between straight front and straight right, you can make it look diagonally to the right.
shorten code for calculate eyeDist
Introduced a new implementation. As a result, when pan(), tilt(), _orbit(), and _orbitFree() move the camera, slerp() now gives the same result. function setup(){
createCanvas(100, 100, WEBGL);
const cam = createCamera();
const cam0 = createCamera();
cam0.camera(20, 30, 40, 0, 2, 4, 0, 1, 8);
const cam1 = createCamera();
cam1.camera(100, 100, 100, 0, 0, 0, 0, 0, -1);
cam.slerp(cam0, cam1, 0);
console.log(confirmcameraMatricesAreTheSame(cam, cam0));
cam.slerp(cam0, cam1, 1);
console.log(confirmcameraMatricesAreTheSame(cam, cam1));
const cam2 = createCamera();
const cam3 = cam2.copy();
cam3.pan(PI*0.9);
const cam4 = cam2.copy();
cam4.pan(PI*0.3);
cam.slerp(cam2, cam3, 1/3);
console.log(confirmcameraMatricesAreTheSame(cam, cam4));
const cam5 = createCamera();
const cam6 = cam5.copy();
cam6.tilt(PI*0.4);
const cam7 = cam5.copy();
cam7.tilt(PI*0.1);
cam.slerp(cam5, cam6, 0.25);
console.log(confirmcameraMatricesAreTheSame(cam, cam7));
const cam8 = createCamera();
const cam9 = cam8.copy();
cam9._orbit(PI*0.8, 0, 0);
const cam10 = cam8.copy();
cam10._orbit(PI*0.3, 0, 0);
cam.slerp(cam8, cam9, 3/8);
console.log(confirmcameraMatricesAreTheSame(cam, cam10));
const cam11 = createCamera();
const cam12 = cam11.copy();
cam12._orbit(0, PI*0.77, 0);
const cam13 = cam11.copy();
cam13._orbit(0, PI*0.22, 0);
cam.slerp(cam11, cam12, 2/7);
console.log(confirmcameraMatricesAreTheSame(cam, cam13));
const cam14 = createCamera();
const cam15 = cam14.copy();
cam15._orbitFree(PI*0.27, PI*0.9, 0);
const cam16 = cam14.copy();
cam16._orbitFree(PI*0.12, PI*0.4, 0);
cam.slerp(cam14, cam15, 4/9);
console.log(confirmcameraMatricesAreTheSame(cam, cam16));
}
function confirmcameraMatricesAreTheSame(cam0, cam1, threshold = 0.0001){
const m0 = cam0.cameraMatrix.mat4;
const m1 = cam1.cameraMatrix.mat4;
let errorSum = 0;
for(i=0;i<16;i++){
errorSum += abs(m0[i] - m1[i]);
if(abs(m0[i] - m1[i]) > threshold){
return false;
}
}
console.log("error: " + errorSum);
return true;
} result:error: 0 This result was not possible in previous implementations due to pan() and tilt() moving the viewpoint. It's clearly improving. |
It may be a little difficult to understand, but you can see that it matches the movement in the case of pan(). (slerpOld() is old slerp()) 2023-07-04.10-38-41.mp4Unrelated, but set() is also useful in this case. Since there was no camera function to set a specific state for the same camera, we had no choice but to use push() ~ pop() in the past, but using set() makes it very easy to return to the prepared state. |
Until now, we used linear interpolation of the center to determine the viewpoint and the center, but this method does not result in a clean interpolation when the center moves. So I've improved the code so that if at least one of eye and center doesn't move, it's an interpolation that doesn't move that. In addition, we have improved the speed by not creating a new vector when only that component is needed. As a result, there is little change in performance.
I want to make unit tests carefully, so I'll work on it after I get home... |
Here are some examples. let cam, cam0, cam1, farCam;
let isFar = false;
function setup(){
createCanvas(600, 600, WEBGL);
pixelDensity(1);
farCam = createCamera();
farCam.camera(200, 200, 500, 0, 0, 250, 0, 0, -1);
cam0 = createCamera();
cam0.camera(100, 0, 0, -100, 0, 0, 0, 0, -1);
cam1 = createCamera();
cam1.camera(0, 100, 100, 0, -100, 100, 0, 0, -1);
cam = createCamera();
}
function draw(){
if (isFar) {
setCamera(farCam);
} else {
setCamera(cam);
}
noStroke();
strokeWeight(2);
lights();
fill("red");
background(255);
cam.slerp(cam0, cam1, (frameCount % 360) / 120);
box(30);
translate(0, 0, 100);
box(30);
translate(0, 0, 100);
box(30);
translate(0, 0, 100);
box(30);
translate(0, 0, 100);
box(30);
translate(0, 0, -300);
if (isFar) {
stroke(0);
line(
cam.eyeX, cam.eyeY, cam.eyeZ,
cam.centerX,cam.centerY, cam.centerZ
);
noStroke();
push();
fill("blue");
translate(cam.eyeX, cam.eyeY, cam.eyeZ);
sphere(8); // small: eye
pop();
push();
fill("green");
translate(cam.centerX, cam.centerY, cam.centerZ);
sphere(24); // big: center
pop();
}
}
function doubleClicked(){
isFar = !isFar;
} 2023-07-04.21-30-29.mp4First, prepare one camera, raise it as it is, and then rotate it by 90 degrees. By interpolating with these two, you can make the field of view rise. In this way, we can achieve rotary motion without using trigonometric functions. The other concerns pan() and tilt(). let cam, cam0, cam1;
function setup(){
createCanvas(400, 400, WEBGL);
pixelDensity(1);
cam0 = createCamera();
cam1 = createCamera();
cam1.pan(PI/7);
// cam1.tilt(PI/7);
cam = createCamera();
noStroke();
}
function draw(){
background(255);
cam.slerp(cam0, cam1, (frameCount % 420) / 30);
lights();
fill("skyblue");
ambientMaterial("skyblue");
push();
rotateX(frameCount*0.08);
box(80);
pop();
translate(0, 0, 680);
push();
rotateY(frameCount*0.06);
box(80);
pop();
translate(0, 0, -680);
translate(340, 0, 340);
push();
rotateZ(frameCount*0.08);
box(80);
pop();
translate(-680, 0, 0);
push();
rotateZ(frameCount*0.08);
box(80);
pop();
} 2023-07-04.21-34-21.mp4After setting the direction of rotation with pan() and tilt(), by executing slerp() outside the [0,1] range, you can use the camera to rotate infinitely. In addition, the second example regarding pan() and tilt() will be strange if it is executed with slerp() before the specification change above. So I'm glad I made the change. |
I'll try to make just one unit test. It is the test about when amt is 0 or 1.
I'll try to make just one unit test. It is the test about when amt is 0 or 1. |
It seems to work fine, so I'll add more. |
It is troublesome to write each cameraMatrix comparison method, so I decided to prepare it globally.
It is troublesome to write each cameraMatrix comparison method, so I decided to prepare it globally. |
add indent and semicolon
It went well. In the case of _orbit() and _orbitFree(), the matrices are not exactly the same, so I thought it would be useful to have this. |
Added unit test for tilt(),_orbit(),_orbitFree()
I forgot to mention that PI/2 rotations are problematic and should not be used.
I forgot to mention that _orbit() shouldn't use for vertical PI/2 rotations. _orbitFree() would be fine, but _orbit() is no good... |
PI/2 rotation destroys the up vector, so it cannot be used. excuse me... I have confirmed that 0.49 and 0.51 are fine. |
Added tests for fixed view, fixed center, and ortho()
use strictEqual
use expect().to.be.closeTo()
Since this function is only for interpolating the view, it is assumed that all the projection matrices match. However, for ortho(), orbitControl() changes the projection matrix a bit, so it is possible to interpolate if there is a difference of that degree. But that's where it gets tricky because it usually doesn't work. I can't do what I can't, so I think this is fine. |
Added explanation about ortho()
I added that it is permissible for the projection matrix to change with orbitControl(). If there are any problems, I will rewrite. |
That's all for the changes, so please leave a review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work on this, it looks like it took a lot of calculations, and the version you arrived on that handles both fixed view and fixed center works really well!
I left some comments about possibly using p5 math objects like vectors and matrices to clean up the code. It looks like that would involve augmenting p5.Matrix in order to do it though, what do you think? If you think it's a good idea but it looks like it'll take time, I'm also not opposed to doing that in a separate PR so that it doesn't feel like it's rushed to get it in in time for 1.7.0.
src/webgl/p5.Camera.js
Outdated
} | ||
|
||
// Calculate the distance between eye and center for each camera. | ||
const dist0 = Math.hypot( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be worth making a p5.Vector out of the eye and center points for both cameras, which would allow us to use .mag()
, .sub()
, .lerp()
, etc below
src/webgl/p5.Camera.js
Outdated
// Create the inverse matrix of mat0 by transposing mat0, | ||
// and multiply it to mat1 from the right. | ||
// This matrix represents the difference between the two. | ||
const m = [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we could make this out of p5.Matrix
to make the transpose + multiply clearer? It looks like we support mat3s there if you do new p5.Matrix('mat3', [...])
. I'm not certain all the operations work on mat3s, so if they don't, this is also fine
src/webgl/p5.Camera.js
Outdated
// Obtain the front vector and up vector by linear interpolation | ||
// and normalize them. | ||
const lerpedFront = new p5.Vector( | ||
(1 - amt) * m0[2] + amt * m1[2], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we do end up using p5.Matrix, maybe we could make functions to get a column as a p5.Vector? Or maybe least as an array to handle both 3x3 and 4x4 matrices, which one can then put into a vector? so then this line could be something like:
const lerpedFront = createVector(...m0.column(2))
.lerp(...m1.column(2), amt)
.normalize();
const lerpedUp = createVector(...m0.column(1))
.lerp(...m1.column(1), amt)
.normalize();
src/webgl/p5.Camera.js
Outdated
// https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294 | ||
let a, b, c, sinTheta; | ||
let invOneMinusCosTheta = 1 / (1 - cosTheta); | ||
if (m[0] > m[4] && m[0] > m[8]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe a diagonal()
method on a matrix could be helpful too, so it's easier to tell which indices you're using?
// Calculate the trace and from it the cos value of the angle. | ||
// An orthogonal matrix is just an orthonormal basis. If this is not the identity | ||
// matrix, it is a centered orthonormal basis plus some angle of rotation about | ||
// some axis. That's the angle. Letting this be theta, trace becomes 1+2cos(theta). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this might not be obvious to readers, maybe we can drop in a link to this: https://en.wikipedia.org/wiki/Rotation_matrix#Determining_the_angle
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I learned for the first time that there is a website that has a specific shape of the procession.
Ok, I'll add it.
src/webgl/p5.Camera.js
Outdated
// Calculates the axis vector and the angle of the difference orthogonal matrix. | ||
// The axis vector is what I explained earlier in the comments. | ||
// similar calculation is here: | ||
// https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use the permalink URL to this line instead? (Obtained through the three dot menu on github when you select a line, I think it just uses the commit hash in the URL instead of the branch, in case this file changes on the dev branch in the future.) https://github.com/mrdoob/three.js/blob/883249620049d1632e8791732808fefd1a98c871/src/math/Quaternion.js#L294
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'll fix that.
// Multiplying mat0 by the first matrix yields mat1, but by creating a state | ||
// in the middle of that matrix, you can obtain a matrix that is | ||
// an intermediate state between mat0 and mat1. | ||
const angle = amt * Math.atan2(sinTheta, cosTheta); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah I see what you mean about quaternions maybe making the implementation more difficult, since you mostly need the angle in order to then pick a different angle based on amt
. Just for my understanding, I suppose you'd want to convert the quaternion to axis/angle, update the angle, then make a new quaternion from that, then apply that to m0, and this current implementation avoids unnecessary work in making the intermediate states?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not familiar with quaternions in the first place, so I'm not sure. Orthogonal matrices are easier to work with, so I've thought about implementing them in that direction. It's very easy because we can get the difference just by taking the transpose and multiplying it. I thought it would be easier to implement this way because it avoids the extra effort of converting to quaternions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I used the word quaternion was because I thought that it was doing something like that in the atmosphere while researching various things. I'm not sure how it would actually be implemented. However, I got the impression that it would take a lot of effort to convert between them.
It doesn't bother me that p5.js doesn't implement quaternions either. If we can handle it with matrices, I thought it would be simpler unless there was a big problem.
src/webgl/p5.Camera.js
Outdated
]; | ||
|
||
// Multiply this to mat0 from left to get the interpolated front vector. | ||
const newFrontX = result[0] * m0[2] + result[1] * m0[5] + result[2] * m0[8]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could maybe add a multiplyVec3
to p5.Matrix
to use here (again only if we use p5.Matrix)
The implementation that gets the components instead of creating a new vector is for performance, but if it's easier to understand, I'll rewrite it. |
I would add two suggestions regarding 3x3 matrix.
I would like to implement the above two. |
Sounds good to me! |
Added methods for 3x3 matrices. In addition, change the specification so that copy() can be used with 3x3 New specifications for transpose3x3 when there are no arguments
remove trailing comma
Make the argument of copy obtained by slice instead of array
add mat3 method to original copy() specification
Direct assignment seems to be safer, so let's rewrite it.
Changed transpose3x3 to return itself transposed if there is no argument. |
test for unit test of p5.Matrix 3x3. copy() for 3x3Matrix.
Let's make a unit test for a 3x3 matrix. If it doesn't work, I give up for this. |
This seems fine, so I'll add more. |
The matrix methods are looking good! |
add other tests for 3x3Matrix
fix indent
fix transpose to transpose3x3
Unit testing is over. Rewrite the method using matrix operations. |
About multiplyVec3(), I add a method that takes a target as an argument. |
Changed to take target as an argument. If there is no target, prepare a copy of the vector to be multiplied.
If there is no target, prepare a copy of the vector to be multiplied. |
If there is a target, set result and return that.
Change title to "implement p5.Camera.slerp(), 3x3Matrix functions and minor spec-change of orbitControl()". |
rewrite with vectors and 3x3Matrix functions
do not use lerp
Requested rewrite completed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updates look great! thanks for adding all the new matrix methods!
Thanks for review and merge! |
Great work @inaridarkfox4231! Thanks @davepagurek for reviewing! |
Resolves #6247
Introduces a new function, slerp(), to p5.Camera. It interpolates the views of the two camera arguments.
Interpolation is the so-called quaternion slerp() and is well known. However, since p5.js does not implement quaternions, it does interpolation directly using orthogonal matrix, which is effectively equivalent to quaternions.
However, when the so-called between-angle (corresponding to the difference of the orthonormal basis) is very small, linear interpolation is used instead. This is to avoid problems by classifying cases including that case because the unit matrix cannot take the axis. This is similar to quaternion interpolation, so I used it as a reference.
Besides that, I made a minor fix regarding orbitControl().
First, only uPMatrix is updated when scaling ortho(), but this does not update the camera's projMatrix. This causes problems when interpolating between ortho() cameras, so I rewrote it to update projMatrix as well.
Also, mouseWheelY is used for the mouse wheel interaction, but since this value depends on the model, there is a concern that using the direct value may cause a significant difference of behavior depends on machine. Therefore, we changed the specification to use only the sign.
The main changes are as above. Here is an example:
Example
p5.Camera.slerp() for PR
2023-07-03.23-03-54.mp4
The main usage is to smoothly switch between multiple cameras, or smoothly restore the camera state changed by orbitControl().
PR Checklist
npm run lint
passes