Implement load/updatePixels() and get() for framebuffers

[davepagurek]

Resolves #6082

This branches off of #5981 (since it also fixes WebGL pixels[] being inverted in the y axis), so we should review/merge that first.

Changes

• Adds support for loadPixels/pixels/updatePixels/get for p5.Framebuffer
  • These do not need to flip the y axis, since we already flip framebuffer content using its camera
  • Like get() for p5.Graphics, the resulting p5.Image will always have a pixel density of 1 even if the source content has higher density
• Adds support for updatePixels for WebGL main canvases or graphics
  • Each WebGL renderer has a framebuffer it uses and caches that it can update the pixels of and then draw to the canvas
  • Just using a p5.Texture would be a tad cheaper, so we could switch to that eventually if we want, but I suspect we'll want to use a framebuffer in the future for filter(). Happy to discuss though!

Screenshots of the change

Some example code drawing in 3D, updating pixels, then drawing some more: (Live version)

function setup() { createCanvas(400, 400, WEBGL); setAttributes({ antialias: true }) } function draw() { background(0); noStroke(); lights(); sphere(100); loadPixels(); for (let y = 200; y < height; y++) { for (let x = 0; x < width; x++) { const idx = (y * width + x) * 4; pixels[idx] = (x / width) * 255; pixels[idx + 1] = (y / height) * 255; pixels[idx + 2] = 255; pixels[idx + 3] = 255; } } updatePixels(); push(); translate(100, 100); rotateX(0.5); rotateY(0.5); box(); pop(); noLoop(); }

Some similar code using a framebuffer (used as an example for the docs of framebuffer's updatePixels): (Live version)

let framebuffer; function setup() { createCanvas(100, 100, WEBGL); framebuffer = createFramebuffer({ antialias: true }); } function draw() { noStroke(); lights(); framebuffer.begin(); background(0); sphere(25); framebuffer.end(); framebuffer.loadPixels(); for (let y = height/2; y < height; y++) { for (let x = 0; x < width; x++) { const idx = (y * width + x) * 4; framebuffer.pixels[idx] = (x / width) * 255; framebuffer.pixels[idx + 1] = (y / height) * 255; framebuffer.pixels[idx + 2] = 255; framebuffer.pixels[idx + 3] = 255; } } framebuffer.updatePixels(); framebuffer.begin(); push(); translate(20, 20); rotateX(0.5); rotateY(0.5); box(20); pop(); framebuffer.end(); image(framebuffer, -width/2, -height/2); noLoop(); }

PR Checklist

• npm run lint passes
• [Inline documentation] is included / updated
• [Unit tests] are included / updated

[aferriss]

This is awesome, thanks for putting this together @davepagurek . I left a couple notes on the PR.

One more thing that comes to mind is to check that if multiple fbo's are in the same sketch that calling loadPixels and updatePixels in arbitrary order still works as expected. For ex:

fbo1.loadPixels();
fbo2.loadPixels();
fbo2.updatePixels();
fbo1.updatePixels();

Another circumstance I wonder if we can catch is when someone tries to manipulate the pixel arrays or call updatePixels before loadPixels has been called. Perhaps a warning in those cases could help solve some programmer error.