Make shaders define what they get used for. #7256

Garima3110 · 2024-09-09T03:45:38Z

This is a work in progress addressing the issue ##6759

PR Checklist

npm run lint passes
Inline documentation is included / updated
Unit tests are included / updated

perminder-17 · 2024-09-09T19:32:20Z

Great job so far, Garima! The idea is that when you call shader(yourShader), it should only apply to fills, regardless of other settings, like the uStrokeShader uniform or whether lights() is used in the sketch.

Currently, methods like isLightShader()

p5.js/src/webgl/p5.Shader.js

Line 983 in 349b7d4

isLightShader() {

check which uniforms are defined, and if any are defined, the function returns true. This allows us to apply fill shaders when lights() is used in sketches.

One possible solution would be to remove these methods that check whether uniforms and attributes are defined. Instead, we can directly set the appropriate user shader based on the method the user calls, such as strokeShader(shader) for stroke shaders.

Also, we should modify the function _getRetainedFillShader() written here:

p5.js/src/webgl/p5.RendererGL.js

Line 1768 in 349b7d4

_getRetainedFillShader() {

We could probably return the user-defined fill shader if present, or light shader if no fill is set. I guess this should work for fill shaders since _getLightShader() is returned when lighting is enabled or a texture is present. That's it for fill shaders, I can leave more reviews for strokeShaders and imageShaders as we proceed further but it's pretty similar for those methods also. Thankyou so much for working on it.

perminder-17 · 2024-09-09T19:36:05Z

src/webgl/p5.Shader.js

@@ -1012,6 +1012,10 @@ p5.Shader = class Shader {
    return this.uniforms.uStrokeWeight !== undefined;
  }

+  isImageShader() {


I think we don't need this method because we don't want our imageShader(yourShader) method to work depending on some uniforms (uImageSampler for your case). We could probably take this one out.

perminder-17 · 2024-09-09T19:36:41Z

src/webgl/material.js

-    this._renderer._useNormalMaterial = false;
-  }
+  // Always set the shader as a fill shader
+  this._renderer.userFillShader = s;


Garima3110 · 2024-09-11T19:17:44Z

Thanks for the suggestions @perminder-17 !
I have made the suggested changes, and the fill shaders seem to be fixed. Please have a look , if everything seems fine I'll work further with the docs and other shaders too.
Thankyou!

perminder-17 · 2024-09-12T09:11:38Z

src/webgl/p5.RendererGL.js

    }
-    return fill;
+    if (this._tex) {


Can we use the OR operator here and combine it into a single if statement? Something like:

if(this._enableLighting || this._tex) return this._getLightShader();

Can this make the code cleaner instead of using two separate if blocks?

perminder-17 · 2024-09-12T09:38:53Z

Great work @Garima3110 so far. I tested it, and it seems to be working well. But to be double sure, I'll test it again to make sure nothing is breaking. In the meantime, you should focus on completing the documentation for the fill shader to meet the first week's target.

The documentation should include: (these are just my thoughts, feel free to add your own explanations if you have any).

A description stating that the fill shader applies only to fills. (explain it in your way)
Reference links to other shaders (strokeShader, imageShader) with a brief explanation, just one line for each of them.
2-3 example sketches for the tutorial, showing how when lights() are enabled, it returns this._getLightShader() (which is a light shader). When lights() are not enabled, it uses the fill shader and returns the fill. Don't go too deep into p5.js's internal system, but give users a basic idea of what’s happening at the backend as well.

Also, talking about the strokeShaders here's the method created

p5.js/src/webgl/p5.Shader.js

Line 1020 in 12e6276

isStrokeShader() {

You can remove the method and update the function here as well: (just you did for fill shaders)

p5.js/src/webgl/p5.RendererGL.js

Line 1710 in 4a2f704

_getImmediateStrokeShader() {

and then we can move to the docs of strokeShader() as well.

Let me know if strokeShader() works for you.

I think we also need to alter some tests created for shaders, will update you for the tests as we completes the docs and code of this project.

Thanks again for your work. :)

Garima3110 · 2024-09-12T11:30:47Z

Thanks for the follow up @perminder-17
I'll work on these and get back to you soon!

Garima3110 · 2024-09-13T07:49:01Z

I have done all changes in docs and examples for fill shaders, please have a look @perminder-17
And just to mention I'm starting the work for strokeShader() too.

perminder-17 · 2024-09-13T15:49:11Z

Great work @Garima3110 so far. I tested it, and it seems to be working well. But to be double sure, I'll test it again to make sure nothing is breaking. In the meantime, you should focus on completing the documentation for the fill shader to meet the first week's target.

Just an update, previously i mentioned the wrong block of code by mistake. I just saw when you pushed it you need to remove the isStrokeShader() code. I have edited the above comment now.

p5.js/src/webgl/p5.Shader.js

Line 1020 in 12e6276

isStrokeShader() {

perminder-17 · 2024-09-14T20:10:27Z

src/webgl/p5.Shader.js

@@ -579,9 +579,6 @@ p5.Shader = class Shader {
    const modelViewProjectionMatrix = modelViewMatrix.copy();
    modelViewProjectionMatrix.mult(projectionMatrix);

-    if (this.isStrokeShader()) {
-      this.setUniform('uPerspective', this._renderer._curCamera.useLinePerspective ? 1 : 0);


I think we use uPerspective uniform in our line.vert file here:

p5.js/src/webgl/shaders/line.vert

Line 31 in 4fbbaf5

uniform int uPerspective;

So, we could keep them without the if block as well. Otherwise we would not be able to use uPerspective unfiorm and could throw errors.

perminder-17 · 2024-09-14T20:13:46Z

src/webgl/material.js

+ * process each pixel on the canvas. This only affects the fill (the inside part of shapes),
+ * not the strokes (outlines).
+ * If you want to apply shaders to strokes or images, use the following methods:
+ * - **[strokeShader()](#/p5/strokeShader)**: Applies a shader to the stroke (outline) of shapes, allowing independent control over the stroke rendering using shaders.


Okay, I was thinking to add these references at the last.
Like at first we could have our fill shader explanation.
At the middle we could have our example sketches how it works.
After the sketches we could have the references of strokeShader() and imageShader(). Let me know if that sound's okay to you?

Yeah that too makes sense, I'll do that no worries!

perminder-17 · 2024-09-14T20:28:45Z

As we're making breaking changes in this PR, we’ll end up with two distinct methods for applying shaders: strokeShader() and imageShader(). I'm considering whether we should also rename shader() to fillShader() for consistency. I'd like to get the WebGL stewards' thoughts on this. Tagging: @davepagurek @lukeplowden @stalgiag @aferriss @aceslowman, @ShenpaiSharma, @teragramgius, @JazerUCSB, @richardegil, @itsjoopark, @Gaurav-1306, @jeanetteandrews

The advantage of renaming it would be creating a clearer, more defined structure: fillShader() for fills, strokeShader() for strokes, and imageShader() for images. However, a potential downside is that users have grown accustomed to using shader() over the years, especially since fill shaders are used most often. This change might cause some confusion for long-time users. What are your thoughts on this?

Here's how it looks as of now:

Screencast.from.15-09-24.01.13.59.AM.IST.webm

Garima3110 · 2024-09-14T21:19:04Z

Thank you for raising this discussion @perminder-17 !

Well, I would also like to add something to your suggestion, I see the potential value in renaming shader() to fillShader() for consistency alongside strokeShader() and imageShader(). However, I believe it would be better to keep the name shader() as it is. My reasoning is that in p5.js, we often retain a feature's basic name based on its default behavior, without explicitly naming it according to that behavior.
The shader() function naturally applies to fills by default, and users are familiar with this behavior. Renaming it could introduce unnecessary complexity or confusion, especially for those who have been using p5.js for a long time. By keeping shader() intact, we maintain simplicity while introducing strokeShader() and imageShader() for more specific cases.

This approach aligns with how we’ve handled similar features, where the core functionality remains intuitively named, while specialized methods are introduced for distinct use cases.
I’d love to hear your thoughts on this approach and whether this strikes a good balance between clarity and consistency.
Thankyou!

lukeplowden · 2024-09-15T09:19:17Z

Looks good @perminder-17 !

I agree with @Garima3110, I don't think shader() should be changed to fillShader(). It's breaking, but I also think it's better to leave the name general because shader() may be used for things which aren't really 'fills'. For example you don't have to draw the results of a shader() call directly to the screen inside of p5.Framebuffer.draw(), it could be a post processing pass, or it could be related to instancing using endShape(). Eventually they might affect fill colour but I think the general function name is more intuitive.

With imageShader() is the idea to affect image() calls? I'm curious to see how that one will work!

davepagurek · 2024-09-15T10:57:20Z

I think I also agree with @lukeplowden and @Garima3110. Maybe the way to think about it is not in terms of fills and strokes, but in terms of materials and strokes. I suppose fill is actually just one component of an object's material, in addition to other properties like texture or shininess. Strokes are a separate system which adds something on top of the base material. In that view of everything, the material system is the main one you interact with and includes the most stuff, so treating it like the default in its naming feels right to me.

One thing that's not super clean about the model I just described is that images are kind of just a separate system, and regular objects that have a material and stroke can't go through the image system, and likewise, images don't get materials and strokes. I think that's fine, since we want images to always draw an image, so making overriding of image() very separate and explicit probably will help avoid accidental overrides and confusion. So I see image shaders as maybe more of a convenient way for developers to change small bits about image drawing and still take advantage of the fit and fill options of image() rather than a core part of the material/stroke system I guess.

…aders

perminder-17 · 2024-09-19T00:40:24Z

Thankyou @lukeplowden @davepagurek and @Garima3110 for your thoughts.

With imageShader() is the idea to affect image() calls? I'm curious to see how that one will work!

We are now planning to create a method that will only affect image calls. In simple terms, we are separating our methods because in many scenarios, we have seen that sometimes fill gets applied to images and sometimes it doesn't, which causes problems and confusion see here: #6327 , #6564 . To avoid accidental shaders being applied and also to avoid confusion, we will create a new method called imageShader() that will specifically handle shaders for images called through the image() function.

I’d also like your thoughts on something related to the question, which may seem obvious but I want to confirm. Do you think the image() function should be influenced by shader()? Or, put simply, should shader() ignore image() calls?

perminder-17 · 2024-09-19T21:00:05Z

Okay, @Garima3110 currently, the imageShader is not working. As we discussed on the call, we need to troubleshoot some functions.

For example, if you see the code

p5.js/src/webgl/p5.Shader.js

Line 1374 in 5fbde13

isTextureShader() {

we check if this.samplers.length > 0 so, when it returns true that means we have our image in our shader and we are currently using it (by calling uniform sampler2D uSampler and also using it).

For now, it’s worth disabling the fill shader effect for images, as @davepagurek's inclination was also towards not using fill shader for effecting image. For doing that, We can update the function

p5.js/src/webgl/p5.RendererGL.js

Line 1768 in 5fbde13

_getRetainedFillShader() {

by adding an extra condition which could handle isTextureShader().

Then, for imageShader() we can create a new function to return _getLightShader() when isTextureShader() is true.

Also, we could add example of by shaderHooks for only strokeShader() by studying this interesting tutorial. #7149 (comment) .

perminder-17 · 2024-09-19T21:02:41Z

src/webgl/material.js

+ * The shader will be used for:
+ * - Strokes only, regardless of whether the uniform `uStrokeWeight` is present.
+ *
+ * @method strokeShader


A question @davepagurek, since we need to add docs for strokeShader utilising shaderHooks. I see we have a method with same name strokeShader in shaderHooks as well. Do you think the method created in shaderHook PR will override this method (because they are with the same name)?

Garima3110 · 2024-09-19T22:19:38Z

Thank you for bringing this up! @perminder-17

I’d also like your thoughts on something related to the question, which may seem obvious but I want to confirm. Do you think the image() function should be influenced by shader()? Or, put simply, should shader() ignore image() calls?

From my perspective, the motivation behind introducing a separate imageShader() method is to bring greater clarity and control to the way shaders are applied in p5.js. Specifically, this new method would target images drawn using the image() function, thereby avoiding the confusion and unintended behavior that can arise when shader() affects image rendering.

Considering the issues mentioned (#6327, #6564), I would say that the shader() function should not influence image() calls. Instead, shader() should be reserved for fills, and strokeShader() should handle strokes. Meanwhile, the newly proposed imageShader() method, which me and @perminder-17 are currently working on, would exclusively manage shaders, for images drawn with the image() function.
I would also like to know what others have to say on this one!
Thankyou.

davepagurek · 2024-09-20T15:37:40Z

Ahh @perminder-17 you're right, those methods are going to clash, so we'll have to rename something. Maybe the hooks ones should be renamed to defaultStrokeShader() instead of just strokeShader()? It's a little verbose, maybe someone else has a better name idea?

perminder-17 · 2024-09-20T15:40:10Z

src/webgl/p5.RendererGL.js

-      }
-    } else if (!fill /* || !fill.isColorShader()*/) {
-      return this._getColorShader();
+    if(fill && !fill.isTextureShader()){


aah....._getRetainedFillShader() is for shapes, not images. You need to modify getImmediateFillShader(), specifically where we are checking

if (this._tex) { ..... }

to properly handle images.

perminder-17 · 2024-09-20T15:46:45Z

src/webgl/p5.RendererGL.js

+  _getRetainedImageShader() {
+    const imageShader = this.userImageShader;
+    if (imageShader && imageShader.isTextureShader()) {
+      return this._getLightShader(); // Return light shader if the shader is a texture shader
+    }
+    return imageShader || this._getColorShader(); // Default to color shader if no image shader is defined
+  }
+


no need for these many conditions.
Just you have to do is:
If a texture is present, it then checks if the imageShader is either missing or not set up for handling textures. If that’s the case, use light shader. Otherwise, it simply returns the existing imageShader.

Also, you should rename it to getImmidiateImageShader

p5.js/src/webgl/p5.RendererGL.Immediate.js

Line 510 in e624307

p5.RendererGL.prototype._drawImmediateFill = function(count = 1) {

Also, we need to create something like _drawImmediateFill for images where we could use _getImmidiateImageShader in the function to make it working.

perminder-17 · 2024-09-20T15:54:56Z

defaultStrokeShader

How about lineShader() @davepagurek ? For hooks?

davepagurek · 2024-09-20T15:57:10Z

That also works for me. I guess the stroke one is the only naming conflict because the shader used internally for images is the material shader, and we're just using shader() to set that?

lukeplowden · 2024-09-20T16:02:16Z

defaultStrokeShader

How about lineShader() @davepagurek ?

I don't think I mind it, but it could be confusing to have two terms for stroke. I noticed that you actually wrote lineShader() here @davepagurek

However couldn't strokeShader() both set the active stroke shader and return it? Given that the hooks version doesn't take any parameters

davepagurek · 2024-09-20T16:07:15Z

oops good catch @lukeplowden, I guess if I can confuse the two already then maybe that's a sign they'd be too similar.

However couldn't strokeShader() both set the active stroke shader and return it? Given that the hooks version doesn't take any parameters

Normally when we do that, it's for something that acts as both a setter and a getter, so if we do that here, I'd expect it to return whatever the active stroke shader is, which might be something set by users. We probably will still want a reliable way to get the default one though. Maybe that could mean defaultStrokeShader() always returns the default, strokeShader() with no args returns the active stroke shader (or the default at first, when a custom one isn't set), and strokeShader(yourShader) would set it?

lukeplowden · 2024-09-20T16:21:47Z

That’s what I was imagining, but then again its also not ideal as it doesn’t align well with the other shader hook functions materialShader(), normal etc which can’t be set in the same way.

lineShader() may just be better

davepagurek · 2024-09-20T16:44:13Z

yeah I think if we rename this one we'd probably also want to rename the others to have the same naming convention, defaultColorShader() and defaultMaterialShader() maybe? Or to be less verbose, base instead of default, e.g. baseMaterialShader?

Garima3110 · 2024-09-20T20:31:34Z

Well. I find all three lineShader() , defaultStrokeShader() and baseStrokeShader() suitable for the names, but my suggestion would be, maybe using baseStrokeShader would be a good name here?!!

perminder-17

Maybe I am not technically 100% correct but what I feel is we are separating our shader in two parts (one with without image and one that will only handle image).

Since there's no method defined for images which you are using in the codebase and it throws errors. Also, there's a need to pass the function which you have made for images here:

p5.js/src/webgl/p5.RendererGL.Immediate.js

Line 195 in e624307

p5.RendererGL.prototype.endShape = function(

I am not 100% sure what could be the condition for that, but I think you can figure it out. That's the main reason why imageShader is still not working for you.

perminder-17 · 2024-09-20T23:30:55Z

src/webgl/p5.RendererGL.Immediate.js

+p5.RendererGL.prototype._drawImmediateImage = function(count = 1) {
+  const gl = this.GL;
+  let shader = this._getImmediateImageShader();
+  this._setImageUniforms(shader);


If I run npm run build this is throwing errors.

I don't think we currently have any method saying _setImageUniforms? Can you please check. Maybe it should be setFillUniform??

perminder-17 · 2024-09-20T23:36:51Z

src/webgl/p5.RendererGL.Immediate.js

+  let shader = this._getImmediateImageShader();
+  this._setImageUniforms(shader);
+
+  for (const buff of this.immediateMode.buffers.image) {


is this.immediateMode.buffers.image is defined??

Nice catch it wasn't, I'll define it in the next code push. Thanks!

perminder-17 · 2024-09-20T23:37:31Z

src/webgl/p5.RendererGL.Immediate.js

+  shader.disableRemainingAttributes();
+  this._setTexUniforms(shader);
+  this._applyColorBlend(
+    this.curImageColor,


should this be going to this.curImageColor. I think that won't work, you need to use fill only for this.

well I think If we want separate control over the color of filled shapes and the color (or tint) of images, then keeping curImageColor might be necessary.
But lets just use fill color for the time being.

perminder-17 · 2024-09-20T23:38:33Z

src/webgl/p5.RendererGL.Immediate.js

+  this._setTexUniforms(shader);
+  this._applyColorBlend(
+    this.curImageColor,
+    this.immediateMode.geometry.hasImageTransparency()


hasImageTransparency() should be hasFillTransparency. It would be really nice of you if could please check once, are these things for e.g. (hasImageTransparency()) are defined in the codebase or not.

shouldn't we have a separate hasImageTransparency() and _setImageUniforms() defined to have more control for imageShaders unlike just using the predefined fill methods here as you suggested?!
Or may be for the time being we can just test the working of imageShaders with predefined hasFillTransparency() and _setFillUniforms...
Let me know your thoughts on this @perminder-17 ?!!

Garima3110 · 2024-09-21T20:22:32Z

Thanks @perminder-17 for the insights,I’m working on these changes ,will push the code super soon..!

Basic methods written and other related changes done

6421777

perminder-17 reviewed Sep 9, 2024

View reviewed changes

Updated code for fill shaders

621dbcb

perminder-17 reviewed Sep 12, 2024

View reviewed changes

Fill shader documentation+code examples complete

c696a4a

Qianqianye added the p5.js 2.0 label Sep 13, 2024

Changes in Stroke shaders

1f64472

perminder-17 reviewed Sep 14, 2024

View reviewed changes

Garima3110 and others added 3 commits September 15, 2024 19:35

Stroke shaders documentation and code changes done

955fcc5

Merge branch 'dev-2.0' of https://github.com/processing/p5.js into sh…

6fbd3e0

…aders

Merge branch 'dev-2.0' into shaders

937d073

perminder-17 reviewed Sep 19, 2024

View reviewed changes

Changes in Image shaders

bb39efe

perminder-17 reviewed Sep 20, 2024

View reviewed changes

Image shaders changes

2f78147

perminder-17 reviewed Sep 21, 2024

View reviewed changes

Garima3110 added 3 commits September 22, 2024 10:52

Minor fixes

d6211f3

fix

2cd7b03

Errors fixed

d5d97a2

Make shaders define what they get used for. #7256

Are you sure you want to change the base?

Make shaders define what they get used for. #7256

Conversation

Garima3110 commented Sep 9, 2024

PR Checklist

perminder-17 commented Sep 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Garima3110 commented Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

perminder-17 commented Sep 12, 2024 • edited Loading

Garima3110 commented Sep 12, 2024

Garima3110 commented Sep 13, 2024

perminder-17 commented Sep 13, 2024 • edited Loading

perminder-17 Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perminder-17 commented Sep 14, 2024

Garima3110 commented Sep 14, 2024

lukeplowden commented Sep 15, 2024 • edited Loading

davepagurek commented Sep 15, 2024

perminder-17 commented Sep 19, 2024

perminder-17 commented Sep 19, 2024

perminder-17 Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

Garima3110 commented Sep 19, 2024

davepagurek commented Sep 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perminder-17 commented Sep 20, 2024 • edited Loading

davepagurek commented Sep 20, 2024

lukeplowden commented Sep 20, 2024

davepagurek commented Sep 20, 2024

lukeplowden commented Sep 20, 2024

davepagurek commented Sep 20, 2024

Garima3110 commented Sep 20, 2024 • edited Loading

perminder-17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Garima3110 Sep 22, 2024 • edited Loading

Choose a reason for hiding this comment

perminder-17 Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Garima3110 commented Sep 21, 2024

perminder-17 commented Sep 9, 2024 •

edited

Loading

Garima3110 commented Sep 11, 2024 •

edited

Loading

perminder-17 commented Sep 12, 2024 •

edited

Loading

perminder-17 commented Sep 13, 2024 •

edited

Loading

perminder-17 Sep 14, 2024 •

edited

Loading

lukeplowden commented Sep 15, 2024 •

edited

Loading

perminder-17 Sep 19, 2024 •

edited

Loading

perminder-17 commented Sep 20, 2024 •

edited

Loading

Garima3110 commented Sep 20, 2024 •

edited

Loading

Garima3110 Sep 22, 2024 •

edited

Loading

perminder-17 Sep 20, 2024 •

edited

Loading