Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

core module UBO #9040

Merged
merged 10 commits into from
Jul 24, 2024
Merged

core module UBO #9040

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5f2b823
Define screen-pass-uniforms
felixpalmer Jul 23, 2024
f41b893
Connect up screenUniforms
felixpalmer Jul 23, 2024
7ba1b02
Tidy
felixpalmer Jul 23, 2024
e7ef730
Improve type
felixpalmer Jul 23, 2024
f2113c4
Add interpolationUniforms
felixpalmer Jul 23, 2024
c7a559d
Tidy
felixpalmer Jul 23, 2024
1a5fd9b
Add springUniforms
felixpalmer Jul 23, 2024
33864f7
Remove uniforms
felixpalmer Jul 23, 2024
dac365d
Tidy
felixpalmer Jul 23, 2024
404cc67
Merge branch 'master' into felix/core-ubo
felixpalmer Jul 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions modules/core/src/effects/post-process-effect.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ export default class PostProcessEffect<ShaderPassT extends ShaderPass> implement
outputBuffer = target;
}
const clearCanvas = !renderToTarget || Boolean(params.clearCanvas);
const moduleSettings = {};
const moduleProps = {};
const uniforms = this.module.passes[index].uniforms;
moduleSettings[this.module.name] = {...this.props, ...uniforms};
passes[index].render({clearCanvas, inputBuffer, outputBuffer, moduleSettings});
moduleProps[this.module.name] = {...this.props, ...uniforms};
passes[index].render({clearCanvas, inputBuffer, outputBuffer, moduleProps});

const switchBuffer = outputBuffer as Framebuffer;
outputBuffer = inputBuffer;
Expand Down Expand Up @@ -76,7 +76,6 @@ function createPasses(device: Device, module: ShaderPass, id: string): ScreenPas
const FS_TEMPLATE_INPUTS = `\
#version 300 es
uniform sampler2D texSrc;
uniform vec2 texSize;

in vec2 position;
in vec2 coordinate;
Expand All @@ -89,14 +88,14 @@ const FILTER_FS_TEMPLATE = (func: string) => `\
${FS_TEMPLATE_INPUTS}
void main() {
fragColor = texture(texSrc, coordinate);
fragColor = ${func}(fragColor, texSize, coordinate);
fragColor = ${func}(fragColor, screen.texSize, coordinate);
}
`;

const SAMPLER_FS_TEMPLATE = (func: string) => `\
${FS_TEMPLATE_INPUTS}
void main() {
fragColor = ${func}(texSrc, texSize, coordinate);
fragColor = ${func}(texSrc, screen.texSize, coordinate);
}
`;

Expand Down
27 changes: 27 additions & 0 deletions modules/core/src/passes/screen-pass-uniforms.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import type {TextureView} from '@luma.gl/core';
import {ShaderModule} from '@luma.gl/shadertools';
import {UniformTypes} from '../shaderlib/misc/uniform-types';

const uniformBlock = `\
uniform screenUniforms {
vec2 texSize;
} screen;
`;

type ScreenBindingProps = {
texSrc: TextureView;
};

type ScreenUniformProps = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really didn't anticipate this type splitting, seems like you have to go through more hoops than I anticipated. Seeing how the system is used in practice is illuminating.

texSize: [number, number];
};

export type ScreenProps = ScreenBindingProps & ScreenUniformProps;

export const screenUniforms = {
name: 'screen',
fs: uniformBlock,
uniformTypes: {
texSize: 'vec2<f32>'
} as const satisfies UniformTypes<Required<ScreenUniformProps>>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think this double type assertion is undesirable. You said you felt it adds clarity. If it is only needed to add clarity in some cases of more "advanced usage" that is one thing. Vut if it always needs to be done I am not so sure I can feel good about it :)

Grist for the mill I suppose...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should review the ShaderModule type, the UniformsT & BindingsT generics don't seem to be that useful IMHO. I think it would be better to have the ShaderModule be generic on PropsT and UniformTypesT, as UniformsT and BindingsT can be obtained by filtering PropsT.

I've had a play and we could remove the nested const satisfies UniformTypes<..> by using the following filtering type helper:

type UniformValue = number | boolean | number[];
type FilterUniformKeys<T> = {[K in keyof T]: T[K] extends UniformValue ? K : never}[keyof T];
type UniformsOnly<T> = {[K in FilterUniformKeys<T>]: T[K]};

We then don't need the ScreenBindingProps & ScreenUniformProps types and can just do:

export const screenUniforms = {
  name: 'screen',
  fs: uniformBlock,
  uniformTypes: {
    texSize: 'vec2<f32>'
  } as const satisfies UniformTypes<UniformsOnly<ScreenProps>>
} as const satisfies ShaderModule<ScreenProps>;

Which could all be encapsulated in luma's ShaderModule definition

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that is some slick typescript.

  • You could perhaps test all this in deck by creating a compatible type or wrapper type for ShaderModule in deck.gl and once you have it working we upstream it?
  • A possible problem is to find a name that covers both "uniforms and bindings". But I suppose UniformsAndBindings is a possible choice.
  • We want to be able to explain that shader modules mapping semantic props to shader uniforms and bindings.
export type ShaderModule<
  PropsT extends Record<string, unknown> = Record<string, unknown>,
  UniformsAndBindingsT extends Record<string, UniformValue | BindingValue> = Record<string, UniformValue | BindingValue>
> = {

} as const satisfies ShaderModule<ScreenProps>;
18 changes: 12 additions & 6 deletions modules/core/src/passes/screen-pass.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type {Device, Framebuffer} from '@luma.gl/core';
import {ClipSpace} from '@luma.gl/engine';
import type {ShaderModule} from '@luma.gl/shadertools';
import Pass from './pass';
import {ScreenProps, screenUniforms} from './screen-pass-uniforms';

type ScreenPassProps = {
module: ShaderModule;
Expand All @@ -16,7 +17,7 @@ type ScreenPassRenderOptions = {
clearCanvas?: boolean;
inputBuffer: Framebuffer;
outputBuffer: Framebuffer | null;
moduleSettings: any;
moduleProps: ShaderModule['props'];
};

/** A base render pass. */
Expand All @@ -27,7 +28,7 @@ export default class ScreenPass extends Pass {
super(device, props);
const {module, fs, id} = props;
const parameters = {depthWriteEnabled: false, depthCompare: 'always' as const};
this.model = new ClipSpace(device, {id, fs, modules: [module], parameters});
this.model = new ClipSpace(device, {id, fs, modules: [module, screenUniforms], parameters});
}

render(params: ScreenPassRenderOptions): void {
Expand All @@ -49,10 +50,15 @@ export default class ScreenPass extends Pass {
*/
protected _renderPass(device: Device, options: ScreenPassRenderOptions) {
const {clearCanvas, inputBuffer, outputBuffer} = options;
const texSize = [inputBuffer.width, inputBuffer.height];
this.model.shaderInputs.setProps(options.moduleSettings);
this.model.setBindings({texSrc: inputBuffer.colorAttachments[0]});
this.model.setUniforms({texSize});
const texSize: [number, number] = [inputBuffer.width, inputBuffer.height];
const screenProps: ScreenProps = {
texSrc: inputBuffer.colorAttachments[0],
texSize
};
this.model.shaderInputs.setProps({
screen: screenProps,
...options.moduleProps
});
const renderPass = this.device.beginRenderPass({
framebuffer: outputBuffer,
parameters: {viewport: [0, 0, ...texSize]},
Expand Down
31 changes: 24 additions & 7 deletions modules/core/src/transitions/gpu-interpolation-transition.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type {Device} from '@luma.gl/core';
import {Timeline, BufferTransform} from '@luma.gl/engine';
import {fp64arithmetic} from '@luma.gl/shadertools';
import {fp64arithmetic, ShaderModule} from '@luma.gl/shadertools';
import {GL} from '@luma.gl/constants';
import Attribute from '../lib/attribute/attribute';
import {
Expand All @@ -13,6 +13,7 @@ import {
import {GPUTransitionBase} from './gpu-transition';

import type {InterpolationTransitionSettings} from '../lib/attribute/transition-settings';
import {UniformTypes} from '../shaderlib/misc/uniform-types';
import type {TypedArray} from '../types/types';

export default class GPUInterpolationTransition extends GPUTransitionBase<InterpolationTransitionSettings> {
Expand Down Expand Up @@ -94,7 +95,8 @@ export default class GPUInterpolationTransition extends GPUTransitionBase<Interp
t = easing(t);
}
const {model} = this.transform;
model.setUniforms({time: t});
const interpolationProps: InterpolationProps = {time: t};
model.shaderInputs.setProps({interpolation: interpolationProps});

this.transform.run({discard: true});
}
Expand All @@ -105,25 +107,39 @@ export default class GPUInterpolationTransition extends GPUTransitionBase<Interp
}
}

const uniformBlock = `\
uniform interpolationUniforms {
float time;
} interpolation;
`;

type InterpolationProps = {time: number};

const interpolationUniforms = {
name: 'interpolation',
vs: uniformBlock,
uniformTypes: {
time: 'f32'
} as const satisfies UniformTypes<Required<InterpolationProps>>
} as const satisfies ShaderModule<InterpolationProps>;

const vs = `\
#version 300 es
#define SHADER_NAME interpolation-transition-vertex-shader

uniform float time;
in ATTRIBUTE_TYPE aFrom;
in ATTRIBUTE_TYPE aTo;
out ATTRIBUTE_TYPE vCurrent;

void main(void) {
vCurrent = mix(aFrom, aTo, time);
vCurrent = mix(aFrom, aTo, interpolation.time);
gl_Position = vec4(0.0);
}
`;
const vs64 = `\
#version 300 es
#define SHADER_NAME interpolation-transition-vertex-shader

uniform float time;
in ATTRIBUTE_TYPE aFrom;
in ATTRIBUTE_TYPE aFrom64Low;
in ATTRIBUTE_TYPE aTo;
Expand All @@ -138,7 +154,7 @@ vec2 mix_fp64(vec2 a, vec2 b, float x) {

void main(void) {
for (int i=0; i<ATTRIBUTE_SIZE; i++) {
vec2 value = mix_fp64(vec2(aFrom[i], aFrom64Low[i]), vec2(aTo[i], aTo64Low[i]), time);
vec2 value = mix_fp64(vec2(aFrom[i], aFrom64Low[i]), vec2(aTo[i], aTo64Low[i]), interpolation.time);
vCurrent[i] = value.x;
vCurrent64Low[i] = value.y;
}
Expand Down Expand Up @@ -177,7 +193,7 @@ function getTransform(device: Device, attribute: Attribute): BufferTransform {
]
}
],
modules: [fp64arithmetic],
modules: [fp64arithmetic, interpolationUniforms],
defines: {
ATTRIBUTE_TYPE: attributeType,
ATTRIBUTE_SIZE: attributeSize
Expand All @@ -196,6 +212,7 @@ function getTransform(device: Device, attribute: Attribute): BufferTransform {
{name: 'aFrom', format: inputFormat},
{name: 'aTo', format: bufferLayout.attributes![0].format}
],
modules: [interpolationUniforms],
defines: {
ATTRIBUTE_TYPE: attributeType
},
Expand Down
37 changes: 30 additions & 7 deletions modules/core/src/transitions/gpu-spring-transition.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type {Device, Framebuffer, Texture} from '@luma.gl/core';
import {Timeline, BufferTransform} from '@luma.gl/engine';
import {ShaderModule} from '@luma.gl/shadertools';
import {
padBuffer,
matchBuffer,
Expand All @@ -11,6 +12,7 @@ import Attribute from '../lib/attribute/attribute';
import {GPUTransitionBase} from './gpu-transition';

import type {SpringTransitionSettings} from '../lib/attribute/transition-settings';
import {UniformTypes} from '../shaderlib/misc/uniform-types';
import type {TypedArray} from '../types/types';

export default class GPUSpringTransition extends GPUTransitionBase<SpringTransitionSettings> {
Expand Down Expand Up @@ -80,10 +82,11 @@ export default class GPUSpringTransition extends GPUTransitionBase<SpringTransit
aCur: buffers[1]
});
transform.transformFeedback.setBuffers({vNext: buffers[2]});
transform.model.setUniforms({
const springProps: SpringProps = {
stiffness: settings.stiffness,
damping: settings.damping
});
};
transform.model.shaderInputs.setProps({spring: springProps});
transform.run({
framebuffer,
discard: false,
Expand All @@ -109,14 +112,33 @@ export default class GPUSpringTransition extends GPUTransitionBase<SpringTransit
}
}

const uniformBlock = `\
uniform springUniforms {
float damping;
float stiffness;
} spring;
`;

type SpringProps = {
damping: number;
stiffness: number;
};

const springUniforms = {
name: 'spring',
vs: uniformBlock,
uniformTypes: {
damping: 'f32',
stiffness: 'f32'
} as const satisfies UniformTypes<Required<SpringProps>>
} as const satisfies ShaderModule<SpringProps>;

const vs = `\
#version 300 es
#define SHADER_NAME spring-transition-vertex-shader

#define EPSILON 0.00001

uniform float stiffness;
uniform float damping;
in ATTRIBUTE_TYPE aPrev;
in ATTRIBUTE_TYPE aCur;
in ATTRIBUTE_TYPE aTo;
Expand All @@ -126,9 +148,9 @@ out float vIsTransitioningFlag;
ATTRIBUTE_TYPE getNextValue(ATTRIBUTE_TYPE cur, ATTRIBUTE_TYPE prev, ATTRIBUTE_TYPE dest) {
ATTRIBUTE_TYPE velocity = cur - prev;
ATTRIBUTE_TYPE delta = dest - cur;
ATTRIBUTE_TYPE spring = delta * stiffness;
ATTRIBUTE_TYPE damper = velocity * -1.0 * damping;
return spring + damper + velocity + cur;
ATTRIBUTE_TYPE force = delta * spring.stiffness;
ATTRIBUTE_TYPE resistance = velocity * spring.damping;
return force - resistance + velocity + cur;
}

void main(void) {
Expand Down Expand Up @@ -168,6 +190,7 @@ function getTransform(device: Device, attribute: Attribute): BufferTransform {
{name: 'aTo', format: attribute.getBufferLayout().attributes![0].format}
],
varyings: ['vNext'],
modules: [springUniforms],
defines: {ATTRIBUTE_TYPE: attributeType},
parameters: {
depthCompare: 'always',
Expand Down
Loading