deck.gl 9 upgrade

[xinaesthete]

I notice that deck.gl (and underlying luma.gl) have a major new version, with some interesting changes - in particular, moving towards WebGPU support. Better typescript support also sounds promising, and there are other features I’d definitely make fairly immediate use of that aren’t directly relevant to Viv. I’m really keen to start using this soon, but it will mean a few changes being needed to Viv.

I have a few other things I should be focussing on first, but if you want I’d be willing to do some work on a PR to upgrade. Some initial notes:

They have an upgrade guide here. It looks as though most of this should be relatively painless, although it’s hard to say without actually trying - as ever, the devil is in the details.

It means that anything directly referring to WebGL APIs needs to change to luma.gl abstractions. Any use of GL constants become strings (sounds nicer for debugging…). Create models with context.device instead of context.gl. WebGL1 is no longer supported at all - you/we could probably get rid of the bits of code with warnings for this - to my knowledge, this isn’t a concern now.

Shaders should require only fairly minimal changes

• deck.gl no longer supports GLSL 1.00 shaders. All shaders must be updated to GLSL 3.00.

No problems anticipated there, may need one or two very minor revisions.

• deck.gl now uses uniform buffers instead of global uniforms. It is not yet required to use uniform buffers but it will be necessary if you would like to run deck.gl on WebGPU in future releases.

Would need a bit of a change, hopefully for the better and not too difficult. Not needed immediately, anyway (but I do want to run on WebGPU ASAP).

• The luma picking module now uses uniform buffers. To access picking state in shaders use picking.isActive rather than picking_isActive

Not applicable to Viv AFAIK, shouldn't be a problem otherwise.

loaders.gl

There’s a major upgrade to v4 of this and I haven’t fully assessed the implications of that - hopefully not a big deal, but wouldn’t like to say.

In general, moderately likely that changing several major versions of dependencies may not be entirely without friction, but definitely worth looking at I think.

[ilan-gold]

Would be very interested in this! I am hoping to have more time to work on Viv starting in May! If you want to get started, great, but there are also others interested!

[xinaesthete]

Hi,

So I made a start on this... shock-horror: not as totally straight-forward as the upgrade guide made out. I'm not sure if I'll finish or not, but I'll share some notes on my progress & if you want to review the code, it's here

.meta_updater/main.mjs changed to the following:

const DECK_VERSION = '~9.0.17';
const LUMAGL_VERSION = '~9.0.14';
const MATHGL_VERSION = '^4.0.1';

Also manual change to Avivator's package.json.

build

A few more changes needed than anticipated;

• ProgramManager being deprecated ostensibly in favour of PipelineFactory but actually when it comes down to it, ShaderAssembler is what's needed for Viv (const programManager = ShaderAssembler.getDefaultShaderAssembler(); //ProgramManager.getDefaultProgramManager(device); - would make sense to also rename the variable in cases like that)
• import GL from '@lumagl/constants' needed to destructure import { GL } ... (but anywhere this is used probably needs to change because of the different way constants are handled, so things currently fail at runtime - this was anticipated).
• new Texture2D(device) needs to be device.createTexture({dimension: '2d', ...}) (similar for 3D), I'm not sure if dimension is necessarily required in all cases, and so far I haven't got as far as any visible rendering of textures on screen.

tests

Quite a lot of tests pass, but it hangs in certain places... I tried bypassing in some cases, but anyway - given that it is clearly broken at runtime it would've been a major red flag if tests passed :)

avivator dev

Avivator building ok & running in dev then I got an error in luma.gl when calling _getModel() in xr-layer.js, which in the course of trying to makeGPUGeometry gets to the following with an attribute object with undefined size.

        attributes[name] = device.createBuffer({ data: attribute.value, id: `${attributeName}-buffer` });
        const { value, size, normalized } = attribute;
        bufferLayout.push({ name, format: getVertexFormatFromAttribute(value, size, normalized) });

The type annotations indicate that attributes?: Record<string, GeometryAttribute | TypedArray>;, however, I found that it seems necessary to change e.g. in xr-layer.js:

        attributes: {
          texCoords: {
            value: new Float32Array([0, 1, 0, 0, 1, 0, 1, 1]),
            size: 4
          }
        }

Seems like it should have been allowed to have attributes: { texCoords: new Float32Array(...) } }, so I'm not sure if that's just a bug in luma.gl.

Now the first errors that I'm seeing appear to be to do with GL constants, but I'm sure there'll be more changes required after that.

other notes

It'd be good to update MaterialUI / React / Zustand etc, but I suppose it's probably sensible to keep that separate.

It might help to catch things like incompatible uses of GL constants if more of the code was in TS, but given the attributes behaviour I wouldn't want to bank on it.

Cheers,
Peter

[xinaesthete]

A couple more comments (I should really stop and get on with some other work soon/now).

I tried renaming xr-layer.js to xr-layer.ts and it does show up a lot of useful errors from the parts that it can infer, which'll probably useful while refactoring even if some things aren't fixed and you end up renaming back to .js. I'd definitely encourage using TS though and it does seem like they've mostly done a good job with that in v9.

[xinaesthete]

for ref https://github.com/visgl/luma.gl/blob/master/docs/legacy/porting-guide.md

[ilan-gold]

@xinaesthete Please open a PR and I'll try to help you out next week if that is ok.

[xinaesthete]

@xinaesthete Please open a PR and I'll try to help you out next week if that is ok.

That's great, I'll do that now, not necessarily assuming either of us'll necessarily have much time for it, but hopefully we can make at least some progress.