texImage2D is not working.

[tech4GT]

We have been trying to use headless-gl for some gl-based server side image processing here
publiclab/image-sequencer#964
, but the recommended way of uploading an Image into the texture is not working for us.
We could really use some guidance and possibly an example usage of a function.
Also it would be nice if someone can point us to the exact part of the code where the parameters are being consumed.
Thanks :)

[dhritzkiv]

See this comment. The last link should lead you to an example snippet

[harshkhandeparkar]

Looks like @Divy123 had opened a similar issue.

[tech4GT]

@dhritzkiv I did try the exact same code but for some reason that seems to not work for us, the output is always a black image.

[tech4GT]

Please have a look at this block of code https://github.com/publiclab/image-sequencer/pull/964/files#diff-0ed30b2ef28487bbffa9087b7aab8ea5R193

The output I am getting is this...

[harshkhandeparkar]

Jywarren had linked to this diff

https://github.com/stackgl/headless-gl/pull/53/files#diff-c24e210c8552e30550810a8b75aa1cdaR101

It seems like this diff is in a pr which isn't merged yet.

cc @tech4GT @dhritzkiv

[dhritzkiv]

A few things to try:

• Pull down this repo locally, run npm install, and then npm test. Check to see that your tests pass.
• as the last argument to gl.texImage2D, instead of using get-pixels try passing in a ArrayBuffer with some hard coded data: [255, 0, 0, 255, ...]

[dhritzkiv]

@harshkhandeparkar Ahh, good catch. Still, I believe texImage2D should be working… I'll check on it this evening.

[tech4GT]

@dhritzkiv Thanks a lot! I'll try this out and circle back here! :)

[harshkhandeparkar]

Is the latest version pushed to npm?

[dhritzkiv]

master branch is what's on npm right now

[tech4GT]

Okay I don't think ArrayBuffer data type is supported here

@dhritzkiv

[tech4GT]

btw I am using a macbook pro on macOS Mojave, could this be a platform issue?

[harshkhandeparkar]

Does it have a proper gpu? Not in-built?

[tech4GT]

Nope, it uses Intel Integrated graphics.

[harshkhandeparkar]

Maybe it doesn't have proper dependencies installed for inbuilt gpu?

[harshkhandeparkar]

Each vendor uses their own version of computing languages

[tech4GT]

OK, well in that case I'll this on a windows machine with a gpu.

[tech4GT]

Well the documentation does say there are no extra dependencies for macOS though.
https://github.com/stackgl/headless-gl#mac-os-x

[dhritzkiv]

The MacBook should be fine. The graphics layer on macOS is pretty stable and consistent.

Regarding ArrayBuffer not working, you should use the specific data type Uint8Array in your case (since you're using gl.UNSIGNED_BYTE)

[tech4GT]

Okay so I also tried installing the project locally and npm install generates this error

[harshkhandeparkar]

Maybe the node-gyp docs will help?

https://github.com/nodejs/node-gyp

[dhritzkiv]

@tech4GT Oops, One more step before running npm install locally: run git submodule update --init

[tech4GT]

aha! One test is failing now @dhritzkiv

[dhritzkiv]

Interesting. That sort of failure (no plan found) isn't indicative of a failure of the suite, but more of the test framework. It varies by machine. Unfortunately, I don't have a fix. Sometimes running it multiple times fixes it.

It actually seems as though some texImage2D suites passed (note the checkmark).

[tech4GT]

No, actually none of they say texImage2d. :/

[harshkhandeparkar]

Can you post a bigger screenshot?

[Divy123]

@dhritzkiv also can you suggest a good way to capture image data out of texture?

[dhritzkiv]

This is how I've gotten the image data out of the gl context (not quite the same, but might give you a place to start:

// render a scene or a texture

const pixels = new Uint8Array(4 * width * height);

gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

const buffer = Buffer.from(pixels.buffer);

await sharp(buffer, {
	raw: {
		width,
		height,
		channels: 4
	}
})
.flip()
.png()
.toFile(outputFileName);

Notes:

• sharp is a image manipulation package for nodejs
• you may be able to accomplish something like for textures with glGetTexImage and glGetBufferSubData
• this is not necessarily the most efficient technique

Sorry I can't be of more help -- I haven't done this stuff in a while and am quite rusty.

[harshkhandeparkar]

Can we get the data back as a buffer or a data-uri or a pixels array?

[jywarren]

Public Lab swarm! 😄 🙌 Thank you @dhritzkiv for being SO helpful! We really appreciate it!

@harshkhandeparkar would we just be able to read directly from pixels instead of using pixels.buffer to fetch it?

[robertleeplummerjr]

As an aside, this works fine in GPU.js.

[aboeglin]

So, do we know what is the status on this ? I have the exact same issue as OP, texture does not show up. Running on a macbook pro as well.