Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SVG support for textures in GLTF specification #2338

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fire
Copy link

@fire fire commented Oct 17, 2023

We believe it is possible to have two implementations: one in three.js and another in Godot Engine.

I am opening this proposal for discussion. Please feel free to share your comments.

@donmccurdy
Copy link
Contributor

donmccurdy commented Oct 20, 2023

Is "SVG as texture" something that is done in engines today? three.js does have an SVG loader, but it generates triangulated geometry, usually as a one-time operation. While we could then draw that geometry to a texture, it may be inefficient as a way to solve the stated goals of this extension. Streaming greater detail at closer distances sounds difficult — more in the territory of OpenSubDiv — and I'm not sure how to implement that on SVG.

Crisp, legible text in a 3D scene is an important problem, and not easily handled in standard formats today. If that's a goal for glTF, I think it would be important that we either (a) offer quality and performance at least as good as SDF/MSDF fonts, or (b) provide the source text and layout hints so that implementations are free to improve over time. I'm worried that transmitting text as SVG would not satisfy either of those goals.

@fire
Copy link
Author

fire commented Oct 20, 2023

Storing the SVG allows for decomposition via importer-time scaling to a raster image, curve conversion to mesh geometry or (multichannel) signed distance fields textures. These method can match SDF/MSDF fonts in quality and performance. However, raw svg is resource-intensive and may not be feasible for real-time applications or systems with limited resources compared to its decomposed forms.

Making mesh geometry efficient is outside the scope of this pr. See meshoptimizer, msft lod and etc. However, it is possible to use a series of LODs to provide performance without using dynamic subdivision on the mesh.

While SVG might not be the optimal format for delivering beziers and raster embeds, its wide availability makes it a practical choice. Another format that is available is lottie for vector graphics animation.

For providing source text and layout hints for future improvements, storing intermediary SVG inputs might not be feasible, especially when they are procedural generated or involve fonts that may not be available on all systems. For reference, PDF/A archive ensures its files are self contained.

Having yet another 2d vector standard inside of GLTF is possible too.

@fire
Copy link
Author

fire commented Oct 23, 2023

Lottie is in the process of standardization with the Linux Foundation group.

@aaronfranke
Copy link
Contributor

@donmccurdy SVG stands for Scalable Vector Graphics. It is also a 2 dimensional format. While you could triangulate it as a mesh, and another extension could do that, the simplest and most natural way to interpret a 2D graphic is an image.

@elalish
Copy link
Contributor

elalish commented Feb 14, 2024

Personally, I find this most interesting in conjunction with animated textures - this is why we started supporting Lottie animations in <model-viewer>: https://modelviewer.dev/examples/scenegraph/#animatedTexturesExample

For static textures, vector graphics have all the problems @donmccurdy points out, since with traditional rendering you would rasterize the graphic at a fixed resolution ahead of time. However, for animated textures, you are re-rasterizing the textures every frame anyway, which gives potential to also adjust the resolution on the fly. It'll still be resource intensive though.

I know various parties have interest in a video texture extension for glTF - maybe that would be a good place to start talking about the potential for vector formats.

@fire
Copy link
Author

fire commented Feb 14, 2024

I have greatly supported @hermet on thorvg, he is working on standardizing lottie. Perhaps there is interested in a lottie gltf extension.

@hermet
Copy link

hermet commented Feb 15, 2024

Hello, in most practical scenarios, rendering SVG/Lottie scenes requires several rendering stages for composition, including blending, masking, and applying layered effects, etc which is significantly more complex than text / video drawing. This means that even if we attempt to draw the vector contents directly on the main surface, it would necessitate generating one or more intermediate frame buffers(textures) for these compositions. Definitely there are pros/cons for introducing texture-based approach, but from my experience, supporting svg/lottie is quite challenging and burdensome due to the extensive nature of its specifications also, as vector contents complexity increases, retaining the scene (texture) for Lottie/SVG can improve performance among various user scenarios.

I think it would be beneficial to discuss whether glTF is likely to support the Lottie feature, thorvg is the most powerful for Lottie support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants