Load and run visual plugin (system) on GUI side #1275

iche033 · 2022-01-08T04:53:39Z

🎉 New feature

Related PR: gazebosim/gz-rendering#533 - Needed for running the shader_param.sdf demo world added in this PR, and also for shader_param_system integration test to pass.

Update: Fix material switching for objects using shaders (ogre2) gz-rendering#533 is merged and ign-rendering 6.2.0 is released. shader_param_system integration test now passes

Summary

This PR allows visual plugins to be loaded on the gui side.

Changes include:

adding a VisualPlugin component to store the <plugin> sdf element
- this component is sent from server to gui
adding a VisualPlugin gui event for GzSceneManager to notify GuiRunner that a new visual plugin is detected
extending GuiRunner to load and run systems (visual plugin)
- this can probably be refactored later to avoid duplicate code with SimulationRunner
adding a SceneUpdate event that systems can connect to on both the server and gui side for rendering
adding a ShaderParam visual plugin (ported from gazebo-classic) to test the whole pipeline

Test it

Run the shader_param.sdf world - this world contains a sphere model with the new ShaderParam system that modifies its appearance using custom shaders. Requires gazebosim/gz-rendering#533

ign gazebo -v 4  -r shader_param.sdf

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Signed-off-by: Ian Chen <[email protected]>

ahcorde · 2022-01-10T08:48:52Z

include/ignition/gazebo/components/Visual.hh

+#include <sdf/Element.hh>
+#include <sdf/parser.hh>


alphabetize

hmm I think this means alphabetize and also take into account case-sensitivity? If so, done in 9f4a753

include/ignition/gazebo/components/Visual.hh

include/ignition/gazebo/gui/GuiEvents.hh

ahcorde · 2022-01-10T08:51:49Z

src/gui/plugins/scene_manager/GzSceneManager.cc

 #include <set>

+#include "../../GuiRunner.hh"


is it fine to add relative paths ?

I think so. It's was done here as well: https://github.com/ignitionrobotics/ign-gazebo/blob/ign-gazebo6/src/gui/plugins/modules/EntityContextMenu.cc#L18

src/systems/shader_param/ShaderParam.cc

test/integration/shader_param_system.cc

test/plugins/TestVisualSystem.hh

chapulina · 2022-01-10T17:27:33Z

I believe this closes #657 and #265

Signed-off-by: Ian Chen <[email protected]>

codecov · 2022-01-21T03:00:13Z

Codecov Report

Merging #1275 (0dead05) into ign-gazebo6 (912e2ce) will decrease coverage by 0.19%.
The diff coverage is 26.27%.

@@               Coverage Diff               @@
##           ign-gazebo6    #1275      +/-   ##
===============================================
- Coverage        62.25%   62.06%   -0.20%     
===============================================
  Files              278      278              
  Lines            23203    23315     +112     
===============================================
+ Hits             14445    14470      +25     
- Misses            8758     8845      +87

Impacted Files	Coverage Δ
include/ignition/gazebo/gui/GuiEvents.hh	`0.00% <ø> (ø)`
include/ignition/gazebo/rendering/RenderUtil.hh	`100.00% <ø> (ø)`
src/SimulationRunner.hh	`100.00% <ø> (ø)`
src/gui/GuiEvents.cc	`44.44% <0.00%> (-12.70%)`	⬇️
src/gui/GuiRunner.hh	`100.00% <ø> (ø)`
src/gui/plugins/scene_manager/GzSceneManager.cc	`19.17% <0.00%> (-8.28%)`	⬇️
include/ignition/gazebo/components/Visual.hh	`12.50% <6.66%> (-87.50%)`	⬇️
src/gui/GuiRunner.cc	`51.03% <30.50%> (-16.00%)`	⬇️
src/SdfEntityCreator.cc	`85.51% <100.00%> (+0.17%)`	⬆️
src/rendering/RenderUtil.cc	`36.53% <100.00%> (+0.20%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 912e2ce...0dead05. Read the comment docs.

ahcorde

LGTM, just some minor style issues

src/gui/GuiRunner.cc

src/gui/plugins/scene_manager/GzSceneManager.cc

src/systems/shader_param/ShaderParam.cc

Signed-off-by: Ian Chen <[email protected]>

srmainwaring · 2022-02-07T23:27:44Z

@iche033 - just checked and that this works for Metal with gazebosim/gz-rendering#554 🙂

Here are the metal shaders:

deform_fs.metal.zip
deform_vs.metal.zip

This commit adds textures: srmainwaring@66a8490, then we can run the waves example

waves-gazebo.mov

ocean_waves.sdf.zip
ocean_waves.zip

* load and run visual plugins on gui end Signed-off-by: Ian Chen <[email protected]> * scene update event emitted on both server and gui side Signed-off-by: Ian Chen <[email protected]> * shader param update working Signed-off-by: Ian Chen <[email protected]> * sim time, constants, full example working Signed-off-by: Ian Chen <[email protected]> * add integration test Signed-off-by: Ian Chen <[email protected]> * code cleanup Signed-off-by: Ian Chen <[email protected]> * more code cleanup Signed-off-by: Ian Chen <[email protected]> * style fixes and add some comments Signed-off-by: Ian Chen <[email protected]> * review changes Signed-off-by: Ian Chen <[email protected]> * require display for shader param test Signed-off-by: Ian Chen <[email protected]> * style and comment Signed-off-by: Ian Chen <[email protected]>

iche033 · 2022-02-08T02:14:49Z

oh nice, looks like you forward ported this to Garden? I can update the shader param system to read metal shaders, e.g. something like:

<shader launguage="glsl">
  <vertex>path_to_vertex_shader.glsl</vertex>
  <fragment>path_to_fragment_shader.glsl</fragment>
</shader> 
<shader launguage="metal">
  <vertex>path_to_vertex_shader.metal</vertex>
  <fragment>path_to_fragment_shader.metal</fragment>
</shader>

This just lets user specify multiple shader files and the system will then select the best shader to use depending on the platform. How does that look?

iche033 · 2022-02-08T05:57:12Z

Implemented the above idea in the shader_param_metal branch. I'll create a PR once I can upload metal shaders to Fuel.

I've also created gazebosim/gz-rendering#558 to accomodate cases like gazebosim/gz-rendering#554 where extra params (shader_reflection_pair_hint) are needed for metal but not glsl

srmainwaring · 2022-02-08T09:24:23Z

oh nice, looks like you forward ported this to Garden?

Yes, just cherry-picked it in. Most of the Metal support needs Garden now because of various ABI breaking interface changes. Did you want me to post a PR for the texture add-on or will you pull the commit into the shader_param_metal branch? Either way is good for me.

Your proposed changes for multi-shader language support and extra param checks look good!

Appreciate everyone is super busy, would you mind asking the other reviewers for the Metal GUI PR if they could take a look? It would be nice to get it merged while it's passing CI clean and before it needs a rebase.

srmainwaring · 2022-02-08T13:56:24Z

@iche033 Just noticed you've already done all the texture changes in #1310! I should have looked more carefully...

osrf-triage · 2022-03-01T18:31:14Z

This pull request has been mentioned on Gazebo Community. There might be relevant details there:

https://community.gazebosim.org/t/new-ignition-releases-2022-03-01-citadel-edifice-fortress/1313/1

osrf-triage · 2022-03-02T20:47:32Z

This pull request has been mentioned on Gazebo Community. There might be relevant details there:

https://community.gazebosim.org/t/using-a-custom-hlms-with-ignition-rendering/1315/2

iche033 added 7 commits December 22, 2021 21:26

load and run visual plugins on gui end

7885195

Signed-off-by: Ian Chen <[email protected]>

scene update event emitted on both server and gui side

10150b1

Signed-off-by: Ian Chen <[email protected]>

shader param update working

324a63c

Signed-off-by: Ian Chen <[email protected]>

sim time, constants, full example working

eedc363

Signed-off-by: Ian Chen <[email protected]>

add integration test

4eec9b8

Signed-off-by: Ian Chen <[email protected]>

code cleanup

ff9b232

Signed-off-by: Ian Chen <[email protected]>

more code cleanup

258e63f

Signed-off-by: Ian Chen <[email protected]>

iche033 requested a review from chapulina as a code owner January 8, 2022 04:53

github-actions bot added the 🏯 fortress Ignition Fortress label Jan 8, 2022

iche033 mentioned this pull request Jan 8, 2022

Fix material switching for objects using shaders (ogre2) gazebosim/gz-rendering#533

Merged

7 tasks

iche033 added 2 commits January 7, 2022 20:54

Merge branch 'ign-gazebo6' into visual_plugin

431e73c

style fixes and add some comments

ea163a7

Signed-off-by: Ian Chen <[email protected]>

ahcorde requested changes Jan 10, 2022

View reviewed changes

chapulina added the needs upstream release Blocked by a release of an upstream library label Jan 10, 2022

iche033 added 3 commits January 10, 2022 15:51

review changes

9f4a753

Signed-off-by: Ian Chen <[email protected]>

Merge branch 'ign-gazebo6' into visual_plugin

f3c7f96

require display for shader param test

92b318c

Signed-off-by: Ian Chen <[email protected]>

This was referenced Jan 25, 2022

Extend ShaderParam system to support textures #1310

Merged

Prepare for 6.2.0 release gazebosim/gz-rendering#551

Merged

Merge branch 'ign-gazebo6' into visual_plugin

006b93c

iche033 removed the needs upstream release Blocked by a release of an upstream library label Jan 29, 2022

ahcorde approved these changes Jan 31, 2022

View reviewed changes

src/gui/GuiRunner.cc Show resolved Hide resolved

src/gui/plugins/scene_manager/GzSceneManager.cc Show resolved Hide resolved

src/systems/shader_param/ShaderParam.cc Outdated Show resolved Hide resolved

style and comment

0dead05

Signed-off-by: Ian Chen <[email protected]>

iche033 mentioned this pull request Feb 4, 2022

Add WaveVisual system osrf/mbzirc#35

Merged

iche033 merged commit 717a7e9 into ign-gazebo6 Feb 4, 2022

iche033 deleted the visual_plugin branch February 4, 2022 01:40

This was referenced Feb 9, 2022

Add Coast world osrf/mbzirc#38

Merged

Extend ShaderParam system to support loading different shader languages #1335

Merged

iche033 mentioned this pull request Feb 24, 2022

Support loading more than one visual plugin on GUI side #1363

Open

chapulina mentioned this pull request Mar 1, 2022

Add plugin to add shaders to a visual #657

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load and run visual plugin (system) on GUI side #1275

Load and run visual plugin (system) on GUI side #1275

iche033 commented Jan 8, 2022 •

edited

Loading

ahcorde Jan 10, 2022

iche033 Jan 10, 2022 •

edited

Loading

ahcorde Jan 10, 2022

iche033 Jan 10, 2022

chapulina commented Jan 10, 2022 •

edited

Loading

codecov bot commented Jan 21, 2022 •

edited

Loading

ahcorde left a comment

srmainwaring commented Feb 7, 2022 •

edited

Loading

iche033 commented Feb 8, 2022

iche033 commented Feb 8, 2022

srmainwaring commented Feb 8, 2022 •

edited

Loading

srmainwaring commented Feb 8, 2022

osrf-triage commented Mar 1, 2022

osrf-triage commented Mar 2, 2022

Load and run visual plugin (system) on GUI side #1275

Load and run visual plugin (system) on GUI side #1275

Conversation

iche033 commented Jan 8, 2022 • edited Loading

🎉 New feature

Summary

Test it

Checklist

ahcorde Jan 10, 2022

Choose a reason for hiding this comment

iche033 Jan 10, 2022 • edited Loading

Choose a reason for hiding this comment

ahcorde Jan 10, 2022

Choose a reason for hiding this comment

iche033 Jan 10, 2022

Choose a reason for hiding this comment

chapulina commented Jan 10, 2022 • edited Loading

codecov bot commented Jan 21, 2022 • edited Loading

Codecov Report

ahcorde left a comment

Choose a reason for hiding this comment

srmainwaring commented Feb 7, 2022 • edited Loading

iche033 commented Feb 8, 2022

iche033 commented Feb 8, 2022

srmainwaring commented Feb 8, 2022 • edited Loading

srmainwaring commented Feb 8, 2022

osrf-triage commented Mar 1, 2022

osrf-triage commented Mar 2, 2022

iche033 commented Jan 8, 2022 •

edited

Loading

iche033 Jan 10, 2022 •

edited

Loading

chapulina commented Jan 10, 2022 •

edited

Loading

codecov bot commented Jan 21, 2022 •

edited

Loading

srmainwaring commented Feb 7, 2022 •

edited

Loading

srmainwaring commented Feb 8, 2022 •

edited

Loading