Reference local schema files #535

jennuine · 2021-04-07T00:44:13Z

Signed-off-by: Jenn Nguyen [email protected]

🦟 Bug fix

Fixes #530

Summary

The xmlschema.rb script converts .sdf description files to xml schema (.xsd files) and now prints the contents of <xsd:include> instead of referencing files from http://sdformat.org/schemas/

Before the fix, running the integration test required internet connection. With the fix, the internet is no longer required.

In the build directory, delete the sdf/1.7 folder
Recompile
Run INTEGRATION_schema_test while disconnected to the internet

Checklist

Signed all commits for DCO
Added tests
Updated documentation (as needed)
Updated migration guide (as needed)
Code check passed (In source directory, run sh tools/code_check.sh)
All tests passed (See
test coverage)
While waiting for a review on your PR, please help review
another open pull request
to support the maintainers

Note to maintainers: Remember to use Squash-Merge

Signed-off-by: Jenn Nguyen <[email protected]>

azeey · 2021-04-07T19:53:05Z

sdf/1.7/light_state.sdf

@@ -1,5 +1,5 @@
 <!-- State information for a light -->
-<element name="light" required="*">
+<element name="light_state" required="*">


I'm not sure if we should be changing these names because they are actually used by Gazebo-classic. If I remember correctly, keeping the original names causes an issue with the generated xsd. I was hoping we can fix that issue without renaming these elements.

Renamed them back in aef3ab8

azeey · 2021-04-07T19:58:41Z

tools/xmlschema.rb

@@ -130,8 +135,7 @@ def printIncludeRef(_file, _spaces, _inc)

 #################################################
 def printInclude(_file, _spaces, _attr)
-  loc = "http://sdformat.org/schemas/"
-  loc += _attr.attributes['filename'].sub("\.sdf","\.xsd")
+  loc = _attr.attributes['filename'].sub("\.sdf","\.xsd")


I should have been more clear in the issue description in that we do still want to be able to have xsd files that can be stored on sdformat.org/schemas. At a glance, this looks like it will embed the absolute paths of the local machine in to the generated schemas. That's useful for tests, but I think we do still want this script to generate xsd that can be uploaded to the website.

I wonder if instead of using <xsd:include>, we just paste the contents and generate one xsd file that can be used for both testing and for the website. This might also solve the problem I mentioned in https://github.com/osrf/sdformat/pull/535/files#r609021282

Thanks for the tip! How's this? aef3ab8

Signed-off-by: Jenn Nguyen <[email protected]>

codecov-commenter · 2021-04-23T23:37:18Z

Codecov Report

Merging #535 (aef3ab8) into sdf9 (066b2fe) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             sdf9     #535   +/-   ##
=======================================
  Coverage   86.67%   86.67%           
=======================================
  Files          61       61           
  Lines        9531     9531           
=======================================
  Hits         8261     8261           
  Misses       1270     1270

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 066b2fe...aef3ab8. Read the comment docs.

Signed-off-by: Jenn Nguyen <[email protected]>

azeey

Looks pretty good so far. Just some minor comments to make pose work properly.

azeey · 2021-06-16T19:39:49Z

sdf/1.7/schema/types.xsd

@@ -26,12 +26,13 @@

  <xsd:simpleType name="pose">
    <xsd:restriction base="xsd:string">
-      <xsd:pattern value="(\s*(-|\+)?(\d+(\.\d*)?|\.\d+|\d+\.\d+[eE][-\+]?[0-9]+)\s+){5}((-|\+)?(\d+(\.\d*)?|\.\d+|\d+\.\d+[eE][-\+]?[0-9]+))\s*"/>
+      <xsd:pattern value="(\s*(-|\+)?(\d+(\.\d*)?|\.\d+|\d+\.\d+)([eE][-\+]?[0-9]+)?\s+){5}(-|\+)?(\d+(\.\d*)?|\.\d+|\d+\.\d+)([eE][-\+]?[0-9]+)?\s*"/>


We need to update this for 1.7 and up because <pose/> is now valid.

Updated 1.7 to accept <pose/>: 75b33b6

Also updated 1.5 & 1.6 regex for pose (does not accept <pose/>) and vector3d

azeey · 2021-06-16T19:47:32Z

tools/xmlschema.rb

+  doc = REXML::Document.new File.new(loc)
+
+  doc.elements.each_with_index("element") do |elem, i|
+    printXSD(_file, _spaces+2, elem)


I think if we should use printElem instead of printXSD here so that the pose element gets the pose type. Right now this is what it generates

<xsd:element name='pose'> <xsd:annotation> <xsd:documentation xml:lang='en'> <![CDATA[A position(x,y,z) and orientation(roll, pitch yaw) with respect to the frame named in the relative_to attribute.]]> </xsd:documentation> </xsd:annotation> <xsd:complexType mixed='true'> <xsd:attribute name='relative_to' type='xsd:string' > <xsd:annotation> <xsd:documentation xml:lang='en'> <![CDATA[ Name of frame relative to which the pose is applied. ]]> </xsd:documentation> </xsd:annotation> </xsd:attribute> </xsd:complexType> </xsd:element>

(1) We have <xsd:complexType mixed='true'> which indicate that <pose> can have child elements, which is not currently true
(2). The xsd type pose is not used at all so it is not validating based on the regex. You can try this by trying to validate 1.0` in an sdf file. It passes currently.

When using printElem, it generates:

<xsd:element name='pose'> <xsd:annotation> <xsd:documentation xml:lang='en'> <![CDATA[A position(x,y,z) and orientation(roll, pitch yaw) with respect to the frame named in the relative_to attribute.]]> </xsd:documentation> </xsd:annotation> <xsd:complexType> <xsd:simpleContent> <xsd:extension base='pose'> <xsd:attribute name='relative_to' type='xsd:string' > <xsd:annotation> <xsd:documentation xml:lang='en'> <![CDATA[ Name of frame relative to which the pose is applied. ]]> </xsd:documentation> </xsd:annotation> </xsd:attribute> </xsd:extension> </xsd:simpleContent> </xsd:complexType> </xsd:element>

This extends the pose type, so validation works properly.

Thanks! 75b33b6

azeey · 2021-06-16T19:49:02Z

sdf/1.7/CMakeLists.txt

-foreach(FIL ${sdfs})
-  get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
-  get_filename_component(FIL_WE ${FIL} NAME_WE)
+set(root root.sdf)


Can you make the same changes to the all the sdf/*/CMakeLists.txt?

75b33b6 I've made the changes for until 1.5 since 1.4 and older do not print out the schemas

azeey · 2021-06-16T19:50:17Z

tools/xmlschema.rb

@@ -72,13 +73,22 @@ def printElem(_file, _spaces, _elem)
      printDocumentation(_file, _spaces+2, _elem.elements["description"].text)
    end

-    _file.printf("%*s<xsd:complexType>\n", _spaces+2, "")
+    if _elem.attributes['name'] == "pose"


We shouldn't encounter pose here because it has type='pose'

removed 75b33b6

azeey · 2021-06-16T19:51:40Z

tools/xmlschema.rb

+     printDocumentation(_file, _spaces+2, _elem.elements["description"].text)
+    end
+
+    if _elem.attributes['name'] == "pose"


If we use printElem as I suggested, I'm not sure if we need a special treatment of pose here.

removed 75b33b6

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine · 2021-06-18T21:46:19Z

Unfortunately, this is not working for nested models 😞.. currently working on a solution

azeey · 2021-06-22T19:27:33Z

We are going to table this work for now and focus on higher priority tasks since we don't have an easy solution that works for nested models.

Signed-off-by: Jenn Nguyen <[email protected]>

azeey · 2024-08-19T16:58:39Z

#1455 might help with this, but we'll need to make several updates to this PR to take advantage of that. I don't think we'll be able to get to it with the limited time we have before the code freeze, so I'll go ahead and remove the beta label.

generate local schema files

5dd4749

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine requested a review from azeey April 7, 2021 00:44

github-actions bot added the 🏰 citadel Ignition Citadel label Apr 7, 2021

azeey reviewed Apr 7, 2021

View reviewed changes

paste contents of xsd:include in xsd files

aef3ab8

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine added 2 commits June 10, 2021 13:44

print contents of include & updated type regex

3abba02

Signed-off-by: Jenn Nguyen <[email protected]>

generate root.xsd only for 1.7

97e291a

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine requested a review from azeey June 10, 2021 21:11

Merge branch 'sdf9' into jennuine/local_schemas

463eb4a

azeey reviewed Jun 16, 2021

View reviewed changes

jennuine added 2 commits June 18, 2021 11:56

Merge branch 'sdf9' into jennuine/local_schemas

84c642c

Signed-off-by: Jenn Nguyen <[email protected]>

printElem, updated types.xsd & CMakeLists

75b33b6

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine marked this pull request as draft June 22, 2021 19:27

azeey mentioned this pull request Jul 23, 2021

naming conflicts in generated schema files (xsd) #632

Open

jennuine added 2 commits July 27, 2021 13:42

added nested model & state tests

6f32833

Signed-off-by: Jenn Nguyen <[email protected]>

world with state file

9b61cb9

Signed-off-by: Jenn Nguyen <[email protected]>

chapulina added the enhancement New feature or request label Jul 23, 2022

azeey added beta Targeting beta release of upcoming collection and removed beta Targeting beta release of upcoming collection labels Jul 31, 2023

azeey mentioned this pull request Nov 6, 2023

Using SDF schema for VSCode SDF validation and intellisense #1326

Open

azeey added the beta Targeting beta release of upcoming collection label Jul 29, 2024

azeey removed the beta Targeting beta release of upcoming collection label Aug 19, 2024

azeey mentioned this pull request Aug 19, 2024

BUG: resolve XSD aliasing and parse missed elements #643

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reference local schema files #535

Reference local schema files #535

jennuine commented Apr 7, 2021 •

edited

Loading

azeey Apr 7, 2021

jennuine Apr 23, 2021

azeey Apr 7, 2021

jennuine Apr 23, 2021 •

edited

Loading

codecov-commenter commented Apr 23, 2021 •

edited

Loading

azeey left a comment

azeey Jun 16, 2021

jennuine Jun 18, 2021

azeey Jun 16, 2021

jennuine Jun 18, 2021

azeey Jun 16, 2021

jennuine Jun 18, 2021

azeey Jun 16, 2021

jennuine Jun 18, 2021

azeey Jun 16, 2021

jennuine Jun 18, 2021

jennuine commented Jun 18, 2021

azeey commented Jun 22, 2021

azeey commented Aug 19, 2024 •

edited

Loading

Reference local schema files #535

Are you sure you want to change the base?

Reference local schema files #535

Conversation

jennuine commented Apr 7, 2021 • edited Loading

🦟 Bug fix

Summary

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jennuine Apr 23, 2021 • edited Loading

Choose a reason for hiding this comment

codecov-commenter commented Apr 23, 2021 • edited Loading

Codecov Report

azeey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jennuine commented Jun 18, 2021

azeey commented Jun 22, 2021

azeey commented Aug 19, 2024 • edited Loading

jennuine commented Apr 7, 2021 •

edited

Loading

jennuine Apr 23, 2021 •

edited

Loading

codecov-commenter commented Apr 23, 2021 •

edited

Loading

azeey commented Aug 19, 2024 •

edited

Loading