Changed default floating point precision to max #872

jennuine · 2022-03-09T03:10:53Z

🎉 New feature

Closes #790

Summary

Previously, there were different precisions as discussed here: #689 (review)

This PR changes the behavior so that all floating types emitted to the output stream are maximum precision by default for consistency. A new PrintConfig::SetOutPrecision option was added so that the precision can be configured as desired.

Test it

./path/to/build/test/integration/INTEGRATION_print_config --gtest_filter=PrintConfig.OutPrecision

or with the command line tool

test.sdf

<sdf version="1.9">
<model name="model1">
  <link name="link1">
    <pose>1 1.0 -1.570796 3.14159265358979323846 0 1.57079632679489667</pose>
  </link>
</model>
</sdf>

ign sdf -p test.sdf
# output (for pose): <pose>1 1 -1.5707960000000001 3.1415926535897931 0 1.5707963267948968</pose>

ign sdf -p --precision 6 test.sdf
# output: <pose>1 1 -1.5708 3.14159 0 1.5708</pose>

ign sdf -p --precision 2 test.sdf
# output: <pose>1 1 -1.6 3.1 0 1.6</pose>

This IEEE 754 floating point converter can be used to determine expected strings: https://baseconvert.com/ieee-754-floating-point

double (64 bit) has a max of 17 digits and float (32 bit) has a max of 9 digits.

There may be a small loss of precision (~2e-16), which is related to gazebosim/gz-math#212

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Signed-off-by: Jenn Nguyen <[email protected]>

codecov-commenter · 2022-03-09T03:34:39Z

Codecov Report

Merging #872 (54056bc) into main (6c9b4ad) will increase coverage by 0.09%.
The diff coverage is 100.00%.

❗ Current head 54056bc differs from pull request most recent head f822dfd. Consider uploading reports for the commit f822dfd to get more accurate results

@@            Coverage Diff             @@
##             main     #872      +/-   ##
==========================================
+ Coverage   88.14%   88.23%   +0.09%     
==========================================
  Files         102      102              
  Lines       14706    14693      -13     
==========================================
+ Hits        12962    12964       +2     
+ Misses       1744     1729      -15

Impacted Files	Coverage Δ
include/sdf/Param.hh	`77.94% <100.00%> (-0.64%)`	⬇️
src/Param.cc	`91.22% <100.00%> (-0.08%)`	⬇️
src/PrintConfig.cc	`100.00% <100.00%> (ø)`
src/ign.cc	`75.53% <100.00%> (+11.38%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c9b4ad...f822dfd. Read the comment docs.

scpeters · 2022-03-09T17:53:32Z

I see some test failures on macOS and windows in NestedModel.NestedModelWithFramesDirectComparison with an expected string not matching:

@@ -5,8 +5,8 @@
       <model name='M1'>
         <frame name='F1'>
-          <pose>0 0 0 1.5707959999999999 0 0</pose>
+          <pose>0 0 0 1.5707960000000001 0 0</pose>
         </frame>
         <frame name='F2'>
-          <pose relative_to='F1'>0 0 0 0 0.78539799999999993 0</pose>
+          <pose relative_to='F1'>0 0 0 0 0.78539800000000004 0</pose>
         </frame>
         <link name='L1'>
@@ -68,5 +68,5 @@
           </link>
         </model>
-        <pose>10 0 0 0 0 1.5708</pose>
+        <pose>10 0 0 0 0 1.5707999999999998</pose>
       </model>
     </model>

jennuine · 2022-03-09T19:00:22Z

I see some test failures on macOS and windows in NestedModel.NestedModelWithFramesDirectComparison with an expected string not matching:

hmm, so the values for macOS and windows were the values I was expecting but the values in linux were off by about ~2e-16. In a VC with @azeey we believed it was related to gazebosim/gz-math#212 since this loss doesn't happen with Vector3 or Color, which I left a comment here: https://github.com/ignitionrobotics/sdformat/blob/7a690da750bae9088512e21fc72b8781c66f3592/test/integration/nested_model.cc#L612-L613

I'm surprised that this loss doesn't happen in macOS or windows.. Instead of doing a direct comparison of sdfParsed and expected string, should I load the expected string into an sdf object, compare each element individually, and use EXPECT_NEAR for poses?

EricCousineau-TRI · 2022-03-10T19:17:09Z

Per VC, maybe related to -lfast-math, as in RobotLocomotion/drake#10253

Signed-off-by: Jenn Nguyen <[email protected]>

EricCousineau-TRI · 2022-04-04T14:16:44Z

Per VC, suggestions:

Should either store only translation, or use quaternion representation to avoid math-based floating bit "noise"
Should check if this conversion is a fixed-point (if rpy goes from x to x', does x' still map back to x'?)

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine · 2022-04-06T18:09:24Z

The precision issue came from ign-math's Quaternion class when converting from euler to quaternion. Specifically, the output of cos function potentially differed by 1 bit between gcc/clang on Linux, MSVC x86 / MSVC x64 on Windows, and clang on macOS. For ex: https://godbolt.org/z/Tx78a3sjs

The workaround is described below.

Should either store only translation, or use quaternion representation to avoid math-based floating bit "noise"

For the nested_model direct string comparison tests, I created a new sdf model that uses translation only so that other tests can check the full poses: e1f6421
I updated the print_config test to use quaternion representation to prevent any potential future issues since there were discrepancies between MSVC versions: fa8e4a2

Now that CI is green, this PR is ready for review.

azeey · 2022-04-09T02:35:38Z

include/sdf/Param.hh

@@ -75,38 +75,48 @@ namespace sdf
  struct ParamStreamer
  {
    const T &val;
+    const int precision;  // Used to set std::ostream's std::setprecision


I suggest we use a constructor for ParamStreamer instead of the template guides below. This would ensure that precision is initialized to zero.

ParamStreamer(const T &_val, int _precision = 0) : val(_val), precision(_precision) {}

Thanks for the suggestion 855cb1d

azeey · 2022-04-09T02:36:04Z

include/sdf/Param.hh

  };

  template<class T> ParamStreamer(T) -> ParamStreamer<T>;
+  template<class T> ParamStreamer(T, bool) -> ParamStreamer<T>;


These template guides can be removed if we create a constructor per my comment above.

azeey · 2022-04-09T03:49:06Z

src/cmd/cmdsdformat.rb.in

@@ -42,14 +42,16 @@ COMMANDS = {  'sdf' =>
                       "  -d [ --describe ] [SPEC VERSION]  Print the aggregated SDFormat spec description. Default version (@SDF_PROTOCOL_VERSION@).\n" +
                       "  -g [ --graph ] <pose, frame> arg  Print the PoseRelativeTo or FrameAttachedTo graph. (WARNING: This is for advanced\n" +
                       "                                    use only and the output may change without any promise of stability)\n" +
-                       "  -p [ --print ] arg                Print converted arg.\n" +
+                       "  -p [ --print ] arg                Print converted arg. Note poses and unit vectors will be normalized.\n" +


It's not clear what is meant by normalization of Poses. It's the quaternion representation of the rotational part of the pose that gets normalized.

azeey · 2022-04-09T04:07:13Z

src/ign.cc

@@ -134,7 +134,8 @@ extern "C" SDFORMAT_VISIBLE int cmdDescribe(const char *_version)

 //////////////////////////////////////////////////
 extern "C" SDFORMAT_VISIBLE int cmdPrint(const char *_path,
-    int inDegrees, int snapToDegrees, float snapTolerance)
+    int _inDegrees, int _snapToDegrees, float _snapTolerance,
+    bool _preserveIncludes, int _outPrecision)


I'm not sure if it's safe to use bool in the function signature of an extern "C" function since bool is not builtin type in C. The parameter _inDegrees is technically a boolean, but we're using int for this reason, I think.

azeey · 2022-04-09T04:09:36Z

src/cmd/cmdsdformat.rb.in

-            Importer.extern 'int cmdPrintPreserveIncludes(const char *)'
-            exit(Importer.cmdPrintPreserveIncludes(File.expand_path(options['print'])))
-          elsif options.key?('snap_to_degrees')
+          precision = -9999


I assume this is a placeholder number for an invalid precision. Usually I see -1 used for that purpose, so seeing -9999 was a little confusing.

Initially, I set it as -9999 since technically negative numbers in setprecision are allowed but silently ignored. I agree that this was a little confusing so I modified it to 0 and updated the -h docs to state the argument must be a positive integer: 855cb1d

Signed-off-by: Jenn Nguyen <[email protected]>

azeey · 2022-04-12T23:26:06Z

@osrf-jenkins run tests please

jennuine added 4 commits March 3, 2022 16:07

set ostream precision opt; default is max

b8e4e8f

Signed-off-by: Jenn Nguyen <[email protected]>

precision command option

22cceb8

Signed-off-by: Jenn Nguyen <[email protected]>

fixed tests

c10f208

Signed-off-by: Jenn Nguyen <[email protected]>

removed _originalStr from ParamPrivate::StringFromValueImpl

7a690da

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine requested review from azeey and scpeters as code owners March 9, 2022 03:10

jennuine self-assigned this Mar 9, 2022

github-actions bot added the 🌱 garden Ignition Garden label Mar 9, 2022

jennuine linked an issue Mar 9, 2022 that may be closed by this pull request

Use consistent float / double precision by using configuring the ostream object precision? #790

Closed

jennuine mentioned this pull request Mar 9, 2022

Updated test for libsdformat behavior change gazebosim/gz-sim#1379

Merged

7 tasks

Merge branch 'main' into jennuine/ostream_precision

7940599

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine added 3 commits April 5, 2022 14:28

Merge branch 'main' into jennuine/ostream_precision

499f65a

Signed-off-by: Jenn Nguyen <[email protected]>

translation only for direct comparison tests

e1f6421

Signed-off-by: Jenn Nguyen <[email protected]>

use quaternions for pose rotation_format

fa8e4a2

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine force-pushed the jennuine/ostream_precision branch from d19c1fd to fa8e4a2 Compare April 5, 2022 21:59

Merge branch 'main' into jennuine/ostream_precision

f822dfd

azeey reviewed Apr 11, 2022

View reviewed changes

jennuine added 2 commits April 12, 2022 09:43

Merge branch 'main' into jennuine/ostream_precision

c1067ba

Signed-off-by: Jenn Nguyen <[email protected]>

addressed reviewer feedback

855cb1d

Signed-off-by: Jenn Nguyen <[email protected]>

jennuine force-pushed the jennuine/ostream_precision branch from 4e5ebff to 855cb1d Compare April 12, 2022 18:00

azeey approved these changes Apr 13, 2022

View reviewed changes

azeey merged commit 65f0341 into main Apr 13, 2022

azeey deleted the jennuine/ostream_precision branch April 13, 2022 15:49

This was referenced Apr 18, 2022

Use consistent float / double precision by using configuring the ostream object precision? #790

Closed

Adding sdf::Error logging to Param #939

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed default floating point precision to max #872

Changed default floating point precision to max #872

jennuine commented Mar 9, 2022

codecov-commenter commented Mar 9, 2022 •

edited

Loading

scpeters commented Mar 9, 2022

jennuine commented Mar 9, 2022

EricCousineau-TRI commented Mar 10, 2022

EricCousineau-TRI commented Apr 4, 2022

jennuine commented Apr 6, 2022

azeey Apr 9, 2022

jennuine Apr 12, 2022

azeey Apr 9, 2022

jennuine Apr 12, 2022

azeey Apr 9, 2022

jennuine Apr 12, 2022

azeey Apr 9, 2022

jennuine Apr 12, 2022

azeey Apr 9, 2022

jennuine Apr 12, 2022

azeey commented Apr 12, 2022

Changed default floating point precision to max #872

Changed default floating point precision to max #872

Conversation

jennuine commented Mar 9, 2022

🎉 New feature

Summary

Test it

Checklist

codecov-commenter commented Mar 9, 2022 • edited Loading

Codecov Report

scpeters commented Mar 9, 2022

jennuine commented Mar 9, 2022

EricCousineau-TRI commented Mar 10, 2022

EricCousineau-TRI commented Apr 4, 2022

jennuine commented Apr 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azeey commented Apr 12, 2022

codecov-commenter commented Mar 9, 2022 •

edited

Loading