-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use consistent float / double precision by using configuring the ostream
object precision?
#790
Comments
Originally was targeting (1) Depending on the condition for poses, the original string may be printed regardless of the precision setting. To make things consistent this will need to be changed so the default precision (6 digits) is used unless the max precision is specified. For ex,
(2) There is an issue with changing the precision for math::Vector3d test(1.5707963267948966, 0, 0)
cout << test; // prints 1.5708 -0 0
cout << setprecision(max) << test; // 1.5707960000000001 -0 0; x should be 1.5707963267948968 |
|
Thanks for bringing this up!
|
The changes are applied to Here are the PRs related to (2) |
Closed by #872. |
Desired behavior
Ensure all objects emit floating point numbers of same precision.
At present, there may be different (default?) precisions in use by composite types in
ign::math
, as encountered by:#689 (comment)
Alternatives considered
N/A
Implementation suggestion
Consider setting precision in
ostream
object inside sdformat, which is then (hopefully) used directly by composite types.Additional context
N/A
\cc @azeey
The text was updated successfully, but these errors were encountered: