Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion of ml/h to itself yields floating point/rounding inaccuracy #65

Open
floriandejonckheere opened this issue Apr 29, 2021 · 1 comment

Comments

@floriandejonckheere
Copy link

floriandejonckheere commented Apr 29, 2021

I've noticed that for some values of mL/h, converting to itself yields an inaccurate result:

Unitwise(4, "mL/h").convert_to("mL/h")
=> #<Unitwise::Measurement value=3.999999993 unit=mL/h>

Oddly enough, this doesn't happen for all values, e.g. 0-3, 5 and 10 yield the correct result. And it seems that mL/h is the only unit where this happens: L, dL, hL, s, min, h in any variation seem to be correct:

[:mL, :L, :dL, :hL]
  .product([:ms, :s, :min, :h])
  .map { |n, d| [n, d].join("/") }
  .index_with { |unit| (0..1000).select { |i| Unitwise(i, unit).convert_to(unit).value != i }.count }

=> {"mL/ms"=>0,
 "mL/s"=>0,
 "mL/min"=>0,
 "mL/h"=>981,
 "L/ms"=>0,
 "L/s"=>0,
 "L/min"=>0,
 "L/h"=>0,
 "dL/ms"=>0,
 "dL/s"=>0,
 "dL/min"=>0,
 "dL/h"=>0,
 "hL/ms"=>0,
 "hL/s"=>0,
 "hL/min"=>0,
 "hL/h"=>0}
@saraid
Copy link

saraid commented Nov 12, 2021

The following diff appears to address this:

diff --git a/lib/unitwise/unit.rb b/lib/unitwise/unit.rb
index d8ad8be..013b6bf 100644
--- a/lib/unitwise/unit.rb
+++ b/lib/unitwise/unit.rb
@@ -88,7 +88,7 @@ module Unitwise
     # @api public
     def scalar(magnitude = 1)
       terms.reduce(1) do |prod, term|
-        prod * term.scalar(magnitude)
+        prod * Number.rationalize(term.scalar(magnitude))
       end
     end
 

It was a problem of a BigDecimal being multiplied with a Rational, looks like, though I won't pretend to understand why that didn't work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants