Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bn_mp_set_double.c does not compile under MacOS plus #159

Closed
buggywhip opened this issue Feb 12, 2019 · 17 comments · Fixed by #476
Closed

bn_mp_set_double.c does not compile under MacOS plus #159

buggywhip opened this issue Feb 12, 2019 · 17 comments · Fixed by #476

Comments

@buggywhip
Copy link

Under MacOS...

$ gcc --version
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix

a simple make returns...

   * cc bn_mp_set_double.o
bn_mp_set_double.c:55:6: warning: "mp_set_double implementation is only available on platforms with IEEE754 floating point format" [-W#warnings]
#    warning "mp_set_double implementation is only available on platforms with IEEE754 floating point format"
     ^
1 warning generated.

...resulting in no binary for mp_set_double(). (mp_get_double() compiles fine.)


Notes:

  • LTM and LTC will not crash if mp_get_double() and mp_set_double() are not compiled. They are not called fm anywhere else in LTM or LTC. ...and only mp_set_double(), demo.c, and mpi.c are conditioned with #if defined(__STDC_IEC_559__) || defined(__GCC_IEC_559)

  • Looking at the list of enabled defines in the attached file ( bn_mp_set_double.i.txt ), make note of those with FLT, DBL, and LDBL.

  • It appears when Apple's defines __DBL_HAS_DENORM__, __DBL_HAS_INFINITY__, and __DBL_HAS_QUIET_NAN__ are true the processor supports IEEE 754/IEC 60559. Perhaps some combination of these 3 defines could be used to set __STDC_IEC_559__?

  • LTM only implements get/set for DBL. Apple supports FLT and LDBL as well. So, for the sake of completeness, perhaps we should also implement mp_get|set_float() and mp_get|set_longdouble()?

  • Finally, out of a concern for some misusing mp_get|set_double(), the doc should provide a strong caution about the possible loss of precision.

  • FWIW, all Macs since 2006 have Intel processors with integrated IEEE-754 compliant FPUs. ...as do all PPC Macs. M680x0 Mac FPUs varies.

@czurnieden
Copy link
Contributor

Apple! *sigh*

It appears when Apple's defines __DBL_HAS_DENORM__, __DBL_HAS_INFINITY__, and __DBL_HAS_QUIET_NAN__ are true the processor supports IEEE 754/IEC 60559. Perhaps some combination of these 3 defines could be used to set __STDC_IEC_559__?

That shouldn't be much of a problem.

According to https://sourceforge.net/p/predef/wiki/OperatingSystems/ the combination

#if ( (defined macintosh) || (defined  Macintosh) || ((defined __APPLE__) && ( defined __MACH__)) )

should suffice to detect an Apple OS (macros like __DBL_HAS_DENORM__ might have been defined by others, so it is better to check for MacOS first) or are you aware of anything different/additional/more exact?

macintosh and Macintosh are MacOS 9. Exclude from get|set_double?

LTM only implements get/set for DBL. Apple supports FLT and LDBL as well. So, for the sake of completeness, perhaps we should also implement mp_get|set_float() and mp_get|set_longdouble()?

binary32 (FLT) can be done, of course, but a long double has a slightly different format, so an implementation will need a bit more work and hence might take a bit longer.

Finally, out of a concern for some misusing mp_get|set_double(), the doc should provide a strong caution about the possible loss of precision.

Also no problem.

M680x0 Mac FPUs varies.

Yes, I was painfully made aware of it in #156 ;-)
And that didn't even touch the many other, uhm, peculiarities of the M6888[12].

@buggywhip
Copy link
Author

buggywhip commented Feb 13, 2019 via email

@czurnieden
Copy link
Contributor

@buggywhip
Copy link
Author

buggywhip commented Feb 14, 2019 via email

@minad
Copy link
Member

minad commented Feb 14, 2019

@buggywhip Doing runtime checking sounds like a good idea. We could use a small set of correct inputs/outputs as tests.
Run the tests once and set a static variable.
In case the tests fail output a message at runtime or abort?
Or just return an error code...

@sjaeckel
Copy link
Member

macintosh and Macintosh are MacOS 9. Exclude from get|set_double?

So yes, for now anyway, I'd exclude.

if there's no hardware to test on, I'd exclude

Doing runtime checking sounds like a good idea.

IMO runtime checking sounds like a bad idea as it brings complexity in our code where the toolchain fails to be set-up properly or something else is not 100% compatible (c.f. #156 ) and neither can nor should we fix all those problems on our side.

We could use a small set of correct inputs/outputs as tests.

IMO runtime checks are not "let's try stuff out to determine if it works and if our testcases work by hazard we decide to make this work all the time" but it should be "let's see what kind of CPU we have and read CPU flags and stuff" and if this CPU type doesn't support those operations we provide a fallback scenario.

@minad
Copy link
Member

minad commented Feb 14, 2019

... but it should be "let's see what kind of CPU we have and read CPU flags and stuff"...

@sjaeckel I don't think that would help. Instead we could just continue tweaking the preprocessor checks. Functional runtime checks would avoid adjusting those checks to every platform and compiler. However you are right that it would increase complexity and it is difficult to chose the right set of test inputs.

Since we are discussing those matters for quite some time now - I would really like to use a simpler solution. What about only activating mp_set_double on platforms we know it works? E.g. checking for x86_64 etc. We should also compile the tests only on those platforms. As time goes by more platforms will be whitelisted...

Edit: I suggest #if defined(__x86_64__) || defined(_M_X64) || defined(_M_AMD64) || defined(__i386__) || defined(_M_X86) || defined(__aarch64__) || defined(__arm__) for now.

@buggywhip
Copy link
Author

  1. Testing for processor and/or FPU type seems pointless because there is nothing to protect. Neither mp_get_double() or mp_set_double() performs FP operations so nothing will blow up. Only the data itself is at issue.

  2. Conditioning based on the type and presence of a compliant FPU only increases the likelihood the double will be correctly formed; it cannot guarantee it. What if the double was taken fm a binary file created elsewhere?

  3. Not yet mentioned, but there are base 2 and base 10 FP formats to consider. I want to confirm how IEEE 754 handles this but cannot right now. ...hard deadlines, sorry.

  4. As previously mentioned, I'm leaning to not conditioning either, but if we condition mp_get_double(), consider conditioning both mp_get|set_double() so as to avoid confusion when one is compiled and the other not.

  5. A long shot (but I need time)... One dirt-simple runtime test would be for mp_set_double() to call mp_get_double() and compare the results. It might be the only test required and it should work on all platforms regardless of the type or even presence of a FPU? ...although it might be preferential for a FPU to do the comparison. ???

  6. Stepping back for a moment, functions in LTC and LTM do not call either mp_get_double() or mp_set_double() and because there are potential precision issues, I'm struggling with the use case behind these functions in the first place. Why do we even need/want them? Any good use cases?

@minad
Copy link
Member

minad commented Feb 14, 2019

2. Conditioning based on the type and presence of a compliant FPU **only increases the likelihood** the double will be correctly formed; it cannot guarantee it.  What if the double was taken fm a binary file created elsewhere?

The function should not protect against invalid data.

4. As previously mentioned, I'm leaning to not conditioning either, but **if** we condition mp_get_double(), consider conditioning both mp_get|set_double() so as to avoid confusion when one is compiled and the other not.

Agree.

5. A long shot (but I need time)...  One dirt-simple runtime test would be for mp_set_double() to call mp_get_double() and compare the results.   It might be the only test required and it should work on all platforms regardless of the type or even presence of a FPU?  ...although it might be preferential for a FPU to do the comparison.  ???

This would not work only if the transformation would round trip, which is generally not the case. However we are probably not going for runtime tests according to @sjaeckel.

6. Stepping back for a moment, functions in LTC and LTM do not call either mp_get_double() or mp_set_double() and because there are potential precision issues, I'm struggling with the use case behind these functions in the first place.  Why do we even need/want them?  Any good use cases?

LTM is a library. The functions might not be used internally but from the outside world. I wrote the patch adding the functions and I need them. GMP also has similar functions. I think these functions are pretty standard and their existence is expected. Look for old issues in the bugtracker. If you don't want the functions available in your build, you can tweak LTM not to include them via macro trickery.

@buggywhip
Copy link
Author

buggywhip commented Feb 14, 2019 via email

@MasterDuke17
Copy link
Collaborator

I just ran into this with MoarVM. The upgrade to LTM v1.2.0 was just merged, so now I'm trying to take advantage of some of its new capabilities. One of those is mp_(get|set)_double(), but I get build failures in our CI environments (Travis and Appveyor).

@minad
Copy link
Member

minad commented Feb 3, 2020

@MasterDuke17 could you paste the failure output here? Compilation failure?

@MasterDuke17
Copy link
Collaborator

Appveyor (windows):

<snip>

compiling 3rdparty\libtommath\bn_mp_set_double.obj
bn_mp_set_double.c
mp_set_double implementation is only available on platforms with IEEE754 floating point format

<snip>

linking moar.dll
   Creating library moar.dll.lib and object moar.dll.exp
bigintops.obj : error LNK2001: unresolved external symbol mp_set_double
moar.dll : fatal error LNK1120: 1 unresolved externals
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.EXE"' : return code '0x460'
Stop.

TravisCI (MacOS only, linux is fine):

<snip>

compiling 3rdparty/libtommath/bn_mp_set_double.o
3rdparty/libtommath/bn_mp_set_double.c:44:6: warning: "mp_set_double implementation is only available on platforms with IEEE754 floating point format" [-W#warnings]

#    warning "mp_set_double implementation is only available on platforms with IEEE754 floating point format"

     ^

1 warning generated.

<snip>

linking 3rdparty/libtommath/libtommath.a

/Applications/Xcode-9.4.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: 3rdparty/libtommath/libtommath.a(bn_mp_set_double.o) has no symbols

linking 3rdparty/libuv/libuv.a

linking libmoar.dylib

Undefined symbols for architecture x86_64:

  "_mp_set_double", referenced from:

      _MVM_bigint_from_num in bigintops.o

ld: symbol(s) not found for architecture x86_64

clang: error: linker command failed with exit code 1 (use -v to see invocation)

make: *** [libmoar.dylib] Error 1

The command "make -j2 install;" exited with 2.

@minad
Copy link
Member

minad commented Feb 3, 2020

Yes ok, so the macro checks should be somehow adapted in order to recognize windows and macos. @MasterDuke17 is it possible that you prepare a patch?

@MasterDuke17
Copy link
Collaborator

My knowledge of the C pre-processor, IEEE754, etc. is very limited, but I'll cobble together something cribbed from the above comments and StackOverflow and see if it works in our CI environments.

MasterDuke17 added a commit to MoarVM/libtommath that referenced this issue Feb 9, 2020
Not all platforms/environments/architectures that support enough of
IEEE 754 for the purposes of mp_set_double() actually support enough
to legitimately define __STDC_IEC_559__, so only relying on that is
too strict. Fixes libtom#159
@MasterDuke17
Copy link
Collaborator

MoarVM@db0d387 makes our CI happy.

minad pushed a commit that referenced this issue Feb 17, 2020
Not all platforms/environments/architectures that support enough of
IEEE 754 for the purposes of mp_set_double() actually support enough
to legitimately define __STDC_IEC_559__, so only relying on that is
too strict. Fixes #159
@minad
Copy link
Member

minad commented Feb 17, 2020

closed by #476

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants