You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This snippet shows that additionally to these known problems, fsqrt.s can return large valid floating-point numbers instead of NaNs, when the input is a non-canonical NaN.
Hence, this is another bug than the reported off-by-one. I do not know at the moment whether the root cause is shared.
Got ft2=0x5b7c271f which corresponds to 7.09747e+16, instead of ft2=0x7fc00000=nan.
Tested with Ariane with cvfpu 0.7.0 (109f9e9ed3adff25464db3aa021cb88119b7bf53).
Thanks!
Flavien
The text was updated successfully, but these errors were encountered:
flaviens
changed the title
FSQRT of non-canonical NaN gives valid large floats
Bug report: FSQRT of non-canonical NaN gives valid large floats
Apr 17, 2023
Hi there!
fsqrt was known to produce some off-by-one errors, and may return 1 on canonical NaN.
This snippet shows that additionally to these known problems,
fsqrt.s
can return large valid floating-point numbers instead of NaNs, when the input is a non-canonical NaN.Hence, this is another bug than the reported off-by-one. I do not know at the moment whether the root cause is shared.
Got
ft2=0x5b7c271f
which corresponds to7.09747e+16
, instead offt2=0x7fc00000=nan
.Tested with Ariane with cvfpu 0.7.0 (109f9e9ed3adff25464db3aa021cb88119b7bf53).
Thanks!
Flavien
The text was updated successfully, but these errors were encountered: