math/big: incorrect string->Float conversion or printing for literals with large exponent

[griesemer]

package main

import (
    "fmt"
    "math/big"
)

func main() {
    var x big.Float
    x.SetString("1e81391777742999")
    fmt.Printf("x = %.6g\n", &x)
}

prints x = 1e+151 which is incorrect.

[griesemer]

Once this is fixed, re-investigate issue #11326.

[ALTree]

The error occurs in the scan function (in math/big/floatconv.go), specifically during the 10**exp10 correction multiplication.

On line 130 we call umul as

z.umul(z, p.pow10(exp10))

umul(x, y) has precondition

x and y must have a non-empty mantissa and valid exponent.

and there's no input checking for +Inf values. Unfortunately p.pow10(exp10) can be Inf (when exp10 is big).

When Parse is called with 1e81391777742999, this results in a call to pow10 that returns an Inf Float. Then umul is called as umul(1, +Inf) and happily returns a bogus value of 1e151.

The fix is simple: if the umul call on line 130 is replaced with a Mul call (which does Inf checking), then

x.SetString("1e81391777742999")
fmt.Printf("x = %.6g\n", &x)

correctly gives

+Inf

The fix also uncovered two wrong test. In the first one (floatconv_test.go, line 370)

{"1e1000000000", 64, 'p', 0, "0x.ecc5f45aa573d3p+1538481529"},

the expected value (with exponent 1538481529) is wrong. 1e1000000000 is not representable as a floating point number with int32 exponent. Again, what is coded as the expected result is a bogus number returned by an umul call with an Inf parameter.

Another test

{"1e-1000000000", 64, 'p', 0, "0x.8a64dd983a4c7dabp-1538481528"}

proves that negative exponents too are mis-handled. In fact, 1e-81391777742999 returns 1e-151. This problem, too, can be fixed by replacing a uquo call (no Inf checking), with a Quo, in line 128 of floatconv.go

I've sent a patch.

[gopherbot]

CL https://golang.org/cl/13778 mentions this issue.