Floating point values.

[BurntSushi]

I believe the treatment of 64 bit integers in the test suite is solid, but I think testing floating point values has holes.

It is partially mitigated by the fact that all values are encoded as JSON strings. This means that we can interpret all floating point values as Go does and perform a proper equality test.

However, different languages may have different floating point representations. Which means that some languages might produce different values---which poses problems for an equality test.

In practice, this may not be so much of an issue since floating point conversion is typically well-handled by the language being used. However, it may be worth it to come up with tests for sufficiently small and large numbers that are not reliably represented in practice.

Overall, I'm not sure. And I have only a passing familiarity with the inner workings of floating point representation to know if this is a real issue or not.

[BurntSushi]

Tip: For Python parsers, I used repr instead of str to convert a float value to a string.

[Julian]

(Not sure what you mean about Python, if this is to interact with this suite, the json module will perfectly well serialize Python floats into JSON numbers).

[BurntSushi]

@Julian It's much more subtle than that. This transcript might make things clearer:

>>> f = 3.141592653589793
>>> print str(f)
3.14159265359
>>> print repr(f)
3.141592653589793

[BurntSushi]

Ah, I might see the source of your confusion now. JSON serialization doesn't really apply here, since the interface of the tester reads all primitive values as strings. So you can't just take a dictionary representation of the interface and dump it as JSON. You've got to make sure all your values are strings.

[Julian]

My comment was before you pointed out that your interface is expecting strings. If that's true, then the json module will encode them as numbers, not strings, so you're up to converting them yourself. Which of str or repr is appropriate depends, but what should be going on is an approximate equality check, which should allow either.

[BurntSushi]

but what should be going on is an approximate equality check, which should allow either.

What value of epsilon do you suggest I use?

[BurntSushi]

It's not clear to me that this is a problem in practice. Moreover, an approximate equality check typically requires some notion of what one believes is accurate. I don't know what that means in this context.

If someone comes across this problem in the real world, please leave a comment and I'll re-open the issue.

[mmakaay]

Not an epsilon-related issue, but an issue related to comparing float values.

When comparing "nan" floats, parsing to a float64 and comparing the expected and actual version using == will not work (it will always return false). You'll have to check if both values are NaN using math.IsNaN(floatValue). In my codebase, my check to see if I've got a float mismatch looks like this:

if expectedFloat != actualFloat && !(math.IsNaN(expectedFloat) && math.IsNaN(actualFloat)) {
    // error handling here...
}