RFC: Add display mode capability

[dpsanders]

The idea is to be able to do the following. Of course, the syntax etc. is not nice at the moment.
We can add output precision, output or not decorations, etc.

julia> a = Interval(3, 4)
[3.0, 4.0]

julia> displaymode(format=:full)

julia> a
Interval(3.0, 4.0)

julia> displaymode(format=:midpoint)

julia> a
3.5 ± 0.5

Now it also does the following:

julia> a = @decorated(1)
[1, 1]

julia> a = @decorated(-pi, pi)
[-3.1416, 3.1416]

julia> displaymode(sigfigs=10)

julia> a
[-3.141592654, 3.141592654]

julia> displaymode(decorations=true)

julia> a
[-3.141592654, 3.141592654]_com

[coveralls]

Coverage decreased (-2.2%) to 92.297% when pulling 132fccd on display_modes into a64e948 on master.

[lbenet]

Nice idea! I'll check the details later today (hopefully).

[dpsanders]

This is now ready to be looked at @lbenet.

[lbenet]

I'm starting to look on this now. A quick question: in your examples above, I am surprised that decorated(1) or decorated(-pi,pi) does not display the decoration type. Is this intended?

[lbenet]

Sorry... I didn't noticed displaymode(decorations=true)...

[lbenet]

I've done some very preliminary tests. Here are some comments about them:

• sigfigs must be somehow bounded (from above). For instance, for Interval{Float64} it makes no sense to have 32 significant digits (or anything beyond 16 digits). An example:

julia> Interval(pi)
[3.1415926535897931159979634685441, 3.1415926535897931159979634685442]

julia> BigFloat(pi)
3.141592653589793238462643383279502884197169399375105820974944592307816406286198

• The displaymode is not working for `Interval{BigFloat}:

julia> ValidatedNumerics.display_params
ValidatedNumerics.DisplayParameters(:standard,false,6)

julia> a = @biginterval(pi)
[3.141592653589793238462643383279502884197169399375105820974944592307816406286198, 3.141592653589793238462643383279502884197169399375105820974944592307816406286233]₂₅₆

• Personally, I think the default for sigfigs should be the default (16 for Float64, 78 for BigFloat with default precision). I prefer to have the whole floating point number used in the lo and hi fields, than ask for them. This is what format=:full does, so I simply propose to have it as default.

[dpsanders]

• Now working with Interval{BigFloat}, thanks. That's why it needs tests, but I wanted to fix the syntax etc. first.
• I don't think it's necessary to fix the maximum number of sigfigs. The following makes sense:

julia> a = @floatinterval(pi)
[3.14159, 3.1416]

julia> displaymode(sigfigs=10)
10

julia> a
[3.141592653, 3.141592654]

julia> displaymode(sigfigs=20)
20

julia> a
[3.1415926535897931159, 3.1415926535897935601]

julia> big(a)
[3.1415926535897931159, 3.1415926535897935601]₂₅₆

julia> displaymode(format=:full)

julia> a
Interval(3.141592653589793, 3.1415926535897936)

julia> big(a)
Interval(3.141592653589793115997963468544185161590576171875000000000000000000000000000000, 3.141592653589793560087173318606801331043243408203125000000000000000000000000000)

• I have added a showall function:

julia> a
[3.14159, 3.1416]

julia> showall(a)
Interval(3.141592653589793, 3.1415926535897936)

This makes it very quick to see all of the digits. I think we're used to immediately seeing all the digits but I don't think it's usually actually useful (except for the authors of the library to check weird corner cases ;) ). I think for the user, a default of 6 (or whatever) sig figs is actually much more useful. In any case, it's a single command to change the default if you don't like it.

[dpsanders]

(By "reasonable" I meant that using more sig figs gives the same answer as converting the Interval{Float64} to anInterval{BigFloat}` in order to see extra digits.)

[lbenet]

I am not convinced that there is no need for an upper bound on the significant figures displayed. Let's get back to the example with pi:

julia> displaymode(sigfigs=32)
32

julia> Interval(pi)
[3.1415926535897931159979634685441, 3.1415926535897931159979634685442]

julia> ans.lo, ans.hi
(3.141592653589793,3.141592653589793)

julia> big(pi)
3.141592653589793238462643383279502884197169399375105820974944592307816406286198

We know the distinction between precision and accuracy. My point is that, since we care about the accuracy of the interval (to a given precision) bounding a result, we should not allow any ambiguity coming from displaying "more significant digits" than what it makes sense. In the example above, Interval(pi) is a thin interval; yet, since I am asking to more significant digits to what it makes sense with 53 digits, it is displayed as an interval with non-zero width, which is incorrect.

I think we're used to immediately seeing all the digits but I don't think it's usually actually useful (except for the authors of the library to check weird corner cases ;) ). I think for the user, a default of 6 (or whatever) sig figs is actually much more useful. In any case, it's a single command to change the default if you don't like it.

I fully agree! It does makes sense to reduce the number of significant digits, but not increasing it!

[lbenet]

Incidentally, tests for Julia 0.4 are not passing! The problem is in @intervalbox tests

[dpsanders]

That is a nice example! The point is that 20 significant figures is not enough to express float(pi) exactly in base 10, so it is correct to round up and round down; it turns out to need 49 significant figures, and then you can see that it is in fact a thin interval given by an exact floating-point number:

julia> displaymode(sigfigs=49)

julia> p = Interval(pi)
[3.141592653589793115997963468544185161590576171875, 3.141592653589793115997963468544185161590576171875]

julia> showall(big(p))
Interval(3.141592653589793115997963468544185161590576171875000000000000000000000000000000, 3.141592653589793115997963468544185161590576171875000000000000000000000000000000)

I will fix the failing test and add more for all these display capabilities.

[coveralls]

Coverage decreased (-0.2%) to 92.754% when pulling 279c8c5 on display_modes into bba7886 on master.

[dpsanders]

Are you happy for this to be merged, @lbenet?

[codecov-io]

Current coverage is 92.75%

Merging #115 into master will decrease coverage by -0.24%

@@             master       #115   diff @@
==========================================
  Files            18         19     +1   
  Lines           857        897    +40   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits            797        832    +35   
- Misses           60         65     +5   
  Partials          0          0          

1. 2 files in src were modified. more
   • Misses +1
   • Hits +2

Powered by Codecov. Last updated by bba7886...324869b

[lbenet]

I still do not agree; you may indeed need to go to 49 significant figures or simply stay with 16, which is the correct amount for Float64. (Incidentally, the reason I used sigfigs=32 was simply because it is the double of 16.)

I agree that the binary representation of pi, using BigFloat is the quoted number. But this is because we have the option of BigFloats. The same example can be casted using BigFloats and there, mpfr sets the limit:

julia> Interval(BigFloat(pi))
[3.14159, 3.1416]₂₅₆

julia> displaymode(format=:full)

julia> Interval(BigFloat(pi))
Interval(3.141592653589793238462643383279502884197169399375105820974944592307816406286198, 3.141592653589793238462643383279502884197169399375105820974944592307816406286198)

julia> displaymode(sigfigs=1000)
1000

julia> Interval(BigFloat(pi))
Interval(3.141592653589793238462643383279502884197169399375105820974944592307816406286198, 3.141592653589793238462643383279502884197169399375105820974944592307816406286198)

The point is that we have not changed the precision of mpfr (ValidatedNumerics.parameters.precision), so mpfr is not changing the number of displayed. If you want more displayed digits, you must change the precision of the calculation. So let me insist that we should not allow the display of any significant figures beyond the number attached to the precision of the calculation.

[lbenet]

Following with the same example:

julia> displaymode(format=:standard)

julia> ValidatedNumerics.display_params
ValidatedNumerics.DisplayParameters(:standard,false,1000)

julia> set_bigfloat_precision(64);

julia> Interval(BigFloat(pi))
[3.14159265358979323851280895940618620443274267017841339111328125, 3.14159265358979323851280895940618620443274267017841339111328125]₆₄

julia> ans.lo
3.14159265358979323851

julia> set_bigfloat_precision(53);

julia> Interval(BigFloat(pi))
[3.141592653589793115997963468544185161590576171875, 3.141592653589793115997963468544185161590576171875]₅₃

julia> ans.lo
3.1415926535897931

How do I understand the figures beyond the precision of the calculation?

[lbenet]

EDIT: Removed the comment; forgot DecoratedInterval....

[lbenet]

I think using Interval(0.1,0.2) yields clearer examples.

[lbenet]

... or any other which is not an Interval with integers.

[dpsanders]

I did not explain properly what the different format settings do.

• format=:full (and also showall(a)) does exactly what the previous display used to do (except with the word Interval instead of square brackets, so that the result may be copied and pasted):
  it uses the standard Julia or MPFR output routines (for Float64 or BigFloat, respectively) to output the number to enough decimal digits to make the result unique as a floating-point number. This is usually 16 figures for Float64, but sometimes 17 are required. Thus it is not affected by the sigfigs setting. This uses the so-called Grisu algorithm, which is complicated but already implemented (and which I do certainly not understand).

In this representation, an interval is thin (diameter=0) if and only if the two outputs are exactly the same.

• format=:standard does something different: it outputs an interval a by rounding a.lo downwards and a.hi upwards at the number of significant figures given by the current sigfigs parameter. This is perfectly well-defined for any value of sigfigs greater than 0: it takes that number of digits in the decimal expansion of the exact floating-point number, which is given by the corresponding sum of powers of two.

This decimal expansion always has a finite number of non-zero digits, but that number of non-zero digits may be large. If sigfigs is less than that number, then the last digit will always appear rounded down or up. This is saying that the exact floating-point number represented (exact sum of powers of 2) falls in the interval between those exact decimal values (exact linear combination of powers of 10).
If sigfigs is big enough, then in principal there should be trailing zeros, but in fact those trailing zeros are not shown, so incrementing sigfigs further will not change the output (which is showing that you already have the exact decimal representation).

e.g.

julia> a = float(pi)
3.141592653589793

julia> a = big(float(pi))
3.141592653589793115997963468544185161590576171875000000000000000000000000000000

Note the trailing zeros showing where the exact decimal representation ends. But:

julia> displaymode(format=:standard, sigfigs=10000)
10000

julia> Interval(float(pi))
[3.141592653589793115997963468544185161590576171875, 3.141592653589793115997963468544185161590576171875]

Here the trailing zeros are not shown.

julia> displaymode(format=:full)

julia> Interval(float(pi))
Interval(3.141592653589793, 3.141592653589793)

This is the behaviour previous to this PR -- it recognises that 3.141592653589793 is sufficient to reconstruct the corresponding Float64 value and so only outputs this many digits.

I hope this clarifies the situation? I guess all this should go in the docs... :/

[lbenet]

I think I had grasped already the ideas behind, though your detailed explanation is certainly welcome! Thanks! (Use it for the docs!)

My concern is related to having more figures that what is compatible with the actual precision (call it 53 bits for Float64, or ValidatedNumerics.parameters.precision for BigFloat). I do understand what goes on with float(pi) and convert this to a BigFloat. But that is the issue. The issue is whether what is displayed is actually meant as a validated result. To cut off significant digits is for convenience in the presentation of the numbers; increasing the number of significant digits is another thing. (I agree that that works for Float64 represented as a BigFloat, if the precision is large enough.) At the moment I am not convinced that that is the case for more significant digits than whatever is allowed by the precision set.

[dpsanders]

I think that is the point: the idea of the :standard format (though it may well need a different name...) is to display an interval that actually contains the interval encoded in the Interval object.

Note that this is not the case before this PR: what you see in

julia> Interval(pi)
[3.141592653589793, 3.141592653589793]

or more explicitly after this PR with

julia> displaymode(format=:full)

julia> Interval(pi)
Interval(3.141592653589793, 3.141592653589793)

is a string that, due to some floating-point / Julia magic, enables you to reproduce the original interval by copying and pasting it in the REPL (basically because there is a hard-coded "precision" in Float64 that fixes how many powers of 2 to use in the representation). However, this displayed interval does not contain the true interval being represented!, which here is a thin interval containing the single floating-point number float(pi), with exact decimal value

3.141592653589793115997963468544185161590576171875

In order to show such a containing interval, you either must show this long string, or show only some of its digits, in which case you must round at that number of displayed digits to get a containing interval.

[Previously we had no way of displaying this true value; now we do. (Although you would probably only rarely, if ever, actually want to display this!)]

[dpsanders]

Arguably there should be another mode that automatically calculates this true string and immediately displays it, without fiddling around finding what sigfigs setting is required. (In fact, John Gustafson basically argues for this in his book The End of Error.) But in practice it seems to me unlikely that you really ever want this...

[lbenet]

Ok, though somewhat hesitating, let's go for it!

I'm merging this!

[lbenet]

Thanks a lot! Specially to patiently deal with my objections!

[lbenet]

Incidentally, have you check what it says the standard about the output?

[dpsanders]

Thanks! I haven't checked what the standard says. In any case this is useful functionality I think