Error during call to eministat:x

[gnuvince]

Given the following sample benchmark program, the call to eministat:x will cause an error by trying to invoke lists:seq/2 with an invalid range. This happens in our own benchmarks, although sometimes by running the benchmark 2-3 times one can get a complete output without errors.

-module(bugreport).

-export([show_bug/1]).

show_bug(N) ->
    D1 = eministat:s("foldl", fun() -> fact1(N) end, 50),
    D2 = eministat:s("recursion", fun() -> fact2(N) end, 50),
    eministat:x(95.0, D1, D2).

fact1(N) ->
    lists:foldl(fun(X, Acc) -> X * Acc end, 1, lists:seq(1, N)).

fact2(0) -> 1;
fact2(N) when N > 0 -> N * fact2(N - 1).

Console output:

Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V7.3  (abort with ^G)
1> bugreport:show_bug(10).
x foldl
+ recursion
** exception error: no function clause matching lists:seq(9,7) (lists.erl, line 242)
     in function  eministat_plot:plot_bar/3 (/home/vfoley/adgear/repos/erlang-microbenchmarks/_build/default/lib/eministat/src/eministat_plot.erl, line 94)
     in call from eministat_plot:'-p/2-fun-1-'/2 (/home/vfoley/adgear/repos/erlang-microbenchmarks/_build/default/lib/eministat/src/eministat_plot.erl, line 28)
     in call from lists:foldl/3 (lists.erl, line 1262)
     in call from eministat_plot:p/2 (/home/vfoley/adgear/repos/erlang-microbenchmarks/_build/default/lib/eministat/src/eministat_plot.erl, line 28)
     in call from eministat:x/3 (/home/vfoley/adgear/repos/erlang-microbenchmarks/_build/default/lib/eministat/src/eministat.erl, line 50)
2> 

OS: Debian GNU/Linux Jessie (8.5) Stable
Kernel: Linux 3.16.0
Erlang: 18.3
eministat: d2b7c99

[jlouis]

(I'm at a conference right now, but will look at this later when I get home again. Perhaps this weekend, though no promises).

Indeed, this looks like a bug. I've seen something like it before and it has to do with the plotting subsystem not being particularly robust. In the limit it fails to procure the right result. Thank you for the counterexample, which can be turned into a test case for the system.

The problem, I think, manifests itself because integers/floating point numbers have an off-by-one bug, situation and suddenly you have calls to lists:seq/2 which are invalid.

[jlouis]

A little bit of work... Pushed the failing test case into a CT suite so we can start tracking the problem.

[jlouis]

There are two problems. Current master (60cdf60) fixes the plotting bug. Your dataset is extremely consistent with its run-time (everything is 0), so it ends up in a situation where it wants to plot everything in the same location. Since the range is calculated to be from X to X for some X, it fails because it wants to run lists:seq(X+1, X-1) in this case. The Erlang library is built such that a sequence like this is an error (the other possible semantics would be to return [] in this case).

Now, the plotting bug done, we are halfway through. If you re-run the test case, you end up with a bug in eministat_resample:quantile/2. This bug occurs because

• You are asking for the 0.0th percentile.
• This is the limit of a normally distributed bell-curve.
• The answer to this query is -infinity.
• Erlang doesn't support -infinity in its Floating Point Numbers (ick).

One possible solution is to throw an exception in this case and then handle it by failing. Or fix the code so it can handle inifinities. The situation occurs whenever you have a dataset [X,X,X,X,...,X]. In systems with IEEE 754 FP math, the computation usually just collapses to an infinity I think, but I'll have to carefully go over the code and rework it in order to handle this situation.