list as input

[bytelinker]

@jagerber48
Hello, data are often organised in pandas df, numy arrays and/or list.
It seems not be possible to use arrays and lists as input for the package.

sform = sciform.Formatter(precision=2, unicode_pm=True)
a_list = [1, 2, 3, 4, 5]
a_unc_list = [0.1, 0.2, 0.3, 0.4, 0.5]
result = sform(a_list, a_unc_list)

Regards

bytelinker

[jagerber48]

This is a good idea, should be possible to implement. Basically the user passes sequence-like or array-like input for value then a sequence or array with the same shape but filled with formatted strings will be returned. If the user passes in a pair of sequence-like or array-like then, if the two elements of the pair have the same shape, another sequenc/array of the same shape will be returned, again filled with formatted strings. At first I don't want to support sequences/arrays with different shapes (i.e. handling broadcasting).

Some cases will be easy to implement, others (multi-dimensional numpy arrays) may or may not be easy to implement, I'll need to see.

[jagerber48]

@bytelinker

A comment: I only plan to support this functionality through the Formatter class. I.e. it won't be possible to format lists of sfloat or vufloat objects using the format specification mini language.

A few questions:

1. The easiest way to accomplish this is to vectorize the formatting functions. The big downside to this is it introduces numpy as a dependency for sciform, which otherwise has no dependencies. How big of a problem is this? I imagine most users of this package will also be using numpy, but it would be nice to keep it as lightweight as possible. Indeed, part of the goal of this package was to have a lightweight alternative to the uncertainties package which has a lot of power for calculating error propagation in addition to just number formatting. What are your, or anyone's, thoughts on including numpy as an optional or required dependency?

Re 1.: My current strategy is to have numpy be an optional dependency and to use the vectorization strategy only if import numpy is able to succeed. Otherwise I manually handle list and tuple inputs.

2. Is it important the type of the output matches the type of the input? For example, is it ok if the user passes in a tuple of values and is returned a list of strings or a numpy array of strings?
   2a. If so, what types needs to be supported? Do list, tuple and numpy array suffice? (I'm not excited to try to include pandas dataframes... that seems to complicated. If sciform can handle lists and numpy arrays then I think internal pandas operations can apply formatting columns or pairs of columns as users need).

[bytelinker]

How big of a problem is this? numpy is quite common, where is the formatting needed and numpy is not available?

Do list, tuple and numpy array suffice? => perhaps pandas.core.series.Series could be added, so a single pandas column could be sent to the package.

[jagerber48]

@bytelinker After looking at this, I've come to the conclusion that it is a very interesting convenience feature, but it feels like I would have to make some slightly arbitrary choices for the implementation and I'm not sure if those choices are justified over having users handle the mapping over sequences/arrays themselves.

Without numpy, users can map sciform formatting using the map function:

from sciform import Formatter, ExpMode

sform = Formatter(exp_mode=ExpMode.SCIENTIFIC, bracket_unc=True)

val_list = [1, 2, 3]
unc_list = [0.1, 0.2, 0.3]

val_str_gen = map(sform, val_list)
val_str_list = list(val_str_gen)
print(val_str_list)
# ['1e+00', '2e+00', '3e+00']

val_unc_str_gen = map(sform, val_list, unc_list)
val_unc_str_list = list(val_unc_str_gen)
print(val_unc_str_list)
# ['(1.0(1))e+00', '(2.0(2))e+00', '(3.0(3))e+00']

With numpy, they can map using the vectorize function

import numpy as np
from sciform import Formatter, ExpMode

sform = Formatter(exp_mode=ExpMode.SCIENTIFIC, bracket_unc=True)

val_arr = np.array([[1, 2, 3], [4, 5, 6]])
unc_arr = np.array([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])

val_str_arr = np.vectorize(sform)(val_arr)
print(val_str_arr)
# [['1e+00' '2e+00' '3e+00']
#  ['4e+00' '5e+00' '6e+00']]

val_unc_str_arr = np.vectorize(sform)(val_arr, unc_arr)
print(val_unc_str_arr)
# [['(1.0(1))e+00' '(2.0(2))e+00' '(3.0(3))e+00']
#  ['(4.0(4))e+00' '(5.0(5))e+00' '(6.0(6))e+00']]

In both cases, the work on the user is one or two lines.

In the first example, the input could have been any iterable, but I had to make the choice to cast the generator resulting from map into a list rather than, e.g. a sequence, or some other sequence. Is this choice justified?

The second example is ok since arrays are cast to arrays, but since vectorize will work over lists or tuples should we use the np.vectorize function for those sorts as well? Then the return type won't match the input type. Is that a problem? Should 1-D numpy arrays be handled the same as N-D numpy arrays, or should they be handled like sequences?

Again, I can see the convenience, it's just not 100% clear to how to make these choices for the end-user.

Or phrased as a question: what specific list input feature would you find useful?

[bytelinker]

np.vectorize can cause issues, when bad inputs cause errors and debugging is needed.

[jagerber48]

For now this discussion is resolved by #120. I've added examples to the documentation showing how sciform formatting can be easily mapped over collections. You can see the examples here.

I'm resistant to building something like this into sciform, I think it is better to keep sciform's scope narrowly focused on formatting numbers and pairs of value/uncertainty numbers.

For this issue I also want to emphasize that it's not clear exactly what interface would be desired for a feature that maps formatting over different collections. A function that accepts a list and returns a list would be easy to build in, but what if someone wants a function that accepts a tuple and returns a tuple? A dict and returns a dict? case-by-case examples like this could easily be built into sciform but maintaining "collections mapping" functions feels out of scope for sciform, especially when it's easy enough for users of sciform to do with python functions.