Tracking issue: stdlib audit

[matthew-healy]

We should work out what the desired standard library interface is. This includes (but is not limited to):

• naming (e.g. fold vs foldLeft)
• typing (e.g. record.values currently has contract { ; Dyn } -> Array Str, but perhaps could have forall r. { ; r } -> Array Str)
• filling in any gaps (e.g. record.map' : forall a b. (Str -> a -> { key: Str, value: b }) -> { _: a } -> { _: b } from today's Nickel Hour)
• removing any functions we don't wish to support long-term

[matthew-healy]

An idea that came up today: there are certain functions involving polymorphic record contratcs which respect parametricity but due to the contracts involved are not supported in Nickel today. An example would be:

removeFoo | forall t r. { foo: t; r } -> { ; r } = fun r => record.remove "foo" r,

This seems valid, but currently fails because record.remove has type forall a. Str -> {_: a} -> {_: a}, and that type gets converted into a contract whose implementation calls %record_map%, which can violate parametricity, so is not allowed.

A workaround is to use the primitive operation %record_remove%, for which we do not generate a contract.

One way of supporting the above program without encouraging users to use primitive ops would be to expose an unsafe module from the standard library, which contained untyped versions of various functions. The above function could then be written as:

removeFoo | forall t r. { foo: t; r } -> { ; r } = fun r => unsafe.record.remove "foo" r,

[matthew-healy]

IMO it's a little weird that join : Str -> Array Str -> Str is accessed via string.join rather than array.join. I get that it's not a "proper" join as we can't express it generically, but I was still surprised to find it there.

[matthew-healy]

We should probably consider #1007 part of this issue.

[yannham]

As discussed before, let's gather here proposals for addition to the stdlib as well. Taking those of #321 here (originally proposed by @silverraven691):

• nums.clamp : Num -> Num -> Num -> Num
• functions.flip : forall a b c. (a -> b -> c) -> b -> a -> c
• lists.optional : Bool -> List -> List
• records.optional : Bool -> { _: Dyn } -> { _: Dyn }
• records.filter : (Str -> Dyn -> Bool) -> { _: Dyn } -> { _: Dyn }
• records.to_list : { _: Dyn } -> List { key: Str, value: Dyn } // or: records.entriesOf
• records.from_list : List { key: Str, value: Dyn } -> { _: Dyn } // or: records.fromEntries
• string.base64_encode : Str -> Str / string.base64_decode : Str -> Str (or replacing one of the type by Base64Str)

[vkleen]

Some more functions to add

• array.intersperse : forall a. a -> Array a -> Array a
• Something like string.join for symbolic strings

[vkleen]

We have started working on this at #1053

[yannham]

Something that I wanted to write an example of generating a record with {servers_1 = ..., server_2 = ..., server_3 = ...} is variant of map/fold functions which also provides the index. It can be done with an additional call to generate right now, at least for map_index, but it could be useful to have as well.

[vkleen]

It would be useful to have a contract helper with the following semantics: Given a predicate pred (or maybe a contract?) check whether the input satisfies pred. If it doesn't apply a normalization function and check again.

[vkleen]

Some more functions:

• array.slice ideally via a new primop
• array.split_at
• array.replicate
• array.range
• a contract for "either an enum tag or a string, that gets normalized into an enum tag"

[vkleen]

• array.fold1 for non-empty arrays
• Contracts for symbolic strings
• Contracts for symbolic strings with a specific tag

[yannham]

array.fold1 for non-empty arrays

To be more in line with the previous naming scheme, I propose fold_left_first and fold_right_last, or fold_left_with_first/fold_right_with_last. Another possibility is reduce or reduce_left/reduce_right. For the record, here is a similar debate for the naming of this function in Rust: rust-lang/rust#68125

[vkleen]

I like reduce_left/reduce_right quite a bit actually. It seems to me that fold_left_first/fold_right_last doesn't really capture the intent, just like foldl1 doesn't really. And the rust community seems to agree, they stabilized the feature as reduce.

[vkleen]

We need more contract helper functions in general. The following ideas came up in a weekly meeting.

• contract.or where this makes sense (say, for predicates)
• contract.Equal for checking value equality

[yannham]

contract.equal for checking contract equality

I think it was rather contract.Equal or EqualsTo to check value equality, as in foo | EqualsTo 5. Unless I'm misremembering something? But I don't think we had any plan to make contract equality accessible to user code, did we?

[vkleen]

You're absolutely right, the perils of not doing things in a timely manner 😅

[matthew-healy]

@yannham @vkleen what do you think about closing this, since 1.0 has released?

[vkleen]

I agree, this issue is as close to completed as it's ever going to get. Let's open more specific ones when necessary.