[Proposal 4] Pipe functions: should they be in a separate proposal?

[js-choi]

I’m making this a separate issue from #84 and #89 because it requires one significant extension to the concept of the “placeholder”.

This issue applies to proposal 4 (Smart Mix) only.

TL;DR: Adding a unary pipe-function operator to Proposal 4’s pipe operator could explain partial application, functional composition, and method extraction. This explainability is a desirable goal that blocked the previous pipe proposal with TC39 last September. Should the pipe-function operator be added to Proposal 4 in the spec/explainer for TC39, or should it be left to a separate follow-up spec/explainer?”

Review of Proposal 4

Proposal 4 has a smart pipe operator |> – a binary operator that evaluates the LHS (here called the “pipeline’s topic”) and then evaluates the RHS (the “pipeline’s body”) using that topic. The body is usually an expression that contains a symbol # (here called a “topic reference”), within which # is bound to the topic’s value, once the topic has been evaluated. If the body is a “simple reference” (an identifier plus optionally a new operator and a property chain), then the body will be used as a unary function or constructor. So the following:

new User.Message(
  capitalize(
    doubledSay(
      (await stringPromise)
        || throw new TypeError(`Expected string from ${stringPromise}`)
    )
  ) + '!'
)

…would become:

stringPromise
  |> await #
  |> # || throw new TypeError()
  |> doubleSay(#, ', ')
  |> capitalize // a bare unary function call
  |> # + '!'
  |> new User.Message // a bare unary constructor call

(The topic reference for this discussion is #, though it could be ?, @, or other choices (see #91).)

I’ve been writing a still-unfinished draft explainer for a smart pipe operator.

It recently occurred to me is that the placeholder concept can support, with just one more operator, the use cases of three other problems – tacit function composition by @gilbert, tacit partial application by @rbuckton, and tacit method extraction by @zenparsing – all at the same time, all using the same “topic concept”.

The similarities between these use cases have been noted many times before over the past years. TC39 has explicitly halted proposals for those three problems plus the current pipeline proposal out of concerns that they all may be solved with less separate syntax. As @littledan has said, explainability of the othe rproposals is a desirable goal.

What might be new here is the use of a single unifying concept: the lexical topic.

The placeholder as a lexical topic

I’ve started start calling the pipe placeholder a topic, on the pattern of other languages’ topic variables. Unlike these languages, which often use a single dynamic variable, this topic is lexically scoped.

Right now, all Lexical Environments might or might not have a topic, depending on whether any of their ancestor Lexical Environments’ Environment Records binds the topic. And right now only the pipe operator’s RHS body binds the topic (i.e., creates a Lexical Environment whose Record binds the topic). However, all blocks except fat-arrow functions create Environments whose Records cancel out their parent Environment’s topic (see the aside at the bottom).

The pipe function

A new type of function, the pipe function -> …, would act as if it were $ => $ |> …, where $ is a hygienically unique variable. A pipe function would also not need a parameter list (whether it could optionally take one is up for debate—see also @bterlson’s proposal for headless arrows).

It would bind its first argument to # within its body. It would also parse its body using the same smart body syntax that the pipe operator uses.

In other words:
-> # is equivalent to x => x |> # and x => x.
-> # + 2 is equivalent to x => x |> # + 2 which is equivalent to x => x + 2.
-> f is equivalent to x => x |> f which is equivalent to x => f(x).
-> f(2, #) is equivalent to x => x |> f(2, #) which is equivalent to x => f(2, x).
-> console.log is equivalent to x => x |> console.log which is equivalent to x => console.log(x).
-> f |> g |> h(2, #) |> # + 2 is equivalent to x => x |> f |> g |> h(2, #) or x => h(2, g(f(x))) + 2.

Just this one more operator seems to solve unary-functional composition, partial application, and method extraction, all with the single concept of a lexical topic.

Functional composition

Functional composition:

const doubleThenSquareThenHalfAsync =
  double +> squareAsync +> half

…becomes:

const doubleThenSquareThenHalfAsync =
  -> double |> await squareAsync(#) |> half

Functional partial application into a unary function

Functional partial application:

const addOne = add(1, ?);
addOne(2); // 3

const addTen = add(?, 10);
addTen(2); // 12

let newScore = player.score
  |> add(7, ?)
  |> clamp(0, 100, ?);

…becomes:

const addOne = -> add(1, #);
addOne(2); // 3

const addTen = -> add(#, 10);
addTen(2); // 12

// with pipeline
let newScore = player.score
  |> add(7, #)
  |> clamp(0, 100, #);

Functional partial application into an n-ary function

Partial application that leaves behind multiple positional parameters is already addressed by the smart pipe and regular fat-arrow functions.

If special syntactic support for n-ary function parameters was required, then multiple topics (#, then ##, ###, ####, though not higher) would be stored in a lexical environment. The pipe operator would only bind the first topic #. But a pipe function -> … would bind all its arguments to their respective positions’ topics #, ##, ###, #### (though not a regular function … => …, which does not shadow any of its outer context’s topics!). In addition, the topic reference ... would be a “rest” topic: it would refer to an array of all topics numbered beyond the maximum number of all topics to which that the context referred by number.

numbers.sort(function (a, b) {
  return a - b
})

…becomes:

numbers.sort(-> # - ##)

And:

[ { x: 22 }, { x: 42 } ]
  .map(el => el.x)
  .reduce((x0, x1) => Math.max(x0, x1), -Infinity)

…becomes:

[ { x: 22 }, { x: 42 } ]
  .map(-> #.x)
  .reduce(-> Math.max(#, ##), -Infinity)

And:

const f = (x, y, z) => [x, y, z];
const g = f(?, 4, ?);
g(1, 2); // [1, 4, 2]

…becomes:

const f = (x, y, z) => [x, y, z];
const g = -> f(#, 4, ##);
g(1, 2); // [1, 4, 2]

And:

const maxGreaterThanZero = Math.max(0, ...);
maxGreaterThanZero(1, 2); // 2
maxGreaterThanZero(-1, -2); // 0

…becomes:

const maxGreaterThanZero = -> Math.max(0, ...);
maxGreaterThanZero(1, 2); // 2
maxGreaterThanZero(-1, -2); // 0

Developers may be expected to not use pipe functions for functions with many parameters. But an alternative, not shown here, would be to use #0, #1, #2, #3 for topic references instead of #, ##, ###, ####. Also note that the syntax is free to decree a maximum number of topics per context: something reasonable, like four. Any more and regular functions with regular identifiers for their parameters should be used instead.

Method extraction

Method extraction would also be addressed by pipe functions:

Promise.resolve(123).then(::console.log);

…becomes:

Promise.resolve(123).then(-> console.log);

…which would be the same as any of these lines:

Promise.resolve(123).then(x => x |> console.log);
Promise.resolve(123).then(x => console.log(x));
Promise.resolve(123).then(console.log.bind(console));

The big question

I’m raising this issue because it seems that addressing these other use cases in a unified fashion is a big barrier. TC39 wants the same concept to address many use cases. (See the notes from when they discussed the pipeline operator on 2017-09.) I think this solution has the potential to address all these use cases in a fairly simple fashion.

In particular, I would like some guidance: Should the Smart-Pipe Proposal include the pipe function, or should it be in a separate add-on proposal? TC39’s concerns may be mollified as long as they know that the other use cases are being addressed through the same conceptual mechanism, but they also may prefer a single unified proposal. If I don’t get an answer, I’ll go see about adding them to my draft after I finish the rest.

(As an aside, one of the goals of my smart-pipe draft is to minimize footguns by keeping lexical scoping simple. In particular, if the lexical origin of a # is unclear or confusing, then that is a footgun. I am making blocks of almost all types – function, for, while, catch, maybe if/else, etc. – cancel out outer scopes’ topics, so that outer contexts’ topics may not be used within those blocks – # would be a ReferenceError. This is also to maintain forward compatibility with possible future proposals for topic binding. The exceptions to block topic shadowing are fat-arrow functions, try and finally blocks and, when they arrive, do blocks; these exceptions allow the use of outer-context topics within their bodies – which would, among other things, make it more convenient to create callbacks in pipelines that use the topic.)

[ljharb]

The headless one imo looks too much like a developer error; that doesn’t seem like a great option.

[js-choi]

@ljharb: Are you talking about headless properties? If so, then this is well noted. I’d just realized before posting this that headless properties are actually a red herring. What I had thought was their most important purpose, method extraction, works with just pipe functions: -> console.log would work as an extracted method console.log.bind(console) without needing any headless property. I’ve edited the issue to delete mentions of headless properties, which here are a distraction.

Pipe functions are much more important to the question here: they are much more likely to be required by TC39 to integrate with the smart-pipe proposal. If by “headless” you also refer to how a pipe function may omit its parameter list, then that would reduce its benefits in pithiness for tacit function composition, tacit partial application, and method extraction by @zenparsing, but it would still help.

My concern is whether these use cases should be addressed with a unified proposal, as the 2017-09 TC39 meeting notes seem to suggest. I’m going to go with yes by default, but only after I finish the rest of the proposal for |> and the topic placeholder first.

[js-choi]

Based on @ljharb’s comment here and @littledan’s comment on IRC, the answer appears to be: hold off on this idea. I’ll confine it to an informative appendix in the explainer as one possible future extension.

(Even without pipe functions, Proposal 4 can still explain function composition ($ => $ |> f |> g), method extraction ($ => $ |> console.log), and partial application into unary functions ($ => $ |> f(2, #)). It’ll just be slightly wordier, with regular arrow functions $ => $ |> … instead of -> …. However, partial application into n-ary functions (example using @rwaldron’s notation: f(?, 3, ?), which would create a binary function) would be tougher to address with just smart pipes, which would allow -> f(#, 3, ##). You would with without pipe functions + multiple lexical topics. You might as well just do ($, $$) => f($, 3, $$).)