What's the status of this proposal?

[dead-claudia]

I haven't seen a ton of activity here in the past few months mod a few , the last repo commit was back in late October of last year, and it's been sitting at spaceship status over in tc39/proposals for over a year. Given the broad interest on the DOM side and (to a lesser extent) Node as well, I'm just curious what the status is.

[benjamingr]

Just my understanding: This is a stage 1 proposal. In order for it to progress forward either @jhusain or @erights who are currently the champions of this proposal need to push it.

In addition @jeffbcross and @robwormald have expressed interest in promoting and advocating this in the past.

I think the biggest issue is the fact a proposal about EventTarget and DOM adoption still hasn't been presented anywhere (as far as I know). As far as I know DOM adoption required showing how observables would be used in that capacity but that work hasn't happened yet.

In addition - this proposal has had a rocky time people-wise with historical disagreements amongst parties and.

The following things contribute to stagnation IMO:

• async-iterators are "ready" and provide a comparable user experience for the "average" case - where it doesn't matter if the stream is push.
• observables aren't really too useful without additional operators unlike promises or async iterators - so library authors and users are likely to keep using rx/xstream/etc
• in the distant past certain individuals* have conducted discussion in a less-than-optimal way regarding the inclusion which has alienated and burned out some people involved in the topic in TC39.

* this is not a criticism of anyone - involved parties have greatly improved in this regard since from what I can tell. Just the status as I see it.

[benjamingr]

Also, unlike async iterators which we have integrated into Node.js core (every readable stream is an async iterator) no one has stepped up to experiment with observables in Node.js core.

Mostly because they don't actually address the use cases we've discussed due to the lack of back pressure support. Node.js is many people - so there might be parties interested.

[dead-claudia]

Okay, fair enough. I do feel a pipeline operator should help eliminate the plague of missing operators, but you would probably agree with me on that. Regarding async iterators, I can see how those could fulfill 99% of use cases, especially with better error handling semantics (not all errors are fatal). It also helps that async iterators have complete native support, whereas observables are still library-only, and Node.js streams are basically observables that happen to permit non-fatal errors, like some Gulp stream errors, and/or errors whose semantics are state- and/or context-dependent, like with FS streams - they represent opening errors before "ready", stream errors after "ready".

The case of DOM event handlers does appear to have some discussion, although it's stalled on the implementor side thanks to the lack of a JS spec. Not like that's stopped them before, but observables are something it appears they're a little more hesitant to add on their own.

[dead-claudia]

I had a thought about backpressure: would that be solvable by allowing observable.next(value) to receive the result of the subscription's .next(value) method? The subscription could choose to return a buffer callback (type: (resolve: () => void) => any) resolved once it's ready and buffer superfluous values to when it continues to receive more, and this would just be an idiom of sorts. It could also enable other forms of useful communication from subscription to subscriber, which could come in handy. (Conveniently, it mirrors generators by being effectively an inverted one.)

Of course, this would be a bit more complicated with subjects and multicast observables (you'd need to reimplement part of Promise.all), but it would work with normal observables.

Alternatively, you could have the observable itself understand the concept of returning a buffer callback and have it buffer the values itself and return an optional buffer callback to the receiver, cached per-observable and with its calls queued for when that entry drains. That would reduce some of the overhead of buffering.

In either case, I'd like to see if there's any way to do a lower-memory variant than that, so it doesn't have to be so GC-heavy.

[benjamingr]

@isiahmeadows if you haven't - check out what RxJava is doing for backpressure: https://github.com/ReactiveX/RxJava/wiki/Backpressure

[staltz]

About backpressure: callbags support it.

[benjamingr]

@staltz I do not think the observable proposal isn't progressing because of backpressure. I was just discussing why we're not pushing for it from Node.js

If you have any reason to believe callbags are the right abstraction for Node.js please do speak up. At the moment we have no plans to incorporate them into Node.js core at any form.

[staltz]

I didn't say the observable proposal isn't progressing because of backpressure, and I didn't say callbags are the right abstraction for Node.js neither do I want to incorporate them into Node.js (because the truth is there is nothing to be incorporated), I merely mentioned callbags because you mentioned RxJava and I believe a JS approach to backpressure would be more approachable to isiahmeadows who was discussing backpressure ideas.

[dead-claudia]

Yeah, I was deviating off-topic with backpressure. (It could hinder broader adoption some if not addressed, but it wouldn't prevent adoption.)

@staltz With that level of functional purity, it seems pretty interesting to me (I want to see if I can create arrows out of it). Conceptually, it reminds me a lot of pull streams, just pushing instead of pulling. I'll stop now before I derail the thread further.

[shobhitg]

One year later... Has this proposal made any progress?
RxJS is an indispensable part of my toolkit. But the biggest hurdle I see is lack of widespread adoption just because Observables are not an official standard. I wish we get some progress on this in the remaining 2019.

[ljharb]

as you can see here: https://github.com/tc39/proposals/blob/master/stage-1-proposals.md Observables haven't been presented on since May 2017, and aren't on any current or upcoming meeting agenda.

[alshdavid]

Has there been any movement on this?

[benjamingr]

@alshdavid yes, @Blesh is picking some of it up, hopefully.

[chicoxyzzy]

@benjamingr you probably meant to mention @benlesh =)

[koper89]

Any news on this topic?

[erights]

No one is putting any effort into advancing it. AFAIK, no one plans to.

[dead-claudia]

Also, there's the more recent https://github.com/tc39/proposal-emitter, but AFAICT even that one's stalled out. Personally, I just want something, but I'm not sure what the proper abstraction is yet, and the community doesn't seem to have arrived on a single specific abstraction, either. (Observables work great for object-oriented APIs, but are almost useless for, say, React, and I haven't yet seen any serious use for servers that aren't better-addressed with async/await.)

Not a TC39 member, just want to make that clear.

[kosich]

As @ljharb noted a year ago and what still holds: this proposal hasn't been presented since 2017 and it is not on TC-39 upcoming agendas.

My understanding of the main reasons for staleness:

• it's not a language core feature, it's a standard toolkit/classes (therefore not critical)
• unlike Promise, Observable is younger, bit bigger API-wise and less adopted/used by the community (imho)
• as @isiahmeadows says, community haven't settled on details (backpressure, termination/abortion, etc)

While there are active threads discussing proposal details in this repo issues, currently they seem rather academical/hypothetical to me

Would be great if proposal champions and active participants could share their vision/understanding ❤️
(maybe, it'd be good to display the state on README)

I'm not a TC39 member, not actively tracking this proposal, not very educated on the topic 🙂.

[erights]

Personally, I just want something, but I'm not sure what the proper abstraction is yet,

Depending on what you need, async interatables may or may not do what you need. It certainly satisfies some of the needs that Observable or Emitter would.

[benjamingr]

I agree with Mark, I think async iterables are super useful and have turned out to model node streams very niftily (with backpressure and graceful termination).

That said there are plenty of use cases where an eager push model is required and you want the producer to control emission and to produce as quickly as possible. Currently we have been using EventEmitter and EventTarget (now also supported in Node!) - those APIs are rather complicated and hard to correctly compose.

We basically need an "async iterable for push" - that might be observables. I think there is a lot of merit in more exploration of this space.

[mattpodwysocki]

@benjamingr that's what I've been

We basically need an "async iterable for push" - that might be observables. I think there is a lot of merit in more exploration of this space.

That's exactly what I'm doing with AsyncObservable in IxJS, the other side of RxJS, which has support for Iterable, AsyncIterable, and soon AsyncObservable as noted in this Pull Request for AsyncObservable. I think that adds the right abstractions around Observable to make both creation and teardown both async for Zalgo purposes, as well as allow for async projections such as on map, filter, etc.

[dead-claudia]

@ any interested: I slapped together an alternate proposal: https://github.com/isiahmeadows/proposal-es-streams

While it's similar in concept, I found it more helpful to deconstruct the problem even further and compare them to generators more directly. And in doing that, I found an even better way to model them, as a lot like iterators, but with pretty much all arrows flipped. I then worked backwards from there, starting by designing the protocol, then iteration, then syntactic sugar, and it all practically wrote itself, edge cases and all.

There are also things about my proposal that make it significantly more powerful, despite being much simpler to spec:

• Allowing streams to receive return values from their consumers gives a very easy channel for managing backpressure and readiness (which inherently requires crosstalk between streams and their consumers). And unlike Node and WHATWG, I don't need separate binary and object types (or modes) to manage it.
• Having an abstraction where streams are aware of async consumers means you can literally create a stream of server request + response pairs that also happen to properly route errors to appropriate responses (like "file not found" to 404, internal errors to 500, etc.).
• Consumers can close their own end very easily without even needing a reference to the stream itself. This is very helpful as streams only need to close their consumers if they themselves get closed separately.

My proposal does have a dependency on the cancellation proposal, but mostly for practical reasons (less to spec and it stays much purer) and the fact I want to maybe help give it a small kick to get some progress going with it again.

Also, there is the caveat that manual definition of streams, like manual definition of iterables, is very verbose. This could be addressed with stream helpers similar to the various proposed iterator helpers, but that's beyond the scope of my proposal. I want to keep it conceptually as simple as pragmatically possible.