Reactive stores

[Rich-Harris]

A proposal for a replacement to Store that integrates cleanly with existing state management libraries.

View formatted RFC

[lukeed]

How are we able to know that user is an observable'd variable and not something else?

If we are able to hook in to a subscribe method, shouldn't we just let the developer set that relationship? It feels like a strong assumption to make and perhaps too magical.

<h1>Hello {firstname}</h1>

<script>
  import { user } from './observes';

  export let firstname = '';
  
  user.subscribe(obj => {
    firstname = obj.firstname; // sets immediately
  });
</script>

TBH I don't think we need this step at all. I get that it's a convenience but I get the sense it may be problematic (re strong assumptions) and I don't have a way to prevent it.

More often than not, a store is to hold values that X-Component will use for computation before rendering – not to be rendered directly.

[lukeed]

Alternatively, why not just allow this & (still) outright skip the magic $ template hookup?

<h1>Hello {$user.firstname}</h1>

<script>
  import { user } from './observes';
  
  let $user = user; // annndd.... done
</script>

At least this way I have an opt-out (by not opting in), AND it's clear what's happening based on what I, the user, have learned from RFC1. There is nothing new here.

All that is new is that variables starting with $ are hints/flags to the compiler that "hey bro, this is an observable~!"

---

Edit: This approach allows me to still go about my business with the first snippet (let firstname) if & when I want. That is beautiful.

[Rich-Harris]

If we are able to hook in to a subscribe method, shouldn't we just let the developer set that relationship?

It's not enough just to subscribe to the value — you also have to unsubscribe when the component is destroyed. Multiply that by all the values you want to observe by all the components you want to observe them in, and you're talking about a lot of boilerplate that $user.firstname bypasses.

Of course, even with the $ prefix you always have the option to do it manually (for example if you need to access the changing value programmatically anyway).

More often than not, a store is to hold values that X-Component will use for computation before rendering – not to be rendered directly.

I haven't found that to be the case — I tend to use a Store for user data (name, email), global volume levels, that kind of thing. All of which show up in the UI and can change over time.

[lukeed]

Maybe we still call it store? We already have state all throughout the Component vocabulary, and varying sub-forms of state. I think adding a new "state" term may be too much.

This is a hot-take, but what about this export signature?

import { observe, readonly, merge } from 'svelte/store';

[Rich-Harris]

Yeah, I like these much better. I wonder about derive or compute rather than merge? But no strong opinions.

[PaulMaly]

I wonder about derive or compute rather than merge?

compose?

[Rich-Harris]

Yeah, I like that a lot.

Relatedly though, one thing I was thinking about is that this RFC doesn't yet show a good way of thinking about async values. One option is simply to model them as streams of promises:

<script>
  import { source, compose } from 'svelte/store';

  const search = source('');

  const suggestions = compose(search, async q => {
    const r = await fetch(`api/suggestions?q=${encodeURIComponent(q)}`;
    return await r.json();
  });
</script>

<input
  list="suggestions"
  value={$search}
  on:input="{e => search.set(e.target.value}"
>

{#if $search}
  {#await $suggestions then suggestions}
    <datalist id="suggestions">
      {#each suggestions as suggestion}
        <option>{suggestion}</option>
      {/each}
    </datalist>
  {/await}
{/if}

But in some situations you might want to model it as a stream of non-promise values. In those cases, you'd want your composer to set new values, rather than returning them:

function alternativeCompose(...args) {
  const fn = args.pop();

  return readonly(set => {
    let running = false;
    const values = [];

    const unsubscribers = args.map((arg, i) => arg.subscribe(value => {
      values[i] = value;
      if (running) fn(...values, set);
    }));

    running = true;
    fn(...values, set);

    return function stop() {
      running = false;
      unsubscribers.forEach(fn => fn());
    };
  });
}

const suggestions = alternativeCompose(search, async (search, set) => {
  const r = await fetch(`api/suggestions?q=${encodeURIComponent(q)}`;
  set(await r.json());
});

Of course, the existing compose can be expressed using alternativeCompose...

function compose(...args) {
  const fn = args.pop();
  return alternativeCompose(...args, (...args2) => {
    const set = args2.pop();
    set(fn(...args2));
  });
}

...in other words it's more 'foundational'. So maybe compose should be reserved for that, and the simpler (to use) function should be something else like map? Though 'map' implies a single input... gah, naming things is hard.

(Not shown in the alternativeCompose implementation: race condition handling and a default initial value.)

[PaulMaly]

combine, conduct, join, connect?

[mindrones]

I'd be confused by compose as it's a functional programming term to mean composing functions, while here it'd be used to add the result of the last arg, a function, to the observables, objects.

merge (https://ramdajs.com/docs/#merge, https://ascartabelli.github.io/lamb/module-lamb.html#merge) might well signify that we rewrite some of the values, so it'd be also a no-go for me.

I always really liked computeand I don't see why changing it ;) but, if you really must, extend would also be clear and unconfusing to me (somehow Lodash aliased it with assignIn but that is a merge too as their merge is a recursive assignIn).
In general it should communicate well that we're adding a dependendent prop.

[mindrones]

Re: using svelte/state: I'd prefer if we sticked to "store".

To me "state" represents the state of the application or of a component ("LOADING", "DRAGGING", "CHECKING_DATA"), rather than the content of a store.
In my experience the state of the application is stored in a state machine which I usually put in the Store, along with the current data (that depends on the current state of the application).

I'd add that I was really happy, when learning Svelte, to see that finally someone called a store... "store" and not "state", let's not follow a bad convention just because it's used in other libraries! :)

[Rich-Harris]

Check the comments towards the bottom of the main thread — we've been thinking about

import { readable, writable, derive } from 'svelte/source';

[tomcon]

import { observe, readonly, derived } from 'svelte/store';
import { observeData, readonlyData, derivedData } from 'svelte/store';
import { observeStore, readonlyStore, derivedStore } from 'svelte/store';

Sorry, but don't agree with this comment at all:

It's arguably simpler to teach than the existing store ...

this implementation is much more complicated imo but brings in readonly and a number of other benefits

[Rich-Harris]

this implementation is much more complicated imo but brings in readonly and a number of other benefits

I know this isn't quite what you meant but I'm going to use it as a cue anyway: it's worth seeing the implementations side-by-side — the core observe function can be expressed in about 20 SLOC, whereas Store is far more involved. observe, readonly and derived together weigh one third what Store does — the implementation is much, much simpler 😀

[tomcon]

My choice of words were misleading @Rich-Harris - apologies
implementation is simpler
user code is more complicated imo
Maybe only a matter of getting used to it

[Rich-Harris]

I think that user code will get simpler in many cases — consider the following:

// current store
import { Store } from 'svelte/store';

class TodoStore extends Store {
  toggleTodo(id) {
    const { todos } = this.get();
    this.set({
      todos: todos.map(todo => {
        if (todo.id === id) return { id, done: !todo.done, description: todo.description };
        return todo;
      });
    });
  }
}

const store = new TodoStore({
  todos: [
    { id: 1, done: false, description: 'water the plants' },
    { id: 2, done: false, description: 'read infinite jest' }
  ]
});

store.toggleTodo(1);

// per this RFC
import { source } from 'svelte/store';

const todos = source([
  { id: 1, done: false, description: 'water the plants' },
  { id: 2, done: false, description: 'read infinite jest' }
]);

function toggleTodo(id) {
  todos.update(todos => {
    return todos.map(todo => {
      if (todo.id === id) return { id, done: !todo.done, description: todo.description };
      return todo;
    })
  });
}

toggleTodo(1);

It's 80% as much code, with less indentation, and because it's just functions it's easier to modularise and compose. For existing Store users I agree that there's a learning curve. My hope is that it's a similar or shallower learning curve for newcomers, who aren't familiar with the specifics of set (which overwrites properties of your data without replacing it — weird!) on on('state', ...) etc

[Rich-Harris]

To recap some discussion from Discord — suggested alternatives to 'observable', which is a loaded word:

• store (we just change the meaning of an existing piece of Svelte jargon)
• source
• stream
• state
• supply
• shareable
• setup

[PaulMaly]

So hot discussion! I like it! 🔥

Actually, I'm not really sure I understand all ideas correctly regarding v3. So, I've a very concrete question:

Example using microstates & rxjs:

states.js

import { from } from "rxjs";
import { create } from "microstates";
import * as models from './models'
...
export const user = from(create(models.User, { firstName: "Jon", lastName: "Dow" }));
...

UserInfo.html

<script>
  import { ondestroy } from 'svelte';
  import { valueOf } from "microstates";
  import { user } from './states.js';
 
  let _user;
  ondestroy(user.subscribe(next => _user = valueOf(next)).unsubscribe);
</script>

<p>{_user.firstName}</p>
<p>{_user.lastName}</p>

UserEdit.html

<script>
  import { ondestroy } from 'svelte';
  import { user } from './states.js';
 
  let currentUser;
  ondestroy(user.subscribe(next => currentUser = next).unsubscribe);

  function changeFirstName(e) {
      currentUser.firstName.set(e.target.value);
  }
  function changeLastName(e) {
      currentUser.lastName.set(e.target.value);
  }
</script>

<input value={currentUser.firstName.state} on:input={changeFirstName}>
<input value={currentUser.lastName.state} on:input={changeLastName}>

Seems, these code probably should work without any svelte-store-observables-stream, right? If so, the biggest value of this RFC subject is that we'llable to use $user instead of additional functions like changeFirstName?

I think it's not reasonable price. There's many implementations of Observers and other patterns. Maybe we should use them instead of built-in solution?

Example using shiz:

states.js

import { value } from "shiz";
...
export const state = value('some shared state');
...

StateInfo.html

<script>
  import { state } from './states.js';
 
  let _state;
 const dependentValue = computed( [ state ], ([ state ]) => state + '...')
  state.on('change', () => _state = state.get());
</script>

<p>{_state}</p>
<p>{dependentValue}</p>

StateEdit.html

<script>
  import { state } from './states.js';
 
  let _state;
  state.on('change', () => _state = state.get());
</script>

<input value={_state} on:input={e => state.set(e.target.value)}>

Or I'm mistaken somewhere?

[Rich-Harris]

It's just syntactic sugar — you'll be able to take it or leave it. The pattern in the examples above certainly works. But if you have to use user in multiple components (say, your nav bar and a settings menu) then you start doubling up on boilerplate. I'd argue it's much nicer to be able to do this:

-import { from } from "rxjs";
+import { from } from "@sveltejs/observable";
+import { derive } from 'svelte/store';
-import { create } from "microstates";
+import { create, valueOf } from "microstates";
import * as models from './models'
...
-export const user = from(create(models.User, { firstName: "Jon", lastName: "Dow" }));
+export const user = derive(
+  from(create(models.User, { firstName: "Jon", lastName: "Dow" })),
+  valueOf
+);
...

<script>
-  import { ondestroy } from 'svelte';
-  import { valueOf } from "microstates";
  import { user } from './states.js';
- 
-  let _user;
-  ondestroy(user.subscribe(next => _user = valueOf(next)).unsubscribe);
</script>

-<p>{_user.firstName}</p>
-<p>{_user.lastName}</p>
+<p>{$user.firstName}</p>
+<p>{$user.lastName}</p>

[lukeed]

How can I leave it? The moment an observed instance is imported it gets auto-injected. It's take it all or leave it all

[Rich-Harris]

No, you only get the auto-subscription if you use it in your markup with the $ prefix. There's no magic behaviour that happens when you import it.

[PaulMaly]

@Rich-Harris Actually, I'm not sure I need it. I think I can simplify my code a little bit just using additional modules. I'm not sure Svelte need to be all-in-one solution. Also, seems v3 claim that it tries to reduce compiler magic. So, this looks like redundant compiler magic for me. Besides, for newcomers, who already familiar with other store solutions, it would additional learning curve.

[lukeed]

Ooh okay, I misunderstood then. I thought the script behavior dictated the template.

So the only requirement is that $user has to match my observable with name of user?

Could we not do my suggestion from above, as it allows control of naming / clarity and is only one additional line of code?

import { user } from '...';
let $user = user;

[PaulMaly]

@lukeed or single line:

import { user as $user } from '...';

[lukeed]

The above also addresses the concern that @PaulMaly and I share about this $ injection is too magical.

We already have a portal from script tag into template (RFC1). I don't think we need to add a second one that only applies to a far narrower sect of variables.

IMO our "observables" should just be handling data back and forth. Let RFC1 be the doorway into template. That also allows one to alias the store-value (or parts OF it) before entering the template.

If boilerplate is a concern, we could build out the impl in a way such that any subscription will auto-close on ondestroy. That's basically all we'd be avoiding. The function we pass into subscribe will update top-level vars.

Perhaps this is a compromise? It sets up a subscription behind the scenes and tears down on destroy.

let $user = bind(user, onUpdate);

[Rich-Harris]

So, this looks like redundant compiler magic for me

That's fine — if you don't use it, you don't pay for it. You can safely ignore this RFC :)

Could we not do my suggestion from above, as it allows control of naming / clarity and is only one additional line of code?

import { user } from '...';
let $user = user;

I think it'd be weird if things that were declared in the component got different treatment based on their name. $user.name works precisely because $user is not declared in the component but user is.

Let me try a different tack — some imaginary documentation:

---

State management

As your application grows beyond a certain size, you may find that passing data between components as props becomes laborious.

For example, you might have an <Options> component inside a <Sidebar> component that allows the user to control the behaviour of a <MainView> component. You could use bindings or events to 'send' information up from <Options> through <Sidebar> to a common ancestor — say <App> — which would then have the responsibility of sending it back down to <MainView>. But that's cumbersome, especially if you decide you want to break <MainView> up into a set of smaller components.

Instead, a popular solution to this problem is to define cross-cutting state outside your component tree altogether, in a way that is observable. Svelte comes with a simple mechanism for defining an observable source of data:

import { source } from 'svelte/data'; // 🐃 still working on this...

export const user = source({
  name: 'Kermit',
  species: 'Frog'
});

const unsubscribe = user.subscribe(value => {
  console.log(`name: ${value.name}`); // logs 'name: Kermit'
});

user.set({ name: 'Rizzo', species: 'Rat' }); // logs 'name: Rizzo'

// later, when we no longer want to receive updates
unsubscribe();

We can use these data sources inside our components:

<script>
  import { ondestroy } from 'svelte';
  import { user } from './sources.js';

  let $user;
  const unsubscribe = user.subscribe(value => {
    $user = value;
  });
  ondestroy(unsubscribe);
</script>

<h1>Hello {$user.name}!</h1>

Whenever you call user.set(...), any components that are subscribing to it will automatically update.

That's a lot of boilerplate though, so Svelte includes some optional syntactic sugar — in your components, if you prefix a known value with $, it will assume that you're referring to an observable data source and handle subscribing/unsubscribing on your behalf:

<script>
-  import { ondestroy } from 'svelte';
  import { user } from './sources.js';

-  let $user;
-  const unsubscribe = user.subscribe(value => {
-    $user = value;
-  });
-  ondestroy(unsubscribe);
</script>

<h1>Hello {$user.name}!</h1>

Read-only data sources

...

Derived data sources

...

[nsivertsen]

I really like this. It's very easy to explain the compiler magic involved, and easy to avoid if you don't want it. Count my vote!

[timhall]

This is so exciting, so much more flexible than the current store while offering a simpler API (and implementation). One suggestion: I think an observe "macro" (in the vein of babel-macros, which I think are really great) would really ease some of the boilerplate and user-complexity concerns and would build on the compiler magic that is svelte:

import { observe } from 'svelte/data';
import { user } from './sources';

let $user = observe(user);

// expands to

let $user;
ondestroy(user.subscribe(value => {
  $user = value;
  __update({ $user: true })
}));

I think it'd be straightforward to add typing to observe (e.g. via source.d.ts), so while it'd be a little strange to mix a "macro" with runtime functions, it wouldn't lose any type support. A small issue is that it wouldn't apply outside of a component, but I think that's a reasonable compromise.

declare module 'svelte/data' {
  // ...

  export function observe<T>(source: Source<T>): T;
}

[PaulMaly]

@Rich-Harris this source will support immutability? How this syntax sugar would work with arrays?

import { source } from 'svelte/data';

export const users = source([]);

<script>
  import { users } from './sources.js';
</script>

<ul>
{#each $users as user}
  <li>{user.name}</li>
{/each}
</ul>

<script>
  import { users } from './sources.js';

  let userName = '',
       userEmail = '';
</script>

<input bind:value=userName>
<input bind:value=userEmail>
<button on:click={ ? }>Add user</button>

[Rich-Harris]

@timhall Thanks — though I'm not sure I follow! Surely that adds both more boilerplate (you need to import and use observe) and magic (we introduce an entirely new concept to learn — compiler macros)?

@PaulMaly with update:

<script>
  import { users } from './sources.js';

  let userName = '',
       userEmail = '';

  function addUser() {
    users.update(users => users.concat({
      name: userName,
      email: userEmail
    }));
  }
</script>

<input bind:value=userName>
<input bind:value=userEmail>
<button on:click={addUser}>Add user</button>

[PaulMaly]

@Rich-Harris Ok, that's nice! I'm on a board.

[timhall]

I was thinking more that $... in templates would still work (and be a really nice shortcut), but if you needed to use the source in js for something like a computed, you could use observe:

<script>
import { observe } from 'svelte/data';
import { user } from './sources';

export let message = 'Howdy';
let $user = observe(user);

const text = () => `${message} ${$user.firstname} ${$user.lastname}`;
</script>

<p>{text()}</p>
<input type="text" bind:value=message />

Maybe you could use derived for the above, but there's a bit of strangeness combining reactives and sources in something like a computed, I'm not sure what the best approach would be for the above.

---

Also, another thought: it may be interesting to have an API to "wrap" a reactive into an observable so that it could be passed around. It would allow a computed to be derived from local reactive values and source values:

<script>
import { derived, wrap } from 'svelte/data';
import { user } from './sources';

export let message = 'Howdy';

const text = derived(wrap(message), user, (message, user) => 
  `${message} ${user.firstname} ${user.lastname}`
)
</script>

<p>{$text}</p>
<input type="text" bind:value=message />

This is a good amount more work, but it gives the opportunity to have a computed macro:

<script>
import { computed } from 'svelte/data';
import { user } from './sources';

export let message = 'Howdy';

let text = computed(user, ($user) => `${message} ${$user.firstname} ${$user.lastname}`);

// let text = computed(() => `${message} ${user.firstname}...
// is preferred, but the above is probably necessary for type support

// expands to 

let text = observe(derived(wrap(message), user, (message, $user) =>
  `${message} ${user.firstname} ${user.lastname}`
))
</script>

<p>{text}</p>

This is probably a bit much, but this could be the "compiler magic" referenced in #4 for computed and remove the need for userland memoization or some other change tracking.

[tony-sull]

Very cool seeing Svelte get observables!

The last few months I got pulled into a project updating one of our Angular v4 projects to the latest (v7). The main focus has been on moving our state management over to NgRX (redux-like Angular library) and shifting everything to RxJS 6 Observables.

I'm still not a fan of many (most?) things Angular does as a framework, but the RxJS integration has been a huge win in my opinion. They were kind of dipping their toes into RxJS in Angular 2, but by v6 they moved everything over to it. Combined with their async pipe it really makes tying the UI to observables an easy process.

My understanding of RxJS 6 is that they cleaned up their module setup to make it a lot easier for bundlers to shake out dead code. If that's the case, what would be the trade-offs for Svelte to use RxJS directly rather than a custom version of observables?

A few RxJS-related observations I've made while upgrading the app, I think they could be helpful for Svelte's observables design:

Subscription objects

RxJS returns an object when calling user$.subscriber(user => {}) instead of directly returning the unsubscribe function. There isn't too much extra functionality in the Subscription class, but it does have a flag for closed and helpers for adding/removing logic to run on unsubscribe()

We may only have a need for the unsubscribe function now, but it's probably worth returning an object here for future use.

Pipeable operators

RxJS 6 moved to pipeable operators, i.e.

numbers$.pipe(
    filter(n => n % 2),
    map(n => n * 100)
);

instead of

numbers$
    .filter(n => n % 2)
    .map(n => n * 100);

I believe TC39 Observables still use the RxJS 5 chaining style, but piping operators really makes it easy to add your own custom operators.

Not sure if this fits directly into how Svelte will handle observables, but thought it may trigger some ideas

Combining subscriptions

const user$ = store.getUser();
const games$ = store.getGames();

user$.pipe(
    filter(user => !!user),
    combineLatest(games$),
).subscribe(([ user, games ]) => { ... });

RxJS has a really handy concept of combineLatest, which re-runs after either stream changes, and withLatestFrom, which only runs when the first stream changes but always grabs the latest value from the second stream.

I can't count how many times I've had to turn to this when updating our codebase. I'm using NgRX to have a redux-like setup and separated the store into features, but many views have some computed values that depend on observables from different store feature areas.

Unsubscribing component subscriptions

One handy trick I found for cleaning up subscriptions when a component is destroyed is the takeUntil operator.

I made a base class to handle it for all my components, but the basic idea is to make an observable for the component's ondestroy method and tell any subscriptions to run until that observable fires.

It kept the component cleanup code really clean. One view has 8 or 10 different subscriptions for various computed logic - instead of having to remember to clear each subscription in ondestroy() I can use takeUntil to tell each observable to handle it for me.

import { Subject } from 'rxjs';
import { filter, map, takeUntil } from 'rxjs/operators';
import { store } from '../store/games';

export class GameHeaderComponent {
    game$: Observable<Game>;
    gameEnded$: Observable<boolean>;

    ondestroy$ = new Subject();

    oncreate() {
        this.game$ = store.getGame();

        this.gameEnded$ = this.game$.pipe(
            filter(game => !!game),
            map(game => game.completed === true),
            takeUntil(this.ondestroy$)
        );
    }

    ondestroy() {
        this.ondestroy$.next();
        this.ondestroy$.complete();
    }
}

[PaulMaly]

@tonyfsullivan You'll be able to use Rxjs with Svelte, perhaps even with its syntax sugar using some kind of adaptor (see RFC). I doubt a little, whether we should have a built-in solution in general, but precisely Svelte shouldn't depend on any existing solution.

[Rich-Harris]

RxJS suffers from the same issues as TC39 Observables — they're not a natural fit for UI, in my opinion. I've been rather surprised to see Angular lean into them so heavily.

Newer versions are definitely a lot smaller and more treeshakeable, but I doubt they can compete with 223 bytes, which is the minified size of the basic implementation with subscribe, set and update 😀

As Paul says, though, it'll be easy to use RxJS Observables in components.

[tomcon]

I think that user code will get simpler in many cases — consider the following:

function toggleTodo(id) {
  todos.update(todos => {
    return todos.map(todo => {
      if (todo.id === id) return { id, done: !todo.done, description: todo.description };
      return todo;
    })
  });

Lots of good stuff in the recent comments but that's quite a bit of code to simply update one todo.done value. Maybe a updateOne() taking a key and value might be useful?

[tomcon]

How would you do it currently though?
function toggleTodo(id) {
   let {todos} = this.get()
   const todo = todos.find(t => t.id === id);
   todo.done = !todo.done;
   this.set({ todos })
}

fair point 🤷‍♂️

[PaulMaly]

@tomcon One more nice thing that you'll be able to pull out modifier function and reuse it anywhere:

function toggleKeyById(id, key) {
  return items => items.map(item => (item.id === id ? {...item, [key]: !item[key]} : item))
}

function toggleTodo(id) {
  todos.update(toggleKeyById(id, 'done'))
});

[tony-sull]

@Rich-Harris 223 bytes is crazy! I'm always up for the smallest bundle size I can find

Out of curiosity I just ran webpack-bundle-analyzer against our Angular 7 codebase - 76kB gzipped for Angular core and 8.2kB gzipped for RxJS. RxJS didn't actually take as much as I was expecting, but Angular core is huge and doesn't even count all the extra template logic they compile down to...

Interesting to hear the issues you've seen with other Observable solutions like TC39 and RxJS. I never really gave a second thought of Observables being a stream of values vs. UIs expecting one value. I always viewed the stream concept more as a way of implicitly tying the UI to the value updates over time, with the UI only ever caring about the latest value.

@PaulMaly That's definitely some code I can get behind. Starts to reminds me of Ramda if you end up generalizing it out to helpers for updating a list with a function to match each item, by id in this case, and a function for doing the update, toggling a key here. Not something that needs to come out of the box, and I don't personally like how big Ramda's codebase is, but its great to see it will be so trivial for me to make my own update helpers in a Svelte v3 app if I want/need to

[arggh]

I just read this RFC and noticed Rich wasn't happy with the naming related confusion to TC39's Observables.

A suggestion: as it is a reactive variable we are talking about, why not call it ReactiveVar? Meteor has a very similar concept under that name.

[Rich-Harris]

An informal poll in Discord suggested that 'source' was the most popular of a range of options. So my current thinking is more along these lines:

import { writable, readable, compose } from 'svelte/source';

const count = writable(0);
count.set(1);
count.update(x => x + 1);

const mousePos = readable(set => {
  const handler = e => set({ x: event.clientX, y: event.clientY });
  document.addEventListener('mousemove', handler);
  return () => document.removeEventListener('mousemove', handler);
});

const baz = compose([foo, bar], ([fooValue, barValue]) => fooValue + barValue);

[acstll]

I like this last name proposal, simple and clear. I would suggest using something like compute or derive instead of compose, which to me sounds like functions calling functions.

[tomcon]

+1 for derive or derived

[Rich-Harris]

Ok, so next question:

In the code shown above, derive takes a function that takes an array of observables and returns a value. That's fine for some things (like, I don't know, determining if a form input is valid) but not for others (anything asynchronous, or anything where some updates should be ignored, e.g. mousePos but only when a key is pressed or whatever).

The latter case can be handled by passing set to the callback:

const suggestions = derive([input], async ([q], set) => {
  const data = await fetch(`suggest?q=${encodeURIComponent(q}`).then(r => r.json());
  set(data.suggestions);
});

Should that be a separate function? Or should derive be overloaded such that if callback.length === 2 it is given set, otherwise it's assumed to return a value synchronously?

And in the common case where you're deriving from a single observable, should there be another overload that does away with the arrays?

const suggestions = derive(input, async (q, set) => {
  // ...
});

Or should those all be separate functions?

[acstll]

Maybe that's impossible, but wouldn't it be nice to be able to determine if the callback is an async function and handle a Promise (or thenable) in return? I'm thinking this would be easier for the user:

const suggestions = derive([input], async ([q]) => {
  const data = await fetch(`suggest?q=${encodeURIComponent(q}`).then(r => r.json());
  return data.suggestions;
});

I would always keep the array, even if there's just a single value. That'd be one thing less to learn.

[Rich-Harris]

In some situations you wouldn't want to wait for the promise to be resolved — you might want to add do this sort of thing...

{#await $suggestions}
  <!-- ... -->
{:then list}
  <!-- ... -->
{:catch err}
  <!-- ... -->
{/await}

...and there'd be no way to tell, if it was handling promises on your behalf. So I think an explicit set is probably better.

[RedHatter]

An informal poll in Discord suggested that 'source' was the most popular of a range of options.

How did I miss that?

I really don't like svelte/source. Makes me think I'm using private svelte internals and doesn't speak to what I actually am importing. IMHO any else would be better, but if everyone else disagrees with me I'll swallow my objections.

[timhall]

I'm thinking the passing set in approach will be relatively unusual and could probably be handled a little more functionally by returning a readable from the mapping function and then "unwrapping" that stream of observables into a single observable (see switchMap in RxJS).

import { writable, readable, derived } from 'svelte/...';

const a = writable();
const b = writable();

const c = derived([a, b], ([a_value, b_value]) => readable(set => {
  set(a_value + b_value);
}));

Also, I really like the single-to-single and array-to-array style for the API (you get out what you pass in). This should be a really simple API, I'm loving it!

function derived(input, map) {
  if (Array.isArray(input)) input = all(input);

  return readable(set => {
    let inner_unsubscribe = null;
    const outer_unsubscribe = input.subscribe(value => {
      if (inner_unsubscribe) {
        inner_unsubscribe();
        inner_unsubscribe = null;
      }

      const result = map(value);
      if (!isSource(result)) return set(result);

      inner_unsubscribe = result.subscribe(inner_value => {
        set(inner_value);
      });
    });

    return () => {
      if (inner_unsubscribe) inner_unsubscribe();
      outer_unsubscribe();
    }
  });
}

// (internal)
function all(observables) {
  return readable(set => {
    // ...
  });
}

// (internal)
function isSource(value) {
  // ...
}

An alternative is separating the inner_unsubscribe stuff into a separate unwrap function that can be used to simplify derived, but would result in a little extra nesting for the readables:

const c = unwrap(
  derived([a, b], ([a_value, b_value]) => readable(set => {
    // ...
  })
);

I kind of like the separate unwrap since it should be a much less common usage and allows it to be tree-shaken if it's not built into derived.

function derived(input, map) {
  if (Array.isArray(input)) input = all(input);
  
  return readable(set => {
    return input.subscribe(value => {
      set(map(value));
    });
  });
}

function unwrap(observable) {
  return readable(set => {
    // ...
  });
}

[marvinhagemeister]

@Rich-Harris This is a great proposal and something I feel like more frameworks will shift to in the coming year. It has the potential to update the target components without having to traverse the view tree, which is awesome 👍

Personally I've used MobX, RxJS and now a custom library (not released yet) that builds on valoo to implement an observable like function in my apps and wanted to share some thoughts/learnings.

Preventing Glitches

If you think about it, state management in current web apps resembles quite often a graph-like structure. This becomes even more apparent the larger an app gets. Because of that one has to make sure that there is no observable is called twice when applying the updates. This is usually referred to as a glitch. A simple example would be:

FirstName -+--------------+--- FirstNameOrFullName
           +-- FullName --+
LastName --+

I don't have the best ascii-art skills, but what's happening is that you have some sort of computed or derived observable that either shows the first name or the fullname. It's quite easy to solve and the author of MobX gave a great talk about this problem. Basically one does a dirty run first after which one has the correct order of the observables and then you do a second run and actually apply the changes. Most smaller observable implementations out there suffer from this problem.

Pull can save unnecessary calculations

Another thing to consider is going for a pull-based approach instead of push. That way one can potentially skip updating observables if nobody subscribes to them. Under the hood it works by marking observables as active or inactive. Only functions at the outer edges can mark them as active. In practice this is usually some form of observe or subscribe that is attached from a component.

Functions are smaller than classes

With my own library I noticed that I could get the size down by not putting the methods on the observables class and instead opting for simple functions. So instead of Observable.subscribe() one can rewrite it as observe(someObservable, value => {..}) which is usually smaller.

Batching writes
This becomes important when one wants to go the extra mile for performance. Batching observable writes is necessary to merge multiple updates into one. Especially in a pull based approach this can save quite a bit of cpu cycles. How it works is that one wraps the code in a function and only applies the updates after the function has completed execution. In most libraries this is referred to as action or transaction and it's in spirit quite similar to transactional updates in a database.

All together, here is another example of how the api could look like:

const foo = observable(42);

// One can specify a custom type, but TypeScript is usually smart enough to infer the type itself
const bar = observable<Person>({ name: "foo", age: 42 });

const surname = observable("JSON");
const lastname = observable("Miller");

const fullname = derive([surname, lastname], (first, last) => first + last); 

// At this point no observable has run, only when we actually subscribe
// via `observe` we'll traverse the graph "upwards" and mark parents as "active"
observe(fullname, value => console.log(value));
// Logs: JSON Miller

// Batching writes
action(() => {
  surName("Rich");
  lastName("Harris");
});

I'm really excited about this proposal and it's awesome to see more frameworks go into this direction 🎉

[marvinhagemeister]

A few more thoughts:

How to observe collections?

There is a time when you need to observe changes to a collection-like structure like an Array/Map/Set. These objects heavily rely on the principle of mutation. If for example the user pushes into an array and has code that does reduce over the items or depends on the length of the array s/he will expect it to be triggered. Mobx does this by proxying all methods which make it a no go for IE 11. For that mobx still maintains the v4 release branch which replaces arrays with a custom ObersvableArray class which has the same Methods as an array.

I'm not sure what the best way to solve this would be. The current RFC seems to rely on immutability and as such expects users to trigger an update to the collection themselves.

Deep observablility

Deep observablility closely ties in to the concept of observable collections. Imagine a nested object structure where the leaf object has an observable property. Should the parent be updated, too?

[TehShrike]

@marvinhagemeister you might be interested in warg, which is a tiny state library that solves the glitch problem in essentially the same way Michel Weststrate suggested (though I hadn't seen his talk until your link 😂).

There's also shiz, which is pull-based.

[timhall]

This doesn't quite match the reference implementation below.

Suggested change

	const total = derive([a, b, c], ([a, b, c]) => a + b + c);
	const total = derive([a, b, c], (a, b, c) => a + b + c);

Although, I think 1-to-1 mapping (array-to-array) makes a little more sense to me. I would prefer the following:

import { join } from '...';

const single = derive(a, a => a + 1);
const joined = derive(join([a, b, c]), ([a, b, c]) => a + b + c);
const separate = derive(a, b, c, (a, b, c) => a + b + c);

[Rich-Harris]

I've updated the example implementation. The reason I've steered away from

s = derive(a, b, c, (a, b, c) => a + b + c);

is that the current implementation allows us to do both of these, which I quite like:

simple = derive([a, b], ([a, b]) => a + b);

complex = derive([a, b], async ([a, b], set) => {
  const data = await fetch_from_api(a, b);
  set(data);
});

[timhall]

This is not really germane to reactive stores in particular, but I've been thinking about how to best handle async values in an inherently sync reactive store (the subscribe callback is called immediately with the current value). I think the following may match TC39 observables a little closer (initially async value and handles errors):

const pending = () => new Promise(() => {});
const ok = value => Promise.resolve(value);
const err = error => Promise.reject(error);

function adaptor(observable) {
  return readable(set => {
    const subscription = observable.subscribe({
      next: value => set(ok(value)),
      error: error => set(err(error))
    });

    return subscription.unsubscribe;
  }, pending());
}

Then you use {#await $value}... to work with them in templates. The subsequent values and errors could be set directly (rather than wrapped in Promise.resolve/reject), but I think it's better to stay consistent and use promises throughout and it allows for error checking.

[Rich-Harris]

@marvinhagemeister belated thanks for these thoughts! Let me address them in turn:

Glitches

Yeah, I kind of glossed over this, thinking maybe it wasn't that common to have that kind of dependency graph, and that people would use more full-featured state management systems if necessary. But I figured I may as well have a crack at it anyway: sveltejs/svelte#1882

Pull can save unnecessary calculations

This is sort of a combination of both push and pull. With readable, you supply a function that is invoked when someone subscribes to the store (e.g. in the case of a hypothetical mousePos store, no mouse event listener is created until there's a subscriber, where 'subscriber' will often mean 'component'). derive is the same. So if you had something like this...

const a = writable(1);
const b = derive(a, n => expensively_compute_b(n));
const c = derive(b, n => expensively_compute_c(n));
const d = derive(c, n => expensively_compute_d(n));

...then nothing would be expensively computed until something subscribed to d (or c or b), no matter how many times a was updated.

It's not pull in the sense of only calculating intermediate values when the final value is read, because I don't think that model makes as much sense for Svelte — the assumption is that if you're subscribing, you're going to be using the value on every change, so values propagate through the graph in a push-based fashion, but only once subscriptions have been set up.

Batching writes

I did wonder about including a transaction mechanism. I figured it probably isn't necessary in Svelte's case, since batching happens at the other end:

<script>
  import { writable, derive } from 'svelte/store.js';

  const firstname = writable('JSON');
  const lastname = writable('Miller');

  const fullname = derive([firstname, lastname], names => names.join(' '));
  fullname.subscribe(value => {
    console.log(value); // this will run twice, and the first time will be incorrect
  });

  setTimeout(() => {
    // this will only result in one *component* update, even
    // though the fullname subscriber will fire twice — i.e.
    // there will never be a time when the DOM contains
    // 'JSON Harris'
    firstname.set('Rich');
    lastname.set('Harris');
  });
</script>

<p>firstname: {$firstname}</p>
<p>lastname: {$lastname}</p>
<p>fullname: {$fullname}</p>

How to observe collections?

You're right that it favours immutability, via the update method. We could definitely create stores with the array/map/set APIs that behaved equivalently, if we were so inclined. I think that's far preferable to mucking about with proxies etc.

Deep observablility

That way madness lies! I think it's much better to keep it simple and encourage having a store for each value that should be observable.