Earthstar v7.0.0 - Stone Soup

[sgwilym]

This is a ground-up rebuild of Earthstar which has been in the making since last April. It brings a lot of new features, the foundations for more to come, and is about as breaking as breaking changes get.

There are new features, APIs, and concepts. It's a lot to take in, so in addition to this new version of Earthstar, we've got a shiny website with explainers, API tours, technical documentation, and more!

So what's new? Lots.

Peer

There is a new Peer class which acts as the point of contact between your application and Earthstar.

• It's used to store, manage, and access replicas, with methods like peer.getReplica.
• And can be used to synchronise with other peers locally or over the network using Peer.sync. This method accepts lots of 'syncable' things like another Peer or a URL to a replica server.

Syncing

Syncing is now protocol-agnostic. Whereas previously you could only sync locally or over HTTP, it's now possible to sync over theoretically any transport. We've included transports for HTTP, Websockets and BroadcastChannel in our new library earthstar-streaming-rpc. We're to add lots more so that Earthstar can synchronise in many different contexts, like local networks.

We've also made it so that two Peers can determine which shares they have in common without revealing what those shares actually are. This is thanks to something we call a 'salted handshake', and makes it so that you can freely associate with other Peers without worrying that your share addresses are leaking.

There is also a low-level Syncer class you can use to handle scenarios and transports not catered to by the Peer.sync convenience method, e.g. replica servers or web workers (I mentioned we added a transport for BroadcastChannel, right?)

Replica

Previously data could be stored in two kinds of 'Storage', one with a synchronous API and one with an asynchronous API. We've replaced both with a single Replica class with an asynchronous API.

The functionality of this new Replica class is much like the previous storage, but it is now far easier to write persistence drivers for using the IReplicaDriver interface. Thanks to this we've been able to add a new IndexedDBdriver and Sqlite drivers for both Node and Deno.

ReplicaCache

There are many contexts where an asynchronous API can be a royal pain. ReplicaCache sticks a synchronous API in front of a backing Replica by keeping a cache of results for queries. The best part is you can subscribe to events where the cache has been updated, and use this to re-query the cache. We suspect this will be a very useful tool in contexts like a React app!

QueryFollower

There are many applications where you will want to subscribe to new events on a Replica, such as new documents coming in. Using a QueryFollower you can subscribe to events filtered by a Query. Callbacks triggered by this abstraction have special qualities like being called blockingly so that they apply backpressure (this is something that makes certain situations much easier to deal with).

New default ed25519 driver

Earthstar uses ed25519 for author verification. Previous versions of Earthstar could use Node's native crypto, or chloride for Node or the browser. Chloride is fast, but adds a lot to a browser bundle. TweetNacl (also exposed via Chloride) is much smaller, but very slow in comparison.

We've added a new default driver for ed25519 operations using noble/ed25519. This driver is smaller than libsodium-wrappers and much faster than TweetNacl, and works in all JS runtimes

Deno support

Earthstar can now be used in Deno projects:

import * as Earthstar from "https://deno.land/x/[email protected]/mod.ts";

Deno is a really great way to write and run Typescript, and we think it will make it far easier to write personal scripts and utilities using Earthstar.

It's also worth mentioning that Deno is now our primary development environment, and that we were able to make that switch without sacrificing Node support. Deno's support for web platform APIs and all-in-one tooling is something we hope will make contributing to Earthstar that much easier.

Web bundle

Not interested in Node or Deno? We also have a new web bundle, which you can use with a <script> tag:

<script type="module">
  import * as Earthstar from "https://cdn.earthstar-project.org/js/earthstar.bundle.v7.0.0.js";
</script>

Thanks

• Thank you to @cinnamon-bun, who led this reworking!
• Thank you to @tlowrimore, who contributed a nice template querying feature!
• Thank you to NLnet, who funded my own work on this new release!

---

This discussion was created from the release Earthstar v7.0.0 - Stone Soup.