crdt-genome

Synopsis

Experiments with Rust CRDTs using Tokio web application framework Axum.

Background

Exploring some ideas of Martin Kleppmann, particularly Local-First software.

See this Podcast

Also this conference talk

Scenario

A group of Axum executeables represents a group of Actors.

Each actor process maintains a genome, represented by a CRDT List.

pub struct Genome {
    genes: ListOfGenes,
}

Each process mutates its genome at random intervals and circulates a CmRDT Op using HTTP POST.

The goal is to observe every genome instance converging to a common value.

Caveat

This little system has serious shortcomings. If a process joins late, or drops out and rejoins, or even loses a single POST request, it will never have the full genome.

Execution

see scripts/run-test.sh

to run some actors, first bring the executeable up to date:

cargo build

Then run instances of the executeable

#!/bin/bash
set -euxo pipefail

target/debug/crdt-genome --actor=0 --count=3 --base=8000 2>&1 | tee actor-0.log &
target/debug/crdt-genome --actor=1 --count=3 --base=8000 2>&1 > actor-1.log &
target/debug/crdt-genome --actor=2 --count=3 --base=8000 2>&1 > actor-2.log &

USAGE:
    crdt-genome --actor <actor> --base <base> --count <count>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -a, --actor <actor>    the actor id of this server
    -b, --base <base>      base port number
    -c, --count <count>    The number of actors

Verification

We want to verify that every genome is converging to the same value.

Since the system is constantly changing, there is no absolute way to do this.

We use HTTP GET to return a string representation of the genome.

$ curl localhost:8000/genome
40fd70816b4f664be2f28766

At a fixed interval, each process polls all the others and compares its genome representation with theirs. For this test, we simply log the result.

$ grep "match" actor-1.log 
Oct 24 15:24:05.500 DEBUG crdt_genome: match count = 6
Oct 24 15:24:10.516 DEBUG crdt_genome: match count = 9
Oct 24 15:24:15.528 DEBUG crdt_genome: match count = 9
Oct 24 15:24:20.540 DEBUG crdt_genome: match count = 9
Oct 24 15:24:25.551 DEBUG crdt_genome: match count = 9
Oct 24 15:24:30.561 DEBUG crdt_genome: match count = 9
Oct 24 15:24:35.573 DEBUG crdt_genome: match count = 9
Oct 24 15:24:40.587 DEBUG crdt_genome: match count = 9
Oct 24 15:24:45.601 DEBUG crdt_genome: match count = 9
Oct 24 15:24:50.613 DEBUG crdt_genome: match count = 9
Oct 24 15:24:55.623 DEBUG crdt_genome: match count = 9
Oct 24 15:25:00.635 DEBUG crdt_genome: match count = 9
Oct 24 15:25:05.646 DEBUG crdt_genome: match count = 9
Oct 24 15:25:10.657 DEBUG crdt_genome: match count = 9