Skip to content

Commit

Permalink
Merge pull request #1 from esl/rebased
Browse files Browse the repository at this point in the history
Add initial code
  • Loading branch information
chrzaszcz authored Mar 23, 2023
2 parents d8808dd + 3d1e1d6 commit d90265d
Show file tree
Hide file tree
Showing 15 changed files with 1,575 additions and 0 deletions.
61 changes: 61 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Erlang CI

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest

container:
image: erlang:25

steps:
- uses: actions/checkout@v3
- name: Compile
run: rebar3 compile
- name: Run tests
run: rebar3 ct --sname=ct1

dialyzer:
runs-on: ubuntu-latest

container:
image: erlang:25

steps:
- uses: actions/checkout@v3
- name: Compile
run: rebar3 compile
- name: Run dialyzer
run: rebar3 dialyzer

xref:
runs-on: ubuntu-latest

container:
image: erlang:25

steps:
- uses: actions/checkout@v3
- name: Compile
run: rebar3 compile
- name: Run xref
run: rebar3 xref

erlfmt:
runs-on: ubuntu-latest

container:
image: erlang:25

steps:
- uses: actions/checkout@v3
- name: Run erlfmt
run: rebar3 fmt --check
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
_build/
# vim temporary files
*.sw*
55 changes: 55 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Cluster ETS

The project adds replication support for Erlang Term Storage (ETS).

It allows to insert or delete objects into ETS tables across several Erlang nodes.

The closest comparison is Mnesia with dirty operations.

Some features are not supported:
- there is no schema
- there is no persistency
- there are no indexes
- there are no transactions (there are bulk inserts/deletes instead).

# Merging logic

When two database partitions are joined together, records from both partitions
are put together. So, each record would be present on each node.

The idea that each node updates only records that it owns. The node name
should be inside an ETS key for this to work (or a pid).

When some node is down, we remove all records that are owned by this node.
When a node reappears, the records are added back.

# API

The main module is cets.

It exports functions:

- `start(Tab, Opts)` - starts a new table manager.
There is one gen_server for each node for each table.
- `insert(Server, Rec)` - inserts a new object into a table.
- `insert_many(Server, Records)` - inserts several objects into a table.
- `delete(Server, Key)` - deletes an object from the table.
- `delete_many(Server, Keys)` - deletes several objects from the table.

`cets_join` module contains the merging logic.

`cets_discovery` module handles search of new nodes.

It supports behaviours for different backends.

It defines two callbacks:

- `init/1` - inits the backend.
- `get_nodes/1` - gets a list of alive erlang nodes.

Once new nodes are found, `cets_discovery` calls `cets_join` module to merge two
cluster partitions.

The simplest `cets_discovery` backend is `cets_discovery_file`, which just reads
a file with a list of nodes on each line. This file could be populated by an
external program or by an admin.
21 changes: 21 additions & 0 deletions rebar.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{dialyzer, [
{warnings, [
no_return,
no_unused,
no_improper_lists,
no_fun_app,
no_match,
no_opaque,
no_fail_call,
no_contracts,
no_behaviours,
no_undefined_callbacks,
unmatched_returns,
error_handling,
underspecs
% overspecs, specdiffs
]}
]}.

%% Enables "rebar3 fmt" command
{project_plugins, [erlfmt]}.
1 change: 1 addition & 0 deletions rebar.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[].
2 changes: 2 additions & 0 deletions run_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/usr/bin/env bash
rebar3 ct --sname=ct1
9 changes: 9 additions & 0 deletions src/cets.app.src
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{application, cets, [
{description, "Clustered Erlang Term Storage"},
{vsn, "0.1"},
{modules, []},
{registered, []},
{applications, [kernel, stdlib]},
{env, []}
% {mod, {cets_app, []}
]}.
Loading

0 comments on commit d90265d

Please sign in to comment.