This is an implementation of an event storage in .NET for running analytical workloads efficiently on a single machine. Core features:
- events are automatically split into the chunks;
- chunks are encrypted and compressed;
- designed for batching operations (high throughput);
- supports single writer and multiple concurrent readers.
This storage takes ideas from the Message Vault, which was based on the ideas of Kafka and append-only storage in Lokad.CQRS
Analytical pipeline on top of this library was deployed at SkuVault to run reporting and data analysis tasks on 1.5B of events (400GB+ of data). You can read more about it in Real-time Analytics with Go and LMDB.
This .NET project was eventually ported to golang for the reasons outlined in the previous article. All further development happens in cellar, this .NET version is currently unmaintained.
3-clause BSD license.