A custom block storage example to manage (store/index etc.) block (aka file/document content) operations in a p2p manner.
blockstorage
is a p2p service that I wrote to understand the internal workings of block management. Important topics that I focusing on:
- Libp2p
- Kademlia DHT
- Content Routing usage
- Peer Routing usage
- P2P Protocol usage
- Peer Discovery (Boostrap Peer, Rendezvous, etc.)
- Kademlia DHT
- IPLD
- Concepts & Spec
- Proof of Concept custom implementation (via protobuf)
- Formats
- Multi Address
- CID (aka content identifier)
-
go-objectstore-lib : Contains base abstraction of storing encoded/serialized objects
-
go-objectstore-fs : Contains file system based implementation/functionality to store encoded/serialized objects
- api/proto : Contains protobuf definitions
- blockpb : Contains protobuf and grpc related objects according to store.proto
- blockpb/store_aux.go : Contains auxiliary functions/definitions to extends proto objects
- util/ctx.go : Contains context cheking helper function and error definitions
- peer : Contains p2p functions and definitions.
- errors.go : Contains
blockstorage
error definitions and error checking functions - grpc : Contains
blockstorage
GRPC endpoint definition and RPC function implementations - impl.go : Contains
BlockStorage
interface implementation and helper functions - options.go : Contains
BlockStorage
construction option definitions - peer.go : Contains p2p related protocol definition and functions
- storage.go : Contains
blockstorage
construction andBlockStorage
interface definition
blockstorage
is still in progress.
- Prepare Step
- Project Structure
- Makefile tasks
- Grpc Step
- proto file
- generate objects
- add makefile task
- implement service endpoint
- Peer Step
- protocol definition
- store definition
- Future
- add block indexing mechanism
- add garbage collection trigger mechanism (to temporaryStore)
- add long term storage trigger