-
Notifications
You must be signed in to change notification settings - Fork 0
Storage Layer
Haifeng Liu edited this page Aug 5, 2024
·
6 revisions
for testing or sandboxing
We need an 'append-only distributed filesystem' for docstore and index snapshots. And there are multiple possible solutions.
-
CubeFS is used as the shared storage infra in the private cloud environments, so all files are stored on CubeFS.
-
In the public clouds such as AWS/Azure/GCP, DFS can still be leveraged as the docstore for fast write performance, and cloud storage like S3, Azure Blob Storage or Google Cloud Storage is used as the index snapshot storage for lower cost.
-
CalmFS, highly tailored for calmseek
concepts: universe, namespace, append-only file (AOF), extent; metaserver, diskserver
operations: CreateUniverse(), CreateNamespace(), CreateFile(), Append(), Read()