From 2021a0d22386467fba38bc6323d786e7bf64e032 Mon Sep 17 00:00:00 2001 From: Shiwei Zhang Date: Fri, 5 Nov 2021 16:19:03 +0800 Subject: [PATCH] high level design doc Signed-off-by: Shiwei Zhang --- docs/Client_Libraries/index.md | 68 +++++++++++++ docs/Client_Libraries/index/polytree.svg | 119 +++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 docs/Client_Libraries/index/polytree.svg diff --git a/docs/Client_Libraries/index.md b/docs/Client_Libraries/index.md index cc46e2d0..cbc9faf0 100644 --- a/docs/Client_Libraries/index.md +++ b/docs/Client_Libraries/index.md @@ -8,6 +8,74 @@ The following languages are currently supported: - [Python](./1_python) (in progress) - [Rust](./2_rust) (in progress) +## Unified Experience + +The objective of ORAS is simple as transferring artifacts from one place to another. + +In the conventional [client-server model](https://en.wikipedia.org/wiki/Client%E2%80%93server_model), the operation of downloading artifacts from the remote registries is referred to as **pull**, and the operation of uploading artifacts to the remote registry is referred to as **push**. + +This model can be generalized by abstracting the client and the server as **targets** so that pull and push can be viewed as **copying** from one target to another (see [**Copy API** oras-project/oras-go#8](https://github.com/oras-project/oras-go/pull/8)). +For instances, + +- Copy from memory to a remote registry. +- Copy from a remote registry to a local file folder. +- Copy from a remote registry to another remote registry. +- Copy from memory to a local file folder. + +### Targets + +Generally, a target is a [content-addressable storage (CAS)](https://en.wikipedia.org/wiki/Content-addressable_storage) with tags. +All blobs in a CAS are addressed by their [descriptors](https://github.com/opencontainers/image-spec/blob/main/descriptor.md). + +To retrieve a blob, + +1. Get a descriptor. Optionally, it can be resolved by a tag. +2. Query the blob with a descriptor. + +To store a blob, + +1. Store the blob directly in the CAS. A descriptor will be returned. +2. Optionally, associate the returned descriptor with a tag. + +It is worth noting that a target is not equal to a registry. + +- Blobs can be tagged in a target but not in a registry. +- Tag list is available in a registry but not always available in a target. + +### Graphs + +Besides plain blobs, it is natural to store [directed acyclic graphs (DAGs)](https://en.wikipedia.org/wiki/Directed_acyclic_graph) in a CAS. +Precisely, all blobs are leaf nodes and most manifests are non-leaf nodes. + +An artifact is a rooted DAG where its root node is an [OCI manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md). +Additionally, artifacts can be grouped by an [OCI index](https://github.com/opencontainers/image-spec/blob/main/image-index.md), which is also a rooted DAG. + +Given a node of a DAG in a CAS, it is efficient to find out all its children. +Since CASs are usually not enumerable or indexed, it is not possible to find the parent nodes of an arbitrary node. +Nevertheless, some CASs choose to implement or partially implement the functionality of parent node finding. +For instances, registries with [Manifest Referrers API](https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md) support are CASs with partially implementation where parent node finding is only available for manifest nodes. + +### Extended Copy + +With the concepts above, we can formally define that + +- **Copy** is a function to replicate a rooted DAG from one CAS to another. +- **Extended Copy** is a function to replicate a DAG from one CAS to another. + +It is worth noting that extended copy is possible only if the source CAS supports parent node finding. +Based on the scenarios, extended copy can have many options such as opting to copy a sub-DAG rooted by a certain node and all its parent nodes of a certain depth with / without their children. + +Optionally, node filters or even node modifiers can be attached to a copy process for advanced scenarios. + +Related issues: + +- [**Support copy of images and associated references** oras-project/oras-go#29](https://github.com/oras-project/oras-go/issues/29) +- [**Copy Artifact Reference Graph** oras-project/oras#307](https://github.com/oras-project/oras/issues/307) + +Hint: A [polytree](https://en.wikipedia.org/wiki/Polytree) is a DAG. + +![polytree](index/polytree.svg) + ## Paradigms 🚧 *Note: this section is contingent on diff --git a/docs/Client_Libraries/index/polytree.svg b/docs/Client_Libraries/index/polytree.svg new file mode 100644 index 00000000..14c2c6e6 --- /dev/null +++ b/docs/Client_Libraries/index/polytree.svg @@ -0,0 +1,119 @@ + + + + + + +polytree + + + +A + +A + + + +F + +F + + + + +C + +C + + + +A->C + + + + + +D + +D + + + +A->D + + + + + +B + +B + + + +E + +E + + + +B->E + + + + + +G + +G + + + + +B->D + + + + + +I + +I + + + + +G->I + + + + + + +D->F + + + + + +D->G + + + + + +H + +H + + + +H->I + + + + +