Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High-level design doc #18

Merged
merged 1 commit into from
Nov 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/Client_Libraries/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,74 @@ The following languages are currently supported:
- [Python](./1_python) (in progress)
- [Rust](./2_rust) (in progress)

## Unified Experience

The objective of ORAS is simple as transferring artifacts from one place to another.

In the conventional [client-server model](https://en.wikipedia.org/wiki/Client%E2%80%93server_model), the operation of downloading artifacts from the remote registries is referred to as **pull**, and the operation of uploading artifacts to the remote registry is referred to as **push**.

This model can be generalized by abstracting the client and the server as **targets** so that pull and push can be viewed as **copying** from one target to another (see [**Copy API** oras-project/oras-go#8](https://github.com/oras-project/oras-go/pull/8)).
For instances,

- Copy from memory to a remote registry.
- Copy from a remote registry to a local file folder.
- Copy from a remote registry to another remote registry.
- Copy from memory to a local file folder.

### Targets

Generally, a target is a [content-addressable storage (CAS)](https://en.wikipedia.org/wiki/Content-addressable_storage) with tags.
All blobs in a CAS are addressed by their [descriptors](https://github.com/opencontainers/image-spec/blob/main/descriptor.md).

To retrieve a blob,

1. Get a descriptor. Optionally, it can be resolved by a tag.
2. Query the blob with a descriptor.

To store a blob,

1. Store the blob directly in the CAS. A descriptor will be returned.
2. Optionally, associate the returned descriptor with a tag.

It is worth noting that a target is not equal to a registry.

- Blobs can be tagged in a target but not in a registry.
- Tag list is available in a registry but not always available in a target.

### Graphs

Besides plain blobs, it is natural to store [directed acyclic graphs (DAGs)](https://en.wikipedia.org/wiki/Directed_acyclic_graph) in a CAS.
Precisely, all blobs are leaf nodes and most manifests are non-leaf nodes.

An artifact is a rooted DAG where its root node is an [OCI manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md).
Additionally, artifacts can be grouped by an [OCI index](https://github.com/opencontainers/image-spec/blob/main/image-index.md), which is also a rooted DAG.

Given a node of a DAG in a CAS, it is efficient to find out all its children.
Since CASs are usually not enumerable or indexed, it is not possible to find the parent nodes of an arbitrary node.
Nevertheless, some CASs choose to implement or partially implement the functionality of parent node finding.
For instances, registries with [Manifest Referrers API](https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md) support are CASs with partially implementation where parent node finding is only available for manifest nodes.

### Extended Copy

With the concepts above, we can formally define that

- **Copy** is a function to replicate a rooted DAG from one CAS to another.
- **Extended Copy** is a function to replicate a DAG from one CAS to another.

It is worth noting that extended copy is possible only if the source CAS supports parent node finding.
Based on the scenarios, extended copy can have many options such as opting to copy a sub-DAG rooted by a certain node and all its parent nodes of a certain depth with / without their children.

Optionally, node filters or even node modifiers can be attached to a copy process for advanced scenarios.

Related issues:

- [**Support copy of images and associated references** oras-project/oras-go#29](https://github.com/oras-project/oras-go/issues/29)
- [**Copy Artifact Reference Graph** oras-project/oras#307](https://github.com/oras-project/oras/issues/307)

Hint: A [polytree](https://en.wikipedia.org/wiki/Polytree) is a DAG.

![polytree](index/polytree.svg)

## Paradigms

🚧 *Note: this section is contingent on
Expand Down
119 changes: 119 additions & 0 deletions docs/Client_Libraries/index/polytree.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.