oras-go to automatically support uplevel artifact manifest and downlevel registries

[SteveLasker]

We're excited to see OCI adopt the ORAS Artifact manifest, and have broader support across all registries.
While Azure will be adding support in the coming weeks, with ECR and Zot support coming as well, it will take time for the breadth of cloud-based registries to add OCI Artifact manifest support. Even longer for on-prem instances that need to be updated, once the projects/products add support for OCI Artifact manifest.

Users work with multiple registries, that's a given. They may build in github, or on-prem and promote to their cloud provider.
To enable users to work interchangeably across various registries, we'd like an automated, easy way for anyone using oras-go to get this capability. Asking various tools to write this code themselves, or putting on the user to figure out will only delay adoption.
Consider the file system APIs. Users can copy files across FAT32 and NTFS without having to think about it, or the days when users had to figure out if their machine was 32 or 64bit to decide which installer they needed.

A user that calls oras attach should work across ACR, ECR, GCR, GHCR, Harbor, JFrog, Docker Hub.
Users that call notation sign, or other projects like uor, helm etc. should also benefit from this uplevel/downlevel capability.

I can see a force flag, but this shouldn't be the default.

Expectation

• oras attach to docker hub works, using the oci image manifest
• oras attach to acr works, using the oci artifact manifest
• notation sign works across docker hub, acr, ghcr, ...

The details for how to use an oci image manifest are captured here: distribution-spec: Backwards Compatibility

[FeynmanZhou]

oras attach to docker hub works, using the oci image manifest

I remember Docker Hub doesn't support OCI image manifest. We might need to consider a special solution to handle it or wait for Docker Hub to support the OCI artifact manifest. cc @shizhMSFT @qweeah @sajayantony for comments

oras attach to acr works, using the oci artifact manifest

This is an intended result when interacting with ACR. It should work after ACR supports the OCI artifact manifest.

notation sign works across docker hub, acr, ghcr, ...

There was an issue in the Notation repo tracking it notaryproject/notation#444 which was planned to RC.2. But IMO Docker Hub is still a special case to support.

[shizhMSFT]

oras-go will not support any kind of implicit manifest conversions, including converting OCI image manifest from / to OCI artifact manifest. The reason is simple and fundamental as manifest conversions change the digest. Thus, all attached artifacts, including signatures, will be invalidated immediately.

Please note that distribution-spec: Backwards Compatibility only specifies how backwards compatibility of Referrers API works. It does not specify how manifest fallback works. As we have investigated in the notation scenarios, we found the fallback methods are application specific. Therefore, oras-go will not support any kind of manifest fallback / conversion.

However, oras CLI is different from oras-go. oras CLI has ORAS specific manifest fallback logic, and thus oras attach, oras push, and many other commands work with OCI-compliant registries (spec 1.0 or above). Currently, oras attach does not work with DockerHub since DockerHub explicitly rejects t the subject field as Feynman mentioned. The support from DockerHub will be there after the GA of OCI spec 1.1.

[SteveLasker]

Some discussion around how to check for support here: notaryproject/notation#444 (comment)

Putting the burden on each client implementation to duplicate this type of checking feels burdensome.

[shizhMSFT]

Closed by oras-project/oras#782