Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion to schema1 does not fail with Zstd layers, making it uncertain we correctly convert to OCI #2181

Closed
mtrmac opened this issue Nov 10, 2023 · 1 comment · Fixed by #2196
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)

Comments

@mtrmac
Copy link
Collaborator

mtrmac commented Nov 10, 2023

With Podman pushing a zstd:chunked image:

DEBU[0000] Manifest has MIME type application/vnd.docker.distribution.manifest.v2+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.v1+prettyjws, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v1+json] 
DEBU[0000] ... will first try using the original manifest unmodified 
…
DEBU[0000] Writing manifest using preferred type application/vnd.docker.distribution.manifest.v2+json failed: creating an updated image manifest: preparing updated manifest, layer "sha256:208a74e284fc3d9c8397655bd66f5b927d94354a7fe5fcdc084df4aa792c61fe": zstd compression is not supported for type "application/vnd.docker.image.rootfs.diff.tar.gzip" 
DEBU[0000] Trying to use manifest type application/vnd.docker.distribution.manifest.v1+prettyjws… 

^^

DEBU[0000] exporting opaque data as blob "sha256:a416a98b71e224a31ee99cff8e16063554498227d2b696152a9c3e0aa65e5824" 
DEBU[0000] Uploading empty layer during conversion to schema 1 
DEBU[0000] Checking /v2/my-busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 
DEBU[0000] HEAD http://localhost:5000/v2/my-busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 
DEBU[0000] ... already exists                           
Writing manifest to image destination
DEBU[0000] PUT http://localhost:5000/v2/my-busybox/manifests/latest 

The conversion has apparently succeeded, and we would be happy with a schema1 upload!

DEBU[0000] Error uploading manifest latest to localhost:5000/my-busybox: received unexpected HTTP status: 500 Internal Server Error while writing manifest "…" 
DEBU[0000] Upload of manifest type application/vnd.docker.distribution.manifest.v1+prettyjws failed: writing manifest: uploading manifest latest to localhost:5000/my-busybox: received unexpected HTTP status: 500 Internal Server Error 

We just got lucky, the registry doesn’t support schema1

DEBU[0000] Trying to use manifest type application/vnd.oci.image.manifest.v1+json… 

… and finally trying to do the right thing.


At the very least, UpdatedImage must refuse to convert to schema1+zstd, or to accept re-compression to zstd. (Probably primarily Schema1.UpdateLayersInfos.)

The copy.determineManifestConversion code (and the multi-platform analogue?) might also want to be aware of compression algorithms, so that we only try OCI and don’t bother with the schema2/schema1 conversions.

@mtrmac mtrmac added the kind/bug A defect in an existing functionality (or a PR fixing it) label Nov 10, 2023
@mtrmac mtrmac changed the title Conversion to schema1 does not fail with Zstd layers, making it uncertain we correctly covert to OCI Conversion to schema1 does not fail with Zstd layers, making it uncertain we correctly convert to OCI Nov 10, 2023
@mtrmac
Copy link
Collaborator Author

mtrmac commented Nov 21, 2023

The copy.determineManifestConversion code (and the multi-platform analogue?) might also want to be aware of compression algorithms, so that we only try OCI and don’t bother with the schema2/schema1 conversions.

That’s actually required: #2195.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant