question: Failed to push image with nydusify

[gaocegege]

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

Version of nydus being used (nydusd --version)

Version: v2.2.1
Git Commit: 04fb92c
Build Time: 2023-05-04T07:28:49.522159639Z
Profile: release
Rustc: rustc 1.66.1 (90743e729 2023-01-10)

Version of nydus-snapshotter being used (containerd-nydus-grpc --version)

Version: v0.9.0
Revision: 2c811d7b2e57a556791c3bedfbaf1db95267131d
Go version: go1.20.3
Build time: 2023-06-17T11:10:15

Kernel information (uname -r)

command result: uname -r

GNU/Linux Distribution, if applicable (cat /etc/os-release)

command result: cat /etc/os-release

containerd-nydus-grpc command line used, if applicable (ps aux | grep containerd-nydus-grpc)

sudo containerd-nydus-grpc --config /etc/nydus/config.toml --log-to-stdout

client command line used, if applicable (such as: nerdctl, docker, kubectl, ctr)

sudo nydusify --debug convert --oci-ref --source gaocegege/gradio-hello-world:v3 --target gaocegege/gradio-hello-world:v3-zran

Screenshots (if applicable)

Details about issue

The image is converted, but cannot be pushed to docker hub. I already logged in with nerdctl.

Logs:

DEBU[2023-07-10T10:38:37Z] checking and pushing to                       digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json size=4287 url="https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran"
DEBU[2023-07-10T10:38:37Z] do request                                    digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=containerd/1.7.0+unknown request.method=HEAD size=4287 url="https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran"
DEBU[2023-07-10T10:38:37Z] fetch response received                       digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json response.header.content-length=17189 response.header.content-type=application/vnd.docker.distribution.manifest.v1+prettyjws response.header.date="Mon, 10 Jul 2023 10:38:37 GMT" response.header.docker-content-digest="sha256:2968069c9d0a97c18813af95c02dac76220eb920a1e877c15c14f0a715ec61db" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=34.122.235.61 response.header.etag="\"sha256:2968069c9d0a97c18813af95c02dac76220eb920a1e877c15c14f0a715ec61db\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" size=4287 url="https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran"
DEBU[2023-07-10T10:38:37Z] do request                                    digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json request.header.content-type=application/vnd.oci.image.manifest.v1+json request.header.user-agent=containerd/1.7.0+unknown request.method=PUT size=4287 url="https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran"
DEBU[2023-07-10T10:38:37Z] fetch response received                       digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json response.header.content-length=55 response.header.content-type=application/json response.header.date="Mon, 10 Jul 2023 10:38:37 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=34.122.235.61 response.header.strict-transport-security="max-age=31536000" response.status="404 Not Found" size=4287 url="https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran"
DEBU[2023-07-10T10:38:37Z] unexpected response                           body="{\"errors\":[{\"code\":\"NOTFOUND\",\"message\":\"not found\"}]}\n" digest="sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54" mediatype=application/vnd.oci.image.manifest.v1+json resp="&{404 Not Found 404 HTTP/1.1 1 1 map[Content-Length:[55] Content-Type:[application/json] Date:[Mon, 10 Jul 2023 10:38:37 GMT] Docker-Distribution-Api-Version:[registry/2.0] Docker-Ratelimit-Source:[34.122.235.61] Strict-Transport-Security:[max-age=31536000]] 0xc00342fa00 55 [] true false map[] 0xc0015d9700 0xc003636210}" size=4287
FATA[2023-07-10T10:38:37Z] push image: failed commit on ref "manifest-sha256:b88b9b9a11d1f31f105d559342c5caf98349d6c42fd36afd46d5232937bd8d54": unexpected status from PUT request to https://registry-1.docker.io/v2/gaocegege/gradio-hello-world/manifests/v3-zran: 404 Not Found 

[gaocegege]

I could push images with nerdctl successfully, thus I think it should be related to https://github.com/dragonflyoss/image-service/blob/master/contrib/nydusify/pkg/converter/provider/provider.go

[imeoer]

@gaocegege It seems related to the code:

https://github.com/containerd/nydus-snapshotter/blob/e02116ca6a986dbee9d20175e7c7e8a12e495871/pkg/converter/convert_unix.go#L1024

Looks like the docker hub doesn't support the OCI reference type (subject field) yet, we will try to add an option to fix it.

[imeoer]

Working on containerd/nydus-snapshotter#509 and https://github.com/goharbor/acceleration-service first to fix it.

[gaocegege]

Thanks in advance!

[imeoer]

Waiting for #1384 to be merged.