-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pulling with digest reference failed from v2 registry #4567
Comments
A question not related to the error, Is this the correct way to access images? Externally I have a route to the registry, registry.myorg.com.au with auth that I can login to, pull/push too no problem. I can tag registry.myorg.com.au/softint/php-dev:latest and push that to the registry. Interally for building I can't seem to access registry.myorg.com.au, I'm guessing I need to authenticate but can't find any documentation on authing to a private docker registry during the build process. Ideally, I could always use registry.myorg.com.au whether Im internal to OS or external in a dev env. |
here are the docs on how to setup docker credentials: if your registry requires auth, perhaps that is why the pull is failing? |
@andrew-jones is 172.30.6.174:5000 the integrated registry you deployed via |
I'm fighting the same thing. Currently running 1.0.5. I'm using the integrated registry deployed via oadm registry. A copy of my failing build is included below. I've set the pullSecret to the value of the pushSecret. The build fails with the same error as above, "pulling with digest reference failed from v2 registry". The registry complains that it didn't login. When I manually run docker pull to fetch the image and then retrigger the build, it runs successfully and pushes the updated image. Both the source and target image are in the same project. I'm using using the docker secrets that were created for the builder serviceaccount. {
"kind": "Build",
"apiVersion": "v1",
"metadata": {
"name": "dias-3-4581510590",
"namespace": "mocks",
"selfLink": "/oapi/v1/namespaces/mocks/builds/dias-3-4581510590",
"uid": "5c9a6afb-5d76-11e5-be07-06e5f2bfc187",
"resourceVersion": "55135",
"creationTimestamp": "2015-09-17T19:57:44Z",
"labels": {
"app": "dias",
"buildconfig": "dias"
}
},
"spec": {
"serviceAccount": "builder",
"source": {
"type": "Git",
"git": {
"uri": "[email protected]:/docker/dias.git"
},
"sourceSecret": {
"name": "gitlabsecret"
}
},
"strategy": {
"type": "Docker",
"dockerStrategy": {
"from": {
"kind": "DockerImage",
"name": "172.30.63.116:5000/mocks/tomcat6@sha256:39a0ba9e84903070fff0e268a7bd7f389d7b4aa887f67fd875fc725aa8259593"
},
"pullSecret": {
"name": "builder-dockercfg-1rgjh"
}
}
},
"output": {
"to": {
"kind": "ImageStreamTag",
"name": "dias:latest"
},
"pushSecret": {
"name": "builder-dockercfg-1rgjh"
}
},
"resources": {}
},
"status": {
"phase": "Failed",
"startTimestamp": "2015-09-17T19:57:45Z",
"completionTimestamp": "2015-09-17T19:57:55Z",
"duration": 10000000000,
"config": {
"kind": "BuildConfig",
"namespace": "mocks",
"name": "dias"
}
}
} |
Yes, and then the build works. However everytime the upstream image is changed, the downstream build kicks off again and fails until I manually pull it. |
@amdonov can you paste the logs from the registry container from around the time the build fails to pull? |
time="2015-09-17T20:39:13Z" level=debug msg="authorizing request" http.request.host="172.30.63.116:5000" http.request.id=84da5094-2181-45dd-86ae-5aef546da00f http.request.method=GET http.request.remoteaddr="10.1.1.1:60625" http.request.uri="/v2/" http.request.useragent="docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.11.1.el7.x86_64 os/linux arch/amd64" instance.id=8a7a5963-0d65-434c-bcad-c876a166d333 |
@ncdc nope, as long as the secret exists on the service account the build will use it. |
@bparees I dunno what's going on, then. This doesn't appear to be a registry issue. It looks like the secret isn't being used. |
if you add the private registry image to FROM in docker build and then
|
the logical conclusion would be that we're not finding a secret on the service account for the pull repository. can you dump the secrets from your builder service account?
@mfojtik the builder has a From definition in it, that is what we should be using for the pull secret and substituting into the dockerfile. if we aren't doing that, it's a bug in how we handle docker builds. |
There's been enough difficulty debugging this sort of issue that we should probably add better logging around the auto selection of pull secrets (at some verbosity level) |
@csrwng could this have something to do with the go-dockerclient auth config changes recently? |
Yes this looks like the same issue, which should be fixed in master. One thing to note when debugging these issues is that docker pull is not the same as docker build because the format of the auth header is different. For pull only the relevant credentials are passed in, while for build, the entire map of credentials is passed. |
Could you point to the relevant files/commits? I'd like to patch my install if possible. |
Should I just update the go-dockerclien dependencyt? |
https://github.com/openshift/origin/pull/4535/commits Was where origin got updated |
The changes are originally here |
Thanks for the help. I upgraded to 1.0.6, and it fixed the problem. |
I just ran into this too. I am running a binary I built from source a week ago. Do I need to upgrade to 1.0.6 too already? This is the output of the docker registry log I am seeing: time="2015-09-21T19:58:29Z" level=debug msg="authorizing request" http.request.host="172.30.131.51:5000" http.request.id=94bd0dc2-aee3-42f9-916c-bc97ebdf64da http.request.method=GET http.request.remoteaddr="172.17.42.1:46916" http.request.uri="/v2/" http.request.useragent="docker/1.7.1 go/go1.4.2 kernel/3.10.0-229.11.1.el7.x86_64 os/linux arch/amd64" instance.id=f759ce2a-f7b1-481f-95c8-a426432b21a0 |
@deanpeterson yes, if you're hitting the same issue/root cause. |
The fix went into the master branch 17 days ago, yet I built from the master only 7 days ago and I am having this problem. Build error: exit status 128 |
Well, that worked. I downloaded the latest 1.0.6 binary and I am able to pull from the registry. |
I have two build configs, one building an apache image on centos 7. The other extending this apache image and installing php.
The apache image builds successfully and I can pull/run the image.
The PHP Dockerfile contains,
FROM 172.30.6.174:5000/softint/apache-centos7:latest
Which is my private repo IP/namespace/image name:tag
My template for the php image builder looks like this,
The build kicks off, and almost immediately returns this error,
I've found a bug reported for Openshift Enterprise,
https://bugzilla.redhat.com/show_bug.cgi?id=1255502
Also this (though I can't see any content),
https://access.redhat.com/solutions/1599503
The text was updated successfully, but these errors were encountered: