Unable to push to harbor image with http #1917
Comments
|
Your total build time is reported as 2m40s which doesn't seem long! Here's a few things to try:
|
You mean
I have tried, same issue
Below is the command i have used it produces too much logs posting here few which I feel is relevant |
|
The network trace wasn't captured. It should have logged HTTP requests and responses like the sample below. Double-check the path to the
You can attach a file to a comment. Just drag and drop. Lastly, In your case, |
|
It's strange a HTTP using 443 port: Execution failed for task :jib.
To my harbor (on premisses) aways works (https and http), but i'm using a robot account. I'm sending robotAccount and token out of POM (my case), in gitlab-ci command line. |
I double checked and triple checked the path and logging.properties but still i'm not getting network logs, below is the properties below is the gradle command i'm using Also to isolate the issue, i have downloaded starter project from https://start.vertx.io/ and added jib, even in the starter project no network logs and push is failing. Attached the sample project, only diff is i have removed username/password, if any of you want to try out i can share my username password privately. |
This comment has been minimized.
This comment has been minimized.
Hmm... Gradle seems weird. I am trying it too, but often I don't get network traces either. |
That was a paste error, i did try with /sample/xyz Also yes i have tried wit absolute path for logging.properties, still no network logs. |
|
Also to note, when I try with AWS ECR it works fine, so it looks like some bug with my harbor instance and maybe because its using http. |
|
Found the issue with network logs, gradle generates log only if i add Attaching the logs, from the logs found 401 errors, so tried removing auth block and used helper from docker even that's giving 401, however push in docker works fine. |
|
@raizoor correctly identified the problem: your registry is serving across HTTP on port 80, and not HTTPS on port 443: I guess your firewall is configured to leave connections dangling. See the docs for |
|
@riyaz Thanks for the logs. (I prefer raw logs though. You don't have to cut them down yourself.) BTW, I was also facing issues in obtaining network logs with Gradle, so thanks for the tip. But it is still very unpredictable with Anyways, the reason you get 401 now is probably because you didn't put @briandealwis the user has been using |
|
@riyaz Run again with -DsendCredentialsOverHttp=true and attach the same logs. |
|
I have attached logs again, below is the command i have used to build |
|
@chanseokoh see bellow, JIB only sent a HTTPS request |
|
Just a few follow-ups:
I meant a second
That's interesting because when I try reproducing the connection fails due to a |
|
Jib currently always tries to connect via HTTPS and only then retries with HTTP when @chanseokoh this might be a motivating case for turning |
|
@riyaz I noticed there is a difference between Jib 1.4.0 and 1.5.0 regarding |
|
@raizoor if you look at the |
|
Regarding no network logs with gradle, --no-daemon seems to work sometime only, if we downgrade the gradle version to 4.10.3 it always works. I have requested our infra team to setup ssl, meanwhile i have switched to aws ecr for now. Attached the log with 1.5, It was keep on trying for more than 20 mins so I had to abort, typically it takes max 30 seconds to upload my image using docker cli. |
|
My test above used Jib 1.2.0 and I had the same result on 1.4.0. With Jib 1.5.0 I do see Jib cascading from HTTPS to HTTP. I see lengthy timeouts though (1m15s or so) on any request though. |
|
@riyaz looked at your last log. No wonder it takes > 20 minutes (with 1.5.0). At least it is indeed working but very inefficient because Jib first tries HTTPS (port 443) before trying HTTP (port 80) for every access to your registry. And this HTTPS-to-HTTP failover can happen only when the HTTPS attempt times out. The log shows that the timeout is about 75 seconds. That is, for every HTTP request Jib makes to your registry, it first has to wait for ~75 seconds. But I do see that the plain That said, if you specify the port But, I strongly recommend you make the port 443 work as a registry and close the insecure registry at port 80. I'm emphasizing again that by sending credentials over HTTP, anyone in the wild can capture and steal your username and password easily.
Given that you don't have a secure registry, ECR seems rather the right choice. I would continue using it, at least until you set up SSL.
Thanks for the info. It's good to know. We are baffled too. |
|
@chanseokoh this timeout (75s) has been returned by @riyaz network or JIB? |
|
@raizoor both the default connection timeout and the read timeout are set to 20 seconds by default (configurable with |
I did an investigation, and I think it is the bug in recent Google HTTP Client versions that does not honor the given connection timeout. (Haven't tested read timeout yet.) Jib 1.5.0 is affected, and I am positive that the timeout will work correctly on 1.4.0. |
|
@riyaz we all know what exactly is happening and why. A few issues were identified:
I'll go ahead and close this issue. As I said, ECR seems the right choice at the moment. I would continue using it, at least until you set up SSL. And try to avoid using |
Environment:
Description of the issue:
I have insecure harbor registry for now and trying to push it and pushing it with property sendCredentialsOverHttp=true but push always times out
Expected behavior:
Should be able to push
Steps to reproduce:
jib-gradle-pluginConfiguration:jib { to { allowInsecureRegistries = true tags = ['latest', '1.0.0-SNAPSHOT'] auth { username = 'xyz' password = 'xyz' } } container { ports = ['8080'] mainClass = "$mainClassName" args = ["run", mainVerticleName, "-conf staging.json"] jvmFlags = ['-Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory'] } }Log output:
Additional Information:
Push works fine with docker command-line
The text was updated successfully, but these errors were encountered: