[bug] conan.bintray.com has invalid/no certificate and fails in redirection loop

[snailcatcher]

Steps to reproduce (Include if Applicable)

Since this morning (12.04.2021) I can't connect to https://conan.bintray.com/ successfully.
One problem is that it seems there is no valid server certificate on the server side.
After disabling verify_ssl in the remotes there will be an exception because of too many redirects:

ERROR: Exceeded 30 redirects.. [Remote: conan-center]

Same error can be reproduced by open the URL via the webbrowser of your choice.

Is this because of the shutdown for bintray? Isn't this dated to 1st May?

[Zuwei-Zhao]

Hi, I also encountered the same error today:
ERROR: Failed requirement 'boost/1.75.0' from 'conanfile.py'
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/ping (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

Unable to connect to conan-center=https://conan.bintray.com

1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
   Then try again.

Any advices on how to fix it? Thanks! @memsharded

[haroal]

Hi,
Same error. I found this : https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

April 12th, 26th, 2021 | We will have some short service brown-outs to remind users about the services that are going away on May 1st. (Specific hours will be advertised in the Bintray status page.)

It seems that Bintray repo is going deprecated.

[memsharded]

Please read our recent blog too: https://blog.conan.io/2021/03/31/Bintray-sunset-timeline.html

It was also announced in our blog some time ago: https://blog.conan.io/2021/02/05/JFrog-announces-sunset-bintray.html, and shared in social channels and mailing lists.

[snailcatcher]

@haroal & @memsharded
You both were right:
Here is the official Schedule from bintray:
https://status.bintray.com/incidents/4mzph0y8l6hk

[memsharded]

The certificates issue is mostly a DNS propagation delay thing. It should probably resume working as soon as DNS propagates.

[snailcatcher]

Ok, thanks for clarifying, I will try again later or tomorrow.

[bldrvnlw]

Isn't there still an issue that the conans/client/cache/remote_registry.py still references conan.bintray.com so a fresh install of conan (1.35) adds conan-center https://conan.bintray.com by default?

[voidbar]

Hey @memsharded, there is a problem with the way this deprecation is being handled, here are a few key points:

• The remotes docs page, and the "bintray is sunsetting" page doesn't specify how to update to the correct configuration and host.
• A fresh install of conan 1.35 doesn't sets the right conan remote by default (or not by default for that matter)

My company's CI processes just broke out of the blue, and fixing this issue doesn't seem to be trivial as it should be.

Can you suggest a hotfix?

Thanks

[Zuwei-Zhao]

Hi, so after the deprecation of bintray, what remote server should we use?

[jgsogo]

URL https://conan.bintray.com will keep working as usual. Besides these DNS-related issues and some maintenance windows during April, everything will be working the same. Default conan-center repository is still https://conan.bintray.com.

[matheusgomes28]

So just to be clear: After the DNS changes take effect, we should still use https://conan.bintray.com as the default remote? From what I understand in the JFrog & Conan blog posts, the bintray remote is getting deprecated. So what's the default remote we should be using ???

[rodolfoap]

Again, bintray is not working. And I find no alternative to conan-center (using https://conan.io). You can check the problem using this gist:

https://gist.github.com/rodolfoap/b0ac9fb02e6d5c8c04bd1612ebd9d207

There, I'm just trying to conan-install catch2 (which can be found using the web site: https://conan.io/center/catch2), but it does not work, even changing the conan-center URL.

How to fix the issue?

[aegis-dev]

What about https://conan.io/center/ ? Is it the main package repository now? Or is it only used for package indexing? If it's the main repository for packages now, shouldn't it be added to conan remote list with conan installation?

[jgsogo]

URL https://conan.bintray.com will keep working as usual. Besides these DNS-related issues and some maintenance windows during April, everything will be working the same. Default conan-center repository is still https://conan.bintray.com.

➡️ URL https://conan.bintray.com will be redirected to a better repository infrastructure. No action will be needed.

[rodolfoap]

https://conan.io/center/ is not a repository address. It gives an "ERROR: <!doctype html>..." using the mentioned Gist https://gist.github.com/rodolfoap/b0ac9fb02e6d5c8c04bd1612ebd9d207

[aegis-dev]

so the question is where is the main repository right now? I am wondering if I have to update dependency package list.
For instance, 'boost/1.71.0@conan/stable'. I found boost package indexed https://conan.io/center/boost?version=1.71.0. Where this package stored? Will I have to update it's reference to just boost/1.71.0?

I am asking questions because whole infrastructure halted and I apparently can't test it myself right now.

[voidbar]

@aegis-dev

so the question is where is the main repository right now? I am wondering if I have to update dependency package list.
For instance, 'boost/1.71.0@conan/stable'. I found boost package indexed https://conan.io/center/boost?version=1.71.0. Where this package stored? Will I have to update it's reference to just boost/1.71.0?

I am asking questions because whole infrastructure halted and I apparently can't test it myself right now.

If I understand correctly, there is no alternative. We just have to sit tight and wait for a fix for conan.bintray.com...
There are internal changes that the jfrog guys are working on on this server.

[briandoodyie]

conan remote update conan-center https://api.bintray.com/conan/conan/conan-center worked for me.

So after may 1st we still will be able to use remote repository? or is there something i'm missing?

[saukijan]

conan remote update conan-center https://api.bintray.com/conan/conan/conan-center worked for me.

So after may 1st we still will be able to use remote repository? or is there something i'm missing?

should be back soon, see bintray status

[rodolfoap]

conan remote update conan-center https://api.bintray.com/conan/conan/conan-center worked for me.

Confirmation, it works. Thanks, @saukijan.

[danimtb]

conan remote update conan-center https://api.bintray.com/conan/conan/conan-center worked for me.

So after may 1st we still will be able to use remote repository? or is there something i'm missing?

Please, do not make this change as the bintray repository will be sunset shortly and stop working. We are handling the redirection/certificate issue on https://conan.bintray.com as this will continue to work after Bintray is gone.

So, again, switching to https://api.bintray.com/conan/conan/conan-center is not recommended as it will just delay the issue. Please be patient as https://conan.bintray.com will be back as soon as possible. Thank you

[vhavel]

Please, are there any updates? https://status.bintray.com/ still says only the scheduled brown-out is taking place, no mention of conan-center being also down.

Sorry to ask for that in github issue, I'm not sure what would be more appropriate channel to contact bintray/conan-center maintainers.

[dburov190]

What a mess. Anyway, it seems working now.

[ghost]

What a mess. Anyway, it seems working now.

I'm going to wait until the 2pm PDT window closes. Some packages are available, but specific versions of other packages are not, so, at least in my own case, not everything is 100% "up" yet.

[jleni]

Any updates? https://conan.bintray.com seems to be redirecting to (itself?) https://conan.bintray.com

[whyman]

[blackliner]

Is this "scheduled brown out" on purpose, just to prove a point? Everyone using conan-center through https://conan.bintray.com will not be affected by the May 1st change, but they have to suffer? I don't get it...

[dburov190]

I am trying to help migrate a part of my companies' team to CMake and Conan. This BS does not provide a good first impression.

Your company could always pay to host their own Artifactory server, either self hosted and in the cloud. If they chose to use the open source conan center, for which they pay nothing, then they choose to take the rough with the smooth.

Try again tomorrow.

So, you're saying, we should all abandon *NIX in our work and switch to Windows? Or, better yet, write our own operating systems? what kind of a BS argument is this?

[jadamwilson2]

Seeing this now:

Please log in to "conan-center" to perform this action. Execute "conan user" command.

If you don't have an account sign up here: https://bintray.com/signup/oss

Remote 'conan-center' username: 

Is THIS expected?

[memsharded]

@blackliner

Is this "scheduled brown out" on purpose, just to prove a point? Everyone using conan-center through https://conan.bintray.com will not be affected by the May 1st change, but they have to suffer? I don't get it...

The problems in ConanCenter were not expected. The plan was that ConanCenter transition was as smooth as possible, and it was aligned with Bintray brownouts, precisely to not be disrupted by such brownout. But things at this scale are always complicated and we failed to transition to the new infrastructure. Something went really wrong with DNS, certificates and redirects. Things have been reverted now, and hopefully when DNS propagate again, we can get things back to normal. We will be writing a post-mortem soon.

[uilianries]

@jadamwilson2 No. It's under investigation. Please be patient.

[planetmarshall]

So, you're saying, we should all abandon *NIX in our work and switch to Windows? Or, better yet, write our own operating systems? what kind of a BS argument is this?

I am saying that if you are not paying for a commercial support contract, but are using the CCI service for free, then perhaps you should be patient and cut them some slack. As mentioned up-thread, this was a planned outage for a period of eight hours. If your systems are mission critical and can't handle an outage for that long, then you should definitely look into a commercial Artifactory solution.

[memsharded]

Thanks @planetmarshall for the support.

Recall that you don't even need to pay for a commercial Artifactory solution! We are offering the Artifactory CE, completely for free, to provide the community with production grade tools that can provide fully self-contained builds to not depend that heavily on other third party free services.

[datalogics-kam]

If your systems are mission critical and can't handle an outage for that long, then you should definitely look into a commercial Artifactory solution.

I would say you could even use the community edition for free. So far, we haven't had any real setbacks because our Artifactory has cached everything we're currently using. Having a local server speeds things up, lets you store your built packages if they vary from the ones CCI makes by default, and bridges over network and server hiccups.

(@memsharded posted while I was typing...but I'll let my end-user experience stand)

[memsharded]

Btw, things seems to start getting back to normal. It seems I can install packages here again (this might be different depending on location, and DNS propagation).

[ghost]

Hmmm, something else may have also regressed. Anonymous access is now asking for creds:

Please log in to "conan-center" to perform this action. Execute "conan user" command.
If you don't have an account sign up here: https://bintray.com/signup/oss
Remote 'conan-center' username:

[memsharded]

@keysight-matthew-giassa that might be temporary hopefully, just tried again and here, it is working normally. Anyone else still experiencing issues with bintray asking for log-in?

[ghost]

@keysight-matthew-giassa that might be temporary hopefully, just tried again and here, it is working normally. Anyone else still experiencing issues with bintray asking for log-in?

10 minutes later and issue is no longer present (and conan CLI commands now working as they did previously).

+1 to on-prem caching BTW (i.e. local, Artifactory, etc.), as another individual noted above.

[ghost]

Would it be reasonable to have a feature added to the CLI tools to dump notices like this (i.e. pipe it to STDERR)? I think some Atlassian tools do this for web UI and CLI tools, and it's pretty awesome to have. Might be helpful with respect to broadening the audience of specific noticed (e.g. planned maintenance, deprecation notices and EoL announcements, etc.).

[uilianries]

@keysight-matthew-giassa Yes, @SSE4 suggested same thing few hours ago. It could be added as a new feature. Thank you for sharing your idea too.

[ghost]

@keysight-matthew-giassa Yes, @SSE4 suggested same thing few hours ago. I could be added as a new feature. Thank you for sharing your idea too.

Nice! Thanks for maintaining this free infra BTW!

[datalogics-kam]

Having some search timeouts, for instance:

❯ conan search -r conan-center boost
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v2/conans/search?q=boost (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)",))

Unable to connect to conan-center=https://conan.bintray.com/
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

[jgsogo]

Having some search timeouts, for instance:

Hi! This is a long-standing issue (#7573, #8398) not related to the ones we've faced today. After moving away from Bintray this will be one of the things we will start to work on in ConanCenter (it requires changes on the server-side that we cannot implement in Bintray).

[datalogics-kam]

Hi! This is a long-standing issue

Ok, thanks for pointing that out. It's not really blocking us; it just happened to be how I checked to see if I can reach Conan Center.

[snailcatcher]

For me the issue is fixed at the moment.
I like the idea with the server message in the CLI.

Thanks to @jgsogo and @memsharded (and all others) for your investigations in this problem.

By the way I recommend everyone to use a cached repository at least for your CI pipelines. If your build nodes are in the same network you will save traffic, gain speed and you can trust your pipeline in case of internet problems. Everything else is not reliable and trustworthy for me.

[bldrvnlw]

@snailcatcher we have our own artifactory with conan-center (conan.bintray.com) as a remote but it's not well documented how you get a particular artifact into the cache.

In the Jfrog doc it suggests: access "http://<host>:<port>/artifactory/<remote-repository-name>/<artifact-path>" and the artifact will be auto-magically cached. However there does not seems to be a way to discover the <artifact-path> in conan-center.

Can you (or anyone else for that matter) provide some insight on how to go about this?

[planetmarshall]

@snailcatcher we have our own artifactory with conan-center (conan.bintray.com) as a remote but it's not well documented how you get a particular artifact into the cache.

Can you (or anyone else for that matter) provide some insight on how to go about this?

We use a couple of strategies.

1. Let conan take care of this for you. Do

conan upload <my_package> --remote <my_remote> --all

And the cache will be uploaded to your remote. List your remote first in ${CONAN_USER_HOME}/remotes.json and conan will look there before CCI. One disadvantage of this approach is you will not get updated recipes from CCI, so I recommend periodically refreshing the recipes from CCI (say once per release cycle) and re-caching them.

2. Use the json manifest feature and lockfiles to do manual caching. We do this on CI to cache all of our dependencies in an Azure blob. It's a bit convoluted, and we'll probably be deprecating this approach shortly as soon as we get our on-site artifactory sorted out.

[solvingj]

For future reference, to be notified of scheduled maintenances in-advance, users can subscribe to notifications here:
https://status.conan.io/

For the Bintray-related maintenance, users can also check this blog post:
https://blog.conan.io/2021/03/31/Bintray-sunset-timeline.html

[DavidAntliff]

https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.gz is returning 403 Forbidden for me this morning - isn't this supposed to be redirected now? I was under the impression that I didn't have to change anything. Even a fresh Conan install doesn't work.

EDIT: fixed the URL to what is shown in the conan trace log. Removed inaccurate mention of remotes.json.

[memsharded]

Hi @DavidAntliff I think there is something wrong in your URL?

If you are looking for the official boost.org downloads, please check: https://www.boost.org/users/download/. They are pointing to new boost server: https://boostorg.jfrog.io/artifactory/main/release/1.76.0/source/boost_1_76_0.tar.bz2

The conan.bintray.com URL keeps working to serve ConanCenter, but this is a completely different thing from the boost sources you are looking for.

[DavidAntliff]

@memsharded something (possibly me) is definitely getting confused. The failing URL is actually https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.gz - I'm not sure where I got that other one from. It looks like the recipe for 1.69 still contains url = "https://dl.bintray.com/boostorg/release/%s/source/%s" % (self.version, zip_name) but I don't see this at https://conan.io/center/boost?version=1.69.0&tab=recipe. I'm building in a fresh container with a fresh Conan install each time and the recipe it ends up with is not the same as the one on ConanCenter.

Anyway, the more I look into this the less it looks like it's related to the original issue so I'll create a new issue rather than pollute this one. Thanks for responding.

[memsharded]

Oh, I see what you mean.

Yes, it seems the ConanCenter recipe hasn't updated yet to then new URLs provided by the official boost.org.
I have reported an issue in conan-center-index so the recipe can be fixed, hopefully soon: conan-io/conan-center-index#5397

In the meantime, to build from source, I would recommend to clone https://github.com/conan-io/conan-center-index, modify the boost conandata.yml to point to the new location and build from there. Please ask for help if you need it.

[DavidAntliff]

Ok, thank you, I'll try your workaround.

I logged this too: #8908