Cargo behind a proxy

[JP-Ellis]

I am using trying to use Cargo from behind a proxy, and it is unable to fetch external repositories despite having all the appropriate environment variables and configurations.

More specifically, I have the following environment variables set:

export HTTP_PROXY=proxy.example.com:8000
export HTTPS_PROXY=proxy.example.com:8000
export FTP_PROXY=proxy.example.com:8000

along with their lowercase counterparts. Git and Cargo both have in their configuration:

[http]
proxy = proxy.example.com:8000

[https]
proxy = proxy.example.com:8000

and git from the command line works fine; however, when Cargo attempts to fetch a repository, I encounter

$ cargo build
    Updating git repository `https://github.com/pistondevelopers/piston`
Unable to update https://github.com/pistondevelopers/piston

Caused by:
  failed to fetch into /home/user/.cargo/git/db/piston-10111ca7958cf505

Caused by:
  [2] Failed to connect to github.com: No route to host

yet if I run git on its own, it works fine:

$ git clone https://github.com/pistondevelopers/piston
Cloning into 'piston'...
remote: Counting objects: 1959, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 1959 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (1959/1959), 2.19 MiB | 575 KiB/s, done.
Resolving deltas: 100% (1252/1252), done.

[tomjakubowski]

Typically these proxy variables are given in the form of URLs (e.g. http://proxy.example.com:8000 or https://proxy.example.com:8443). It's possible that libgit2 is more strict about this than git or curl is.

[JP-Ellis]

I had actually thought of that, and I did try this with all the relevant proxy variables set with the full URLs. I.e., ~/.gitconfig and ~/.cargo/config containing:

[http]
proxy = "http://proxy.example.com:8000/"

[https]
proxy = "https://proxy.example.com:8000/"

and the environment variables being

export HTTP_PROXY=http://proxy.example.com:8000/
export HTTPS_PROXY=https://proxy.example.com:8000/
export FTP_PROXY=ftp://proxy.example.com:8000/

but the result remains the same.

[tomjakubowski]

Taking a quick look through libgit2's source, it looks like they handle reading HTTP proxy settings in a private-looking function, but only ever call this function from the winhttp transport.

And in fact, @alexcrichton has opened an issue for that here: libgit2/libgit2#2555

[alexcrichton]

Aha yes indeed! I would very much like cargo to support proxies. I believe that all of our own personal HTTP requests are routed through proxies when necessary (mostly when dealing with the registry), but as @tomjakubowski found out this has yet to move over to libgit2 on unix.

My plan at this point is to make a git2-curl package (or something like that) which provides an HTTP transport powered through curl-rust. That way we can do maximum configuration with curl-rust to support things like HTTP/SOCKS proxies, etc.

Thanks for opening this! It's good to have a tracker.

[bartavelle]

Is there a way to manually do the cloning ? I am new to rust, so I don't know where cargo is supposed to store the dependent repos ...

[bartavelle]

Ok I just cloned it and used a file:/// type url ..

[hamstergene]

Until this issue is fixed, you can use alias cargo='http_proxy= https_proxy= proxychains4 cargo' to workaround the problem, it works well for me. On OS X, proxychains can be installed via brew install proxychains-ng and configured by editing /usr/local/etc/proxychains.conf.

[aldanor]

Same story here -- behind a corporate proxy with SSL certificate injection (and both HTTP_PROXY/HTTPS_PROXY set to the same address):

$ cargo build --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
Unable to update registry https://github.com/rust-lang/crates.io-index

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  SSL error: error:140E0114:SSL routines:SSL_shutdown:uninitialized

E.g., I normally get git and wget working by settings http.sslverify=false and --no-check-certificate as defaults, but with cargo there doesn't seem a way around this.

Is this also caused by libgit2? If not and if it's caused by ssl verification, it'd be nice to have an option to ignore it (same as in `git).

[alexcrichton]

@aldanor yes that's coming from libgit2, but I don't think that it's coming from http.sslverify as I believe that libgit2 just doesn't handle connecting to HTTP proxies.

They did have support for http.sslverify awhile back, but it looks like they semi-recently removed it. We could always add support for it in Cargo as well though!

Are you sure that the only error there is http.sslverify, and not the fact that cargo isn't connecting to a proxy though?

[aldanor]

@alexcrichton Nope, not sure it's exactly the same problem (other than the OP's error was "no route to host" and not SSL-related), wonder how to confirm it? It's just I've faced this same problem with most tools and package managers (git, npm, wget, pip to name a few) and in each case it was solved by disabling ssl verification, hence the assumption it was also the problem here.

If I enable http.sslverify on git, it yields:

$ git config --global http.sslverify true
$ git clone https://github.com/rust-lang/cargo.git
Cloning into 'cargo'...
error: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed while accessing https://github.com/rust-lang/cargo.git/info/refs
fatal: HTTP request failed

which is a different SSL routine and a different error code, hm...

So maybe libgit2 fails at connecting to proxy in the first place, like you said.

[alexcrichton]

Ok, thanks for checking @aldanor! I've opened #1180 as well to track http.sslverify which is somewhat separable from this issue. It's definitely something we should fix!

[Jojoshua]

+1

C:\master\src>cargo build
Updating registry https://github.com/rust-lang/crates.io-index
An unknown error occurred

To learn more, run the command again with --verbose.

C:\master\src>cargo build --verbose
Updating registry https://github.com/rust-lang/crates.io-index
An unknown error occurred

Caused by:
failed to send request: A connection with the server could not be established

[alexcrichton]

Ok, as a status update to this I've finished binding libgit2's custom transport API and I've written a simple HTTP backend using libcurl: https://crates.io/crates/git2-curl.

I'll try to hook this up into cargo and then we should get proxy support for free because libcurl supports it out of the box. Unfortunately the implementation is not super efficient (reads an entire network operation, aka repository, into memory) due to the current design of curl-rust, but it should serve as an adequate enough stopgap for now.

[rohitjoshi]

Great!! This would defiantly help/

[aldanor]

Wonder if this is supposed to work now? (not quite clear from the merge mentioned above). Are cargo/libgit2 now expected to respect HTTP_PROXY? Has anyone had any success with that?

// Weird thing, I'm not getting a "connection refused" error while using the same proxies as before (were previously causing "ssl certificate failed").

[alexcrichton]

@aldanor yeah this should in theory work now, could you gist the errors you're seeing?

[worker2345234]

Hi, I'm working behind our company firewall using a Windows 7 PC. I've no futher Network skills. I took the Proxy IP from my Internet Explorer and tried:

C:\Data\Projects\rust\rustprj>cargo --version
cargo 0.3.0-nightly (06dbe65 2015-05-29) (built 2015-06-01)

C:\Data\Projects\rust\rustprj>SET http_proxy=http://10.3.3.35:8080/

C:\Data\Projects\rust\rustprj>SET https_proxy=https://10.3.3.35:8080/

C:\Data\Projects\rust\rustprj>cargo build --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
Unable to update registry https://github.com/rust-lang/crates.io-index

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [12] SSL connect error

C:\Data\Projects\rust\rustprj>

Any ideas?

THX, Mark

[alexcrichton]

@worker2345234 could you try also setting the .cargo/config values? Trying this out locally it does indeed use the proxy, so it could be that your proxy is rejecting the connections regardless.

[worker2345234]

Ok, I created a small .cargo/config file:

[http]
proxy = "10.3.3.35:8080"  # HTTP proxy to use for HTTP requests (defaults to none)
timeout = 60000      # Timeout for each HTTP request, in milliseconds

The cargo result is the same as yesterday. I tried to load https://github.com/rust-lang/crates.io-index directly in the InternetExplorer, works. A screenshot from the InternetExplorer-Settings:

[cosmo0920]

@minecrawler It seems that it is cargo regression. Does this workaround work for your environment? #598 (comment)

[alexcrichton]

@minecrawler the [https] section isn't read by Cargo, but it looks like [http] is being picked up and the proxy is being used. It then looks like Cargo is rejecting the SSL certificate coming back, which presumably means that the corporate proxy is intercepting TLS traffic. It also looks like the certificate presented isn't valid according for your system.

In short, it looks like the problem is that Cargo asserts the validity of certificates but the certificate it's working with isn't valid. Cargo doesn't currently respect sslVerify like git does. Unfortunately the http.cainfo option which is normally used to configure what certificates Cargo trusts doesn't work on Windows (it's not compatible with schannel).

The tl;dr; I believe is that on Windows right now there's no way to get Cargo to accept an invalidate SSL certificate. Ideally the relevant certificates would be added to your OS certificate store, but we still need to implement custom validation.

[minecrawler]

@cosmo0920 Thank you for your hint, but unfortunately that workaround does not work for me.

@alexcrichton Thank you for the explanation. There is just one thing I do not understand: Why would it work on an OS other than Windows? How would you go about it on Linux and why shouldn't something like that be a viable option on Windows?

[alexcrichton]

Each platform has its own method of establishing and validating SSL connections. On Windows the HTTP library we're using, libcurl, doesn't read cainfo but on Linux/OSX it does.

[worker2345234]

I just downloaded the nightly aaaaannnnddd.... finally it works (after almost 2 years....trying,trying,trying)!

First time that I could call successfully e.g. 'cargo install racer' , thanks for the #3699 fix.

Highfive Alex, I'm sure this will give rust a kick into company environements.

Here a brief summery how I could get it to work (Win7 inside my company behind company Firewall with NTLM authentification):

1.) download and install CNTLM proxy on my PC to manage the authentification for our company firewall
2.) in c:\Users\foo.cargo\config I added:

[http]
proxy = "127.0.0.1:3128" # use my cntlm proxy for authentification
timeout = 60000     # Timeout for each HTTP request, in milliseconds
check-revoke = false # please ignore the man in the middle attack of our IT department

@alex: Would be nice, if you will give the rustup team a hint that they also implement a check-revoke=false option

[alexcrichton]

@worker2345234 glad it's working, thanks for the confirmation!

Want to file an issue over at rust-lang-nursery/rustup.rs for the revocation issue with rustup?

[worker2345234]

rustup also uses curl lib to download the binaries. That's why the rustup tool often doesn't work at companies.
As workaround I always download the binary rust-installers but the prefered way would be rustup. Especially for cross compiling...
I guess you might explain them better why it is important to turn the ssl-revocation off.

[alexcrichton]

Oh sure, an issue is just a good tracking point for integration

[minecrawler]

was excited to hear that someone got it working, however it's still not working for me :(

PS C:\> cargo --version
cargo 0.18.0-nightly (5db6d64 2017-03-03)
PS C:\> cargo install amethyst_tools --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): [12/-20] early EOF
warning: spurious network error (1 tries remaining): [12/-20] early EOF
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [12/-20] early EOF

with cargo config:

[http]
proxy = "http://localhost:3128"
timeout = 60000
check-revoke = false
sslVerify = false

[worker2345234]

I used the same cargo Version:
C:\>cargo --version cargo 0.18.0-nightly (5db6d64 2017-03-03)
But I had another error message: error: [35] SSL connect error (schannel: next InitializeSecurityContext failed: Unknown error (0x80092012....

[worker2345234]

My error message with check-revoke=true is:
C:\>cargo install --force rustfmt Updating registry 'https://github.com/rust-lang/crates.io-index' warning: spurious network error (1 tries remaining): [12/-2] [35] SSL connect error (schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - Die Sperrfunktion konnte keine Sperrp r?fung f?r das Zertifikat durchf?hren.) error: failed to fetch 'https://github.com/rust-lang/crates.io-index'

with check-revoke=false, everything is ok

[aldanor]

Can confirm that the issue has been resolved for me -- Linux, SSL_CERT_FILE and HTTP_PROXY both set; curl backend seems to pick it up fine, git-sourced crates work fine as well; no changes to .cargo/config required. Whichever was the change that fixed it, it seems to work now.

[alexcrichton]

@minecrawler oh dear, and to confirm you placed the configuration in ~/.cargo/config, right?

[minecrawler]

@alexcrichton yes, I should have edited the right one, because before I upgraded, I had the local registry configured in that file, which worked (at least for compilation)

[alexcrichton]

@minecrawler you may be running into a different issue then unfortunately :(

[minecrawler]

@alexcrichton is there any way to get more information? For example the reply from the proxy or the reply from the destination? That's the stuff I would expect from --verbose! I don't really want to start reverse-engineering the packet contents, but I think they might provide important information regarding the actual problem...

[alexcrichton]

@minecrawler your error looks like it's happening somewhere deep inside libgit2, which I unfortunately don't know how to coax more debugging information out of :(

You could also try building Cargo with debug information and stepping through libgit2, but short of that I unfortunately don't have many ideas :(

[ddickstein]

cargo config tip here was great. solved my problem with installing ripgrep.

[bocc]

I also have an issue related to this: corporate proxy requires authentication, so if I set up the https_proxy env variable with user@host format, it gets back a 407 - authentication failed. If I supply my password, it works, but naturally I want to avoid that if possible. Is there a way make cargo/rustup prompt for the proxy password?

[minecrawler]

@bocc I am on Windows, so I don't know if this is helpful.

I don't know, what triggered the changes, but lately, the cargo behind a proxy works for me. It just does. However, I use one little tool, which helps me get many tools around our corporate proxy (npm, bower, git,...). It's called CNTLM. I run it as a portable application, using a scheduled script to start it. You can also install it as a service, though (requires admin rights). CNTLM is a local proxy which you can supply with a hashed version of your credentials in a config file. Then you route all traffic over that proxy. Your applications will not require authentication for CNTLM, and CNTLM will do the authentication with the corporate proxy, which means your applications will gain internet access (yay).

[alexcrichton]

@bocc unfortunately Cargo doesn't support prompting for a password right now, it needs to be configured somehow to be stored via other means (like @minecrawler is mentioning I believe)

[rishavs]

I am having a similar issue;

PS C:\Users\Mockingbird\DevSpace\Actix\hello> cargo run
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: The operation timed out
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: The operation timed out
; class=Os (2)
error: failed to load source for a dependency on `actix-web`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: The operation timed out
; class=Os (2)

I am not behind any proxy and every network based app works fine here. I am on windows 10 and have avast as the antivirus. I have already tried disabling the antivirus but that didn't fix anything.

My Rust version is stable-x86_64-pc-windows-msvc - rustc 1.28.0 (9634041f0 2018-07-30)

[rohitjoshi]

@alexcrichton I am getting error SSL peer certificate or SSH remote key was not OK; It is behind the corporate proxy.

root@df7e775e36a9:/tmp/test-project# cargo build
    Updating crates.io index
warning: spurious network error (2 tries remaining): [60] SSL peer certificate or SSH remote key was not OK; class=Net (12)
warning: spurious network error (1 tries remaining): [60] SSL peer certificate or SSH remote key was not OK; class=Net (12)
error: failed to load source for a dependency on `chrono`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [60] SSL peer certificate or SSH remote key was not OK; class=Net (12)

[kylixs]

it's also work fine with socks5 proxy:

[http]
        proxy="socks5://127.0.0.1:1080"
[https]
        proxy="socks5://127.0.0.1:1080"

[ketut]

add proxy config to ~/.cargo/config was working for me
[http]
proxy = "http://myuser:[email protected]:3128"

[https]
proxy = "https://myuser:[email protected]:3128"

[deg0nz]

I had the same error with a GitLab CI runner in Windows Docker container. I just needed to set the CARGO_HTTP_CHECK_REVOKE environment variable to false to fix the error.

Edit: ...and set the Environment variables http_proxy and https_proxy of course

[spascoe]

When running behind a corporate proxy that uses self-signed certificates, there are several cargo configuration features that will help

1. https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli uses git instead of libgit
2. http.cafile="c:\some\path\to\a\file\of\pem\certificates.pem"

Extract your Corporate proxy's public certificate chain to base64 encoded.cer files and then using a text editor paste them all into a single file named something like "certificates.pem" (Name is unimportant)
Use the path to that file in step 2 above.