HTTPRequest fails on sites with IPv6

[CosineP]

Godot version: 3.0.2 Stable

OS/device including version: Windows 10. I have IPv6 enabled.

Issue description: Accessing some sites with HTTPRequest fails with Error 2

Steps to reproduce:
Try to HTTPRequest a website with IPv6 support, with IPv6 enabled. (I believe)

Minimal reproduction project:
httprequest-fail.zip

Here's what it outputs on my Windows 10:

Loaded certs from 'res://ca-certificates.crt':  151
https://www.reddit.com/ succeeds.
It uses its own servers and DigiCert CA

http://api.myjson.com/bins/pnmlz succeeds.
It uses unknown hosting, no SSL

https://api.myjson.com/bins/pnmlz succeeds.
It uses the same site, with GEOTRUST CA

https://www.humblebundle.com/ fails with error #2
It uses Google App Engine and COMODO CA

http://lunaphippscostin.com/ fails with error #2
It uses Google App Engine but no SSL

https://letsencrypt.org/ fails with error #2
It uses unknown hosting and Let's Encrypt CA

On linux (idk if I have IPv6 enabled though) it works perfectly:

CERT STR: /C=US/ST=California/L=San Francisco/O=Reddit Inc./CN=*.reddit.com
VALID: 1
CONNECTION RESULT: 1
cert_ok: 1
https://www.reddit.com/ succeeds.
It uses its own servers and DigiCert CA

http://api.myjson.com/bins/pnmlz succeeds.
It uses unknown hosting, no SSL

CERT STR: /CN=api.myjson.com
VALID: 1
CONNECTION RESULT: 1
cert_ok: 1
https://api.myjson.com/bins/pnmlz succeeds.
It uses the same site, with GEOTRUST CA

CERT STR: /serialNumber=4903485/jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/C=US/postalCode=94108/ST=CA/L=San Francisco/street=Floor 11/street=201 Post St/O=Humble Bundle, Inc./OU=COMODO EV SSL/CN=www.humblebundle.com
VALID: 1
CONNECTION RESULT: 1
cert_ok: 1
https://www.humblebundle.com/ succeeds.
It uses Google App Engine and COMODO CA

http://lunaphippscostin.com/ succeeds.
It uses Google App Engine but no SSL

CERT STR: /CN=www.letsencrypt.org
VALID: 1
CONNECTION RESULT: 1
cert_ok: 1
https://letsencrypt.org/ succeeds.
It uses unknown hosting and Let's Encrypt CA

The ca-certificates is the one from linux /etc/whatever

[LinuxUserGD]

Is there any change if you use the certificate from the godot repo?
https://github.com/godotengine/godot/blob/master/thirdparty/certs/ca-certificates.crt

[CosineP]

No change. I don't think it's an SSL issue because when I forget the certs / have an SSL problem it's usually Error 4. Plus it fails on lunaphippscostin.com with no SSL.

[Faless]

@CosineP can you post the output that Godot gives in the console when it fails?

[CosineP]

@Faless I already posted it in the OP. No further output or errors are shown.

It's the same output if I copy the godot binary into the source directory and run it directly.

[Faless]

@CosineP , I'm unable to reproduce either on Linux or Windows 10 with Godot 3.0.2 .

Are you on an IPv6 network? Can you try compiling the latest master branch of Godot? (I recently changed a small thing that might affect behaviour of ip resolution on IPv6 network but it's not cherry picked in 3.0.x)

[Faless]

@CosineP can you add:

	var host = sites[index].url.replace("http://", "").replace("https://", "")
	host = host.substr(0, host.find("/"))
	print(IP.resolve_hostname(host))

at the beginning of your _request_completed function and post the output?

[CosineP]

@Faless You hit the nail on the head with IPv6! I disabled IPv6 on my network and things started working! (Except lunaphippscostin.com.... see [1]).

I added those lines, and I'll post the results for IPv6 on and off. It seems to be failing to access servers at IPv6 addresses.

Here's IPv6 OFF

151.101.117.140
https://www.reddit.com/ succeeds.
It uses its own servers and DigiCert CA

192.73.252.244
http://api.myjson.com/bins/pnmlz succeeds.
It uses unknown hosting, no SSL

192.73.252.244
https://api.myjson.com/bins/pnmlz succeeds.
It uses the same site, with GEOTRUST CA

172.217.10.51
https://www.humblebundle.com/ succeeds.
It uses Google App Engine and COMODO CA

2001:4860:4802:32:0:0:0:15
http://lunaphippscostin.com/ fails with error #2
It uses Google App Engine but no SSL

23.205.23.89
https://letsencrypt.org/ succeeds.
It uses unknown hosting and Let's Encrypt CA

And here's IPv6 ON again

151.101.117.140
https://www.reddit.com/ succeeds.
It uses its own servers and DigiCert CA

192.73.252.244
http://api.myjson.com/bins/pnmlz succeeds.
It uses unknown hosting, no SSL

192.73.252.244
https://api.myjson.com/bins/pnmlz succeeds.
It uses the same site, with GEOTRUST CA

2607:f8b0:4006:803:0:0:0:2013
https://www.humblebundle.com/ fails with error #2
It uses Google App Engine and COMODO CA

2001:4860:4802:32:0:0:0:15
http://lunaphippscostin.com/ fails with error #2
It uses Google App Engine but no SSL

2600:1400:d:188:0:0:0:ce0
https://letsencrypt.org/ fails with error #2
It uses unknown hosting and Let's Encrypt CA

[1] lunaphippscostin.com is my site, so it might be my fault. But I was able to load it in my browser even after turning off IPv6... Here are the DNS records for that domain if that helps (these are the vanilla suggestions given to me by Google App Engine for registering a custom domain).

I haven't compiled from master.... I don't have any tooling for compiling on Windows (I use Windows for testing and Overwatch only ^.^) so I'd really rather not.... but if you really believe that change will fix things, I'll totally set it up.

Thanks for all your help so far!

[Faless]

@CosineP you can always cross compile for windows from other platforms using mingw64 or you can download an unofficial nightly build provided by a member of the community.

[CosineP]

@Faless Wow, that nightly build is awesome, thanks!! (Whenever I think "I want x for godot", Calinou has done it ^.^) Unfortunately, the nightly build behaves exactly like 3.0.2 did :(

Would a workaround be to resolve the houstname with , IP.TYPE_IPV4 before HTTPRequest? I'll try this when I have a moment.

[CosineP]

@Faless No, I can't resolve the hostname first because shared hosting gleans information from the hostname used. So my only hope is a fix for HTTPRequest failing on IPv6, or, a way to specify which protocol to use on HTTPRequest.

BTW, I tried again with the most recent nightly build, and it still has the same behaviour.

[Faless]

@CosineP can you try my branch: https://github.com/Faless/godot/tree/ipv6_detect and tell me if that fixes the problem for you? Basically, what I'm trying to do, is detecting is IPv6 is actually available on the machine before making the DNS request.

EDIT: I will also work on forcing the IP resolution type for HTTPRequest, but I'm trying to have it fixed with default value first.

[Faless]

I'm still unable to reproduce this.
Just tried with an IPv6 tunnel (no IPv6 from my ISP :( ) on Windows 7 (windows 10 broke tunnels in April update, should be back in Fall maybe?).

Here's the result (btw, I had to update the certs file, which was outdated and failing. Also works with the new embed certs feature)

151.101.13.140
https://www.reddit.com/ succeeds.
It uses its own servers and DigiCert CA

192.73.252.244
http://api.myjson.com/bins/pnmlz succeeds.
It uses unknown hosting, no SSL

192.73.252.244
https://api.myjson.com/bins/pnmlz succeeds.
It uses the same site, with GEOTRUST CA

2a00:1450:4002:80a:0:0:0:2013
https://www.humblebundle.com/ succeeds.
It uses Google App Engine and COMODO CA

2001:4860:4802:38:0:0:0:15
http://lunaphippscostin.com/ succeeds.
It uses Google App Engine but no SSL

2a02:26f0:de:3a0:0:0:0:ce0
https://letsencrypt.org/ succeeds.
It uses unknown hosting and Let's Encrypt CA

[akien-mga]

Closing as this is an old issue and we haven't been able to replicate it.