Cannot talk to GitHub #78

vinayak-kukreja · 2022-08-13T16:36:58Z

Hey, could you please take a look at this issue? Looks like its not just for the YAML packages.

a-hilaly · 2022-08-13T20:11:00Z

We're experiencing the same issue here https://prow.ack.aws.dev/view/s3/ack-prow-logs/pr-logs/pull/aws-controllers-k8s_rds-controller/109/rds-unit-test/1558538624394334208

SVilgelm · 2022-08-13T22:40:14Z

I see same issue:

go: gopkg.in/[email protected]: unrecognized import path "gopkg.in/check.v1" (parse https://gopkg.in/check.v1?go-get=1: no go-import meta tags ())
go: gopkg.in/[email protected]: unrecognized import path "gopkg.in/yaml.v2" (parse https://gopkg.in/yaml.v2?go-get=1: no go-import meta tags ())

blackknight467 · 2022-08-14T00:23:08Z

same issue here:

[gopkg.in/[email protected]:](http://gopkg.in/[email protected]) unrecognized import path "[gopkg.in/yaml.v3](http://gopkg.in/yaml.v3)": reading [https://gopkg.in/yaml.v3?go-get=1:](https://gopkg.in/yaml.v3?go-get=1) 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get [https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack:](https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack) net/http: request canceled (Client.Timeout exceeded while awaiting headers)
The command '/bin/sh -c go mod download' returned a non-zero code: 1

mahendrabishnoi2 · 2022-08-14T05:33:55Z

I'm also facing same issue.

package gopkg.in/gorp.v2: unrecognized import path "gopkg.in/gorp.v2": reading https://gopkg.in/gorp.v2?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-gorp/gorp.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

guillermo · 2022-11-02T13:05:01Z

I am having the same issue right now. Any hints about how this issue was solved back in the time?

mdmourao · 2022-11-02T14:51:05Z

I am having the same issue right now. I think the problem is related to Github but the status is showing all Operational
https://stackoverflow.com/questions/74290916/cannot-obtain-refs-from-github-github-down-status-not-showing-any-issues

EvanBoyle · 2022-11-02T15:41:22Z

Also hitting this

metal3d · 2022-11-04T19:06:06Z

And the problems is repeated fro days now... impossible to work on my apps.

A “replace” directive didn't work for me... That's a very serious problem.

Maybe I can make it work on local machine and change /etc/hosts to fake the server...

johnpekcan · 2022-11-04T19:40:34Z

Having this same issue for multiple days

/go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/ini/codec.go:9:2: unrecognized import path "gopkg.in/ini.v1": reading https://gopkg.in/ini.v1?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-ini/ini.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
/go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/yaml/yaml3.go:6:8: unrecognized import path "gopkg.in/yaml.v3": reading https://gopkg.in/yaml.v3?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

johnpekcan · 2022-11-04T19:40:53Z

lots of comments in #63 as well

metal3d · 2022-11-04T20:25:50Z

I managed some things by forking repo and changed references to not use gopkg.in - but here, with Fyne.io, there are too many problems to solve.

Really, the gopkg.in idea is good, but if the service doesn't respond, that a SPOF for thouthands of projects.

vwbusguy · 2022-11-04T23:03:59Z

Really, the gopkg.in idea is good, but if the service doesn't respond, that a SPOF for thouthands of projects.

Totally agree. I wonder if it's feasible to backup these repos/versions on a gitea or a CDN and failover to it. This just ate up my afternoon debugging why the go installs were failing at different points trying to resolve deps while GitHub was having issues (again).

mdmourao · 2022-11-04T23:08:06Z

This is a critical problem i wasn't able to deploy my workload for hours. But i think the problem is related to the way this tool communicates with github. Its weird we don't hear anything from the developers. We real need to find a work around for this.

niemeyer · 2022-11-06T10:34:41Z

I've been off for two weeks now on intense company wide events that happen twice a year, and we still have the Ubuntu Summit starting tomorrow. That's why you haven't heard from me. That said, it's for exactly this kind of reason that this service is under the supervision of a 24x7 team at Canonical, and not me personally. I'll reach out to them.

niemeyer · 2022-11-06T13:03:50Z

Here is some info from the team:

We’ve had some issues one of our transit providers affecting gopkg and a number of other services. We’ve weighted/prepended them out for now so traffic is going via our other providers links.

sivag73 · 2022-11-08T18:49:44Z

@niemeyer , I am seeing this issue again now.
Pls let me know if you a new issue needs to be created for this one.

niemeyer · 2022-11-08T20:44:00Z

Indeed, there was an issue today:

https://status.canonical.com/

niemeyer · 2022-11-08T20:47:09Z

And again it was an infrastructure-wide issue, so probably the same link misbehaving again. 😕

sanghvisagar · 2022-11-14T18:33:02Z

@niemeyer Getting the same issue again
go: gopkg.in/natefinch/[email protected]: unrecognized import path "gopkg.in/natefinch/lumberjack.v2": https fetch: Get "https://gopkg.in/natefinch/lumberjack.v2?go-get=1": dial tcp 185.125.188.113:443: i/o timeout

zekenautilus · 2022-11-21T17:55:47Z

just saw this today:

go: gopkg.in/DataDog/[email protected]: unrecognized import path "gopkg.in/DataDog/dd-trace-go.v1": reading https://gopkg.in/DataDog/dd-trace-go.v1?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/DataDog/dd-trace-go.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

siddhanta-rath · 2022-11-29T17:41:59Z

@niemeyer this is happening again today:

/.go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/ini/codec.go:9:2: unrecognized import path "gopkg.in/ini.v1": reading https://gopkg.in/ini.v1?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-ini/ini.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
/.go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/yaml/yaml3.go:6:8: unrecognized import path "gopkg.in/yaml.v3": reading https://gopkg.in/yaml.v3?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

alecholmez · 2022-11-29T19:02:08Z

Yup I see the same thing:

remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
	fatal: unable to access 'https://gopkg.in/yaml.v3/': The requested URL returned error: 502

EDIT:

It's fixed for me now

zdandoh · 2022-11-29T19:14:36Z

Seeing the same right now, not fixed yet.

Mansh05 · 2022-11-29T19:21:10Z

Same thing for me as well.

niemeyer · 2022-11-29T19:46:47Z

Sorry for the trouble. I'm in touch with our IS team and apparently it's a connectivity issue again. If that's confirmed, I've suggested moving the service out of this provider to give gopkg.in a break as this is getting shameful. 😕

hiyunfengzhao · 2022-11-29T22:02:00Z

anyone still seeing the issue? I still get server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-check/check.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Mansh05 · 2022-11-29T22:02:46Z

@niemeyer When would this be resolved in your opinion. Not pushing you but rather need to inform my teammates. Thanks

jimmyraywv · 2022-11-30T01:30:45Z

So, is there any option to get around gopkg.in?

adamdavis40208 · 2022-11-30T03:11:36Z

I had a successful build, if it gives anyone hope.

pelletier197 · 2022-12-06T16:53:32Z

Think this might be here again. Am I the only one?

mdmourao · 2022-12-06T16:57:21Z

Its down again: https://status.canonical.com/
Actually was down 3 times today.

siddhanta-rath · 2023-05-17T02:53:42Z

Today I am facing the issue again: the build is keep on falling even if gopkg.in came up after the major incident happened on today.

/.go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/ini/codec.go:9:2: unrecognized import path "gopkg.in/ini.v1": reading https://gopkg.in/ini.v1?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-ini/ini.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
/.go/pkg/mod/github.com/spf13/[email protected]/internal/encoding/yaml/yaml3.go:6:8: unrecognized import path "gopkg.in/yaml.v3": reading https://gopkg.in/yaml.v3?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

curtismenmuir · 2023-06-01T09:52:27Z

+1 seeing this issue today:

remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe
fatal: unable to access 'https://gopkg.in/yaml.v2/': The requested URL returned error: 502

remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-playground/colors.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken 
fatal: unable to access 'https://gopkg.in/go-playground/colors.v1/': The requested URL returned error: 502

harshit777 · 2023-06-01T13:46:17Z

Seeing the same issue as @curtismenmuir

remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/natefinch/lumberjack.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe fatal: unable to access 'https://gopkg.in/natefinch/lumberjack.v2/': The requested URL returned error: 502

remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe fatal: unable to access 'https://gopkg.in/yaml.v2/': The requested URL returned error: 502

curtismenmuir · 2023-06-01T13:56:42Z

@niemeyer is this something you could help us with?

j-maltese · 2023-06-01T13:57:09Z

Reporting the same issue as @curtismenmuir and @harshit777 on our pipelines this morning - issue is intermittent (sometimes fails, sometimes succeeds):

unrecognized import path "gopkg.in/yaml.v2": reading https://gopkg.in/yaml.v2?go-get=1: 502 Bad Gateway server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe

harshit777 · 2023-06-01T14:53:08Z

Gopkg was affected today : Issue

jalilalchy · 2023-06-01T18:25:57Z

Still appears to be ongoing. @niemeyer

mahsaelasticpath · 2023-06-01T19:08:32Z

Having the same issue : @niemeyer
go: gopkg.in/DataDog/[email protected] requires
gopkg.in/[email protected]: unrecognized import path "gopkg.in/yaml.v2": reading https://gopkg.in/yaml.v2?go-get=1: 502 Bad Gateway
server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe

curtismenmuir · 2023-06-02T09:19:14Z

Work around for this: https://github.com/orgs/community/discussions/56932#discussioncomment-6069091

We got this working by setting the GOPROXY Env Var to https://proxy.golang.org,direct :)

EG add GOPROXY=https://proxy.golang.org,direct to Jenkinsfile env vars

Tulsishah · 2023-10-16T05:41:16Z

I am getting this issue, my GOPROXYEnv Var to https://proxy.golang.org,direct as default. Is there any fix for this?

opohorel · 2023-10-16T13:08:20Z

Running into the same issue today with
GOPROXY="direct"

rjcoupe · 2023-10-16T13:57:29Z

Hitting this issue too, GOPROXY=https://proxy.golang.org,direct set.

j-maltese · 2023-10-16T15:03:52Z

Having similar issues:
remote: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: write: broken pipe
fatal: unable to access 'https://gopkg.in/yaml.v2/': The requested URL returned error: 502
package gopkg.in/yaml.v2: exit status 128

curtismenmuir · 2023-10-16T16:38:05Z

Fixed this previously by setting GOPROXY Env Var to https://proxy.golang.org,direct

We have not seen this issue since.

rweir · 2023-10-16T18:46:32Z

I'm still seeing this intermittently from my Github Actions, even with the GOPROXY set. I note that Canonical status says there was a brief outage of about 14 hours ago - presumably this is related.

suntong · 2023-10-16T19:47:21Z

I'm getting this from my Github Actions just now. Thanks for the confirmation @rweir.

niemeyer · 2023-10-20T10:22:24Z

We still see issues from time to time. Some of them are just either side (Canonical or GitHub) having connectivity issues, but there's also some sort of longstanding issue which seems related to problematic caching of idle connections after some other event. The code on gopkg itself is quite simple, though, so unless I'm missing something obvious it doesn't feel like this should persist the way it does. In either case, I've added a small attempt to workaround the issue by detecting this situation and dropping the idle connections. This has been deployed today, together with updated dependencies to bring them to recent releases. Let's see what happens next.

a-hilaly mentioned this issue Aug 13, 2022

correct delta_pre_compare for DBInstance aws-controllers-k8s/rds-controller#109

Merged

bcmills mentioned this issue Aug 17, 2022

cmd/go: remove test dependency on gopkg.in service golang/go#54503

Open

johnpekcan mentioned this issue Nov 4, 2022

Cannot talk to GitHub go-yaml/yaml#887

Closed

niemeyer closed this as completed Nov 6, 2022

niemeyer mentioned this issue Nov 6, 2022

"cannot talk to GitHub" #63

Closed

jiechen0826 mentioned this issue May 31, 2023

added nitro check to prefix provider and pod controller aws/amazon-vpc-resource-controller-k8s#234

Merged

haouc mentioned this issue Jun 2, 2023

fix an incorrect argument passed to SendNodeEvent aws/amazon-vpc-resource-controller-k8s#241

Merged

Cannot talk to GitHub #78

Cannot talk to GitHub #78

Comments

vinayak-kukreja commented Aug 13, 2022

a-hilaly commented Aug 13, 2022

SVilgelm commented Aug 13, 2022

blackknight467 commented Aug 14, 2022

mahendrabishnoi2 commented Aug 14, 2022

guillermo commented Nov 2, 2022

mdmourao commented Nov 2, 2022

EvanBoyle commented Nov 2, 2022

metal3d commented Nov 4, 2022

johnpekcan commented Nov 4, 2022 • edited Loading

johnpekcan commented Nov 4, 2022

metal3d commented Nov 4, 2022

vwbusguy commented Nov 4, 2022

mdmourao commented Nov 4, 2022

niemeyer commented Nov 6, 2022

niemeyer commented Nov 6, 2022

sivag73 commented Nov 8, 2022 • edited Loading

niemeyer commented Nov 8, 2022

niemeyer commented Nov 8, 2022

sanghvisagar commented Nov 14, 2022

zekenautilus commented Nov 21, 2022

siddhanta-rath commented Nov 29, 2022

alecholmez commented Nov 29, 2022 • edited Loading

zdandoh commented Nov 29, 2022

Mansh05 commented Nov 29, 2022

niemeyer commented Nov 29, 2022

hiyunfengzhao commented Nov 29, 2022

Mansh05 commented Nov 29, 2022

jimmyraywv commented Nov 30, 2022

adamdavis40208 commented Nov 30, 2022

pelletier197 commented Dec 6, 2022

mdmourao commented Dec 6, 2022

siddhanta-rath commented May 17, 2023

curtismenmuir commented Jun 1, 2023

harshit777 commented Jun 1, 2023 • edited Loading

curtismenmuir commented Jun 1, 2023

j-maltese commented Jun 1, 2023

harshit777 commented Jun 1, 2023

jalilalchy commented Jun 1, 2023 • edited Loading

mahsaelasticpath commented Jun 1, 2023 • edited Loading

curtismenmuir commented Jun 2, 2023

Tulsishah commented Oct 16, 2023

opohorel commented Oct 16, 2023

rjcoupe commented Oct 16, 2023

j-maltese commented Oct 16, 2023

curtismenmuir commented Oct 16, 2023

rweir commented Oct 16, 2023

suntong commented Oct 16, 2023

niemeyer commented Oct 20, 2023

johnpekcan commented Nov 4, 2022 •

edited

Loading

sivag73 commented Nov 8, 2022 •

edited

Loading

alecholmez commented Nov 29, 2022 •

edited

Loading

harshit777 commented Jun 1, 2023 •

edited

Loading

jalilalchy commented Jun 1, 2023 •

edited

Loading

mahsaelasticpath commented Jun 1, 2023 •

edited

Loading