Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All machines at nearform are offline #2451

Closed
mhdawson opened this issue Oct 1, 2020 · 9 comments
Closed

All machines at nearform are offline #2451

mhdawson opened this issue Oct 1, 2020 · 9 comments
Labels

Comments

@mhdawson
Copy link
Member

mhdawson commented Oct 1, 2020

Looks like all the machines hosted by nearform are offline in jenkins. I could still ssh into test-nearform_intel-ubuntu1604-x64-1 eariler

I'd tried restarting the agent earlier when I thought it was just one, and the failure was that it could not connect to the CI.

I checked that the IP was in the firewall so thinking its a problem on the nearform network side.

@jasnell can you check if there is something going on?

@mhdawson
Copy link
Member Author

mhdawson commented Oct 5, 2020

@jasnell, @mcollina can you see if there is any known issue on the Nearform side

@jasnell
Copy link
Member

jasnell commented Oct 5, 2020

Checking... Will get back once I hear from our sysadmin

@mcollina
Copy link
Member

mcollina commented Oct 5, 2020

We are undergoing some maintenance on our own network.

@jasnell
Copy link
Member

jasnell commented Oct 6, 2020

Ok, so... it is sounding like the fans on the intel benchmarking machine may be going out. The machine was shutdown, possibly due to overheating. We're going to continue to monitor it but it should be up now if you can give it a try.

The mac minis appear to be running fine. Please let me know if you're having issues with those.

@jasnell
Copy link
Member

jasnell commented Oct 7, 2020

Can someone confirm if they are able to access the servers now

@AshCripps
Copy link
Member

@jasnell I can log in to the systems however none of them can connect to our jenkins server - they are in the firewall so they should connect just fine.

@jasnell
Copy link
Member

jasnell commented Oct 8, 2020

hmmm... ok... let me connect you directly with our sysops.... expect an email soon

@mhdawson
Copy link
Member Author

mhdawson commented Oct 8, 2020

@AshCripps thanks for following up. What you reported was the same as I saw. We could ssh in, but they could not connect to the CI despite being in the firewall config.

@AshCripps
Copy link
Member

Eamonn restarted the firewall at nearform's end and the machines are back up and now connecting to the CI 🎉 I also checked and brought the release mac back online. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants