Please create new Helix queue ubuntu.2004.ppc64le.experimental.open #9567
Comments
ilyas1974
added
Ops - First Responder
Detected By - Customer
|
Assigning to @oleksandr-didyk, @premun - please give Alex the context here. Thanks! |
ilyas1974
added
the
Operations
|
IBM have given us some PPC VMS, to do along with their current work on a PPC port. All the same details as our existing s390x engagement, with a different team & a different architecture. IBM has lots of architectures
…Sent from my iPhone
On Jun 7, 2022, at 11:44 AM, Tomáš Kapin ***@***.***> wrote:
Assigning to @oleksandr-didyk, @premun - please give Alex the context here. Thanks!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.
|
|
@directhex just to make sure - do I understand it correctly that there is one specific VM somewhere (in Azure?) and we will want to install the Helix agent there and connect it to the new queue? |
|
It's four VMs in a cloud operated by an IBM partner, running PPC64 hardware. We just want to install the Helix agent & connect it to the new queue, for test runs via runtime-community.yml |
|
@directhex and will you install the agent on the VMs or should we provide a pubkey and install it ourselves? |
|
I'm happy to do it myself, but I'll need access to the keyvault secrets demanded by the helix setup script. If y'all would prefer to do it, I'm happy to do that too. |
|
As for your question @directhex, I think @premun has more context to make the best decision here |
directhex
changed the title
|
After a quick chat with @premun, we decided that if would be better if we installed the agent ourselves. I'll reach out to you with the pubkey that needs to be added. Also, just in case - would it be possible to roll-back the VM to some snapshot in the low-low case that we mess something up during the installation? |
|
I don't have direct access to the VM management (I didn't request it), but I can file a ticket for that if needed |
|
@oleksandr-didyk looks like we have 4 attempts for now then 😅 should be doable |
|
Just a quick update on task status - PR was approved and merged, waiting on resolution of an issue with the build (related to the move to the new datacenter). Once that stops blocking us, we will continue with installing the agent on the VMs and will let you know once they are ready to accepts tasks from the queue |
|
@directhex we successfully installed the agent on one of the VMs. Could you please submit some proper job to the queue so we can test it E2E properly before we procced with installing it to the other VMs. Thanks |
|
PR for that is dotnet/runtime#70734 let's see how it goes |
|
Hm. What did the queue name end up as? I don't see it on helix.dot.net, and none of the variations I've tried work. (Above value from the dotnet-helix-machines.git PR) |
|
And was using "ppc64el" (the Debian/Ubuntu architecture name) rather than "ppc64le" (IBM's preferred name) intentional? |
|
@directhex the queue hasn't been released yet and it's in staging only (https://helix.int-dot.net/). You will have to point the Helix SDK onto staging: <HelixBaseUri>https://helix.int-dot.net</HelixBaseUri>However, maybe don't run all of the tests on staging so either change this property for that specific RID or please comment out the other legs if possible. We will rename the queue, that was a mistake (@oleksandr-didyk ^^) |
|
Yeah, my bad, forgot to mention that it was still on Sure, will rename it to |
|
I'm not going to change |
|
@directhex it is possible that @oleksandr-didyk hasn't moved the VM he bootstrapped yesterday into the renamed queue yet |
|
Yeah, the queue is up and the agent seems to be processing the messages without any issues. I apologize for the delay and will continue with adding the agents to the other VMs. Once they are ready, I'll let you know |
|
Thanks @oleksandr-didyk What's the process going to be for getting this into the prod instance? The initial smoke test submitted from my dev machine looks promising, jobs are running and tests are passing https://helix.int-dot.net/api/jobs/019947f6-a77e-4a08-ae41-e2c395e82d0d/workitems?api-version=2019-06-17 |
|
We just give it a different configuration file and it will connect to the PROD queue. So machines that are already set up and working can be moved once we roll out - so most probably next Wednesday |
|
I just installed the agent on the rest of the VMs, everything seems to be A-OK. As @premun mentioned, once the PROD queue is up after next rollout, I'll switch the configuration files and will notify here |
|
The new queue is up and we should move the machines into PROD. @oleksandr-didyk is this something you could do in between the bootcamp? I believe it's only you who has access to these VMs? |
|
@directhex the VMs were updated to listen to the PROD queue and seem to be working / hearbeating just fine. Could you please verify on your end that its A-OK so that the ticket can be |
|
Yep, queue in prod is working |
As part of IBM's engagement with us, we'd like to add some IBM-provided PPC64 little endian VMs they've provided to a Helix queue so they can get early & easy visibility on failures - this mirrors their engagement on s390x, and the corresponding ubuntu.2004.s390x.experimental.open queue.
Right now the only authorized SSH key on the provided VMs is my own, I can add anyone else's if there's a pubkey documented somewhere I should add.
The text was updated successfully, but these errors were encountered: