Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add official signed build pipeline #1016

Merged
merged 9 commits into from
Dec 19, 2019
Merged

Add official signed build pipeline #1016

merged 9 commits into from
Dec 19, 2019

Conversation

dagood
Copy link
Member

@dagood dagood commented Dec 18, 2019

Adds runtime-official.yml.

The new pipeline builds the subsets in release mode. It includes an AllConfigurations libraries job, which fans out into each installer job in addition to the platform-specific artifacts. After the installer build, a utility stage runs to sign and prepare the artifacts for publish. Then Arcade validation and publish stages run.

I tweaked Libraries publish a bit to let AllConfigurations bits get published. testhost doesn't seem to get built that way, causing a copy step to fail, plus publishing in an additional layout to get the OOB packages signed.

Feedback items on the runtime live CI build (#749) are not generally addressed in this PR. I changed skiptests from a parameter to a variable so that I could skip Installer tests, but not much else.

I've run this all the way through green, here: https://dev.azure.com/dnceng/internal/_build/results?buildId=460871

That build didn't have any channels auto-assigned to the branch, so no publishing happened. I'm working on trying out a validation channel in a new build now.

Tested publishing via the validation channel here: https://dev.azure.com/dnceng/internal/_build/results?buildId=461236. Looks reasonable to me. It didn't publish to dotnetcli blob storage but I believe Arcade will do that based on the channel the build's assigned to. master is already set up to default-assign to .NET Core 5 Dev, so building after merging will try that out.

@dotnet/runtime-infrastructure @mmitche

I'd removed this to run the same official build ID multiple times. Adding it back.
Copy link
Member

@safern safern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than a few comments/questions, LGTM.

@dagood dagood merged commit 385121f into dotnet:master Dec 19, 2019
@dagood dagood deleted the official branch December 19, 2019 00:55
@safern
Copy link
Member

safern commented Dec 19, 2019

@dagood I notices that the allConfigurations artifacts are downloaded in every installer build step... is that expected?

Should they only be downloaded in 1 build?

Which build is the one that will sign them?

@dagood
Copy link
Member Author

dagood commented Dec 19, 2019

That's normal, that's what I meant by fan out in this bit of the description:

It includes an AllConfigurations libraries job, which fans out into each installer job in addition to the platform-specific artifacts.

Some of the AllConfigurations packages are used to build the platform-specific installers assets, so all jobs need the bits.

Which build is the one that will sign them?

All the allconfigurations nupkgs are recursively signed by the "Prepare for Publish" stage. If the installer jobs need to pull out anything and put them in the Windows installers, those specific bits are signed then (because the Windows installers can't be recursively signed later). I don't know off the top of my head what's being pulled in this way though.

There might be some time saving potential there, but AFAIK we don't know yet if unnecessary round trips or redundant signing is worse yet.

@safern
Copy link
Member

safern commented Dec 19, 2019

Thanks for explaining.

Some of the AllConfigurations packages are used to build the platform-specific installers assets, so all jobs need the bits.

Do we know which ones?

@dagood
Copy link
Member Author

dagood commented Dec 19, 2019

Some of the AllConfigurations packages are used to build the platform-specific installers assets, so all jobs need the bits.

Do we know which ones?

@jkoritzinsky might, I think at least Microsoft.NETCore.Platforms and Microsoft.NETCore.Targets.

@safern
Copy link
Member

safern commented Dec 19, 2019

The reason why I ask is because the drop of allConfigurations packages is heavy. So we might instead be able to just upload to a different drop the ones we know are needed.

@jkoritzinsky
Copy link
Member

The two packages @dagood mentioned are the packages that we need.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants