Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use sparse checkout for eng/common sync PRs #1594

Merged
8 commits merged into from
May 12, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions eng/common/pipelines/templates/steps/create-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ parameters:
PRTitle: not-specified
PRBody: ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this change is worth flushing through eng/common but I guess if you are doing the sparse-checkout then it can come along for the ride.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's getting lumped in with the pre-existing display name changes. Figured it was easier to just let the whitespace fix tag along (rather than me unstaging this file every time I open it and my editor auto-deletes whitespace).

ScriptDirectory: eng/common/scripts
GHReviewersVariable: ''
GHReviewersVariable: ''
GHTeamReviewersVariable: ''
GHAssignessVariable: ''
# Multiple labels seperated by comma, e.g. "bug, APIView"
Expand All @@ -33,7 +33,7 @@ steps:
PushArgs: ${{ parameters.PushArgs }}
WorkingDirectory: ${{ parameters.WorkingDirectory }}
ScriptDirectory: ${{ parameters.ScriptDirectory }}
SkipCheckingForChanges: ${{ parameters.SkipCheckingForChanges }}
SkipCheckingForChanges: ${{ parameters.SkipCheckingForChanges }}

- task: PowerShell@2
displayName: Create pull request
Expand Down
1 change: 1 addition & 0 deletions eng/common/pipelines/templates/steps/sparse-checkout.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ steps:
- pwsh: |
$dir = "${{ coalesce(repo.WorkingDirectory, format('{0}/{1}', '$(System.DefaultWorkingDirectory)', repo.Name)) }}"
New-Item $dir -ItemType Directory -Force
displayName: Create ${{ repo.Name }} directories

- pwsh: |
git clone --no-checkout --filter=tree:0 git://github.com/${{ repo.Name }} .
Expand Down
15 changes: 10 additions & 5 deletions eng/pipelines/eng-common-sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ parameters:
default: eng/common
- name: BaseBranchName
type: string
default: $(Build.SourceBranchName)
default: $(Build.SourceBranch)
- name: Repos
type: object
default:
Expand Down Expand Up @@ -47,6 +47,11 @@ stages:
displayName: Sync ${{ parameters.DirectoryToSync }} Directory

steps:
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
Paths:
- ${{ parameters.DirectoryToSync }}

- pwsh: |
Set-PsDebug -Trace 1
$patchDir = "$(Build.ArtifactStagingDirectory)/patchfiles"
Expand Down Expand Up @@ -135,13 +140,13 @@ stages:
Add-Content -Path "$(Build.ArtifactStagingDirectory)/${{ parameters.PRDataFileName }}" -Value $PRData
displayName: Write Sync PR Data to Artifact File
condition: succeeded()

- task: PublishPipelineArtifact@1
condition: succeeded()
displayName: Publish ${{ parameters.PRDataFileName }}
displayName: Publish ${{ parameters.PRDataFileName }}
inputs:
artifactName: ${{ parameters.ArtifactName }}
path: $(Build.ArtifactStagingDirectory)/${{ parameters.PRDataFileName }}
path: $(Build.ArtifactStagingDirectory)/${{ parameters.PRDataFileName }}

- stage: VerifyAndMerge
jobs:
Expand Down Expand Up @@ -213,4 +218,4 @@ stages:
DirectoryToSync: ${{ parameters.DirectoryToSync }}
UpstreamBranchName: "sync-${{ parameters.DirectoryToSync }}"
BaseBranchName: ${{ parameters.BaseBranchName }}
Repos: ${{ parameters.Repos }}
Repos: ${{ parameters.Repos }}
40 changes: 29 additions & 11 deletions eng/pipelines/templates/steps/sync-directory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,16 @@ parameters:

steps:
- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
benbp marked this conversation as resolved.
Show resolved Hide resolved
parameters:
Paths:
- ${{ parameters.DirectoryToSync }}
Repositories:
- ${{ each repo in parameters.Repos }}:
- Name: Azure/${{ repo }}
benbp marked this conversation as resolved.
Show resolved Hide resolved
WorkingDirectory: $(System.DefaultWorkingDirectory)/${{ repo }}

- ${{ each repo in parameters.Repos }}:
- pwsh: |
Set-PsDebug -Trace 1
git clone --branch ${{ parameters.BaseBranchName }} https://github.com/azure/${{ repo }}
displayName: Clone ${{ repo }}
workingDirectory: $(System.DefaultWorkingDirectory)
- template: /eng/common/pipelines/templates/steps/set-default-branch.yml
- pwsh: |
$repoPath = "${{ repo }}/${{ parameters.DirectoryToSync }}"
Expand All @@ -31,7 +35,7 @@ steps:
{
Write-Host $file.FullName
git -c user.name="azure-sdk" -c user.email="[email protected]" am -3 $file.FullName
if ($lastExitCode -ne 0) {
if ($lastExitCode -ne 0) {
git -c user.name="azure-sdk" -c user.email="[email protected]" am --show-current-patch=diff
Write-Error "##vso[task.LogIssue type=warning;]Failed to properly apply patch files to [https://github.com/azure/${{ repo }}]"
exit 1
Expand Down Expand Up @@ -121,24 +125,38 @@ steps:
-AuthToken "$(azuresdk-github-pat)"

- ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
Paths:
- ${{ parameters.DirectoryToSync }}
Repositories:
- Name: $(Build.Repository.Name)
Commitish: $(Build.SourceVersion)
WorkingDirectory: $(System.DefaultWorkingDirectory)/$(Build.Repository.Name)
- ${{ each repo in parameters.Repos }}:
- Name: Azure/${{ repo }}
benbp marked this conversation as resolved.
Show resolved Hide resolved
Commitish: ${{ parameters.BaseBranchName }}
WorkingDirectory: $(System.DefaultWorkingDirectory)/${{ repo }}

- ${{ each repo in parameters.Repos }}:
- pwsh: |
Set-PsDebug -Trace 1
git clone --branch ${{ parameters.BaseBranchName }} https://github.com/azure/${{ repo }}
$repoPath = "${{ repo }}/${{ parameters.DirectoryToSync }}"
if (!(Test-Path $repoPath)) { mkdir $repoPath }
Remove-Item -v -r $repoPath
Copy-Item -v -r $(Build.SourcesDirectory)/${{ parameters.DirectoryToSync }} $repoPath
Remove-Item -v -r -ErrorAction Ignore $repoPath
Copy-Item -v -r `
$(System.DefaultWorkingDirectory)/$(Build.Repository.Name)/${{ parameters.DirectoryToSync }} `
$repoPath
Get-ChildItem -r $repoPath
displayName: Copy ${{ parameters.DirectoryToSync }} from azure-sdk-tools to ${{ repo }}
workingDirectory: $(System.DefaultWorkingDirectory)

- template: /eng/common/pipelines/templates/steps/create-pull-request.yml
parameters:
RepoName: ${{ repo }}
BaseBranchName: ${{ parameters.BaseBranchName }}
PRBranchName: sync-${{ parameters.DirectoryToSync }}
CommitMsg: ${{ parameters.CommitMessage }}
PRTitle: ${{ parameters.CommitMessage }}
PushArgs: -f
WorkingDirectory: $(System.DefaultWorkingDirectory)/${{ repo }}
ScriptDirectory: $(System.DefaultWorkingDirectory)/eng/common/scripts
ScriptDirectory: $(System.DefaultWorkingDirectory)/$(Build.Repository.Name)/eng/common/scripts
benbp marked this conversation as resolved.
Show resolved Hide resolved