Skip to content

Commit

Permalink
Enables integration tests
Browse files Browse the repository at this point in the history 
Integration tests run platform builds with IA32 and X64 architectures
for targets DEBUG and RELEASE for platforms OvmfPkg, ArmVirtPkg, and
EmulatorPkg.

Signed-off-by: Joey Vagedes <[email protected]>
  • Loading branch information
@Javagedes
Javagedes committed Sep 28, 2022
1 parent 2a50cdf commit e8c7426
Show file tree
Hide file tree
Showing 6 changed files with 717 additions and 0 deletions.
5 changes: 5 additions & 0 deletions integration_test/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/test
/report
stderr.txt
stdout.txt
/edk2/
199 changes: 199 additions & 0 deletions integration_test/Shared_Keywords.robot
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
*** Settings ***
Documentation A shared set of common keywords for stuart operations and git operations
#
# Copyright (c), Microsoft Corporation
# SPDX-License-Identifier: BSD-2-Clause-Patent

Library Process
Library OperatingSystem

# Suite Setup

*** Variables ***

#Test output location
${TEST_OUTPUT} ${TEST_OUTPUT_BASE}

*** Keywords ***

### list comparison helper ###
Confirm same contents
[Arguments] ${actual} ${expected}
@{epkgs}= Split String ${expected} ,
@{apkgs}= Split String ${actual} ,

FOR ${a} IN @{apkgs}
Should Contain ${expected} ${a}
END

FOR ${b} IN @{epkgs}
Should Contain ${actual} ${b}
END

### Git operations ###
Clone the git repo
[Arguments] ${git_url} ${ws_name}

Log To console cloning ${git_url} to ${TEST_OUTPUT}
${result}= Run Process git.exe clone ${git_url} ${ws_name}
... cwd=${TEST_OUTPUT} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}

${result}= Run Process git fetch --all --prune
... cwd=${TEST_OUTPUT}${/}${ws_name} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Reset git repo to default branch
[Arguments] ${ws} ${default_branch_name}

# checkout remote tag for origin/<default branch>
${result}= Run Process git checkout origin/${default_branch_name}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

# clean non ignored files quietly to avoid log overflow
${result}= Run Process git clean -qfd cwd=${ws}
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

# reset to restore files
${result}= Run Process git reset --hard
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Get default branch from remote
[Arguments] ${ws}

# Set origin head to auto
${result}= Run Process git remote set-head origin --auto
... cwd=${ws}
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

# get the head
${result}= Run Process git rev-parse --abbrev-ref origin/HEAD
... cwd=${ws}
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

# Strip off origin/ from the branch because all other commands
# add the remote name.
${branch}= Get Substring ${result.stdout} 7

[Return] ${branch}

Make new branch
[Arguments] ${name} ${ws}
${result}= Run Process git checkout -b ${name}
... cwd=${ws} shell=True
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Delete branch
[Arguments] ${name} ${to_branch} ${ws}
Run Keyword Switch branch ${to_branch} ${ws}
${result}= Run Process git branch -D ${name}
... cwd=${ws} shell=True
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Switch branch
[Arguments] ${name} ${ws}
${result}= Run Process git checkout ${name}
... cwd=${ws} shell=True
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stage changed file
[Arguments] ${file_path} ${ws}
${result}= Run Process git add ${file_path}
... cwd=${ws} shell=True
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Commit changes
[Arguments] ${msg} ${ws}
${result}= Run Process git commit -m ${msg}
... cwd=${ws} shell=True
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

### Stuart operations ###
Stuart setup
[Arguments] ${setting_file} ${arch} ${target} ${packages} ${tool_chain} ${ws}
Log to console Stuart Setup
${result}= Run Process stuart_setup
... -c ${setting_file} -a ${arch} TOOL_CHAIN_TAG\=${tool_chain} -t ${target} -p ${packages} TARGET\=${target}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart ci setup
[Arguments] ${setting_file} ${arch} ${target} ${packages} ${tool_chain} ${ws}
Log to console Stuart CI Setup
${result}= Run Process stuart_ci_setup
... -c ${setting_file} -a ${arch} TOOL_CHAIN_TAG\=${tool_chain} -t ${target} -p ${packages} TARGET\=${target}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart update
[Arguments] ${setting_file} ${arch} ${target} ${packages} ${tool_chain} ${ws}
Log to console Stuart Update
${result}= Run Process stuart_update
... -c ${setting_file} -a ${arch} TOOL_CHAIN_TAG\=${tool_chain} -t ${target} -p ${packages} TARGET\=${target}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart platform build
[Arguments] ${setting_file} ${arch} ${target} ${tool_chain} ${ws}
Log to console Stuart Build
${result}= Run Process stuart_build
... -c ${setting_file} -a ${arch} TOOL_CHAIN_TAG\=${tool_chain} TARGET\=${target}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart platform run
[Arguments] ${setting_file} ${arch} ${target} ${tool_chain} ${addtional_flags} ${ws}
Log to console Stuart Build Run
${result}= Run Process stuart_build
... -c ${setting_file} -a ${arch} TOOL_CHAIN_TAG\=${tool_chain} TARGET\=${target} --FlashOnly ${addtional_flags}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart CI build
[Arguments] ${setting_file} ${archs} ${targets} ${packages} ${tool_chain} ${ws}
Log to console Stuart CI Build
${result}= Run Process stuart_ci_build
... -c ${setting_file} -a ${archs} -t ${targets} -p ${packages} TOOL_CHAIN_TAG\=${tool_chain}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0

Stuart pr evaluation
[Documentation] Run Pr Eval
[Arguments] ${setting_file} ${packages} ${base_ref} ${other_build_flags} ${ws}

${result}= Run Process stuart_pr_eval
... -c ${setting_file} -p ${packages} --pr-target origin/${base_ref}
... --output-csv-format-string {pkgcsv}
... cwd=${ws} stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Return From Keyword ${result.stdout}

### Edk2 BaseTools Build operations ###
Build BaseTools
[Arguments] ${tool_chain} ${ws}
Log to console Compile basetools
${result}= Run Process python
... BaseTools/Edk2ToolsBuild.py -t ${tool_chain}
... cwd=${ws} shell=True stdout=stdout.txt stderr=stderr.txt
Log Many stdout: ${result.stdout} stderr: ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
85 changes: 85 additions & 0 deletions integration_test/azure-pipelines/robot-integration-test.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
##
# Azure Pipeline integration test using robot framework
# and edk2 / Project Mu public repositories
#

# Copyright (c) 2019, Microsoft Corporation
# SPDX-License-Identifier: BSD-2-Clause-Patent
##

jobs:
- job: Integration_Test

#Use matrix to speed up the build process
strategy:
matrix:
EDK2_VS2019:
Tag: 'VS2019'
Image: 'windows-2019'
EDK2_GCC5:
Tag: 'GCC5'
Image: 'ubuntu-18.04'

workspace:
clean: all

pool:
vmImage: $(Image)

steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.10.x'
architecture: 'x64'

- script: pip install --upgrade -r requirements.txt
displayName: 'Install requirements'

- script: pip install --upgrade -r integration_test/pip-requirements.txt
displayName: 'Install Integration test requirements'

- script: pip install -e .
displayName: 'Install from Source'

- powershell: choco install qemu --version=2020.08.14; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
condition: and(contains(variables.Tag, 'AndQemu'), succeeded())

- script: git config --global user.email "[email protected]"
displayName: Configure git user user.email

- script: git config --global user.name "Your Name"
displayName: Configure git user user.name

- script: python -m robot.run -v TEST_OUTPUT_BASE:$(Build.BinariesDirectory) -d report --xunit xunittestresults.xml --include $(Tag) -L TRACE .
displayName: 'Run Robot Tests'
workingDirectory: "integration_test"

# Copy the build logs to the artifact staging directory
- task: CopyFiles@2
displayName: "Copy logs"
inputs:
targetFolder: '$(Build.ArtifactStagingDirectory)'
SourceFolder: 'integration_test/report'
contents: |
log.html
output.xml
report.html
flattenFolders: true
condition: succeededOrFailed()

# Publish build artifacts to Azure Artifacts/TFS or a file share
- task: PublishBuildArtifacts@1
continueOnError: true
displayName: "Publish logs"
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: 'Logs $(System.JobName)'
condition: succeededOrFailed()

- task: PublishTestResults@2
inputs:
testResultsFiles: xunittestresults.xml
searchFolder: 'integration_test/report'
condition: succeededOrFailed()
displayName: 'Publish Test Results'
Loading

0 comments on commit e8c7426

Please sign in to comment.