Skip to content

Commit

Permalink
Merge pull request #504 from ppodgorsek/issue-503
Browse files Browse the repository at this point in the history 
Issue #503 - Add CI pipeline example for Azure DevOps
  • Loading branch information
@ppodgorsek
ppodgorsek authored Aug 30, 2024
2 parents 8ccb571 + 814968e commit 4b291b6
Showing 1 changed file with 167 additions and 93 deletions.
260 changes: 167 additions & 93 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
* [Installing additional dependencies](#installing-additional-dependencies)
* [Security consideration](#security-consideration)
* [Continuous integration](#continuous-integration)
* [Azure DevOps pipeline](#ci-azure-devops)
* [Jenkins pipeline](#ci-jenkins)
* [Defining a test run ID](#defining-a-test-run-id)
* [Upload test reports to an AWS S3 bucket](#upload-test-reports-to-an-aws-s3-bucket)
* [Testing this project](#testing-this-project)
Expand Down Expand Up @@ -74,10 +76,12 @@ As stated by [the official GitHub project](https://github.com/robotframework/Sel

This container can be run using the following command:

docker run \
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:<version>
```sh
docker run \
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:<version>
```

<a name="switching-browsers"></a>

Expand Down Expand Up @@ -114,9 +118,11 @@ It is possible to use different directories to read tests from and to generate r

It is possible to parallelise the execution of your test suites. Simply define the `ROBOT_THREADS` environment variable, for example:

docker run \
-e ROBOT_THREADS=4 \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e ROBOT_THREADS=4 \
ppodgorsek/robot-framework:latest
```

By default, there is no parallelisation.

Expand All @@ -126,20 +132,24 @@ By default, there is no parallelisation.

When using parallelisation, it is possible to pass additional [pabot options](https://github.com/mkorpela/pabot#command-line-options), such as `--testlevelsplit`, `--argumentfile`, `--ordering`, etc. These can be passed by using the `PABOT_OPTIONS` environment variable, for example:

docker run \
-e ROBOT_THREADS=4 \
-e PABOT_OPTIONS="--testlevelsplit" \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e ROBOT_THREADS=4 \
-e PABOT_OPTIONS="--testlevelsplit" \
ppodgorsek/robot-framework:latest
```

<a name="passing-additional-options"></a>

### Passing additional options

RobotFramework supports many options such as `--exclude`, `--variable`, `--loglevel`, etc. These can be passed by using the `ROBOT_OPTIONS` environment variable, for example:

docker run \
-e ROBOT_OPTIONS="--loglevel DEBUG" \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e ROBOT_OPTIONS="--loglevel DEBUG" \
ppodgorsek/robot-framework:latest
```

<a name="testing-emails"></a>

Expand All @@ -157,9 +167,11 @@ This project is meant to allow your tests to run anywhere. Sometimes that can be

To set the timezone used inside the Docker image, you can set the `TZ` environment variable:

docker run \
-e TZ=America/New_York \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e TZ=America/New_York \
ppodgorsek/robot-framework:latest
```

<a name="installing-additional-dependencies"></a>

Expand All @@ -169,10 +181,12 @@ It is possible to install additional dependencies dynamically at runtime rather

To do so, simply mount a text file containing the list of dependencies you would like to install using `pip`: (by default, this file is empty if not mounted)

docker run \
-v <local path to the dependency file>:/opt/robotframework/pip-requirements.txt:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:latest
```sh
docker run \
-v <local path to the dependency file>:/opt/robotframework/pip-requirements.txt:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:latest
```
The file must follow [Pip's official requirements file format](https://pip.pypa.io/en/stable/reference/requirements-file-format/).

Expand All @@ -191,9 +205,11 @@ rpa==1.50.0
By default, containers are implicitly run using `--user=1000:1000`, please remember to adjust that command-line setting accordingly, for example:
docker run \
--user=1001:1001 \
ppodgorsek/robot-framework:latest
```sh
docker run \
--user=1001:1001 \
ppodgorsek/robot-framework:latest
```

Remember that that UID/GID should be allowed to access the mounted volumes in order to read the test suites and to write the output.

Expand All @@ -208,42 +224,90 @@ This is a good security practice to make sure containers cannot perform unwanted

## Continuous integration

<a name="ci-azure-devops"></a>

### Azure DevOps pipeline

Azure DevOps provides a `Docker@2` task but it only allows a limited number of commands, it is therefore required to use a `script` instead:

```yml
- job: FunctionalRegressionTests
displayName: Robot Framework tests
steps:
- script: |
set -x
mkdir -p $(Build.Repository.LocalPath)/robot-framework-reports
chmod -R ugo+rwx $(Build.Repository.LocalPath)/robot-framework-reports
# The ROBOT_OPTIONS allow to generate a specific output file that can be
# used to publish test results in the Azure DevOps pipeline run
# Remember to replace ${{ parameters.robotFrameworkFolder }} by the correct
# folder name in your repository
docker run --rm \
--shm-size=1g \
-v $(Build.Repository.LocalPath)/robot-framework-reports:/opt/robotframework/reports:Z \
-v $(Build.Repository.LocalPath)/${{ parameters.robotFrameworkFolder }}:/opt/robotframework/tests:Z \
-e ROBOT_OPTIONS="-x outputxunit.xml" \
-t docker.io/ppodgorsek/robot-framework:latest
displayName: Run Robot Framework tests
- task: PublishTestResults@2
condition: always()
displayName: Publish Robot Framework test results
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/outputxunit.xml'
searchFolder: '$(Build.Repository.LocalPath)/robot-framework-reports'
failTaskOnFailedTests: true
failTaskOnFailureToPublishResults: true
failTaskOnMissingResultsFile: true
```
<a name="ci-jenkins"></a>
### Jenkins pipeline
It is possible to run the project from within a Jenkins pipeline by relying on the shell command line directly:
pipeline {
agent any
stages {
stage('Functional regression tests') {
steps {
sh "docker run --shm-size=1g -e BROWSER=firefox -v $WORKSPACE/robot-tests:/opt/robotframework/tests:Z -v $WORKSPACE/robot-reports:/opt/robotframework/reports:Z ppodgorsek/robot-framework:latest"
}
}
}
```yml
pipeline {
agent any
stages {
stage('Functional regression tests') {
steps {
sh "docker run --shm-size=1g -e BROWSER=firefox -v $WORKSPACE/robot-tests:/opt/robotframework/tests:Z -v $WORKSPACE/robot-reports:/opt/robotframework/reports:Z ppodgorsek/robot-framework:latest"
}
}
}
}
```

The pipeline stage can also rely on a Docker agent, as shown in the example below:

pipeline {
agent none
stages {
stage('Functional regression tests') {
agent { docker {
image 'ppodgorsek/robot-framework:latest'
args '--shm-size=1g -u root' }
}
environment {
BROWSER = 'firefox'
ROBOT_TESTS_DIR = "$WORKSPACE/robot-tests"
ROBOT_REPORTS_DIR = "$WORKSPACE/robot-reports"
}
steps {
sh '''
/opt/robotframework/bin/run-tests-in-virtual-screen.sh
'''
}
}
}
```yml
pipeline {
agent none
stages {
stage('Functional regression tests') {
agent { docker {
image 'ppodgorsek/robot-framework:latest'
args '--shm-size=1g -u root' }
}
environment {
BROWSER = 'firefox'
ROBOT_TESTS_DIR = "$WORKSPACE/robot-tests"
ROBOT_REPORTS_DIR = "$WORKSPACE/robot-reports"
}
steps {
sh '''
/opt/robotframework/bin/run-tests-in-virtual-screen.sh
'''
}
}
}
}
```
<a name="defining-a-test-run-id"></a>
Expand All @@ -257,9 +321,11 @@ For that purpose, the `ROBOT_TEST_RUN_ID` variable was introduced:
It can simply be passed during the execution, such as:
docker run \
-e ROBOT_TEST_RUN_ID="feature/branch-name" \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e ROBOT_TEST_RUN_ID="feature/branch-name" \
ppodgorsek/robot-framework:latest
```
By default, the test run ID is empty.
Expand All @@ -269,48 +335,54 @@ By default, the test run ID is empty.
To upload the report of a test run to an S3 bucket, you need to define the following environment variables:
docker run \
-e AWS_ACCESS_KEY_ID=<your AWS key> \
-e AWS_SECRET_ACCESS_KEY=<your AWS secret> \
-e AWS_DEFAULT_REGION=<your AWS region e.g. eu-central-1> \
-e AWS_BUCKET_NAME=<name of your S3 bucket> \
ppodgorsek/robot-framework:latest
```sh
docker run \
-e AWS_ACCESS_KEY_ID=<your AWS key> \
-e AWS_SECRET_ACCESS_KEY=<your AWS secret> \
-e AWS_DEFAULT_REGION=<your AWS region e.g. eu-central-1> \
-e AWS_BUCKET_NAME=<name of your S3 bucket> \
ppodgorsek/robot-framework:latest
```
<a name="testing-this-project"></a>
## Testing this project
Not convinced yet? Simple tests have been prepared in the `test/` folder, you can run them using the following commands:
# Using Chromium
docker run \
-v `pwd`/reports:/opt/robotframework/reports:Z \
-v `pwd`/test:/opt/robotframework/tests:Z \
-e BROWSER=chrome \
ppodgorsek/robot-framework:latest

# Using Firefox
docker run \
-v `pwd`/reports:/opt/robotframework/reports:Z \
-v `pwd`/test:/opt/robotframework/tests:Z \
-e BROWSER=firefox \
ppodgorsek/robot-framework:latest
```sh
# Using Chromium
docker run \
-v `pwd`/reports:/opt/robotframework/reports:Z \
-v `pwd`/test:/opt/robotframework/tests:Z \
-e BROWSER=chrome \
ppodgorsek/robot-framework:latest
# Using Firefox
docker run \
-v `pwd`/reports:/opt/robotframework/reports:Z \
-v `pwd`/test:/opt/robotframework/tests:Z \
-e BROWSER=firefox \
ppodgorsek/robot-framework:latest
```
For Windows users who use **PowerShell**, the commands are slightly different:
# Using Chromium
docker run \
-v ${PWD}/reports:/opt/robotframework/reports:Z \
-v ${PWD}/test:/opt/robotframework/tests:Z \
-e BROWSER=chrome \
ppodgorsek/robot-framework:latest

# Using Firefox
docker run \
-v ${PWD}/reports:/opt/robotframework/reports:Z \
-v ${PWD}/test:/opt/robotframework/tests:Z \
-e BROWSER=firefox \
ppodgorsek/robot-framework:latest
```sh
# Using Chromium
docker run \
-v ${PWD}/reports:/opt/robotframework/reports:Z \
-v ${PWD}/test:/opt/robotframework/tests:Z \
-e BROWSER=chrome \
ppodgorsek/robot-framework:latest
# Using Firefox
docker run \
-v ${PWD}/reports:/opt/robotframework/reports:Z \
-v ${PWD}/test:/opt/robotframework/tests:Z \
-e BROWSER=firefox \
ppodgorsek/robot-framework:latest
```
Screenshots of the results will be available in the `reports/` folder.
Expand Down Expand Up @@ -384,11 +456,13 @@ As mentioned on the [Docker Hub](https://hub.docker.com/r/ppodgorsek/robot-frame

As mentioned in the official documentation, [Podman](https://docs.podman.io/en/latest/markdown/podman-run.1.html#platform-os-arch) and [Docker](https://docs.docker.com/build/building/multi-platform/) provide a `--platform` option which selects a given application architecture, such as:

docker run \
--platform linux/amd64 \
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:<version>
```sh
docker run \
--platform linux/amd64 \
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
ppodgorsek/robot-framework:<version>
```

Please note: builds and automated tests of this project will remain performed on a `linux/amd64` architecture so such emulation might not work, depending on your device and operating system.

Expand Down

0 comments on commit 4b291b6

Please sign in to comment.