Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate CircleCI workflows to GitHub Actions (2/3) #37

Closed
wants to merge 3 commits into from
Closed

Migrate CircleCI workflows to GitHub Actions (2/3) #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fb2d6e1
adding remaining CI workflows
Dec 16, 2020
965978a
removing sleep from common.sh
Dec 16, 2020
91dc67e
removing windows-msi
Dec 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
211 changes: 211 additions & 0 deletions .github/workflows/build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,13 @@ jobs:
uses: actions/[email protected]
with:
go-version: 1.15
- name: Restore module cache
uses: actions/cache@v2
env:
cache-name: cache-go-modules
with:
path: /home/runner/go/pkg/mod
key: go-pkg-mod-${{ runner.os }}-${{ hashFiles('./go.mod') }}
- name: Setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
Expand Down Expand Up @@ -125,3 +132,207 @@ jobs:
with:
name: collector-binaries
path: ./bin
loadtestpre:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.splitloadtest.outputs.matrix }}
steps:
Comment on lines +135 to +139
Copy link
Author

@AzfaarQureshi AzfaarQureshi Dec 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loadtestpre does some pre-processing so we can simulate CircleCI testsharding on GitHub Actions. This cuts down the runtime of loadtests from 15mins to ~5mins
This job does the following:

  1. Get a list of all loadtest tests using make testbed-list-loadtest
  2. Split tests into groups of two (TestIdleMode|TestBallastMemory, TestLog10kDPS|TestMetric10kDPS etc)
  3. Create a JSON containing each test grouping. This JSON will be consumed by the strategy.matrix of the loadtest job using fromJSON(). The strategy.matrix is how we get to run the loadtests in parallel

- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Go
uses: actions/[email protected]
with:
go-version: 1.15
- name: Setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Restore module cache
uses: actions/cache@v2
env:
cache-name: cache-go-modules
with:
path: /home/runner/go/pkg/mod
key: go-pkg-mod-${{ runner.os }}-${{ hashFiles('./go.mod') }}
- name: split loadtest jobs
id: splitloadtest
run: |
TESTS="$(make -s testbed-list-loadtest | xargs echo|sed 's/ /|/g')"
TESTS=(${TESTS//|/ })
MATRIX="{\"include\":["
curr=""
for i in "${!TESTS[@]}"; do
if (( i > 0 && i % 2 == 0 )); then
curr+="|${TESTS[$i]}"
else
if [ -n "$curr" ] && (( i>1 )); then
MATRIX+=",{\"test\":\"$curr\"}"
elif [ -n "$curr" ]; then
MATRIX+="{\"test\":\"$curr\"}"
fi
curr="${TESTS[$i]}"
fi
done
MATRIX+="]}"
echo "::set-output name=matrix::$MATRIX"
Comment on lines +160 to +177
Copy link
Author

@AzfaarQureshi AzfaarQureshi Dec 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section of the code is what creates the strategy.matrix JSON

loadtest:
runs-on: ubuntu-latest
needs: [setup-environment, loadtestpre]
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.loadtestpre.outputs.matrix) }}
env:
TEST_RESULTS: testbed/tests/results/junit/results.xml
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Go
uses: actions/[email protected]
with:
go-version: 1.15
- name: Download tool binaries
uses: actions/download-artifact@v2
with:
name: tool-binaries
path: /home/runner/go/bin
- name: Add execute permissions to tool binaries
run: chmod -R +x /home/runner/go/bin
- name: Setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Restore module cache
uses: actions/cache@v2
env:
cache-name: cache-go-modules
with:
path: /home/runner/go/pkg/mod
key: go-pkg-mod-${{ runner.os }}-${{ hashFiles('./go.mod') }}
- name: Install fluentbit
if: ${{ contains(matrix.test, 'Log10kDPS') }}
run: |
wget https://packages.fluentbit.io/ubuntu/bionic/pool/main/t/td-agent-bit/td-agent-bit_1.5.3_amd64.deb
sudo dpkg -i ./td-agent-bit*.deb
echo "/opt/td-agent-bit/bin" >> $GITHUB_PATH
sudo ln -s /opt/td-agent-bit/bin/td-agent-bit /usr/local/bin/fluent-bit
- run: mkdir -p results && touch results/TESTRESULTS.md
- name: Loadtest
run: make testbed-loadtest
env:
TEST_ARGS: "-test.run=${{ matrix.test }}"
- name: Create test result archive # some test results have invalid characters
if: ${{ failure() || success() }}
continue-on-error: true
run: tar -cvf test_results.tar testbed/tests/results
Comment on lines +223 to +226
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • making a tarball of the test results because some tests have filenames with invalid characters (namely the 0*0bytes test) which causes the upload action to fail.

  • if: ${{ failure() || success() }} this ensures that test results are uploaded even if the loadtest fails which can be useful for debugging

  • continue-on-error: true Sometimes (rarely, but still) the upload action fails due to a network error on github's side. This makes sure the entire job doesnt fail if the test results fail to upload

- name: Upload test results
if: ${{ failure() || success() }}
continue-on-error: true
uses: actions/upload-artifact@v2
with:
name: test-results
path: test_results.tar
- name: GitHub issue generator
if: ${{ failure() && github.ref == 'ref/head/master' }}
run: |
go run cmd/issuegenerator/main.go $TEST_RESULTS
correctness:
runs-on: ubuntu-latest
needs: [setup-environment]
env:
TEST_RESULTS: testbed/tests/results/junit/results.xml
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Go
uses: actions/[email protected]
with:
go-version: 1.15
- name: Download tool binaries
uses: actions/download-artifact@v2
with:
name: tool-binaries
path: /home/runner/go/bin
- name: Setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Add execute permissions to tool binaries
run: chmod -R +x /home/runner/go/bin
- name: Restore module cache
uses: actions/cache@v2
env:
cache-name: cache-go-modules
with:
path: /home/runner/go/pkg/mod
key: go-pkg-mod-${{ runner.os }}-${{ hashFiles('./go.mod') }}
- name: Loadtest
run: make testbed-correctness
- name: GitHub issue generator
if: ${{ failure() && github.ref == 'ref/head/master' }}
run: |
go run cmd/issuegenerator/main.go $TEST_RESULTS
build-package:
runs-on: ubuntu-latest
needs: [cross-compile]
strategy:
fail-fast: false
matrix:
package_type: ["deb", "rpm"]
steps:
- name: Checkout Repo
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Setup Go
uses: actions/[email protected]
with:
go-version: 1.15
- name: Setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
mkdir bin/
- name: Install Ruby
uses: actions/setup-ruby@v1
with:
ruby-version: '2.6'
- name: Install fpm and dependencies
run: gem install --no-document fpm -v 1.11.0
- name: Download tool binaries
uses: actions/download-artifact@v2
with:
name: tool-binaries
path: /home/runner/go/bin
- name: Download collector binaries
uses: actions/download-artifact@v2
with:
name: collector-binaries
path: ./bin
- name: Add execute permissions to downloaded binaries
run: |
chmod -R +x /home/runner/go/bin
chmod -R +x ./bin
- name: Set Release Tag
id: github_tag
run: |
TAG="${GITHUB_REF##*/}"
if [[ $TAG =~ ^v[0-9]+\.[0-9]+\.[0-9]+.* ]]
then
echo "::set-output name=tag::$TAG"
fi
- name: Build ${{ matrix.package_type }} amd64 package
run: ./internal/buildscripts/packaging/fpm/${{ matrix.package_type }}/build.sh "${{ steps.github_tag.outputs.tag }}" "amd64" "./dist/"
- name: Build ${{ matrix.package_type }} arm64 package
run: ./internal/buildscripts/packaging/fpm/${{ matrix.package_type }}/build.sh "${{ steps.github_tag.outputs.tag }}" "arm64" "./dist/"
- name: Test ${{ matrix.package_type }} package
run: |
if [[ "${{ matrix.package_type }}" = "deb" ]]; then
./internal/buildscripts/packaging/fpm/test.sh dist/otel-collector*amd64.deb
else
./internal/buildscripts/packaging/fpm/test.sh dist/otel-collector*x86_64.rpm
fi
- name: Upload Packages
uses: actions/upload-artifact@v2
with:
name: build-packages
path: ./dist
2 changes: 1 addition & 1 deletion testbed/testbed/receivers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ type DataReceiverBase struct {
Port int
}

const DefaultHost = "localhost"
const DefaultHost = "127.0.0.1"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line was causing an error with the fluentbit log test since localhost could not be resolved on the github actions runners.


func (mb *DataReceiverBase) ReportFatalError(err error) {
log.Printf("Fatal error reported: %v", err)
Expand Down
4 changes: 2 additions & 2 deletions testbed/tests/e2e_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ func TestIdleMode(t *testing.T) {
)
defer tc.Stop()

tc.SetResourceLimits(testbed.ResourceSpec{ExpectedMaxCPU: 4, ExpectedMaxRAM: 50})
tc.SetResourceLimits(testbed.ResourceSpec{ExpectedMaxCPU: 4, ExpectedMaxRAM: 55})
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the GitHub Action runners the RAM utilization is consistently higher (probably due to some other background processes on the GitHub machine). Loadtests don't pass unless these values are adjusted a little 😞

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this increase enough?

tc.StartAgent()

tc.Sleep(tc.Duration)
Expand All @@ -53,7 +53,7 @@ func TestBallastMemory(t *testing.T) {
ballastSize uint32
maxRSS uint32
}{
{100, 50},
{100, 60},
{500, 70},
{1000, 100},
}
Expand Down
4 changes: 2 additions & 2 deletions testbed/tests/metric_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ func TestMetric10kDPS(t *testing.T) {
testbed.NewOTLPDataReceiver(testbed.GetAvailablePort(t)),
testbed.ResourceSpec{
ExpectedMaxCPU: 50,
ExpectedMaxRAM: 60,
ExpectedMaxRAM: 65,
},
},
{
Expand All @@ -76,7 +76,7 @@ func TestMetric10kDPS(t *testing.T) {
testbed.NewOTLPHTTPDataReceiver(testbed.GetAvailablePort(t)),
testbed.ResourceSpec{
ExpectedMaxCPU: 50,
ExpectedMaxRAM: 60,
ExpectedMaxRAM: 65,
},
},
}
Expand Down