Migrate CircleCI workflows to GitHub Actions (2/3) #37
Conversation
AzfaarQureshi
force-pushed
the
migrating-to-gha-2
branch
2 times, most recently
from
cbb2dd3
to
8b1029b
Compare
AzfaarQureshi
force-pushed
the
migrating-to-gha-2
branch
from
ff2f05b
to
fb2d6e1
Compare
AzfaarQureshi
force-pushed
the
migrating-to-gha-2
branch
2 times, most recently
from
247c5f7
to
5f89284
Compare
AzfaarQureshi
changed the title
AzfaarQureshi
force-pushed
the
migrating-to-gha-2
branch
from
5f89284
to
91dc67e
Compare
| loadtestpre: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| matrix: ${{ steps.splitloadtest.outputs.matrix }} | ||
| steps: |
There was a problem hiding this comment.
loadtestpre does some pre-processing so we can simulate CircleCI testsharding on GitHub Actions. This cuts down the runtime of loadtests from 15mins to ~5mins
This job does the following:
- Get a list of all loadtest tests using
make testbed-list-loadtest - Split tests into groups of two (
TestIdleMode|TestBallastMemory,TestLog10kDPS|TestMetric10kDPSetc) - Create a JSON containing each test grouping. This JSON will be consumed by the strategy.matrix of the loadtest job using
fromJSON(). The strategy.matrix is how we get to run the loadtests in parallel
| TESTS="$(make -s testbed-list-loadtest | xargs echo|sed 's/ /|/g')" | ||
| TESTS=(${TESTS//|/ }) | ||
| MATRIX="{\"include\":[" | ||
| curr="" | ||
| for i in "${!TESTS[@]}"; do | ||
| if (( i > 0 && i % 2 == 0 )); then | ||
| curr+="|${TESTS[$i]}" | ||
| else | ||
| if [ -n "$curr" ] && (( i>1 )); then | ||
| MATRIX+=",{\"test\":\"$curr\"}" | ||
| elif [ -n "$curr" ]; then | ||
| MATRIX+="{\"test\":\"$curr\"}" | ||
| fi | ||
| curr="${TESTS[$i]}" | ||
| fi | ||
| done | ||
| MATRIX+="]}" | ||
| echo "::set-output name=matrix::$MATRIX" |
There was a problem hiding this comment.
This section of the code is what creates the strategy.matrix JSON
| - name: Create test result archive # some test results have invalid characters | ||
| if: ${{ failure() || success() }} | ||
| continue-on-error: true | ||
| run: tar -cvf test_results.tar testbed/tests/results |
There was a problem hiding this comment.
-
making a tarball of the test results because some tests have filenames with invalid characters (namely the
0*0bytestest) which causes the upload action to fail. -
if: ${{ failure() || success() }}this ensures that test results are uploaded even if the loadtest fails which can be useful for debugging -
continue-on-error: trueSometimes (rarely, but still) the upload action fails due to a network error on github's side. This makes sure the entire job doesnt fail if the test results fail to upload
| @@ -60,7 +60,7 @@ type DataReceiverBase struct { | |||
| Port int | |||
| } | |||
|
|
|||
| const DefaultHost = "localhost" | |||
| const DefaultHost = "127.0.0.1" | |||
There was a problem hiding this comment.
this line was causing an error with the fluentbit log test since localhost could not be resolved on the github actions runners.
| @@ -42,7 +42,7 @@ func TestIdleMode(t *testing.T) { | |||
| ) | |||
| defer tc.Stop() | |||
|
|
|||
| tc.SetResourceLimits(testbed.ResourceSpec{ExpectedMaxCPU: 4, ExpectedMaxRAM: 50}) | |||
| tc.SetResourceLimits(testbed.ResourceSpec{ExpectedMaxCPU: 4, ExpectedMaxRAM: 55}) | |||
There was a problem hiding this comment.
In the GitHub Action runners the RAM utilization is consistently higher (probably due to some other background processes on the GitHub machine). Loadtests don't pass unless these values are adjusted a little 😞
Which problem is this solving?
As part of issue, this pull request migrates the loadtest, correctness and build-package jobs to GitHub Actions.
windows-msiand the publish jobs will be migrated in PR 3/3Migration Plan
We suggest having CircleCI and GitHub Action jobs run in parallel for a few weeks. After the GitHub Actions jobs are running fine for a week or so and then remove the CircleCI workflows from config.yml
cc- @alolita, @shovnik