Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gradle.yml: use XXL runners but only if gradle related files are changed #35548

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 18 additions & 28 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,62 +20,52 @@ on:
- synchronize

jobs:
run-check:
# The gradle check task which we will run is embarrassingly parallelizable.
# We therefore run this on a machine with a maximum number of cores.
# We pay per time and per core, so there should be little difference in total cost.
# The latency overhead of setting up gradle prior to running the actual task adds up to about a minute.
runs-on: connector-test-large
name: Gradle Check
timeout-minutes: 30
changes:
runs-on: ubuntu-latest
steps:
- name: Checkout Airbyte
uses: actions/checkout@v3
# IMPORTANT! This is necessary to make sure that a status is reported on the PR
# even if the workflow is skipped. If we used github actions filters, the workflow
# would not be reported as skipped, but instead would be forever pending.
#
# I KNOW THIS SOUNDS CRAZY, BUT IT IS TRUE.
#
# Also it gets worse
#
# IMPORTANT! DO NOT CHANGE THE QUOTES AROUND THE GLOBS. THEY ARE REQUIRED.
# MAKE SURE TO TEST ANY SYNTAX CHANGES BEFORE MERGING.
- name: Get changed files
uses: tj-actions/changed-files@v39
id: changes
with:
# Include java connectors and java CDK.
# Adding all *.java and *.gradle files gets us most of the way there.
# We're take a bit more strict for the java CDK, to make sure that
# the tests run when they should, for instance when changing the contents
# of a text file used as a resource.
files_yaml: |
gradlecheck:
- '**/*.java'
- '**/*.gradle'
- 'airbyte-cdk/java/**/*'
outputs:
gradle: ${{ steps.changes.outputs.gradlecheck_any_changed }}

run-check:
needs:
- changes
if: needs.changes.outputs.gradle == 'true'
# The gradle check task which we will run is embarrassingly parallelizable.
# We therefore run this on a machine with a maximum number of cores.
# We pay per time and per core, so there should be little difference in total cost.
# The latency overhead of setting up gradle prior to running the actual task adds up to about a minute.
runs-on: connector-test-xxlarge
name: Gradle Check
timeout-minutes: 30
steps:
- name: Checkout Airbyte
uses: actions/checkout@v3
- uses: actions/setup-java@v3
if: steps.changes.outputs.gradlecheck_any_changed == 'true'
with:
distribution: "zulu"
java-version: "21"
- name: Install Pip
if: steps.changes.outputs.gradlecheck_any_changed == 'true'
run: curl -fsSL https://bootstrap.pypa.io/get-pip.py | python3
- name: Install Pyenv
if: steps.changes.outputs.gradlecheck_any_changed == 'true'
run: python3 -m pip install virtualenv --user
- name: Docker login
if: steps.changes.outputs.gradlecheck_any_changed == 'true'
# Some tests use testcontainers which pull images from DockerHub.
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Run Gradle Check
if: steps.changes.outputs.gradlecheck_any_changed == 'true'
uses: burrunan/gradle-cache-action@v1
env:
CI: true
Expand Down
Loading