fix bundle_v2 integration tests failures due to running out of space #428
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The previous space freeing method (easimon/maximize-build-space) at some point circa 2024-01 stopped freeing as much space, likely due to changes in the runner (~29GB free after it freed space). Not sure why this happened, but jlumbroso/free-disk-space at time of this commit would get us up to ~45GB free on the runner without negative effects so we've switched to that.
ca-scribner
force-pushed
the
KF-5553-fix-ci-running-out-of-space-v2
branch
from
65bd118
to
33e7c3c
Compare
ca-scribner
changed the title
This was referenced Apr 9, 2024
Closed
misohu
approved these changes
Apr 10, 2024
|
@misohu I fully agree. I would suggest that we focus afterwards on being able to use the self hosted runners. I'll merge for now to unblock the rest, but as next step I'd like us to sit down in the sprint for 2hours tops to revisit the blockers for self hosted. |
NohaIhab
pushed a commit
that referenced
this pull request
May 9, 2024
switch to jlumbroso/free-disk-space for freeing runner space (#428) The previous space freeing method (easimon/maximize-build-space) at some point circa 2024-01 stopped freeing as much space, likely due to changes in the runner (~29GB free after it freed space). Not sure why this happened, but jlumbroso/free-disk-space at time of this commit would get us up to ~45GB free on the runner without negative effects so we've switched to that. Support functionality to override default images for kfp-profile-controller (#416) * Support functionality to override default images for kfp-profile-controller ci: remove destructive mode from integration tests (#441) The charmcraft issues that forced us to use destructive mode are now fixed. build: install `jinja2` from binary (#443) This commit installs jinja2 (an install dependency of charmed-kubeflow-chisme) as a binary to avoid running into the build time issues described in canonical/bundle-kubeflow#883. Part of canonical/bundle-kubeflow#883 refactor: use k8s_service_info lib instead of SDI (#436) * refactor: use k8s_service_info lib instead of SDI Use the k8s_service_info for receiving the MLMD GRPC Service info instead of using the SDI, as it will stop being supported soon. This commit also ensures that mlmd runs with trust=True in the integration tests. Fixes #413 fix: Pin integration test dependencies in main (#434) * pin integration test dependencies Co-authored-by: Daniela Plascencia <[email protected]> feat: Integrate ROCK in metadata-writer charm (#439) chore: Bump o11y libs and remove obsolete juju topology (#446) Ref canonical/bundle-kubeflow#880 Ref canonical/bundle-kubeflow#849 ci: bump juju to 3.5
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#426 documents that the bundle_v2 integration tests fail due to lack of space on the runner. This appears to be in part because the action we've used for freeing space in our GH runners (
easimon/maximize-build-space) no longer frees as much space as it used to. It used to leave us with >40GB free, but now its around 29GB free.Testing with an alternate action for freeing space (
jlumbroso/free-disk-space) yielded ~45GB free without any adverse side effects noticed, so this PR changes to using that new action.Closes #426