-
Notifications
You must be signed in to change notification settings - Fork 867
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Improve image build speed #1919
chore: Improve image build speed #1919
Conversation
Signed-off-by: Justin Marquis <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #1919 +/- ##
==========================================
+ Coverage 82.36% 82.40% +0.03%
==========================================
Files 119 119
Lines 16872 16910 +38
==========================================
+ Hits 13897 13934 +37
- Misses 2284 2285 +1
Partials 691 691
Continue to review full report at Codecov.
|
Signed-off-by: Justin Marquis <[email protected]>
Kudos, SonarCloud Quality Gate passed! |
@@ -23,13 +23,13 @@ jobs: | |||
- name: Set up Docker Buildx | |||
uses: docker/setup-buildx-action@v1 | |||
|
|||
- name: Cache Docker layers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this intentional to remove cache?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the cache is only caching the exported layers in the final build stage. Which provides no real benefit. This could be changed to mode=max to export all interminate layers but GitHub has a limit of 10GB across a single repo. Also the time to prossess approximately 20gb of cache and compress it really outweighs the benefits. A full cache will also not prossess RUN commands in the dockerfile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also this cache is being accessed during pull requests. It can create conflicts and slow the processes down.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
got it..thanks
thanks for the PR |
@harikrongali @jessesuen Thank you for the review and merge. Image build times are much better, but the plugin-image should have been built from the runners cache of the controller-image. I will look into this and have a solution by Monday. We should be able to cut off an additional 20+ minutes. |
Signed-off-by: Justin Marquis <[email protected]>
Signed-off-by: Justin Marquis <[email protected]> Signed-off-by: Ravi Hari <[email protected]>
* chore: Improve image build speed #2 Signed-off-by: Justin Marquis <[email protected]> * chore: Added test-arm-image test for arm64 Signed-off-by: Justin Marquis <[email protected]> * chore: enabled buildkit for docker build commands Signed-off-by: Justin Marquis <[email protected]> * chore: support for releases and further docker optimizations Signed-off-by: Justin Marquis <[email protected]>
Signed-off-by: Justin Marquis <[email protected]> Signed-off-by: Juan Enciso <[email protected]>
* chore: Improve image build speed argoproj#2 Signed-off-by: Justin Marquis <[email protected]> * chore: Added test-arm-image test for arm64 Signed-off-by: Justin Marquis <[email protected]> * chore: enabled buildkit for docker build commands Signed-off-by: Justin Marquis <[email protected]> * chore: support for releases and further docker optimizations Signed-off-by: Justin Marquis <[email protected]>
Signed-off-by: Justin Marquis [email protected]
Disabled cache, it is configured to only cache the export layers that are in the final build stages. Not taking advantage of the cache on disk for the second run of the
kubectl-argo-rollouts
image This will reduce build times by half. I recommend a second PR to cleanup the existing cache steps from the docker-publish.yml as they are not needed anymore.I have tested these builds locally. I'm expecting the complete Workflow should be complete in approximately 20-25 minutes on GitHub runners.
The flowchart below shows how the Dockerfile is processed using cross-compilation. Orange nodes are x86_64 and opaque orange nodes are arm64.
Checklist:
(a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or(c) this is a chore."fix(controller): Updates such and such. Fixes #1234"
.