-
Notifications
You must be signed in to change notification settings - Fork 400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build go binaries in parallel #314
Comments
FWIW I tried to leverage MultiWrite in #238 and evidence suggested it was slower, since it required builds to finish before publishing. I would believe building all binaries at once is faster, but it means we sit idle on pushing until all builds complete, which might eat all our savings. If you're interested in hacking on this it could be useful to get an idea about how much / whether it saves real time |
Yeah this seems like a great change to make. It's a little bit harder than I'd like because of multi-arch builds and how the code is currently structured, but we might be able to work around that a bit... The way things are currently laid out, we call Beyond that, a single For We'll want to fix #192 as well... I'm also a bit worried about streaming use cases -- it may be the case that we are reading yaml from stdin that never "ends" because of something like our own
My optimizations in #267 and #269 are mostly around no-op builds, which would help for this case, but what you're proposing would help a lot in general. This might be what @dprotaso was getting at in an earlier conversation, and I just didn't understand. My initial thoughts on a solution for this would look something like:
One final thing I'm worried about is name collisions. For example, if we have to build |
Oh one more thing: from inside There is definitely a lot of "waste" from having multiple
vs.
vs.
|
This issue is stale because it has been open for 90 days with no |
/lifecycle frozen I think there are some gains to be made - though non-obvious how to approach it atm. |
Ok - I have no power here lol |
Compiling multiple binaries in a single go command is much faster than individually. For example, both incremental builds:
Along with #236, this would improve the speed of
ko
quite a bit I think.Currently building these 3 binaries in parallel then building and pushing in parallel with
docker buildx bake
, warm builds take ~8s compared to ~23s forko
The text was updated successfully, but these errors were encountered: