Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oci_image uses significantly more disk space than container_image from rules_docker #439

Closed
SanjayVas opened this issue Dec 15, 2023 · 5 comments
Labels
enhancement New feature or request

Comments

@SanjayVas
Copy link

oci_image appears to copy all base image layer blobs to its Bazel output tree. Additionally it temporarily takes up even more space due to running a separate registry for each target.

I believe container_image from rules_docker only has the created layer(s) in its output.

Anecdotally, this blew up my CI. My compressed Bazel output cache went from ~3 GiB to ~20 GiB when migrating from rules_docker to rules_oci.

@thesayyn thesayyn added the enhancement New feature or request label Dec 15, 2023
@alexeagle
Copy link
Collaborator

I think this is fixed by that #440 - @SanjayVas if you have time, could you patch that in and verify?

@thesayyn
Copy link
Collaborator

@alexeagle partially. we still copy blobs from base image (copy_to_directory) into oci_image's oci-layout (treeartifact) which causes duplication of the blobs.
we should symlink blobs from base image into the oci_image's output to prevent copies.

@thesayyn
Copy link
Collaborator

but that's a bigger fix and i don't have time to contribute. happy to review if somebody else is willing to contribute it.

SanjayVas added a commit to world-federation-of-advertisers/cross-media-measurement that referenced this issue Dec 16, 2023
rules_oci outputs additional layer blobs that result in the Bazel cache being too large. See bazel-contrib/rules_oci#439
@SanjayVas
Copy link
Author

I think this is fixed by that #440 - @SanjayVas if you have time, could you patch that in and verify?

Sorry, the migration to rules_oci was prefactoring for migrating our tree of repos to Bzlmod. I can possibly try after that is done, but it will of course be easier if a version with that change has been published to BCR.

@thesayyn
Copy link
Collaborator

thesayyn commented May 8, 2024

Okay now that #560 #559 landed, this can be closed.

@thesayyn thesayyn closed this as completed May 8, 2024
ple13 pushed a commit to world-federation-of-advertisers/cross-media-measurement that referenced this issue Aug 16, 2024
rules_oci outputs additional layer blobs that result in the Bazel cache being too large. See bazel-contrib/rules_oci#439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants