Proper support for installing packages with apt

[alexeagle]

https://github.com/chainguard-dev/rules_apko provides a nice solution for Alpine. We may want something similar for Debian.

apt-get does things like:

• install transitive packages
• write to /var/lib/dpkg/status.d (see https://github.com/GoogleContainerTools/distroless/blob/0aaf3a9a08c66b7f3dda323be174251a98e11ced/private/pkg/dpkg_status.go#L80)

We would like to read a lockfile telling us where to download the .deb files, but apt-get doesn't create one. Something like https://github.com/TrevorSundberg/apt-lock looks promising, if we could rely on it.

[betaboon]

I'll just leave this here: https://github.com/betaboon/rules_debian_packages

[alexeagle]

Ooh! What's your plan with long term maintenance and support of that repo?

[betaboon]

i have no explicit plans. it's working for us as is.

I'm fine with working on it to help this case.
maybe forking it as a basis for a rules_apto :P
or donating it.
whatever helps. let me know ;)

[alexeagle]

Nice, it looks like a pretty good shape to me. Maybe we can donate that ruleset to bazel-contrib and get it on the BCR. @thesayyn any thoughts on it?

[thesayyn]

Look promising. This is what distroless team does. I have done something similar in the past; https://github.com/bazel-contrib/rules_oci/tree/apt/experimental/apt, a pure bazel replacement using repository rules. The problem with apt packages is that they are rolling so it's impossible to get reproducible builds even if you generate locks etc, the remote .deb may disappear when there is a new version.

[thesayyn]

debian snapshot is what we want but they pretty much unreliable.

[betaboon]

snapshots is what I'm using in rules_debian_packages

[alexeagle]

I think users would need to "patch over" this problem with a reliable, immutable registry. Something like PackageCloud, Artifactory, etc. There's already a need for such a thing to back the Bazel downloader, and I think we can provide this to Aspect Pro users and everyone else will have to build their own.

Which is to say, users of "bazel-contrib/rules_debian_pkg" or whatever we call it will be required to supply the package registry they want to use, and choosing snapshot.debian.org is one choice they could make but we warn them that it's non-reproducible.

[betaboon]

tbh i found debian snapshots to be quite reliable in terms of old packages being available.
the one aspect that snapshots.debian.org is unreliable at is the occasional timeout.
so i would heavily advice a caching proxy like squid et al

also, the lockfile-format is heavily inspired by the lockfile that distroless uses. (it might even be compatible)

[thesayyn]

that it's non-reproducible.

problem with debian snapshot is that it's unreliable in terms of uptime. you will get frequent interruptions because the load that registry is enormous.

[betaboon]

just a quick update, I'm currently working on making rules_debian_packages usable with rules_oci.

i got this working on a dev-branch:

load("@apt//:packages.bzl", "debian_package_layer")
load("@rules_oci//oci:defs.bzl", "oci_image", "oci_tarball")
load("@rules_pkg//pkg:mappings.bzl", "pkg_mklink")
load("@rules_pkg//pkg:tar.bzl", "pkg_tar")

pkg_mklink(
    name = "usr_bin_bash",
    link_name = "/usr/bin/bash",
    target = "/bin/bash",
)

pkg_tar(
    name = "usr_bin_bash_layer",
    srcs = [":usr_bin_bash"],
)

oci_image(
    name = "image",
    base = "@distroless_cc",
    tars = [
        # required to replicate distroless/python
        debian_package_layer("libbz2-1.0"),
        debian_package_layer("libcom-err2"),
        debian_package_layer("libcrypt1"),
        debian_package_layer("libexpat1"),
        debian_package_layer("liblzma5"),
        debian_package_layer("libreadline8"),
        debian_package_layer("zlib1g"),
        # required by rules_py
        debian_package_layer("bash"),
        debian_package_layer("coreutils"),
        debian_package_layer("grep"),
        ":usr_bin_bash_layer",
    ],
)

oci_tarball(
    name = "image.tar",
    image = ":image",
    repo_tags = ["python_base:latest"],
)

[betaboon]

just wanted to let you know that i just merged the rules_oci support in rules_debian_packages mentioned above.

I'm still missing bzlmod support, and could use some some help on that, as i have no experience with bzlmod yet (see wip PR )

[alexeagle]

That's neat! We are working with the distroless team who already has something similar for working with debian packages, I'll leave for @thesayyn to propose how https://github.com/GoogleContainerTools/rules_distroless might relate to rules_debian_packages.

[thesayyn]

Closing is favor of bazel-contrib/SIG-rules-authors#88