From fbbfc09d74042a8cae691d48d5d32cfd1cef4da6 Mon Sep 17 00:00:00 2001 From: Ed Morley <501702+edmorley@users.noreply.github.com> Date: Mon, 29 Apr 2024 15:47:20 +0100 Subject: [PATCH 1/2] Heroku-24: Remove unnecessary packages from build image A number of development packages currently being installed in the build image are actually for use-case that either do not make sense in a container context, or are not used by the dependencies of the app itself (we only need libs for app dependencies, not for the compiling runtimes themselves use-case). Many of these packages have been simply been copy-pasted from one new stack to the next, even though they are not actually needed in the image. As such, the following have been removed from the build image: - `libacl1-dev`: Used for the management of POSIX Access Control Lists. Whilst many Linux system tools use `libacl` (including `coreutils`), app dependencies won't be typically linking against it, so we don't need the headers for it. - `libapt-pkg-dev`: Development files for APT's `libapt-pkg` and `libapt-inst`. Neither our buildpacks nor apps need to link against `libapt` since they use the `apt` CLI instead. - `libaudit-dev`: For Linux Audit, whose `libaudit` is used by things like the `login` command, but otherwise not typically needed for app dependencies to link against. - `libcap-dev`: For management of POSIX 1003.1e capabilities, an alternative to the superuser model of privilege under Linux. Not something needed for app dependencies to link against. - `libkeyutils-dev`: For the management of keys in the kernel. Not useful in a non-root container context. - `libkmod-dev`: For the management of Linux Kernel modules. Not useful in a non-root container context. Also, the `libkmod` lib is not present in the run image, so this wouldn't be usable at runtime anyway. - `libpopt-dev`: For parsing cmdline parameters. Whilst many Linux system tools use `libpopt`, app dependencies won't be typically linking against it, so we don't need the headers for it. - `libreadline-dev`: Provides functionality for entering text in interactive scenarios, and the management of terminal history. Whilst many tools/runtimes will use the library, application dependencies themselves won't typically link against it, so we don't need the headers for it in the build image. (See note below about compiling runtimes.) - `libseccomp-dev`: Provides a high level interface to Linux seccomp filter. Was added in #105 to support a demo of an internal project that has since been abandoned (see https://github.com/heroku/build-team/issues/83). IMO this demo use-case should instead have used the APT buildpack instead of requesting addition to the base image while experimenting. - `libsemanage-dev`: For SELinux policy manipulation. This is not something app dependencies typically ever need to link against. - `libsystemd-dev`: Development files for SystemD. Not typically needed in a container context, plus not something app dependencies typically ever need to link against. Also, the `libsystemd0` lib is not present in the run image, so this wouldn't be usable at runtime anyway. - `libudev-dev`: For enumerating and introspecting local devices. Not useful in a container context. Note: For use-cases like compiling Python/Ruby/PHP binaries for upload to S3, any required headers (such as `libreadline-dev`) that aren't also needed for *application* dependencies should instead be installed in the image being used to compile those binaries, rather than being included in the build image here. GUS-W-15159536. --- heroku-24-build/installed-packages-amd64.txt | 17 ----------------- heroku-24-build/installed-packages-arm64.txt | 17 ----------------- heroku-24-build/setup.sh | 12 ------------ 3 files changed, 46 deletions(-) diff --git a/heroku-24-build/installed-packages-amd64.txt b/heroku-24-build/installed-packages-amd64.txt index 6181401d..339e4af0 100644 --- a/heroku-24-build/installed-packages-amd64.txt +++ b/heroku-24-build/installed-packages-amd64.txt @@ -93,11 +93,9 @@ keyboxd krb5-multidev less libacl1 -libacl1-dev libaec0 libaom-dev libaom3 -libapt-pkg-dev libapt-pkg6.0t64 libarchive13t64 libargon2-1 @@ -107,9 +105,7 @@ libass9 libassuan0 libatomic1 libattr1 -libattr1-dev libaudit-common -libaudit-dev libaudit1 libbinutils libblkid-dev @@ -131,8 +127,6 @@ libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libcairo2-dev -libcap-dev -libcap-ng-dev libcap-ng0 libcap2 libcap2-bin @@ -268,10 +262,7 @@ libk5crypto3 libkadm5clnt-mit12 libkadm5srv-mit12 libkdb5-10t64 -libkeyutils-dev libkeyutils1 -libkmod-dev -libkmod2 libkrb5-3 libkrb5-dev libkrb5support0 @@ -324,8 +315,6 @@ libmpc3 libmpfr6 libmysqlclient-dev libmysqlclient21 -libncurses-dev -libncurses6 libncursesw6 libnetpbm11t64 libnettle8t64 @@ -368,7 +357,6 @@ libpng-dev libpng16-16t64 libpoppler-glib8t64 libpoppler134 -libpopt-dev libpopt0 libpq-dev libpq5 @@ -382,7 +370,6 @@ libquadmath0 librabbitmq-dev librabbitmq4 libraw23t64 -libreadline-dev libreadline8t64 librhash0 librsvg2-2 @@ -394,12 +381,10 @@ libsasl2-2 libsasl2-dev libsasl2-modules libsasl2-modules-db -libseccomp-dev libseccomp2 libselinux1 libselinux1-dev libsemanage-common -libsemanage-dev libsemanage2 libsepol-dev libsepol2 @@ -421,7 +406,6 @@ libssl3t64 libstdc++-13-dev libstdc++6 libsvtav1enc1d1 -libsystemd-dev libsystemd0 libsz2 libtasn1-6 @@ -438,7 +422,6 @@ libtirpc3t64 libtool libtsan2 libubsan1 -libudev-dev libudev1 libunbound8 libunibreak5 diff --git a/heroku-24-build/installed-packages-arm64.txt b/heroku-24-build/installed-packages-arm64.txt index d5e032fd..9983c5b7 100644 --- a/heroku-24-build/installed-packages-arm64.txt +++ b/heroku-24-build/installed-packages-arm64.txt @@ -89,11 +89,9 @@ keyboxd krb5-multidev less libacl1 -libacl1-dev libaec0 libaom-dev libaom3 -libapt-pkg-dev libapt-pkg6.0t64 libarchive13t64 libargon2-1 @@ -103,9 +101,7 @@ libass9 libassuan0 libatomic1 libattr1 -libattr1-dev libaudit-common -libaudit-dev libaudit1 libbinutils libblkid-dev @@ -127,8 +123,6 @@ libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libcairo2-dev -libcap-dev -libcap-ng-dev libcap-ng0 libcap2 libcap2-bin @@ -261,10 +255,7 @@ libk5crypto3 libkadm5clnt-mit12 libkadm5srv-mit12 libkdb5-10t64 -libkeyutils-dev libkeyutils1 -libkmod-dev -libkmod2 libkrb5-3 libkrb5-dev libkrb5support0 @@ -317,8 +308,6 @@ libmpc3 libmpfr6 libmysqlclient-dev libmysqlclient21 -libncurses-dev -libncurses6 libncursesw6 libnetpbm11t64 libnettle8t64 @@ -361,7 +350,6 @@ libpng-dev libpng16-16t64 libpoppler-glib8t64 libpoppler134 -libpopt-dev libpopt0 libpq-dev libpq5 @@ -374,7 +362,6 @@ libpython3.12-stdlib librabbitmq-dev librabbitmq4 libraw23t64 -libreadline-dev libreadline8t64 librhash0 librsvg2-2 @@ -386,12 +373,10 @@ libsasl2-2 libsasl2-dev libsasl2-modules libsasl2-modules-db -libseccomp-dev libseccomp2 libselinux1 libselinux1-dev libsemanage-common -libsemanage-dev libsemanage2 libsepol-dev libsepol2 @@ -413,7 +398,6 @@ libssl3t64 libstdc++-13-dev libstdc++6 libsvtav1enc1d1 -libsystemd-dev libsystemd0 libsz2 libtasn1-6 @@ -430,7 +414,6 @@ libtirpc3t64 libtool libtsan2 libubsan1 -libudev-dev libudev1 libunbound8 libunibreak5 diff --git a/heroku-24-build/setup.sh b/heroku-24-build/setup.sh index 78730f97..9bcd6976 100755 --- a/heroku-24-build/setup.sh +++ b/heroku-24-build/setup.sh @@ -13,15 +13,11 @@ packages=( gettext # Internationalization utils used by Django, Rails etc. git jq - libacl1-dev - libapt-pkg-dev libargon2-dev - libaudit-dev libbsd-dev libbz2-dev libc-client2007e-dev libcairo2-dev - libcap-dev libcurl4-openssl-dev libdb-dev libev-dev @@ -37,8 +33,6 @@ packages=( libicu-dev libidn-dev libjpeg-dev - libkeyutils-dev - libkmod-dev libkrb5-dev libldap-dev liblz4-dev @@ -50,18 +44,12 @@ packages=( libmysqlclient-dev libnetpbm10-dev libonig-dev - libpopt-dev libpq-dev librabbitmq-dev - libreadline-dev librtmp-dev - libseccomp-dev - libsemanage-dev libsodium-dev libssl-dev - libsystemd-dev libtool - libudev-dev libuv1-dev libwrap0-dev libxml2-dev From 2ce90e63fa048fe37731c4959aa09ef6857d183b Mon Sep 17 00:00:00 2001 From: Ed Morley <501702+edmorley@users.noreply.github.com> Date: Sun, 5 May 2024 23:09:28 +0100 Subject: [PATCH 2/2] Add `libncurses6` to the run image Since it was present in Heroku-22 via a transitive dependency, but no longer pulled in on Heroku-24. --- heroku-24-build/installed-packages-amd64.txt | 1 + heroku-24-build/installed-packages-arm64.txt | 1 + heroku-24/installed-packages-amd64.txt | 1 + heroku-24/installed-packages-arm64.txt | 1 + heroku-24/setup.sh | 1 + 5 files changed, 5 insertions(+) diff --git a/heroku-24-build/installed-packages-amd64.txt b/heroku-24-build/installed-packages-amd64.txt index 339e4af0..f62d3fdb 100644 --- a/heroku-24-build/installed-packages-amd64.txt +++ b/heroku-24-build/installed-packages-amd64.txt @@ -315,6 +315,7 @@ libmpc3 libmpfr6 libmysqlclient-dev libmysqlclient21 +libncurses6 libncursesw6 libnetpbm11t64 libnettle8t64 diff --git a/heroku-24-build/installed-packages-arm64.txt b/heroku-24-build/installed-packages-arm64.txt index 9983c5b7..be525658 100644 --- a/heroku-24-build/installed-packages-arm64.txt +++ b/heroku-24-build/installed-packages-arm64.txt @@ -308,6 +308,7 @@ libmpc3 libmpfr6 libmysqlclient-dev libmysqlclient21 +libncurses6 libncursesw6 libnetpbm11t64 libnettle8t64 diff --git a/heroku-24/installed-packages-amd64.txt b/heroku-24/installed-packages-amd64.txt index d9f8f8ab..7830049c 100644 --- a/heroku-24/installed-packages-amd64.txt +++ b/heroku-24/installed-packages-amd64.txt @@ -170,6 +170,7 @@ libmnl0 libmount1 libmp3lame0 libmysqlclient21 +libncurses6 libncursesw6 libnettle8t64 libnghttp2-14 diff --git a/heroku-24/installed-packages-arm64.txt b/heroku-24/installed-packages-arm64.txt index d9f8f8ab..7830049c 100644 --- a/heroku-24/installed-packages-arm64.txt +++ b/heroku-24/installed-packages-arm64.txt @@ -170,6 +170,7 @@ libmnl0 libmount1 libmp3lame0 libmysqlclient21 +libncurses6 libncursesw6 libnettle8t64 libnghttp2-14 diff --git a/heroku-24/setup.sh b/heroku-24/setup.sh index bf515675..fc921884 100755 --- a/heroku-24/setup.sh +++ b/heroku-24/setup.sh @@ -72,6 +72,7 @@ packages=( libmemcached11 # Used by the PHP Memcached extension. libmp3lame0 # Used by FFmpeg in heroku-buildpack-activestorage-preview. libmysqlclient21 + libncurses6 # Used by the Ruby runtime. libonig5 # Used by the PHP runtime. libopencore-amrnb0 # Used by FFmpeg in heroku-buildpack-activestorage-preview. libopencore-amrwb0 # Used by FFmpeg in heroku-buildpack-activestorage-preview.