Heroku-24: Remove gcc, make and libc6-dev from the run image #273
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GCC was added to our run images back in #127 in order to support Ruby 2.6's then new MJIT feature: https://www.ruby-lang.org/en/news/2018/12/25/ruby-2-6-0-released/ However, since then: - The Ruby MJIT feature hasn't really resulted in significant performance benefits for real world use-cases like a Rails app. - Ruby's MJIT has since been superseded by YJIT, which is faster and doesn't need GCC at runtime: https://shopify.engineering/yjit-just-in-time-compiler-cruby https://shopify.engineering/ruby-yjit-is-production-ready - The image size impact of including build tools in our run images has increased considerably (#127 quoted it as 84 MB, but measuring now it's 203 MB). - In a CNB world, image size is much more of a concern than in the S3 `.img` + slug model, so we need to be more selective over what packages we include. As such, this removes `gcc`, `make` and `libc6-dev` from the run image for a 203 MB saving (they are still present in the build image, hence zero changes to `installed-packages-*.txt` for that image). Richard (Ruby owner) has confirmed he's fine with this change. Note: I'm intentionally not adding `binutils` back (which was a transitive dependency), since its 15 MB cost is not worth it for the ~once a year platform operator debugging use-case. Before: ``` -----> Size breakdown... heroku/heroku:24 661MB heroku/heroku:24-build 1.13GB ``` After: ``` -----> Size breakdown... heroku/heroku:24 458MB heroku/heroku:24-build 1.13GB ``` Towards #266. GUS-W-15159536.
edmorley
changed the title
runesoerensen
approved these changes
Mar 21, 2024
edmorley
added a commit
that referenced
this pull request
May 21, 2024
Python's `ctypes.util.find_library` is used to find libraries at runtime. `find_library()` calls out to various external programs to help with this task, initially trying `ldconfig`, and then falling back to `gcc` and `ld` if needed - as documented here: https://docs.python.org/3/library/ctypes.html#finding-shared-libraries Even after the removal of compilation tools from the run image in #273, `ldconfig` still exists in the run image, so libraries can still be found by `ctypes.util.find_library` in the base image. However, whilst the `find_library()` docs say that it supports `LD_LIBRARY_PATH`, the docs fail to mention that the env var is only honoured when `ld` can be found: https://github.com/python/cpython/blob/e870c852c0ea96fa4e4569e9c39c7ceb80ce858d/Lib/ctypes/util.py#L315-L320 As such, when only `ldconfig` is available, `find_library()` only returns libraries from the base image, and not any installed via eg the APT or other buildpacks (which rely upon `LD_LIBRARY_PATH` since they have to do userland "installs" of libraries due to not having root permissions). One such popular use of `ctypes.util.find_library` is in Django's GIS feature, which uses it to locate the GDAL library at runtime: https://github.com/django/django/blob/4971a9afe5642569f3dcfcd3972ebb39e88dd457/django/contrib/gis/gdal/libgdal.py#L53-L66 Therefore, we sadly have to include `binutils` in the run image after all - though thankfully doing so only forgoes 19 MB of the original 203 MB run image size reduction from removing `gcc`, `make` and `libc6-dev`. GUS-W-15821115.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This is a first step towards reducing image size for Heroku-24: #266)
GCC was added to our run images back in #127 in order to support Ruby 2.6's then new MJIT feature:
https://www.ruby-lang.org/en/news/2018/12/25/ruby-2-6-0-released/
However, since then:
.img+ slug model, so we need to be more selective over what packages we include.As such, this removes
gcc,makeandlibc6-devfrom the run image for a 203 MB saving (they are still present in the build image, hence zero changes toinstalled-packages-*.txtfor that image).Richard (Ruby owner) has confirmed he's fine with this change.
Note: I'm intentionally not adding
binutilsback (which was a transitive dependency), since its 15 MB cost is not worth it for the ~once a year platform operator debugging use-case. (More discussion on this at:https://salesforce-internal.slack.com/archives/C02GZCPPV38/p1709204898451809?thread_ts=1709054633.272659&cid=C02GZCPPV38)
Before:
After:
Towards #266.
GUS-W-15159536.