Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve cache handling #1679

Merged
merged 1 commit into from
Oct 31, 2024
Merged

Improve cache handling #1679

merged 1 commit into from
Oct 31, 2024

Conversation

edmorley
Copy link
Member

@edmorley edmorley commented Oct 30, 2024

Refactors cache handling to dedicated functions under lib/cache.sh (rather than being scattered around the buildpack), and makes the following improvements:

  • Ensures the cache is now also discards the cache when the package manager (or its version) changes.
  • Improves the build log output shown when restoring or discarding the cache. For example, if the cache was invalidated all reasons are now shown.
  • Stops performing unnecessary cache file copies when the cache is due to be invalidated. This required moving the cache restoration step to after the bin/pre_compile hook runs.
  • Fixes cache restoration in the case where an app's requirements.txt was formerly a symlink.
  • Adds buildpack metrics for the status of the cache and duration of cache restoration/saving.

Note: The .heroku/ namespaced cache contents are considered an internal Heroku buildpack implementation detail and should not be directly modified by any bin/{pre,post}_compile hooks or third-party buildpacks.

Fixes #1673.
Fixes #1674.
Fixes #1675.
Fixes #1676.
Fixes #1677.
Fixes #1678.
Prep for #796.
Unblocks upgrading pip (since #1674 prevents pypa/pip#12950).
GUS-W-16811131.

@edmorley edmorley self-assigned this Oct 30, 2024
@edmorley edmorley marked this pull request as ready for review October 30, 2024 21:04
@edmorley edmorley requested a review from a team as a code owner October 30, 2024 21:04
@edmorley edmorley enabled auto-merge (squash) October 30, 2024 21:05
@edmorley edmorley enabled auto-merge (squash) October 30, 2024 21:10
Refactors cache handling to dedicated functions under `lib/cache.sh`
(rather than being scattered around the buildpack), and makes the
following improvements:
- Ensures the cache is now also discards the cache when the package
  manager (or its version) changes.
- Improves the build log output shown when restoring or discarding the
  cache. For example, if the cache was invalidated all reasons are now
  shown.
- Stops performing unnecessary cache file copies when the cache is due
  to be invalidated. This required moving the cache restoration step to
  after the `bin/pre_compile` hook runs.
- Fixes cache restoration in the case where an app's `requirements.txt`
  was formerly a symlink.
- Adds buildpack metrics for the status of the cache and duration of
  cache restoration/saving.

Fixes #1673.
Fixes #1674.
Fixes #1675.
Fixes #1676.
Fixes #1677.
Fixes #1678.
Prep for #796.
Unblocks upgrading pip (since #1674 prevents pypa/pip#12950).
GUS-W-16811131.
@edmorley edmorley merged commit f00f258 into main Oct 31, 2024
7 checks passed
@edmorley edmorley deleted the cache-refactor branch October 31, 2024 14:56
@heroku-linguist heroku-linguist bot mentioned this pull request Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment