Skip to content

Commit

Permalink
Set PYTHONHOME to work around relocation issue with uWSGI (#25)
Browse files Browse the repository at this point in the history
Our Python runtime is relocated (installed into a different location to which is was
originally compiled) which Python itself handles well, since it recalculates its actual
location at startup:
https://docs.python.org/3.11/library/sys_path_init.html

However, the uWSGI package uses the wrong `sysconfig` APIs so tries to reference
the old compile location, unless we override that by setting `PYTHONHOME`:
unbit/uwsgi#2525

This is a standard Python env var, and setting it is pretty harmless (now that the
stack images no longer contain Python 2, so we don't have the dual install issue),
so even though this is a uWSGI bug, it makes sense for us to work around it for now.
(The classic Python buildpack also sets this env var, albeit that's primarily due to
build and run time having different paths, and Python resolving symlinks unless
`PYTHONHOME` is set.)

See also:
https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONHOME

If this issue is ever fixed in uWSGI, we can always reconsider whether we need to
set this env var - however, the issue will still exist in older uWSGI releases, plus there
may be other packages similarly affected.

No test has been added, since:
- uWSGI doesn't ship wheels, and compiling it is slow in CI
- I've tested the change works locally
- `PYTHONHOME` is a built-in Python concept, so not something that really needs a uWSGI-specific test.

The cache hasn't been force-invalidated (which would normally be required any
time the env vars set by the buildpack change), since it's already due to be
invalidated in the next buildpack release, due to the change in setuptools/wheel
versions in #24).

Fixes #18.
GUS-W-12703344.
  • Loading branch information
edmorley authored Apr 11, 2023
1 parent 789dbac commit 2f81aff
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 0 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Updated setuptools from 67.5.0 to 67.6.1. ([#24](https://github.com/heroku/buildpacks-python/pull/24))
- Updated wheel from 0.38.4 to 0.40.0. ([#24](https://github.com/heroku/buildpacks-python/pull/24))

### Fixed

- The `PYTHONHOME` environment variable is now set to work around uWSGI not handling relocated Python installs correctly. ([#25](https://github.com/heroku/buildpacks-python/pull/25))

## [0.1.0] - 2023-03-06

### Added
Expand Down
18 changes: 18 additions & 0 deletions src/layers/python.rs
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,19 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
"PKG_CONFIG_PATH",
":",
)
// Our Python runtime is relocated (installed into a different location to which is was
// originally compiled) which Python itself handles well, since it recalculates its actual
// location at startup:
// https://docs.python.org/3.11/library/sys_path_init.html
// However, the uWSGI package uses the wrong `sysconfig` APIs so tries to reference the old
// compile location, unless we override that by setting `PYTHONHOME`:
// https://github.com/unbit/uwsgi/issues/2525
.chainable_insert(
Scope::All,
ModificationBehavior::Override,
"PYTHONHOME",
layer_path,
)
// Disable Python's output buffering to ensure logs aren't dropped if an app crashes.
.chainable_insert(
Scope::All,
Expand Down Expand Up @@ -520,6 +533,7 @@ mod tests {
("LANG", "C.UTF-8"),
("PIP_DISABLE_PIP_VERSION_CHECK", "1"),
("PKG_CONFIG_PATH", "/layers/python/lib/pkgconfig"),
("PYTHONHOME", "/layers/python"),
("PYTHONUNBUFFERED", "1"),
("SOURCE_DATE_EPOCH", "315532801"),
]
Expand All @@ -531,6 +545,7 @@ mod tests {
("LANG", "C.UTF-8"),
("PIP_DISABLE_PIP_VERSION_CHECK", "1"),
("PKG_CONFIG_PATH", "/layers/python/lib/pkgconfig"),
("PYTHONHOME", "/layers/python"),
("PYTHONUNBUFFERED", "1"),
]
);
Expand All @@ -543,6 +558,7 @@ mod tests {
base_env.insert("LANG", "this-should-be-overridden");
base_env.insert("PIP_DISABLE_PIP_VERSION_CHECK", "this-should-be-overridden");
base_env.insert("PKG_CONFIG_PATH", "/base");
base_env.insert("PYTHONHOME", "this-should-be-overridden");
base_env.insert("PYTHONUNBUFFERED", "this-should-be-overridden");
base_env.insert("SOURCE_DATE_EPOCH", "this-should-be-preserved");

Expand All @@ -563,6 +579,7 @@ mod tests {
("LANG", "C.UTF-8"),
("PIP_DISABLE_PIP_VERSION_CHECK", "1"),
("PKG_CONFIG_PATH", "/layers/python/lib/pkgconfig:/base"),
("PYTHONHOME", "/layers/python"),
("PYTHONUNBUFFERED", "1"),
("SOURCE_DATE_EPOCH", "this-should-be-preserved"),
]
Expand All @@ -574,6 +591,7 @@ mod tests {
("LANG", "C.UTF-8"),
("PIP_DISABLE_PIP_VERSION_CHECK", "1"),
("PKG_CONFIG_PATH", "/layers/python/lib/pkgconfig:/base"),
("PYTHONHOME", "/layers/python"),
("PYTHONUNBUFFERED", "1"),
("SOURCE_DATE_EPOCH", "this-should-be-preserved"),
]
Expand Down

0 comments on commit 2f81aff

Please sign in to comment.