Add documentation #11
Labels
classic buildpack parity
Features required for parity with the classic Heroku Python buildpack
documentation
Improvements or additions to documentation
Comments
edmorley
added
documentation
edmorley
added a commit
that referenced
this issue
Sep 3, 2024
The Python package manager Poetry is now supported for installing app dependencies: https://python-poetry.org To use Poetry apps must have a `poetry.lock` lockfile, which can be created by running `poetry lock` locally, after adding Poetry config to `pyproject.toml` (which can be done either manually or by using `poetry init`). Apps must only have one package manager file (either `requirements.txt` or `poetry.lock`, but not both) otherwise the buildpack will abort the build with an error (which will help prevent some of the types of support tickets we see in the classic buildpack). Poetry is installed into a build-only layer, so is not available at run-time to reduce image size. The app dependencies are installed into a virtual environment (the same as for pip, after #257), which is on `PATH` so does not need explicit activation when using the app image. As such, use of `poetry run` or `poetry shell` is not required at run-time to use dependencies in the environment. When using Poetry, pip is not explicitly installed, since Poetry includes its own bundled copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls). Both the Poetry and app dependencies layers are cached, however, the Poetry download/wheel cache is not cached, since using it is slower than caching the dependencies layer (for more details see the comments on `poetry_dependencies::install_dependencies`). The `poetry install --sync` command is run using `--only main` so as to only install the main dependencies group and not any other groups (such as test/dev/... groups). Relevant Poetry docs: - https://python-poetry.org/docs/cli/#install - https://python-poetry.org/docs/configuration/ - https://python-poetry.org/docs/managing-dependencies/#dependency-groups Work that will be handled later: - Support for selecting Python version via `tool.poetry.dependencies.python`: #260 - Build output and error messages polish/CX review (this will be performed when switching the buildpack to the new logging style). - More detailed user-facing docs: #11 Closes #7. GUS-W-9607867. GUS-W-9608286. GUS-W-9608295.
edmorley
added a commit
that referenced
this issue
Sep 4, 2024
The Python package manager Poetry is now supported for installing app dependencies: https://python-poetry.org To use Poetry, apps must have a `poetry.lock` lockfile, which can be created by running `poetry lock` locally, after adding Poetry config to `pyproject.toml` (which can be done either manually or by using `poetry init`). Apps must only have one package manager file (either `requirements.txt` or `poetry.lock`, but not both) otherwise the buildpack will abort the build with an error (which will help prevent some of the types of support tickets we see in the classic buildpack with users unknowingly mixing and matching pip + Pipenv). Poetry is installed into a build-only layer (to reduce the final app image size), so is not available at run-time. The app dependencies are installed into a virtual environment (the same as for pip after #257, for the reasons described in #253), which is on `PATH` so does not need explicit activation when using the app image. As such, use of `poetry run` or `poetry shell` is not required at run-time to use dependencies in the environment. When using Poetry, pip is not installed (possible thanks to #258), since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls). Both the Poetry and app dependencies layers are cached, however, the Poetry download/wheel cache is not cached, since using it is slower than caching the dependencies layer (for more details see the comments on `poetry_dependencies::install_dependencies`). The `poetry install --sync` command is run using `--only main` so as to only install the main `[tool.poetry.dependencies]` dependencies group from `pyproject.toml`, and not any of the app's other dependency groups (such as test/dev groups, eg `[tool.poetry.group.test.dependencies]`). I've marked this `semver: major` since in the (probably unlikely) event there are any early-adopter projects using this CNB that have both a `requirements.txt` and `poetry.lock` then this change will cause them to error (until one of the files is deleted). Relevant Poetry docs: - https://python-poetry.org/docs/cli/#install - https://python-poetry.org/docs/configuration/ - https://python-poetry.org/docs/managing-dependencies/#dependency-groups Work that will be handled later: - Support for selecting Python version via `tool.poetry.dependencies.python`: #260 - Build output and error messages polish/CX review (this will be performed when switching the buildpack to the new logging style). - More detailed user-facing docs: #11 Closes #7. GUS-W-9607867. GUS-W-9608286. GUS-W-9608295.
edmorley
added a commit
that referenced
this issue
Sep 4, 2024
The Python package manager Poetry is now supported for installing app dependencies: https://python-poetry.org To use Poetry, apps must have a `poetry.lock` lockfile, which can be created by running `poetry lock` locally, after adding Poetry config to `pyproject.toml` (which can be done either manually or by using `poetry init`). Apps must only have one package manager file (either `requirements.txt` or `poetry.lock`, but not both) otherwise the buildpack will abort the build with an error (which will help prevent some of the types of support tickets we see in the classic buildpack with users unknowingly mixing and matching pip + Pipenv). Poetry is installed into a build-only layer (to reduce the final app image size), so is not available at run-time. The app dependencies are installed into a virtual environment (the same as for pip after #257, for the reasons described in #253), which is on `PATH` so does not need explicit activation when using the app image. As such, use of `poetry run` or `poetry shell` is not required at run-time to use dependencies in the environment. When using Poetry, pip is not installed (possible thanks to #258), since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls). Both the Poetry and app dependencies layers are cached, however, the Poetry download/wheel cache is not cached, since using it is slower than caching the dependencies layer (for more details see the comments on `poetry_dependencies::install_dependencies`). The `poetry install --sync` command is run using `--only main` so as to only install the main `[tool.poetry.dependencies]` dependencies group from `pyproject.toml`, and not any of the app's other dependency groups (such as test/dev groups, eg `[tool.poetry.group.test.dependencies]`). I've marked this `semver: major` since in the (probably unlikely) event there are any early-adopter projects using this CNB that have both a `requirements.txt` and `poetry.lock` then this change will cause them to error (until one of the files is deleted). Relevant Poetry docs: - https://python-poetry.org/docs/cli/#install - https://python-poetry.org/docs/configuration/ - https://python-poetry.org/docs/managing-dependencies/#dependency-groups Work that will be handled later: - Support for selecting Python version via `tool.poetry.dependencies.python`: #260 - Build output and error messages polish/CX review (this will be performed when switching the buildpack to the new logging style). - More detailed user-facing docs: #11 Closes #7. GUS-W-9607867. GUS-W-9608286. GUS-W-9608295.
edmorley
added a commit
that referenced
this issue
Sep 4, 2024
After #254, pip is now installed into its own layer rather than into the system site-packages directory inside the Python layer. This means its now possible to exclude pip from the final app image, by making the pip layer be a build-only layer. Excluding pip from the final app image: - Prevents several classes of user error/confusion/bad app design patterns seen in support tickets (see #255 for more details). - Reduces app image supply chain surface area. - Reduces app image size by 13 MB and layer count by 1, meaning less to have to push to the remote registry. - Matches the approach used for Poetry, where we don't make Poetry available at run-time either. Users that need pip at run-time for a temporary debugging task can run `python -m ensurepip --default-pip` in the container at run-time to make it available again (this command doesn't even have to download anything - it uses the pip bundled with Python). Or if pip is an actual run-time dependency of the app, then the app can add `pip` to its `requirements.txt` (which much more clearly conveys the requirements of the app, and also allows the app to pick what pip version it needs at run-time). Should we find that pip's absence causes confusion in the future, we could always add a wrapper/shim `pip` script in the app image which does something like: ``` echo "pip isn't installed at run-time, if you need it temporarily run 'python -m ensurepip --default-pip' to install it" exit 1 ``` ...to improve discoverability. We'll also document pip (and Poetry) being available at build-time only in the docs that will be added by #11. Closes #255.
edmorley
added a commit
that referenced
this issue
Sep 9, 2024
After #254, pip is now installed into its own layer rather than into the system site-packages directory inside the Python layer. This means its now possible to exclude pip from the final app image, by making the pip layer be a build-only layer. Excluding pip from the final app image: - Prevents several classes of user error/confusion/bad app design patterns seen in support tickets (see #255 for more details). - Reduces app image supply chain surface area. - Reduces app image size by 13 MB and layer count by 1, meaning less to have to push to the remote registry. - Matches the approach used for Poetry, where we don't make Poetry available at run-time either. Users that need pip at run-time for a temporary debugging task can run `python -m ensurepip --default-pip` in the container at run-time to make it available again (this command doesn't even have to download anything - it uses the pip bundled with Python). Or if pip is an actual run-time dependency of the app, then the app can add `pip` to its `requirements.txt` (which much more clearly conveys the requirements of the app, and also allows the app to pick what pip version it needs at run-time). Should we find that pip's absence causes confusion in the future, we could always add a wrapper/shim `pip` script in the app image which does something like: ``` echo "pip isn't installed at run-time, if you need it temporarily run 'python -m ensurepip --default-pip' to install it" exit 1 ``` ...to improve discoverability. We'll also document pip (and Poetry) being available at build-time only in the docs that will be added by #11. Closes #255. GUS-W-16697386.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Whilst the Python CNB already has pretty comprehensive user-facing build log output and error messages, plus lots of developer-facing code comments, we also need:
pack build)Internal tracking epic
The text was updated successfully, but these errors were encountered: