How to get version from pyproject.toml from python app?

[sobolevn]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Question

I have a basic python app, and inside this app there's a --version flag.
I want to show the same version as I have in pyproject.toml, but without duplicating this information.

What should I do here?
Thanks!

[roywes]

The pkg_resources module enables package information introspection. Here's how it can be used to get the version of an installed package.

>>> import pkg_resources
>>> my_version = pkg_resources.get_distribution('my-package-name').version

[piotr-dobrogost]

Related: Single-sourcing the package version from Python Packaging User Guide

[sobolevn]

Thanks. I am using #273 (comment)

[nchammas]

Using pkg_resources.get_distribution() implies a dependency on setuptools, no? I think one of the goals of Poetry is to eliminate the need for setuptools.

For what it's worth, I see that Poetry itself simply duplicates the version.

I think there currently isn't a "proper" solution for single-sourcing the application version in a project packaged by Poetry.

[timonbimon]

any update on how to do this best? does following how Poetry does it itself really mean updating the version.py file by hand?

[sobolevn]

I am using @roywes solution with a fallback to
https://github.com/wemake-services/wemake-python-styleguide/blob/master/docs/conf.py#L22-L37

when the package is not installed.

[Eric-Arellano]

You can now use importlib.metadata and its backport importlib_metadata: https://docs.python.org/3/library/importlib.metadata.html#distribution-versions.

It has a convenient function importlib.metadata.version('my-distribution') that will return a str read from your pyproject.toml configuration.

[insysion]

@Eric-Arellano Just for clarity, importlib.metadata isn't reading directly from the pyproject.toml, is it?

The docs seem to say that a package built by poetry and installed will have the metadata version set from the pyproject.toml field

[ErikBjare]

@insysion The metadata is built from the pyproject.toml (into these locations in the sdist: setup.py, pyproject.toml, PKG-INFO), but the importlib.metadata probably gets it from the version set in the poetry-built setup.py.

[cglacet]

It seems both pkg_resources.get_distribution('my-package-name').version and importlib.metadata.version('my-package-name') needs the package to be installed which seems odd when you are building the doc for that particular package (from local sources). Is there a way to instead simply get the version that is currently built?

[sinoroc]

@cglacet You could directly parse the TOML in pyproject.toml.
What is the context exactly? Why can't you install or at least build the project?

[cglacet]

@sinoroc Yes I could, but isn't including a package as its own dependency weird?

[sinoroc]

isn't including a package as its own dependency weird?

@cglacet I am not following.

[cglacet]

@sinoroc That's probably not clear because it's not clear in my head haha. If I need to install a package to get its version, then, to get the current version of the package I'm currently building it needs to install itself (so it can read its own version number). In my case, I want to include the version number in the package documentation (sphinx configuration).

And I also want that particular piece of code to run even while in development (because I will build the documentation locally a lot before its built by my CI pipeline)

[cglacet]

Like you said, I could very well parse the pyproject.toml file, but I'm surprised my use case doesn't already have a solution. I'm not trying to do anything fancy 😄

[sinoroc]

I'm surprised my use case doesn't already have a solution

I know that it is a question that keeps coming up again and again, and there is still no one true answer. It all depends on what the context and conditions are. There are multiple solutions, all with their pros and cons.

---

Since you mention building sphinx documentation I will tell here how it is solved in one of my projects...

The conf.py for sphinx contains something like:

import importlib.metadata

_DISTRIBUTION_METADATA = importlib.metadata.metadata('MyProjectName')

author = _DISTRIBUTION_METADATA['Author']
project = _DISTRIBUTION_METADATA['Name']
version = _DISTRIBUTION_METADATA['Version']

and the readthedocs.yml file contains something like:

version: 2
python:
  install:
    - method: 'pip'
      path: '.'

So, yes it means that the project always has to be installed beforehand (at least in editable (aka develop) mode). It is a bit of a shame, I reckon, but it works well, so I leave it like that for now.

Although I believe it would be relatively easy, to extract the version string (and other metadata) from a build artefact of the project, instead of from a complete installation.

---

In the case of poetry projects, since the source of truth for the version string is in pyproject.toml file:
Instead, in Sphinx's conf.py you could import a toml parser and extract the metadata (version string, etc.) out of pyproject.toml. It would avoid building and installing the project itself.

---

If I need to install a package to get its version, then, to get the current version of the package I'm currently building it needs to install itself (so it can read its own version number)

This bit, I do not understand what you mean. Where is the issue? Usually libraries or applications do not really need to know their own version string. And if they do have this need, then I would definitely think that it is at run-time, meaning that they are already installed. So using version = importlib.metadata.version('MyProjectName') should be straightforward, right?

[andyleejordan]

Wow, thank you @sinoroc. Your example helped me immensely.

[abolourian]

I'm looking for a solution to run in poetry_script.py and the solutions above doesn't work. This worked for me though.
subprocess.run(['poetry', 'version', '-s'], capture_output=True, text=True).stdout.rstrip()

[sinoroc]

@abolourian What are you trying to do? How is your use case different than others' use cases? What is the poetry_script.py you are talking about? What have you tried exactly that didn't work for you?

[abolourian]

@sinoroc In a CICD process, I'm using a method in poetry_script.py to (1) bump the [tool.poetry].version in pyproject.toml file, (2) extract the version and write it to _version.py file, and (3) create zip files for AWS Lambda packages. I need the statement I mentioned in my previous comment for step 2.

I defined a method in a poetry_script.py file and added the following lines in pyproject.toml. I was not clear enough.
[tool.poetry.scripts]
build = "poetry_scripts:build"

[sinoroc]

@abolourian

Basically everything I am saying is: don't write a _version.py, and don't write your version string in __version__ of your main top level package's __init__.py either, or anything like that. There is no need for that. My recommendation is to call importlib.metadata.version('LibraryName') at run-time instead.

If really you want to do it, fair enough, no problem. You approach with calling poetry version --short in a subprocess seems good enough. My personal opinion is that I don't think it's a good practice, and I also believe the default project skeleton created by poetry new should not have this __version__ string. If I remember right some of the maintainers also think it might be time to get rid of this.

Also I would like to mention the project poetry-scmver. I don't recommend it, but it exists and it seems to work.

You might also want to read this thread: #693

And take note that there is a project of a real plugin system in poetry, scheduled for v1.2. Which might allow to do such things in a much cleaner fashion.

Other discussion on the topic: python/importlib_metadata#248

[sinoroc]

@maresb My point is: this should be solved in the development tooling, this does not belong in the code itself. And it is made even worse by the fact that it is running unconditionally on import.

Maybe the Poetry commands such as poetry version, poetry run, and so on should do this kind of check and warn: "the content of pyproject.toml has changed since last run of `poetry install" or something like that.

I also believe that having a __version__ variable is unnecessary. At the very minimum the value should be computed lazily. So that the retrieval of the version string happens when __version__ is actually accessed, not when the module is imported.

[maresb]

this should be solved in the development tooling

I don't see how Poetry could address this since a version change in pyproject.toml can be easily triggered without any interaction from the poetry CLI. In particular, git checkout or git pull may change the version number. I suppose one could attack this with post-checkout and post-merge commits, but that sort of tooling is fairly uncommon.) I'm fairly sure that any robust solution must involve changing the way Python handles metadata for editable installs.

I also believe that having a __version__ variable is unnecessary. At the very minimum the value should be computed lazily. So that the retrieval of the version string happens when __version__ is actually accessed, not when the module is imported.

I find __version__ to be a very practical quasistandard. I like being able to simply type something like np.__version__ to inspect version numbers. But I do agree it would be nice to compute it lazily. Note that PEP 562 provides a mechanism which could accomplish this, but that would require some really gnarly looking boilerplate.

[sinoroc]

Use importlib.metadata.version('numpy') instead of np.__version__. This is standard. I guess you @maresb know this already (and you might have your own reason to want to do things differently), but I repeat it any chance I get anyway because people less informed than you keep doing the same mistake of ignoring this advice thinking their own use case is special when really it is not.

Maybe a post checkout git hook could help indeed, no idea if it exists. My thinking is that sooner or later you will need to run a poetry command, so maybe poetry could do a quick check to make sure that the editable installation metadata is up-to-date with the content of pyproject.toml. Honestly this kind of thing was never a problem for me, I am not sure in which circumstances it can happen to be an issue worth this much worry and workaround.

[rcousens]

I don't claim to be a Python master by any means, but I wrote this in my __version__.py file:

import importlib.metadata
import tomllib

try:
    with open("pyproject.toml", "rb") as f:
        pyproject = tomllib.load(f)
    __version__ = pyproject["tool"]["poetry"]["version"]
except Exception as e:
    __version__ = importlib.metadata.version('bop')

Seems to work for local development when the package is installed in edit mode, and then when built, distributed and pip installed. Version number is correct and allows me to centralise on pyproject.toml. Awesome!

[MaxLenormand]

@MaxLenormand @facundopadilla This can not possibly work once installed. The pyproject.toml file is not in the wheel, and is of course never installed.

Late to the party but yes, I quickly realised this wasn't going to work

I have dropped this since. I'm not sure what the best approach is to handle this

[sinoroc]

I'm not sure what the best approach is to handle this

@MaxLenormand What do you mean? It is a solved problem, use importlib.metadata.version('NameOfInstalledDependency').

[vengroff]

Here is what I put in __init__py in my top-level package. It's a combination of some of the ideas above:

import importlib.metadata
from pathlib import Path


def _package_version() -> str:
    """Find the version of the package."""
    package_version = "unknown"

    try:
        # Try to get the version of the current package if
        # it is running from a distribution.
        package = Path(__file__).parent.name
        metadata = importlib.metadata.metadata(package)

        package_version = metadata["Version"]
    except importlib.metadata.PackageNotFoundError:
        # Fall back on getting it from a local pyproject.toml.
        # This works in a development environment where the
        # package has not been installed from a distribution.
        import toml

        pyproject_toml_file = Path(__file__).parent.parent / "pyproject.toml"
        if pyproject_toml_file.exists() and pyproject_toml_file.is_file():
            package_version = toml.load(pyproject_toml_file)["tool"]["poetry"][
                "version"
            ]
            # Indicate it might be locally modified or unreleased.
            package_version = package_version + "+"

    return package_version


version = _package_version()

I suppose I could move the private function _package_version to a tiny self-contained project and put
it on pypi. Then any project could import it and set the version with a one-liner.

[sinoroc]

@vengroff Why? Why is this code necessary? Why is having a version variable necessary? Anyone who wants to know the version string of your library can run importlib.metadata.version('name-of-library'). And, no Path(__file__).parent.name is not a reliable way to obtain the name of the library (you likely get the name of the import package, but that is not the same thing, they might be equal, but not the same thing). I really do not understand which problem this whole thing is trying to solve, and at what cost. This could all be a single function call to the standard library.

[vengroff]

@sinoroc Thanks, you are right about the first half. I should use importlib.metadata.version. The second half is to deal with the case when it is in a development environment and importlib.metadata can't find anything.

[sinoroc]

@vengroff Be mindful that this creates a run time cost (import time even!) on all production users only for some slight developer comfort. I do not have any use case in mind where it would make sense to compute the value of version at import or even to have the version variable at all.

[juanmirocks]

@sinoroc I personally need to have access to the version variable in various cases:

• Show library version in CLI help (or print out version in general)
• Show API version in REST API documentation
• Return API version in some/all REST API outputs (for better transparency with the client)

I don't think I'm alone and so I find having easy access to the version variable important.

[sinoroc]

@juanmirocks All those are legitimate use cases and can be done with importlib.metadata.version('name-of-library'). And I do not see any reason to store this value in a global variable (computed at import-time). Maybe the value could be cached in order to avoid reaching to importlib.metadata more than once, but I have no idea if it is necessary or worth it.

[vengroff]

I forgot to follow up with what I did following the previous messages in this issue from a couple weeks ago.

For the reasons @juanmirocks mentioned among others, it can be nice to have the version available as package.version or package.__version__. And if, as in the API case mentioned, you want to easily tell whether you are running a dev or released server, you can make the version of unpackaged code come from pyproject.toml but look different, you have more work to do. If you don't want to execute that code at import time, which was @sinoroc's objection, you can hide it in a function triggered by getting the attribute with package.__getattr__. And you can cache it so the work only happens once per process. I put all those together and ended up with this in my __init__.py:

from typing import Any
import importlib.metadata
from pathlib import Path

__package_version = "unknown"


def __get_package_version() -> str:
    """Find the version of this package."""
    global __package_version

    if __package_version != "unknown":
        # We already set it at some point in the past,
        # so return that previous value without any
        # extra work.
        return __package_version

    try:
        # Try to get the version of the current package if
        # it is running from a distribution.
        __package_version = importlib.metadata.version("my_package_name")
    except importlib.metadata.PackageNotFoundError:
        # Fall back on getting it from a local pyproject.toml.
        # This works in a development environment where the
        # package has not been installed from a distribution.
        import toml

        pyproject_toml_file = Path(__file__).parent.parent / "pyproject.toml"
        if pyproject_toml_file.exists() and pyproject_toml_file.is_file():
            __package_version = toml.load(pyproject_toml_file)["tool"]["poetry"][
                "version"
            ]
            # Indicate it might be locally modified or unreleased.
            __package_version = __package_version + "+"

    return __package_version


def __getattr__(name: str) -> Any:
    """Get package attributes."""
    if name in ("version", "__version__"):
        return __get_package_version()
    else:
        raise AttributeError(f"No attribute {name} in module {__name__}.")

[sinoroc]

@vengroff Yes, this seems like this would have less technical side effects and drawbacks.

But this is a lot of code for -- in my opinion -- very little gain, and this gain is strictly limited to the development phase, which -- again in my opinion -- is definitely not what one should optimize for.

[juanmirocks]

From my experience, many packages are in the "development phase" FOREVER or for a VERY LONG TIME. They never get "distributed" and yet they keep running for a long time.

For those scenarios, either we have some code similar as to what @vengroff suggested (which IMHO is not long; it's mostly comments), or otherwise developers need to keep remembering to install from time to time their dev packages (poetry install) and then they can make use of the updated importlib.metadata.version("<my_package>").

I guess that second option is acceptable though feels strange to me at least. Nonetheless, if that's so far the "recommended" approach, it would be nice adding it to the poetry docs somewhere. Do you agree? I guess some people don't know such option exists (me included before) and that's why the confusion and the many comments in this issue.

[epogrebnyak]

There should be a package for this. пт, 5 янв. 2024 г., 00:33 Darren Erik Vengroff ***@***.***>:

…

I forgot to follow up with what I did following the previous messages in this issue from a couple weeks ago. For the reasons @juanmirocks <https://github.com/juanmirocks> mentioned among others, it can be nice to have the version available as package.version or package.__version__. And if, as in the API case mentioned, you want to easily tell whether you are running a dev or released server, you can make the version of unpackaged code come from pyproject.toml but look different, you have more work to do. If you don't want to execute that code at import time, which was @sinoroc <https://github.com/sinoroc>'s objection, you can hide it in a function triggered by getting the attribute with package.__getattr__. And you can cache it so the work only happens once per process. I put all those together and ended up with this in my __init__.py: from typing import Anyfrom .impl.exceptions import CensusApiExceptionimport importlib.metadatafrom pathlib import Path __package_version = "unknown" def __get_package_version() -> str: """Find the version of this package.""" global __package_version if __package_version != "unknown": # We already set it at some point in the past, # so return that previous value without any # extra work. return __package_version try: # Try to get the version of the current package if # it is running from a distribution. __package_version = importlib.metadata.version("my_package_name") except importlib.metadata.PackageNotFoundError: # Fall back on getting it from a local pyproject.toml. # This works in a development environment where the # package has not been installed from a distribution. import toml pyproject_toml_file = Path(__file__).parent.parent / "pyproject.toml" if pyproject_toml_file.exists() and pyproject_toml_file.is_file(): __package_version = toml.load(pyproject_toml_file)["tool"]["poetry"][ "version" ] # Indicate it might be locally modified or unreleased. __package_version = __package_version + "+" return __package_version def __getattr__(name: str) -> Any: """Get package attributes.""" if name in ("version", "__version__"): return __get_package_version() else: raise AttributeError(f"No attribute {name} in module {__name__}.") — Reply to this email directly, view it on GitHub <#273 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACGWBLR7IKDGAB7UDTUIT7DYM4N27AVCNFSM4FHXZN4KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBXG43TQOJZGY3Q> . You are receiving this because you commented.Message ID: ***@***.***>

[vengroff]

There is a package for this now. I factored a variation of the code above to a new package called usingversion. See https://github.com/vengroff/usingversion.

Things get just a little more complex when you want to move this functionality out of a package's __init__.py. The usage
is two lines (see the README.md) but it still feels a bit clunkier than it needs to be. If you have suggestions or other comments, open an issue or discussion in the usingversion repo.

[sinoroc]

it would be nice adding it to the poetry docs

Note that it is not specific to Poetry. No build backend or dev workflow tool that I know of does it any better than Poetry, because that is the state of the art as far as I can tell. If packaging metadata changes (which includes the version) then the thing must be reinstalled (to take the new metadata into account), even if the thing is installed as "editable".

I am not saying things can not be improved. Maybe someone will come up with a good solution to handle updating metadata transparently for editable installations.

From my experience, many packages are in the "development phase" FOREVER or for a VERY LONG TIME. They never get "distributed" and yet they keep running for a long time.

Without thinking much about this... I wonder what is the point of bumping the version number in such a use case then? How does that work? You git pull some new code, but do not run poetry install right after?

[juanmirocks]

@sinoroc I see your points. I was referring rather to developers working on some tool, e.g. a REST API web server, which returns the version somewhere, expecting to see the version changed in pyproject.toml immediately reflected in the API output.

From a user point of view, the version property in the pyproject.toml is poetry's "responsibility", after all the property is called tool.poetry.version. And that's why I think some developers using Poetry expect to get that "live" version for a developing/running application in some way. That's why I still think that a small documentation comment somewhere would be helpful. My 2 cents.

[sinoroc]

So I just tested poetry version major and it does not trigger the update of the installed metadata (python run pip show my-project). Maybe it should. Maybe it has already been suggested, I do not know. Right now, one still needs to run poetry install for the installed metadata to be updated. On the other hand poetry version shows the new version (i.e. not what is the installed metadata).

And yes, maybe it is worth a note somewhere that updating the installed metadata (via editing pyproject.toml directly or via the command line), requires running poetry install. I do not know what is the best location for such a note in the documentation.

Maybe in the output of the poetry version major and any other command that changes the metadata (such as poetry add some-library), there should also be a note that poetry install is required to update the metadata.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.