Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.11 Support #7476

Closed
mawl opened this issue Sep 7, 2023 · 1 comment · Fixed by #7598
Closed

Python 3.11 Support #7476

mawl opened this issue Sep 7, 2023 · 1 comment · Fixed by #7598
Labels
analyzer About the analyzer tool

Comments

@mawl
Copy link

mawl commented Sep 7, 2023

Hi there,

As a follow-up to #7333:

How can I teach poetry / pip / python-inspector to use python version 3.11 instead of 3.10?

#5740 says, that this should work:

.ort.yml

analyzer:
  package_managers:
    PIP:
      options:
        operatingSystem: 'linux'
        pythonVersion: '3.11'

I have build your Dockerfile with actual python components:

      --build-arg PYTHON_VERSION=3.11.4
      --build-arg PYENV_GIT_TAG=v2.3.23
      --build-arg PIPTOOL_VERSION=23.2.1      
      --build-arg PYTHON_POETRY_VERSION=1.5.1 

pyenv versions says:
* 3.11.4 (set by /opt/python/version)

But this get's executed and the dependency tree stays empty:

python-inspector --python-version 310 --operating-system linux ...

Full log:

13:24:30.432 [main] INFO  org.ossreviewtoolkit.analyzer.Analyzer - Starting Poetry analysis.
13:24:30.438 [DefaultDispatcher-worker-1] INFO  org.ossreviewtoolkit.analyzer.PackageManager - Using Poetry to resolve dependencies for path 'test/python/poetry/poetry.lock'...
13:24:30.448 [DefaultDispatcher-worker-1] INFO  org.ossreviewtoolkit.plugins.packagemanagers.python.Poetry - Generating 'requirements-from-poetry.txt' file in '/builds/company/compliance/license-scanning/test/python/poetry' directory...
13:24:30.455 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.EnvironmentVariableFilter - Filtering out these variables from the environment: [...
13:24:30.459 [DefaultDispatcher-worker-1] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'poetry export --without-hashes --format=requirements.txt' in '/builds/company/compliance/license-scanning/test/python/poetry'...
13:24:30.997 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - dnspython==2.4.1 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - pendulum==2.1.2 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - pymongo==4.4.1 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - python-dateutil==2.8.2 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - pytzdata==2020.1 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - six==1.16.0 ; python_version >= "3.11" and python_version < "3.12"
13:24:30.998 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - 
13:24:31.009 [DefaultDispatcher-worker-1] INFO  org.ossreviewtoolkit.plugins.packagemanagers.python.Pip - Resolving dependencies for '/builds/company/compliance/license-scanning/test/python/poetry/requirements-from-poetry.txt' with Python version '3.10' and operating system 'linux'.
13:24:31.011 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.EnvironmentVariableFilter - Filtering out these variables from the environment: [...
13:24:31.013 [DefaultDispatcher-worker-1] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'python-inspector --python-version 310 --operating-system linux --json-pdt /tmp/ort-PythonInspector14862222632561284951/python-inspector7097629943133964626.json --analyze-setup-py-insecurely --requirement /builds/company/compliance/license-scanning/test/python/poetry/requirements-from-poetry.txt --verbose' in '/builds/company/compliance/license-scanning/test/python/poetry'...
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - Resolving dependencies...
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - Using netrc file /home/ort/.netrc
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - direct_dependencies:
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='dnspython==2.4.1; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='pendulum==2.1.2; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='pymongo==4.4.1; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='python-dateutil==2.8.2; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='pytzdata==2020.1; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.864 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  DependentPackage(purl='pkg:pypi/[email protected]', extracted_requirement='six==1.16.0; python_version >= "3.11" and python_version < "3.12"', scope='install')
13:24:31.865 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - environment: Environment(python_version='310', operating_system='linux')
13:24:31.865 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - repos:
13:24:31.865 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture -  PypiSimpleRepository(index_url='https://pypi.org/simple', credentials=None)
13:24:31.865 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - done!
13:24:31.865 [DefaultDispatcher-worker-1] DEBUG org.ossreviewtoolkit.utils.common.ProcessCapture - 

Thanks for your help.

@fviernau
Copy link
Member

I've also investigated this a bit. I found that

  1. Specifying a python version (range) in pyproject.toml is mandatory
  2. The poetry export ... command used by Poetry.kt to generate a `requirements.txt also exports the
    version constraints for the python version.

Once ORT calls python-inspector to analyze the generated requirements.txt file with a python version (-p option) not in the python version range specified in pyproject.toml then the result would always be empty. So, I wonder if we should change the logic such that Poetry.kt selects an arbitrary python version from available python versions matching the range, in case no version has been specified.

Looking for feedback on this...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer About the analyzer tool
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants