Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ORT not able to analyze any Python based project (setuptool breaking) #5159

Closed
Tracked by #4637
meeraa247 opened this issue Mar 17, 2022 · 9 comments
Closed
Tracked by #4637
Labels
analyzer About the analyzer tool bug Issues that are considered to be bugs help wanted An issue where third-party help is wanted on

Comments

@meeraa247
Copy link

Hi @ all,

first of all: We appreciate this community a all and would like to thank you for all the support. Our team heavily relies on scanning & analyzing projects using ORT.

Recently we came across a small project based on Python (v.3.8.x).
During the ANALYZE Step, we identified the following ERRORS (from the logs):

[DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'Python_v3.8.8/Lib/site-packages/numpy/ma/setup.py' failed with: IOException: Running '/tmp/ort-Pip-ma-virtualenv14989541890154123245/bin/pip --trusted-host pypi.org --trusted-host pypi.python.org install --no-warn-conflicts --prefer-binary .' in '/project/Python_v3.8.8/Lib/site-packages/numpy/ma' failed with exit code 1:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-t9xwYP/
You are using pip version 18.1, however version 20.3.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

At first, it seemed to be an issue with the so called "virtualenv". @heliocastro also checked and confirmed this behaviour from side.

We also tried to specify a dedicated python version for ORT (using the docker build process), but that did not help.

Any help would be highly appreciated as this blocks our processes for this project at the moment.

Thank you.

@heliocastro
Copy link
Contributor

heliocastro commented Mar 18, 2022

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference:
https://github.com/fastavro/fastavro

Look like some discrepancy on the manipulating the setuptools new versions, as our dependency analyzer tool is pydep is really old. To be investigated.

Here's the output of analyzer error

09:23:47.125 [DefaultDispatcher-worker-3] DEBUG org.eclipse.jgit.util.SystemReader - loading config FileBasedConfig[/workspace/.gitconfig]
09:23:47.181 [DefaultDispatcher-worker-3] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'developer_requirements.txt' failed with: IOException: Running '/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py info .' in '/worspace/code' failed with exit code 1:
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptoolsalso replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>
main()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 43, in main
args.func(args)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 68, in info
setup_dict, err = pydep.setup_py.setup_info_dir(args.dir)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 41, in setup_info_dir
return setup_info(setupfile), None
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 53, in setup_info
setuptools_mod = __import__('setuptools')
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in <module>
import _distutils_hack.override  # noqa: F401
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
__import__('_distutils_hack').do_override()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 72, in do_override
ensure_local_distutils()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
assert '_distutils' in core.__file__, core.__file__
AssertionError: /opt/python/versions/3.8.13/lib/python3.8/distutils/core.py

@sschuberth sschuberth added bug Issues that are considered to be bugs analyzer About the analyzer tool labels Mar 25, 2022
@sschuberth
Copy link
Member

To be investigated.

@heliocastro are you planning to work on this?

@sschuberth sschuberth added the help wanted An issue where third-party help is wanted on label Apr 7, 2022
sschuberth added a commit that referenced this issue Apr 11, 2022
The original `pydep` tool is archived by now [1] and running it with
newer Python versions might cause problems [2]. So replace it by simply
"running" `setup.py` with the option for the metadata field to get.

Resolves #2816.

[1]: https://github.com/sourcegraph/pydep
[2]: #5159 (comment)

Signed-off-by: Sebastian Schuberth <[email protected]>
sschuberth added a commit that referenced this issue Apr 11, 2022
The original `pydep` tool is archived by now [1] and running it with
newer Python versions might cause problems [2]. So replace it by simply
"running" `setup.py` with the option for the metadata field to get.

Resolves #2816.

[1]: https://github.com/sourcegraph/pydep
[2]: #5159 (comment)

Signed-off-by: Sebastian Schuberth <[email protected]>
sschuberth added a commit that referenced this issue Apr 11, 2022
The original `pydep` tool is archived by now [1] and running it with
newer Python versions might cause problems [2]. So replace it by simply
"running" `setup.py` with the option for the metadata field to get.

Resolves #2816.

[1]: https://github.com/sourcegraph/pydep
[2]: #5159 (comment)

Signed-off-by: Sebastian Schuberth <[email protected]>
@sschuberth sschuberth changed the title ORT not able to analyze any Python based project (setuptool breaking ) ORT not able to analyze any Python based project (setuptool breaking) Apr 11, 2022
@sschuberth
Copy link
Member

@meeraa247, while your original post does not detail the actual error message, I assume it's really the same as in @heliocastro's post. So, can you guys give #5247 a try to confirm it fixes this specific issue?

@sschuberth
Copy link
Member

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference:
https://github.com/fastavro/fastavro

Interestingly, that project exclusively uses a developer_requirements.txt file, which does even trigger the code path to run pydep in ORT. So I'm wondering how you could be seeing

File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>

sschuberth added a commit that referenced this issue Apr 12, 2022
The original `pydep` tool is archived by now [1] and running it with
newer Python versions might cause problems [2]. So replace it by simply
"running" `setup.py` with the option for the metadata field to get.

Resolves #2816.

[1]: https://github.com/sourcegraph/pydep
[2]: #5159 (comment)

Signed-off-by: Sebastian Schuberth <[email protected]>
@sschuberth
Copy link
Member

So, can you guys give #5247 a try to confirm it fixes this specific issue?

Feel free to reopen if this exact same issue still appears.

@HardikRa
Copy link

I ran into the same error while trying to scan the repository for Django
ort/cli/build/install/ort/bin/ort -P ort.analyzer.allowDynamicVersions=true analyze -i /home/complab304pc30/Documents/django -o /home/djangoORTAnalyzerOutput -f JSON

This is the output for the same

/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
15:57:24.423 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.648 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
15:57:24.658 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.662 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.677 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.679 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.688 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:25.312 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.122189590s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.

The following is the output when run with --stacktrace

 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:07.932 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.129 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
16:08:08.161 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.166 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.178 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.180 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.186 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
16:08:08.784 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.031969722s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.

@sschuberth
Copy link
Member

@HardikRa, the ORT distribution (in contrast to the Docker image) does not come with third-party tools as required by the analyzer. In particular, you need to ensure that the python-inspector Python package is installed and available in the PATH environment of the user running ort.

@HardikRa
Copy link

I have installed the python-inspector package and added it to the PATH, it resolved most errors, however there is still one error that popped up

11:38:27.784 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IllegalArgumentException: Unable to determine a project name for '/home/complab304pc30/Documents/django/setup.py'.

@sschuberth
Copy link
Member

sschuberth commented Feb 21, 2023

@HardikRa that's a new issue and I've tracked it as such.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer About the analyzer tool bug Issues that are considered to be bugs help wanted An issue where third-party help is wanted on
Projects
None yet
Development

No branches or pull requests

4 participants