ORT not able to analyze any Python based project (setuptool breaking) #5159

meeraa247 · 2022-03-17T14:26:35Z

Hi @ all,

first of all: We appreciate this community a all and would like to thank you for all the support. Our team heavily relies on scanning & analyzing projects using ORT.

Recently we came across a small project based on Python (v.3.8.x).
During the ANALYZE Step, we identified the following ERRORS (from the logs):

[DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'Python_v3.8.8/Lib/site-packages/numpy/ma/setup.py' failed with: IOException: Running '/tmp/ort-Pip-ma-virtualenv14989541890154123245/bin/pip --trusted-host pypi.org --trusted-host pypi.python.org install --no-warn-conflicts --prefer-binary .' in '/project/Python_v3.8.8/Lib/site-packages/numpy/ma' failed with exit code 1:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-t9xwYP/
You are using pip version 18.1, however version 20.3.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

At first, it seemed to be an issue with the so called "virtualenv". @heliocastro also checked and confirmed this behaviour from side.

We also tried to specify a dedicated python version for ORT (using the docker build process), but that did not help.

Any help would be highly appreciated as this blocks our processes for this project at the moment.

Thank you.

The text was updated successfully, but these errors were encountered:

heliocastro · 2022-03-18T09:28:41Z

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference:
https://github.com/fastavro/fastavro

Look like some discrepancy on the manipulating the setuptools new versions, as our dependency analyzer tool is pydep is really old. To be investigated.

Here's the output of analyzer error

09:23:47.125 [DefaultDispatcher-worker-3] DEBUG org.eclipse.jgit.util.SystemReader - loading config FileBasedConfig[/workspace/.gitconfig]
09:23:47.181 [DefaultDispatcher-worker-3] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'developer_requirements.txt' failed with: IOException: Running '/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py info .' in '/worspace/code' failed with exit code 1:
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptoolsalso replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>
main()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 43, in main
args.func(args)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 68, in info
setup_dict, err = pydep.setup_py.setup_info_dir(args.dir)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 41, in setup_info_dir
return setup_info(setupfile), None
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 53, in setup_info
setuptools_mod = __import__('setuptools')
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in <module>
import _distutils_hack.override  # noqa: F401
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
__import__('_distutils_hack').do_override()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 72, in do_override
ensure_local_distutils()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
assert '_distutils' in core.__file__, core.__file__
AssertionError: /opt/python/versions/3.8.13/lib/python3.8/distutils/core.py

sschuberth · 2022-04-07T08:36:19Z

To be investigated.

@heliocastro are you planning to work on this?

The original `pydep` tool is archived by now [1] and running it with newer Python versions might cause problems [2]. So replace it by simply "running" `setup.py` with the option for the metadata field to get. Resolves #2816. [1]: https://github.com/sourcegraph/pydep [2]: #5159 (comment) Signed-off-by: Sebastian Schuberth <sebastian.schuberth@bosch.io>

sschuberth · 2022-04-11T20:06:12Z

@meeraa247, while your original post does not detail the actual error message, I assume it's really the same as in @heliocastro's post. So, can you guys give #5247 a try to confirm it fixes this specific issue?

sschuberth · 2022-04-12T07:19:30Z

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference:
https://github.com/fastavro/fastavro

Interestingly, that project exclusively uses a developer_requirements.txt file, which does even trigger the code path to run pydep in ORT. So I'm wondering how you could be seeing

File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>

The original `pydep` tool is archived by now [1] and running it with newer Python versions might cause problems [2]. So replace it by simply "running" `setup.py` with the option for the metadata field to get. Resolves #2816. [1]: https://github.com/sourcegraph/pydep [2]: #5159 (comment) Signed-off-by: Sebastian Schuberth <sebastian.schuberth@bosch.io>

sschuberth · 2022-04-21T12:40:35Z

So, can you guys give #5247 a try to confirm it fixes this specific issue?

Feel free to reopen if this exact same issue still appears.

HardikRa · 2023-02-20T10:56:51Z

I ran into the same error while trying to scan the repository for Django
ort/cli/build/install/ort/bin/ort -P ort.analyzer.allowDynamicVersions=true analyze -i /home/complab304pc30/Documents/django -o /home/djangoORTAnalyzerOutput -f JSON

This is the output for the same

/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
15:57:24.423 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.648 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
15:57:24.658 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.662 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.677 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.679 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.688 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:25.312 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.122189590s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.

The following is the output when run with --stacktrace

 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:07.932 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.129 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
16:08:08.161 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.166 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.178 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.180 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.186 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
16:08:08.784 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.031969722s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.

sschuberth · 2023-02-20T11:25:46Z

@HardikRa, the ORT distribution (in contrast to the Docker image) does not come with third-party tools as required by the analyzer. In particular, you need to ensure that the python-inspector Python package is installed and available in the PATH environment of the user running ort.

HardikRa · 2023-02-21T06:12:50Z

I have installed the python-inspector package and added it to the PATH, it resolved most errors, however there is still one error that popped up

11:38:27.784 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IllegalArgumentException: Unable to determine a project name for '/home/complab304pc30/Documents/django/setup.py'.

sschuberth · 2023-02-21T08:44:32Z

@HardikRa that's a new issue and I've tracked it as such.

meeraa247 mentioned this issue Mar 21, 2022

Unknown Scanner Error for Python Files in packages #5168

Closed

sschuberth added bug Issues that are considered to be bugs analyzer About the analyzer tool labels Mar 25, 2022

sschuberth mentioned this issue Mar 25, 2022

Improve resolution of Python / PIP dependencies #4637

Closed

10 tasks

sschuberth added the help wanted An issue where third-party help is wanted on label Apr 7, 2022

sschuberth changed the title ~~ORT not able to analyze any Python based project (setuptool breaking )~~ ORT not able to analyze any Python based project (setuptool breaking) Apr 11, 2022

sschuberth closed this as completed Apr 21, 2022

sschuberth mentioned this issue Feb 21, 2023

Unable to determine a project name for Python project #6524

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ORT not able to analyze any Python based project (setuptool breaking) #5159

ORT not able to analyze any Python based project (setuptool breaking) #5159

meeraa247 commented Mar 17, 2022

heliocastro commented Mar 18, 2022 •

edited

Loading

sschuberth commented Apr 7, 2022

sschuberth commented Apr 11, 2022

sschuberth commented Apr 12, 2022

sschuberth commented Apr 21, 2022

HardikRa commented Feb 20, 2023

sschuberth commented Feb 20, 2023

HardikRa commented Feb 21, 2023

sschuberth commented Feb 21, 2023 •

edited

Loading

ORT not able to analyze any Python based project (setuptool breaking) #5159

ORT not able to analyze any Python based project (setuptool breaking) #5159

Comments

meeraa247 commented Mar 17, 2022

heliocastro commented Mar 18, 2022 • edited Loading

sschuberth commented Apr 7, 2022

sschuberth commented Apr 11, 2022

sschuberth commented Apr 12, 2022

sschuberth commented Apr 21, 2022

HardikRa commented Feb 20, 2023

sschuberth commented Feb 20, 2023

HardikRa commented Feb 21, 2023

sschuberth commented Feb 21, 2023 • edited Loading

heliocastro commented Mar 18, 2022 •

edited

Loading

sschuberth commented Feb 21, 2023 •

edited

Loading