Replace pipdeptree with python-inspector #1
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Status of the python-inspector integration in ORT:
Locally we can also run this way:
|
This was referenced Aug 2, 2022
The previous algorithm exhibits the following issues when run on graphs with cycles, in particular when the amount of edges is large: 1. Cycles of length 1 lead to infinite recursion. 2. The algorithm is inefficient in terms of execution time and memory allocation, as it creates a copy of the `predecessorNodes` set per recursion. 3. When run against [1] the execution of `toPackageReferenceForest()` didn't finish within 15 minutes, causing high CPU load and memory consumption. If GoMod used the dependency graph format already, the solution would be as simple as calling `DependencyGraphBuilder.breakCycles()`. Fix the problem by copying the code of `DependencyGraphBuilder.breakCycles()` as a temporary quick fix. This saves effort, because refactoring GoMod to use the dependency graph is planned anyway [2], which will allow for removing that copied code again. [1] https://github.com/ossf/scorecard [2] oss-review-toolkit#4249 Fixes oss-review-toolkit#5627. Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
The rule is nested inside a package rule, whereas only the inner rule may flag violations. The rule name of the outer rule is unused. Align both names for consistency with other rules and use a slightly shorter name which does not mention all aspects of the `require` conditions. This relaxes the need for renaming the rule in case of adjustments to the `require` condition. Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
While at it, remove the accidental empty line. Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Frank Viernau <frank_viernau@epam.com>
Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
The submodules are found by traversing the `node_modules` directory which can become quite large. To avoid doing this if it is not required, use a cache for the result.1 Additionally, factor out the `loadWorkspaceSubmodules()` method, so the discovery of submodules can be done differently in NPM's subclasses. Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
PNPM [1] is an alternative package manager for JavaScript / TypeScript projects. To detect PNPM projects, the `pnpm-lock.yaml` file is being used. By using the `--shamefully-hoist` option when installing packages using PNPM, a directory structure similar to NPM and Yarn is created for the `node_modules` directory. This enables ORT to reuse the NPM implementation to resolve dependencies. [1]: https://pnpm.io Resolves oss-review-toolkit#2024. Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
The issue [1] was resolved. [1]: oss-review-toolkit#3741 Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
…re License 1.1.1' Found in [1], mapped based on [2] and [3]. [1]: https://repo.maven.apache.org/maven2/org/jboss/modules/jboss-modules/1.11.0.Final/jboss-modules-1.11.0.Final.pom [2]: http://www.bearcave.com/software/java/xml/xmlpull_license.html [3]: https://github.com/nexB/scancode-toolkit/blob/ffc47b117ef382d42311bff1fc8708ea42241ff2/src/licensedcode/data/licenses/indiana-extreme.LICENSE Signed-off-by: Thomas Steenbergen <thomas_steenbergen@epam.com>
Found in [1]. [1]: https://repo.maven.apache.org/maven2/org/jboss/forge/roaster/roaster-parent/2.19.2.Final/roaster-parent-2.19.2.Final.pom Signed-off-by: Thomas Steenbergen <thomas_steenbergen@epam.com>
…GPLv2)' Found in [1]. [1]: https://repo.maven.apache.org/maven2/com/unboundid/unboundid-ldapsdk/3.2.0/unboundid-ldapsdk-3.2.0.pom Signed-off-by: Thomas Steenbergen <thomas_steenbergen@epam.com>
…ion 2.1 (LGPLv2.1)' Found in [1]. [1]: https://repo.maven.apache.org/maven2/com/unboundid/unboundid-ldapsdk/3.2.0/unboundid-ldapsdk-3.2.0.pom Signed-off-by: Thomas Steenbergen <thomas_steenbergen@epam.com>
[1]: https://saxonica.plan.io/projects/saxon/issues?utf8=%E2%9C%93&set_filter=1&sort=id%3Adesc&f%5B%5D=status_id&op%5Bstatus_id%5D=c&f%5B%5D=cf_6&op%5Bcf_6%5D=%3D&v%5Bcf_6%5D%5B%5D=85&f%5B%5D=&c%5B%5D=tracker&c%5B%5D=status&c%5B%5D=priority&c%5B%5D=subject&c%5B%5D=assigned_to&c%5B%5D=updated_on&group_by=&t%5B%5D= Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
The list of copyrights was compiled by a combination of a source code search for for `Copyright (C)` and from the information in `.reuse/dep5`. Signed-off-by: Marcel Bochtler <marcel.bochtler@bosch.io>
TG1999
force-pushed
the
pyinsp
branch
2 times, most recently
from
a56816f
to
9137a68
Compare
Do not set the exit code to 2 if a tool version cannot be determined, and the tool does set a version requirement. Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.io>
The configured precision of 36 never really had any effect as the number refers to the number of name components, not characters, and no logger name exceeded 36 components. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
These versions could theoretically be different, and independent versioninig eases the migration to a different logging implementation only while keeping the API. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
This change is not so much about the past security flaws in the Log4j implementation [1], but more about generally switching (back) to a more lightweight logger that also plays more nicely with GraalVM [2]. However, stick to Log4j on the API side as it is more powerful than SLF4J, see [3]. This commit restores the Logback configuration files from bac29f6 and updates them with changes meanwhile done to `log4j2.xml`. [1]: https://blog.ltgt.net/migrating-off-log4j2/ [2]: micronaut-projects/micronaut-core#6041 (comment) [3]: https://kajalrawal.medium.com/log4j2-is-it-worth-to-use-slf4j-with-log4j2-21e83d0d792c Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
See [1]. Once [2] is resolved the official Gradle plugin should be used instead of the current Palantir plugin [3] to benefit from automatic download of reachability-metadata [4]. Note that still more work is required to make the native-image build succeed. [1]: https://github.com/oracle/graalvm-reachability-metadata/blob/master/metadata/ch.qos.logback/logback-classic/1.2.11/reflect-config.json [2]: graalvm/native-build-tools#100 [3]: https://github.com/palantir/gradle-graal [4]: https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html#metadata-support Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
The format was changed in 4ea9863. Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.io>
The TODO belongs to the PostgreSQL connection parameters which were moved to a separate class in 4ea9863. Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.io>
Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.io>
Add a warning announcing that the experimental scanner will replace the current default scanner implementation. Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.io>
…try fails This allows to handle packages that were never published to the NPM registry but only ever referenced by (short) repository URL. Fixes oss-review-toolkit#5632. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
This is the same code as in the base class. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Newer versions depend on newer versions of the graphql-java library [1] that limit the number of tokens to be parsed. ORT's GitHub client library exceeds that token limit, so the plugin needs to make the maximum configurable. [1]: https://github.com/graphql-java/graphql-java Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
This fixes a deployment issue [1] with version 1.12.6 [2]. [1]: mockk/mockk#884 [2]: https://github.com/mockk/mockk/releases/tag/1.12.6 Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
See: https://github.com/kotest/kotest/releases/tag/v5.4.2 Signed-off-by: Marcel Bochtler <git@bochtler.io>
TG1999
force-pushed
the
pyinsp
branch
3 times, most recently
from
d9bf6b5
to
76f44ff
Compare
This PR replaces pipdeptree with python-inspector to resolve Python packages dependencies found in requirement files. python-inspector can resolve dependencies for any target Python version and OS (and not only the one running the tool). In this integration in ORT, it replaces pipdeptree pretty much in place as python-inspector implements a similar output data structure by design to ease the integration. Reference: https://github.com/nexB/python-inspector Reference: oss-review-toolkit#4637 Reference: oss-review-toolkit#3671 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> Signed-off-by: Tushar Goel <tushar.goel.dav@gmail.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> Signed-off-by: Tushar Goel <tushar.goel.dav@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Tushar Goel tushar.goel.dav@gmail.com