Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework get_installation_order to allow for dependency cycles #8127

Merged
merged 9 commits into from
May 3, 2020

Conversation

pradyunsg
Copy link
Member

@pradyunsg pradyunsg commented Apr 24, 2020

Switches our ordering strategy to "weight" nodes to be based on: The length of the longest path, from root to the node (ignoring any paths containing cycles).

Our graph is a directed cyclic graph, so, that's fun to think about. :)

@pradyunsg pradyunsg added the skip news Does not need a NEWS file entry (eg: trivial changes) label Apr 24, 2020
@uranusjr
Copy link
Member

If you mean the root node in graph being None, my intention is to use it as a defined behaviour, not internal 🙂 Feel free to depend on it.

@pradyunsg
Copy link
Member Author

If you mean the root node in graph being None, my intention is to use it as a defined behaviour, not internal 🙂 Feel free to depend on it.

Alrighty! IMO we should document this expectation on AbstractResolver. :)


All good then, this is ready for review... once I fix the failing test.

I don't think we'd ever get a graph where some edges aren't accessible from the root node. passed to the get_installation_order function, which is what the test is doing (which can be seen by renaming the packages). :)

        (
            [
                (None, "pkg1"),
                (None, "pkg2"),
                (None, "pkg3"),
                (None, "pkg4"),
                ("pkg5", "pkg1"),
                ("pkg6", "pkg1"),
                ("pkg7", "pkg1"),
                ("pkg7", "pkg5"),
                ("pkg7", "pkg6"),
            ],
            [
                "pkg1==0.0.1",
                "pkg5==0.0.1",
                "pkg6==0.0.1",
                "pkg7==0.0.1",
            ],
        ),

@uranusjr
Copy link
Member

I don't think we'd ever get a graph where some edges aren't accessible from the root node.

You won’t, the resolver guarentees this. (Maybe worthwhile to add a test on ResolveLib though? Not sure.)

@pfmoore
Copy link
Member

pfmoore commented Apr 24, 2020

Maybe worthwhile to add a test on ResolveLib though?

Property based testing would be ideal for this :-)

@pradyunsg
Copy link
Member Author

pradyunsg commented Apr 24, 2020

We want the length for the longest path to any node from root, ignoring any paths containing cycles.

Hmm...

A good test case for this is taking a graph like None -> A -> B -> C -> D -> E. We should have the same/similar result to this, even after adding any of the following edge-choices: None -> [BCDE], [AB] -> D, D -> [ABC].

@pradyunsg pradyunsg marked this pull request as draft April 24, 2020 20:31
@pradyunsg
Copy link
Member Author

Sitting and working on this now.

@pradyunsg
Copy link
Member Author

IT WORKS! IT WORKS! IT WORKS!

@pradyunsg pradyunsg marked this pull request as ready for review April 24, 2020 22:39
We take the length for the longest path to any node from root, ignoring
any paths that contain any node twice (i.e. cycles).
O(n) list lookups to O(1) set lookups
@pradyunsg pradyunsg requested a review from uranusjr April 24, 2020 22:55
@uranusjr
Copy link
Member

I can’t say I grok the logic, but can’t pick on an obvious problem either. All YAML tests on circular dependencies pass as well. Can you check what is failing this one?

tests/functional/test_yaml.py::test_yaml_based[install/conflicting_triangle]

I categorsied it under get_installation_order as well, but that is still failing. But maybe it’s for another reason now.

@pradyunsg
Copy link
Member Author

pradyunsg commented Apr 25, 2020

tests/functional/test_yaml.py::test_yaml_based[install/conflicting_triangle]

Ah, yes.

That's a test that's supposed to be testing for what conflicts we report (there's a conflicting "response" there). We don't have support for checking for that, in the YAML tests themselves, so... it'll stay the way it is. It's being skipped for all resolvers right now, but exists as a optimistic "we'll get there one day" statement. :)

Copy link
Member

@pfmoore pfmoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the way that algorithm turned out! It's pretty comprehensible, which isn't always the case for graph algorithms 🙂

@pfmoore pfmoore merged commit 2483fb6 into pypa:master May 3, 2020
@pradyunsg pradyunsg deleted the resolver/installation-order branch May 4, 2020 13:42
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
skip news Does not need a NEWS file entry (eg: trivial changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants