Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/entry points by group and name #278

Merged
merged 20 commits into from
Feb 24, 2021

Conversation

jaraco
Copy link
Member

@jaraco jaraco commented Jan 23, 2021

In pypa/twine#728, I learned that the dict hack for resolving a group of entry points by name is unintuitive, violates static type checks, and still doesn't provide an elegant way to resolve a set of entry points by group and name.

This finding inspired me to create this patch to provide an even nicer experience. See the changes to the docs and tests for examples of the improved usage.

The change is backward compatible for usages under test, but deprecates the __iter__ on EntryPoint and dict-based construction.

@jaraco
Copy link
Member Author

jaraco commented Jan 23, 2021

The change is backward compatible for usages under test,

Hyrum's law strikes again. Seems Flake8 is relying on the output of entry_points() being a proper dict and having a .get method.

@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from 04b3d28 to 28adeb8 Compare January 23, 2021 03:44
@jaraco
Copy link
Member Author

jaraco commented Jan 23, 2021

Seems Flake8 is relying on the output of entry_points() being a proper dict and having a .get method.

So this finding begs the question, what other use-cases are there out there that expect entry_points to return a mutable mapping? Are there use-cases that rely on __iter__ to return the group names? Are there use-cases that mutate the dict after? Are those use-cases dominant enough to demand backward-compatibility and deprecation warnings (like I've done for .get in 342a94b)?

@jaraco jaraco requested a review from warsaw January 23, 2021 17:09
@jaraco
Copy link
Member Author

jaraco commented Jan 23, 2021

Here's my answer to those questions - I recommend to merge this change as written, cut a release of importlib_metadata, and see if users report any issues. If not, then it's probably safe to merge into CPython 3.10. Otherwise, I'll add compatibility shims to support the reported issues as well.

@jaraco
Copy link
Member Author

jaraco commented Jan 23, 2021

@sigmavirus24: Would you care to review this change, given that it was inspired by a twine issue and affects flake8?

@jaraco
Copy link
Member Author

jaraco commented Jan 27, 2021

I plan to merge this as early as Friday unless I hear from Barry or Ian that they'd like me to hold off and await their review.

@sigmavirus24
Copy link

@asottile you rewrote flake8 to use importlib-metadata. Thoughts?

importlib_metadata/__init__.py Outdated Show resolved Hide resolved
"""

def __getitem__(self, group) -> EntryPoints:
return EntryPoints(ep for ep in self if ep.group == group)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit unfortunate that this is O(N) on every key access -- this will likely surprise consumers who may access this in a loop

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Perhaps there's a more efficient way to handle it. If there's a use-case that demands it, I'd be happy to investigate an optimization. I do expect this approach to be less complex than sorting and grouping by group as before.

importlib_metadata/__init__.py Show resolved Hide resolved
@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from 00f10d9 to 9448e13 Compare February 16, 2021 00:02
@asottile
Copy link
Contributor

and actually, taking a step back you argued exactly the opposite point when I pointed this out originally (that it can't be removed): #97 (comment)

@jaraco
Copy link
Member Author

jaraco commented Feb 16, 2021

and actually, taking a step back you argued exactly the opposite point when I pointed this out originally (that it can't be removed): #97 (comment)

I'm not following this concern exactly (I'm unsure what point was argued). I re-read that thread from that comment and it seems that I was not adamant about the approach, but ultimately settled on the current compromise that I'd now like to amend. At the time, I didn't have another proposal that met the goals, but I believe this approach does. Your concerns were relevant then and are now. If you have a concern that's not addressed in another thread, could you re-state it and what you recommend?

@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch 2 times, most recently from 25c9634 to c9adbca Compare February 16, 2021 02:30
@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from c9adbca to 71fd4a7 Compare February 16, 2021 18:56
@jaraco
Copy link
Member Author

jaraco commented Feb 16, 2021

I've been considering an alternative experience that uses parameters to entry_points to solicit the group and name. In particular, something like:

def entry_points(group=All, name=All):
    return (ep for ep in dist.entry_points() for dist in distributions() if ep.name == name and ep.group == group)

In this form, the user will always get an iterator of EntryPoint objects and they can filter on group or name but otherwise just get an iterator of EntryPoints. The result would need to be wrapped in some object to provide backward compatibility, but that wrapper would eventually be removed.

A materialized list could be used in place of an iterator, although that approach would give the caller less control over the execution.

Would that approach be preferable?

… single collection that can select on 'name' or 'group' or possibly other attributes. Expose that selection in the 'entry_points' function.
@jaraco
Copy link
Member Author

jaraco commented Feb 21, 2021

I'm unsubscribing this thread since you're not listening to me.

With all due respect, I am listening, and I care very much about your opinion, and I want to find a solution and preferably one that doesn't take several years to implement. Thusfar, your recommendations have primarily been "don't change anything," which is why I've focused on explaining the motivations for the change. I've since added more context in separate issues to help distill the motivations and elucidate the challenges. I'm sorry you've disengaged, but I'll try to push forward without you and honor the concerns you've raised.

a quick search on github shows that hundreds of others are using the returned dictionary and return list as dictionaries and lists -- please don't break them

I believe that's what I've done. The current approach retains the __getitem__ and get support and acknowledges there may be other uses that could break. I'd love to be able to discover those early and see if a design could be implemented to provide compatibility. I wish you'd saved the query or provided some characteristic references. I tried doing the search myself and found it very difficult to find meaningful results.

My plan currently is to come up with a design that's largely compatible, has easy adjustments for rare incompatibilities, and declares this change with a backward-incompatible release.

@jaraco jaraco closed this Feb 21, 2021
@jaraco jaraco reopened this Feb 21, 2021
domdfcoding added a commit to domdfcoding/domdf_python_tools that referenced this pull request Feb 22, 2021
@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from e411927 to 9d55a33 Compare February 22, 2021 13:48
@jaraco
Copy link
Member Author

jaraco commented Feb 23, 2021

I've been thinking about the compatibility concerns and I have a new idea - it should be possible to cut a fully compatible release of these changes, exposing the new .select behavior, and then defer the backward-incompatible behavior for a future release.

@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from e0ec362 to a474726 Compare February 23, 2021 15:06
…rd compatibilty to the new interfaces without sacrificing backward compatibility.
@jaraco jaraco force-pushed the feature/entry-points-by-group-and-name branch from a474726 to 2db4dad Compare February 23, 2021 15:09
@jaraco
Copy link
Member Author

jaraco commented Feb 24, 2021

In this latest version, I've come up with a fully compatible release with forward compatibility for the new, simpler, and more flexible interface. I'd like to get it out to ensure it's stable. Thanks for the comments - they really helped me figure out a better transition here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants