CEP 4 - Plugins Mechanism Implementation #32

beeankha · 2022-07-06T14:13:25Z

This CEP is for the implementation of a new plugins mechanism for the conda codebase. It is about the initial mechanism of plugins, versus for any specific new plugins (those will be proposed and discussed in their own separate CEPs after the implementation of plugins has been completed).

This pull request is intended as a public forum for discussing the plugins feature proposal.

cep-plugin-infrastructure.md

travishathaway · 2022-08-03T19:43:48Z

cep-plugin-infrastructure.md

+
+### Hook
+
+Below is an example of a very basic plugin "hook":


Do we want to define different types of hooks in this CEP? For example, I did some quick brainstorming was able to come up with the following:

Hook Description

Sub-command hook Allows plugin authors to create an entirely new sub-command as is currently described in this open pull-request: conda/conda#11435

Exception handling hook Allows a plugin author to hook in to the way that exceptions are handled and open up the possibility to add telemetry for exception logging (e.g. via services like Sentry)

HTTP/CondaSession hook Allows a plugin author to completely override the way that HTTP requests are handled in conda. This opens up the possibility for more complex authentication mechanisms that we currently cannot support (e.g. OAuth workflows). This too could be used for telemetry for logging.

Solver hook Allows plugin authors to either modify or completely override the solver used in conda.

pre/post command hook Allows plugin authors to run something either before or after a list of specified commands (i.e. arbitrary command decorators)

I would like to see us detail as many of these hooks as possible in this CEP and leave the door open for defining new hooks when we need.

If we do end up just leaving the sub-command hook here as the only hook, could we at least refer to it as the "Sub-Command Hook" and announce that there will be more hooks added as necessary?

I think we'd want to include the ones that make sense but avoid trying to create an exhaustive list. Somewhat of YAGNI at play, we could try to spend too much time thinking of future scenarios that might not come up. We know we won't be able to capture everything so I'd suggest we outline some pragmatic choices with perhaps details on how to document new hooks in the future.

I am also skeptical about adding a lot of hooks at once. Each hook is an open door we might not be able to close in the future, and I would think very carefully about adding hooks (and where exactly in the codebase).

could we at least refer to it as the "Sub-Command Hook" and announce that there will be more hooks added as necessary?

This approach sounds perfect to me!

LtDan33 · 2022-08-03T23:11:33Z

Some things we could consider:

Where plugins will live
How someone could find them
How to see which plugins are installed

jezdez · 2022-08-05T13:01:32Z

@conda-incubator/steering

This PR falls under the Enhancement Proposal Approval policy of the conda governance policy, please vote and/or comment on this PR.

This PR needs 60% of the Steering Council to vote yea to pass.

To vote please leave Approve (yea) or Request Changes (nay) pull request reviews.

If you would like changes to the current language please leave a comment (in the PR) or push to this branch.

This vote will end on 2022-08-12.

marcelotrevisani · 2022-08-05T19:12:19Z

As I mentioned before, I have a lot of interest in this topic and it would be really nice to have several pre and pos hooks for conda actions, not just the solver

#1 (comment)

cep-4.md

brettcannon · 2022-08-10T16:49:35Z

cep-4.md

+_setup.py_
+```python
+import setuptools
+
+if __name__ == "__main__":
+    setuptools.setup()
+```


setup.py is actually optional with setuptools now if you define pyproject.toml.

brettcannon · 2022-08-10T16:51:22Z

cep-4.md

+_pyproject.toml_
+```toml
+[build-system]
+requires = ["setuptools", "setuptools-scm"]


Based on my personal experience, I would avoid explicitly defining a full example of how to package a project like this and instead focus on how to specify an entry point. Otherwise I could see people copy-and-pasting either example for all their plugins as the way to package conda plugins.

I would also list the pyproject.toml example first as that's the standard.

brettcannon · 2022-08-10T16:54:07Z

cep-4.md

+dependencies = ["conda"]
+
+[project.entry-points."conda"]
+my-conda-plugin = "my_plugin.module:function"


Is there a specific reason this is different than the setup.py example? That one points at a module while this points as a function in a submodule.

That was my mistake! I've updated it to be more consistent with the other example, thank you for pointing this out.

jezdez · 2022-08-12T17:48:23Z

@conda-incubator/steering

This vote falls under the Enhancement Proposal Approval policy of the conda governance policy, please vote and/or comment on this PR.

This vote needs 60% of the Steering Council to vote yea to pass.

This vote presently has 6, and needs 4 more for quorum.

It is proposed that this vote will time out and be evaluated with the current votes in 7 days, on 2022-08-19.

To vote please leave Approve (yea) or Request Changes (nay) reviews.

- pyproject.toml needs to be listed first, as it is the standard - fix inconsistency in the project.entry-points section of pyproject.toml - remove setup.py file for pyproject.toml example

chenghlee · 2022-08-24T14:51:45Z

Voting Results

This was a standard, non-timed-out vote.

Among Steering Council members there are 10 "yes", 0 "no", and no abstentions.
No Emeritus Steering member voted.

This vote has reached quorum (10 + 0 = 10 which is at least 60% of 16).

It has also passed since it recorded 10 "yes" votes and 0 "no" votes giving 10/10 which is greater than 60% of 15.

It should be noted that a request for change was recorded in the pull request about minor implementation details that do not invalidate the previous votes. The author made the requested change.

Initial commit of plugin mechanism CEP

a8bff64

beeankha marked this pull request as ready for review July 6, 2022 15:10

Update register decorator and function name in example

c228569

beeankha force-pushed the initial_plugins_cep branch from 30422b2 to c228569 Compare July 7, 2022 00:00

jezdez reviewed Jul 25, 2022

View reviewed changes

cep-plugin-infrastructure.md Outdated Show resolved Hide resolved

jezdez reviewed Jul 25, 2022

View reviewed changes

cep-plugin-infrastructure.md Outdated Show resolved Hide resolved

jezdez reviewed Jul 25, 2022

View reviewed changes

cep-plugin-infrastructure.md Outdated Show resolved Hide resolved

beeankha added 2 commits July 26, 2022 12:29

Add email addresses, refer to CEP 2 in 'Abstract' section

9f848bf

Add an example for packaging using a pyproject.toml file

0b85e99

beeankha mentioned this pull request Jul 29, 2022

Draft a CEP for conda Plugins Standards and Best Practices #33

Open

travishathaway reviewed Aug 3, 2022

View reviewed changes

Assigning CEP 4 to the proposal.

8ec571c

jezdez changed the title ~~Plugins Mechanism Implementation CEP~~ CEP 4 - Plugins Mechanism Implementation Aug 5, 2022

jezdez added the vote Voting following governance policy label Aug 5, 2022

jezdez requested a review from a team August 5, 2022 13:04

jezdez approved these changes Aug 5, 2022

View reviewed changes

marcelotrevisani reviewed Aug 5, 2022

View reviewed changes

cep-4.md Show resolved Hide resolved

marcelotrevisani reviewed Aug 5, 2022

View reviewed changes

cep-4.md Show resolved Hide resolved

beckermr approved these changes Aug 5, 2022

View reviewed changes

msarahan approved these changes Aug 6, 2022

View reviewed changes

ocefpaf approved these changes Aug 6, 2022

View reviewed changes

jaimergp reviewed Aug 10, 2022

View reviewed changes

cep-4.md Outdated Show resolved Hide resolved

jaimergp reviewed Aug 10, 2022

View reviewed changes

cep-4.md Outdated Show resolved Hide resolved

jezdez reviewed Aug 10, 2022

View reviewed changes

cep-4.md Outdated Show resolved Hide resolved

jezdez reviewed Aug 10, 2022

View reviewed changes

cep-4.md Outdated Show resolved Hide resolved

jezdez added 2 commits August 10, 2022 17:06

Update cep-4.md

903bbff

Update cep-4.md

53db476

xhochy approved these changes Aug 10, 2022

View reviewed changes

brettcannon reviewed Aug 10, 2022

View reviewed changes

chenghlee approved these changes Aug 11, 2022

View reviewed changes

kkraus14 approved these changes Aug 12, 2022

View reviewed changes

marcelotrevisani approved these changes Aug 12, 2022

View reviewed changes

jaimergp approved these changes Aug 17, 2022

View reviewed changes

goanpeca approved these changes Aug 19, 2022

View reviewed changes

beeankha force-pushed the initial_plugins_cep branch from 01139da to 5aac33f Compare August 19, 2022 14:30

Update packaging examples

b3cba41

- pyproject.toml needs to be listed first, as it is the standard - fix inconsistency in the project.entry-points section of pyproject.toml - remove setup.py file for pyproject.toml example

beeankha force-pushed the initial_plugins_cep branch from 5aac33f to b3cba41 Compare August 19, 2022 14:31

Add 'Accepted' status and update Resolution section

aeb0a1d

beeankha force-pushed the initial_plugins_cep branch from 40700ae to aeb0a1d Compare August 22, 2022 23:21

chenghlee merged commit b1aa942 into conda:main Aug 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CEP 4 - Plugins Mechanism Implementation #32

CEP 4 - Plugins Mechanism Implementation #32

beeankha commented Jul 6, 2022

travishathaway Aug 3, 2022 •

edited

Loading

LtDan33 Aug 3, 2022

jaimergp Aug 10, 2022

LtDan33 commented Aug 3, 2022

jezdez commented Aug 5, 2022 •

edited

Loading

marcelotrevisani commented Aug 5, 2022

brettcannon Aug 10, 2022

brettcannon Aug 10, 2022

brettcannon Aug 10, 2022

beeankha Aug 19, 2022

jezdez commented Aug 12, 2022

chenghlee commented Aug 24, 2022

Hook	Description
Sub-command hook	Allows plugin authors to create an entirely new sub-command as is currently described in this open pull-request: conda/conda#11435
Exception handling hook	Allows a plugin author to hook in to the way that exceptions are handled and open up the possibility to add telemetry for exception logging (e.g. via services like Sentry)
HTTP/CondaSession hook	Allows a plugin author to completely override the way that HTTP requests are handled in `conda`. This opens up the possibility for more complex authentication mechanisms that we currently cannot support (e.g. OAuth workflows). This too could be used for telemetry for logging.
Solver hook	Allows plugin authors to either modify or completely override the solver used in `conda`.
pre/post command hook	Allows plugin authors to run something either before or after a list of specified commands (i.e. arbitrary command decorators)

CEP 4 - Plugins Mechanism Implementation #32

CEP 4 - Plugins Mechanism Implementation #32

Conversation

beeankha commented Jul 6, 2022

travishathaway Aug 3, 2022 • edited Loading

Choose a reason for hiding this comment

LtDan33 Aug 3, 2022

Choose a reason for hiding this comment

jaimergp Aug 10, 2022

Choose a reason for hiding this comment

LtDan33 commented Aug 3, 2022

jezdez commented Aug 5, 2022 • edited Loading

marcelotrevisani commented Aug 5, 2022

brettcannon Aug 10, 2022

Choose a reason for hiding this comment

brettcannon Aug 10, 2022

Choose a reason for hiding this comment

brettcannon Aug 10, 2022

Choose a reason for hiding this comment

beeankha Aug 19, 2022

Choose a reason for hiding this comment

jezdez commented Aug 12, 2022

chenghlee commented Aug 24, 2022

Voting Results

travishathaway Aug 3, 2022 •

edited

Loading

jezdez commented Aug 5, 2022 •

edited

Loading