-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid entry point lookups in Node/Group metaclasses #6091
Avoid entry point lookups in Node/Group metaclasses #6091
Conversation
Benchmarks against the current main branch
|
The one failing Rabbitmq test is confusing. It only happened for RMQ version 3.7, but it happened twice already so I am not sure if it is just a flaky test or not. |
ea71dc4
to
cce6d05
Compare
These class attributes require a look up whether the `Node` class has a registered entry point which can have a non-negligible cost. These attributes were defined in the `AbstractNodeMeta` class, which is the metaclass of the `Node` class, which would cause this code to be executed as soon as the class was imported. Here, the `AbstractNodeMeta` metaclass is removed. The `_plugin_type_string` and `_query_type_string` class attributes are changed to class properties. The actual value is stored in the private attribute analog which is defined lazily the first time the property is accessed.
This is a follow-up on previous commit aiming to speedup the import of the `aiida.orm` by avoiding costly entry point lookups. Here we completely remove the `GroupMeta` metaclass and move its logic into the `_typestring` classproperty, which avoids the code being executed on import while being backwards compatible.
This forces the import of `aiida.cmdline` in `aiida.orm` which doesn't just slow down, but also is conceptually wrong. The problem of the `with_dbenv` decorator is also that it cannot be imported inside a method to avoid the import cost when importing `aiida.orm` but has to be imported at the top in order to be used.
cce6d05
to
9301cdd
Compare
There were two more top-level imports of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @danielhollas
Thanks @sphuber! I'll work on submitting the other PRs which should be less risky than this one. |
It turns out a big part of why aiida.orm import is so slow are costly entry point lookups
in AbstractNodeMeta and GroupMeta metaclasses.
Commit 1:
_plugin_type_string
and_query_type_string
These class attributes require a look up whether the
Node
class has aregistered entry point which can have a non-negligible cost. These
attributes were defined in the
AbstractNodeMeta
class, which is themetaclass of the
Node
class, which would cause this code to beexecuted as soon as the class was imported.
Here, the
AbstractNodeMeta
metaclass is removed. The_plugin_type_string
and_query_type_string
class attributes arechanged to class properties. The actual value is stored in the private
attribute analog which is defined lazily the first time the property is
accessed.
Commit 2:
__type_string
This attribute was defined in the GroupMeta metaclass.
Because this was the only purpose of this metaclass,
we can fully remove it and use the same strategy as above.
Commit 3: Avoiding
aiida.cmdline
importThere are some further entry point lookups in metaclasses defined in
aiida.cmdline
used for the click interface.It's not that clear how to get rid of those, but here we at least avoid importing
aiida.cmdline
fromaiida.orm
andaiida.manage
,which also speeds up
import aiida
.With this last change, there are no entry point lookups during
aiida.orm
import. 🎉Benchmarks
TODO: I will add benchmarks shortly...