Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor SimpleNodeMixin. #4

Merged
merged 1 commit into from
Apr 28, 2021

Conversation

avolkov1
Copy link

Refactored meta/ports resolve and extended logic.

Split the original SimpleNodeMixin class into the following classes:

  • NodeExtensionMixin - handles the ports and metadata resolve logic previously done in SimpleNodeMixin.update.
  • NodeTaskGraphExtensionMixin - handles the ports and meta extended logic previously done in SimpleNodeMixin.ports_setup and SimpleNodeMixin.meta_setup.
  • TemplateNodeMixin - Uses the APIs of NodeExtensionMixin and NodeTaskGraphExtensionMixin to implement the templating logic of SimpleNodeMixin for nodes that inherit from it.

Other miscellaneous fixes and updates had to do with dynamic ports logic, some greenflow lab server side updates, and unit tests updates.

Tested all the unit tests. Tested all the notebooks under gquant_plugin.

Refactored meta/ports resolve and extended logic.
Copy link
Author

@avolkov1 avolkov1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self review.

This refactoring essentially boils down to the principle of programming to interfaces not implementation. In my opinion the SimpleNodeMixin felt like programming to implementation. With the refactoring I hoped to adhere better to the principle of programming to interfaces. The following is my explanation of how this is so.

I wanted to organize the original SimpleNodeMixin class into separate components i.e. split into multiple classes. These classes ideally would make it clearer about intent of usage of the convenience syntax for ports and meta. The TemplateNodeMixin is the corresponding implementation of what SimpleNodeMixin did. Prior to refactoring developers could not re-use the core logic of SimpleNodeMixin to implement custom Nodes without re-using SimpleNodeMixin verbatim.

The refactoring makes it possible to re-use via interfaces the logic of resolving ports/meta, dynamic ports and ports_setup_ext/meta_setup_ext, outside of just SimpleNodeMixin. By inheriting from Node all the core logic interfaces are available without having to know any particular implementation details. Developers can use the core logic (resolving ports/meta, dynamic ports and ports_setup_ext/meta_setup_ext) to implement custom Nodes without using TemplateNodeMixin.

@yidong72
Copy link
Owner

Looks good. I will merge as is and do changes in my branch.

@yidong72 yidong72 merged commit 94314af into yidong72:dask-performance Apr 28, 2021
@avolkov1 avolkov1 deleted the dask-performance branch April 30, 2021 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants